Polling Inquiry – FAQs
Inquiry into the Performance of the Opinion Polls at the 2019 Federal Election
Why did AMSRO launch a polling inquiry?
The Federal Election result on May 18, 2019 caught almost everyone by surprise – the key reason being the mismatch between the result on the day and what the reported results of the leading polling organisations had been predicting over the preceding three years.
Since then, there has been widespread criticism of political polling, with no shortage of people inside and outside our profession providing their diagnoses of what went wrong and how to fix it.
Two things were clear:
- There is no consensus from the polling companies, or any other quarter, as to ‘what went wrong.’
- The credibility of opinion polling with the Australian public appears to be at an all-time low.
The Association of Market and Social Research Organisations (AMSRO) and the Statistical Society of Australia (SSA) believed it to be of vital importance to address public concerns about the accuracy of the polls because concerns about polling have the potential to undermine confidence in the wider market research industry.
At the time (in Australia) there was no specific body representing opinion polling organisations. AMSRO is the closest thing we have in Australia to such an association (some, but by no means all, companies conducting political polling are AMSRO members). It therefore fell to AMSRO and the SSA to examine the accuracy of the polls in the lead up to the election, determine the cause/s of any inaccuracies identified and suggest improvements to how polls are conducted and how their results are communicated to the general public. In this way, it is hoped that the credibility of polling can be restored in the eyes of the Australian public.
Key to restoring credibility was an inquiry process that everyone can have confidence in. In establishing the Polling Inquiry, AMSRO sought to proceed so as to maximise the prospects of achieving the ultimate aim of restoring credibility with polling and market research. The view being that the inquiry process is impartial, robust, and comprehensive so that it can produce results that have credibility.
On the Monday after the May 2019 election AMSRO announced that it would immediately commence an inquiry, with the initial tasks being to establish Terms of Reference and an Inquiry Panel.
On 7 June, AMSRO announced the formation of the independent Inquiry Panel and draft Terms of Reference. The Terms of Reference and the make-up of the Inquiry Panel and Advisory Board are publicly available on AMSRO’s website: www.amsro.com.au. Further information is available here.
What is AMSRO’s role?
- Appoint the Inquiry Panel and Advisory Board
- Determine the Terms of Reference following feedback from the Advisory Board and Inquiry Panel, and other relevant stakeholders.
- Encourage, promote, and support the work of the review – provide (non-binding) comments on reporting, provide additional resources from AMSRO member companies as requested by the Inquiry Panel, reimburse travel and other reasonable expenses incurred
- Once the Inquiry Panel provides its final report – to publish, promote and support the final report and its recommendations.
- To field media queries during the Inquiry process and after the release of the report – so that the Inquiry Panel may conduct its work with a minimum of distraction.
Who is on the Inquiry Panel?
The Inquiry Panel is comprised of technical specialists in polling, statistics, sampling, and related fields. No individuals working for organisations currently conducting political polling in Australia are represented on the panel. This is to provide the polling companies with the reassurance that they are not being judged by their competitors, as it is vital that they participate in the Inquiry.
What the role of the Inquiry Panel?
Darren Pennay is the Chair of the Inquiry, a task for which he is well qualified and equipped. Darren is the founder and immediate past-CEO of The Social Research Centre (SRC), part of the ANU, and one of Australia’s leading social research organisations. Darren leads a panel of nine other members with relevant experience who are all highly regarded in their fields. AMSRO is fortunate to have been able to access some of the best-qualified experts in polling and survey statistics, including international experts who worked on the Polling Inquiries undertaken in the wake of the 2015 UK General Election and the 2016 US Presidential Election.
Who governs the Inquiry?
The Inquiry periodically report back to AMSRO on progress but acted independently of AMSRO. The Inquiry Panel, not AMSRO, determine the final report and the Inquiry’s recommendations. The independence of the panel is important, particularly given that some polling companies are AMSRO members while others are not.
Why is there an Advisory Board?
AMSRO felt it prudent to establish an Advisory Board, whose role is to provide suggestions and advice to the Inquiry Panel. The Advisory Board is comprised of senior figures with a range of relevant backgrounds. They include experts in statistics, political science, commercial market research and the media.
Who is on the Advisory Board?
AMSRO established an independent Advisory Board with diverse expertise in relevant areas for the Inquiry Panel to draw on for broad strategic guidance and input.
The role of the Advisory Board is to: review and comment on Inquiry Terms of Reference; review and provide (non-binding) comments on Inquiry Panel reports (including preliminary reporting, if any); provide input to the Inquiry Panel on additional relevant lines of inquiry.
AMSRO also invited a Publisher representative from each of Nine Entertainment (Sydney Morning Herald/The Age) and NewsCorp to participate in an advisory capacity as part of a special Reference Group, however they declined due to time constraints.
What co-operation did the Inquiry receive from the polling companies?
Ipsos, YouGov, Essential and Lonergan all participated in the Inquiry to some extent.
Ipsos supported the panel Inquiry’s work from the outset. They participated in an interview, provided comments on draft materials, and also provided access to their polling questionnaires and cross tabulations of their polling results. YouGov and Essential made written submissions and commented on draft materials. Lonergan research also participated in an interview and provided a questionnaire.
Ultimately, however, the Inquiry Panel advised that it was the lack of access to raw data sets and detailed descriptions of the survey methods used by the pollsters that distinguished this review from similar reviews undertaken overseas and hampered the Inquiry panel’s ability to conclusively identify the specific factors that contributed to the relative inaccuracy of the 2019 polls and identify improvements.
The final Report was released publicly on 11 November 2020.
For further information:
Sarah Campbell
Executive Director
AMSRO
