Media Release: Political Polling Inquiry Final Report released today

The Association of Market and Social Research Organisations (AMSRO) today announced that the Inquiry into the Performance of the Opinion Polls at the 2019 Australian Federal Election (AMSRO Polling Inquiry) has released its final report, including recommendations for how to improve political polling in Australia.

The report follows AMSRO’s decision, along with the Statistical Society of Australia, to instigate an independent and impartial review of election polling in Australia to determine why all the published polls incorrectly called the outcome at 2019’s May Federal Election and how polling might be improved in the future.

As evidenced by the Inquiry, from 2007 until 2016, Australian pollsters had, in aggregate, a 96% success rate (that is, 25 out of 26 final week polls called the right result) across four Federal elections. The 2019 May election, however, was a notable anomaly and AMSRO and the Statistical Society of Australia, on behalf of the wider industry, wanted to know what went wrong.  AMSRO invited pollsters, media organisations and others who commission election and political polling to contribute to the inquiry.

The independent inquiry report concluded that the collective performance of the published national election polls in 2019 constituted a ‘polling failure’ rather than simply a ‘polling miss’, as the polls erred in their estimate of the vote in a manner that was statistically significant and erred in the same direction and at a similar level; and found that the source of errors lay in the polls themselves and was not a result of a last-minute shift in preferences among voters.

The AMSRO Polling Inquiry Chair, Darren Pennay (Founder and former CEO of The Social Research Centre) said: “The first preference voteswere either underestimated (LNP) or overestimated (ALP) because of inadequately adjusted, unrepresentative samples. It is very likely the polls were skewed towards over-representing more politically engaged and better educated voters and this bias was not corrected. As a result, the polls over-represented Labor voters. 

“As a consequence, pollsters should seek to better understand the biases in their samples and to develop more effective sample balancing and/or weighting strategies to improve representativeness, by looking at education or other variables.”

The Inquiry Panel also found some evidence that the reporting of the polls failed to consistently meet the basic disclosure guidelines for editors and journalists set out by the Australian Press Council.

The Inquiry report has made 10 key recommendations aimed at improving polling methods and their disclosure. Recommendations include calling for an enforceable Code of Conduct for Election Polling; consistent, enforceable disclosure standards; and recommendations to improve polling methods and better educate the media and public on polling.

The full report and recommendations can be found here. 

AMSRO Board member (and AMSRO President when the Inquiry was launched), Craig Young, said: “Election and political polling are consequential and influential and play an important role in the democratic process.  AMSRO welcomes this final report from the Inquiry Panel and is exceedingly grateful to all those who have volunteered their time and expertise to support this important Inquiry.

“We also welcome the recent formation of the Australian Polling Council and recognise that their stated objectives are aligned with the recommendations of the Inquiry report regarding the need for greater transparency, and the creation of a Code of Conduct and polling guidelines, as well as better education of the media and public about polling.  We look forward to supporting Council members as they develop their Code of Conduct.  A code that delivers greater transparency and disclosure regarding how the published polls are conducted, is critical to re-establishing public confidence in polling. As the research industry association, AMSRO supports extending such disclosure standards to cover all published market research, not just political polling. 

“We hope that this report ultimately provides the polling companies with useful information, ideas and further lines of inquiry to assist them in their efforts to improve the accuracy – and public perceptions of the accuracy – of election and political polling in Australia. AMSRO looks forward to working with the polling companies and other important stakeholders to support the creation of appropriate standards to bring about these much-needed changes.” 

The final report is released today and further information on the AMSRO Polling Inquiry can be found at: http://dataandinsights.com.au/amsro-polling-inquiry-final-report

Notes for editors:

The Inquiry Panel:

• Darren Pennay (Chair) – Founder and past CEO of the Social Research Centre, Campus Visitor at the ANU Centre for Social Research and Methods and Adjunct Professor at the Institute for Social Science Research, University of Queensland.
• Professor Murray Goot – Emeritus Professor of Politics at Macquarie University, a leading expert in public opinion polling, voter behaviour and politics.
• Dr Phil Hughes – Asia-Pacific Head of Statistical Consulting at Engine and one of Australia’s most experienced and respected applied statisticians.
• Dr Dina Neiger – Chief Statistician, the Social Research Centre and Accredited Statistician, Statistical Society of Australia.
• Dr Jill Sheppard – lecturer in the School of Politics and International Relations, ANU, with special research expertise in political behaviour (voting, participation, and attitudes).
• Mr John Stirton – independent and widely respected polling expert, who ran the Fairfax Poll between 1997 and 2014.

The Advisory Board:

• Dennis Trewin AO (Chair) – Dr Trewin was Australian Statistician (Head of the Australian Bureau of Statistics) and an Australian Electoral Commissioner from 2000 to 2007 and a former President of the International Statistical Institute
• Dr John Henstridge – one of Australia’s most eminent Statisticians, previous president of the Statistical Society of Australia.
• Ian McAllister – Distinguished Professor of Political Science, Australian National University (ANU).  Professor McAllister is recognised as an international expert on opinion polling.
• Kerry O’Brien – Journalist and author. Foundation editor and host of the ABC’s national 7.30 Report, foundation host of Lateline, host of Four Corners and anchor of the ABC’s federal and state election night coverage for more than 20 years.
• Travyn Rhall – After training as a statistician, Mr Rhall spent a decade in statistical consulting at the Australian Bureau of Statistics, before embarking on a career in applied (commercial) market research. He was a Director of Newspoll for several years. His last full-time executive role was Global Chief Executive of Kantar Insights, which operated in 70+ countries with revenues of over AUD$3.5 billion.

International Advisers:

• Dr Paul Lavrakas – internationally renowned expert in survey methodology and survey error, specifically in relation to public opinion research and election polling. Past President of the American Association for Public Opinion Research (AAPOR).
• Dr Patrick Moynihan – Associate Director of international research methods at Pew Research Center and previously senior polling analyst for ABC News.
•  – Professor and Vice Chair of the Department of Sociology at the University of Nebraska-Lincoln. She is a Fellow at the American Association for Public Opinion Research (AAPOR) and worked on the 2016 US Polling Inquiry.
• TheLondon School of Economics, who headed a polling inquiry after the 2015 UK General Election.

Ends

For more information please contact:

Sarah Campbell                                                                                
Executive Director, AMSRO                                                           
0417 665 144                                                                  
sarah@amsro.com.au    

Rochelle Burbury
Third Avenue Consulting
0408 774 577
rochelle@thirdavenue.com.au   

About AMSRO

The Association of Market & Social Research Organisations (AMSRO) is the peak body for the market and social research, data and insights industry. AMSRO works in partnership with its company members, plus privacy authorities, business, government, and the community to protect and promote the industry and uphold the highest ethical and privacy market and social research standards. Since its establishment in 1989, AMSRO has grown to more than 90 members, who employ over 5,000 people, representing 70% of the industry’s annual (data collection) turnover.  In 2003, AMSRO pioneered its own privacy code for members which won an Australian Privacy Award in 2009.  www.amsro.com.au

About the Statistical Society of Australia The Society was founded in 1962 as a national umbrella’ organisation to support and further the work of state statistical societies already in existence, to establish a national journal and newsletter and to host national conferences. The national Society now represents Australian and overseas statisticians with state branches in six states and territories providing the focus and organisation of local activities. The overall objective of the Society is to further the study, application and good practice of statistical theory and methods in all branches of learning and enterprise.