Methodology
The 2021 Lowy Institute Poll reports the results of a national survey of 2222 Australian adults between 15 and 29 March 2021. The survey was conducted by the Social Research Centre (SRC), using the Life in Australia™ panel — currently the only probability-based online panel in Australia. Members of the panel were randomly recruited via their landline or mobile telephone or via their address (rather than being self-selected volunteers) and agreed to provide their contact details to take part in surveys on a regular basis. SRC uses a mixed-mode approach for the panel, including online surveys (95% of respondents) and computer-assisted telephone interviewing (5% of respondents) to provide coverage of the offline population (households without internet access). The order of questions in the questionnaire was different from the order presented in this report.
On a simple random sample of 2222 responses, the margin of error is 2.1%. Where a complex sample is used, the ‘design effect’ measures the additional variance in comparison with a simple random sample. The design effect for this survey is estimated at 1.97. For the 2021 Lowy Institute Poll survey, a completion rate of 83.8% was achieved. Taking into account the recruitment rate to the panel and attrition from the panel, the cumulative response is 7.1%, which compares favourably with many international probability-based panels. Unlike other commercial online panels in Australia, the probability basis of the Life in Australia™ sampling method means results are generalisable to the national population and sampling errors and confidence intervals can be calculated.
In 2019, the Lowy Institute completed a three-year transition in the methodology for Lowy Institute polling, which until 2017 was conducted solely by telephone. From 2005 to 2011, the Poll was conducted by landline only. From 2012 to 2017, it was conducted using both landline and mobile numbers. In 2017, the SRC administered four key questions from the Poll to an online sample of 2513 respondents in parallel with the telephone survey of 1200 respondents, which was reported in the 2017 Lowy Institute Poll. This parallel survey provided valuable comparison information between the two methodologies. In 2018, the Lowy Institute Poll was a combination of a telephone-only sample of 600 respondents and an online sample of 600 respondents drawn from Life in Australia™. In 2019, the Lowy Institute Poll was administered in the same way as in 2020: with approximately 90% of respondents online and 10% offline.
In order to ensure comparability of the 2018 Poll with the telephone responses of our 2005–17 Polls, the response sets were weighted and blended using the following approach: first, weights for the telephone respondents (50% of the sample) were calculated accounting for the dual chances of being contacted by landline or mobile, and reflecting key population characteristics. Then, for each online respondent, the most similar telephone respondent was found using a range of survey variables, and that person’s telephone weighting was used as the ‘base weight’ for the next step. Finally, the telephone and online responses were then combined into a single data set, and the results then weighted to reflect the demographic profile of the Australian population aged 18 years and over based on Australian Bureau of Statistics population data.
The transition to a predominantly online survey panel for Lowy Institute polls mirrors shifts in survey methods by highly respected polling organisations internationally. The Pew Research Center has moved the majority of its US polling online over the past five years1, primarily through its American Trends Panel2, another probability-based online panel. The Chicago Council on Global Affairs3, which has been conducting nationwide surveys on foreign policy since 1974, moved to online polling in 2004.
The decision to move to an online survey methodology was made because of the declining productivity of telephone surveys. The number of fixed-line telephones is steadily decreasing, and more of the population has moved to mobile-only households, which facilitates call-screening. A decline in contact rates and, to a lesser extent, cooperation rates, has led to reduced response rates for landline and mobile phones.
Some questions in the 2021 Lowy Institute Poll are new and are not affected by the possibility of ‘mode’ differences (where respondents may answer the same question differently in an in-person telephone interview compared with an online survey, which removes the human element and provides more time for consideration of responses). However, the majority of questions in the 2021 survey are ‘tracking’ questions that have been asked in previous Lowy Institute polls, allowing us to compare public opinion on a single issue over time. Footnotes and dotted lines have been used in the charts in the Poll to represent this change of mode, which can elicit slightly different responses.
Data quality checks for the online portion of the sample included checks for ‘speeding’ (completing the survey rapidly), ‘straight-lining’ (providing the same answer to every question of a bank of items with the same response options), and the number of non-substantive responses given (don’t know or refused). There was no single determining factor in deciding whether to exclude a case: factors were considered together and included the degree of speeding and the difficulty of straight-lining a particular bank of items.
Climate Poll 2021
Climate Poll 2021 by the Lowy Institute reports the results of a nationally representative online and telephone survey conducted by the SRC between 12 and 26 April 2021, with a sample size of 3286 Australian adults. The order of questions in the survey was different from the order presented in this report. The survey was also conducted using the Life in Australia™ panel as explained above. SRC uses a mixed-mode approach for the panel, including online surveys (95% of respondents) and computer-assisted telephone interviewing (5% of respondents) to provide coverage of the offline population (households without internet access).
On a simple random sample of 3286 responses, the margin of error is approximately 1.7%. Where a complex sample is used, the ‘design effect’ measures the additional variance in comparison with a simple random sample. The design effect for this survey is estimated at 1.62. A completion rate of 82.1% was achieved. Taking into account the recruitment rate to the panel and attrition from the panel, the cumulative response is 7.1%
Attitudes to global warming was asked in the annual Lowy Institute Poll, as per the above methodology.
November 2020 poll
As a part of the Lowy Institute’s Being Chinese in Australia report, released in March 2021, SRC conducted a parallel national survey on 3029 adults between 9 and 23 November 2020. The survey was conducted by the SRC, using the Life in Australia™ panel. SRC uses a mixed-mode approach for the panel, including online surveys (89% of respondents) and computer-assisted telephone interviewing (11% of respondents) to provide coverage of the offline population (households without internet access). The questions on democracy, foreign influence and Australian media reporting about China that are included in this poll were fielded during this survey.
- Courtney Kennedy and Claudia Deane, “What our Transition to Online Polling Means for Decades of Phone Survey Trends”, Fact Tank, Pew Research Center, 27 February 2019, https://www.pewresearch.org/fact-tank/2019/02/27/what-our-transition-to-online-polling-means-for-decades-of-phone-survey-trends/.
- Scott Keeter, “Growing and Improving Pew Research Center’s American Trends Panel”, Pew Research Center, 27 February 2019, https://www.pewresearch.org/methods/2019/02/27/growing-and-improving-pew-research-centers-american-trends-panel/.
- Dina Smeltz and Craig Kafura, “Record Number of Americans Endorse Benefits of Trade”, The Chicago Council on Global Affairs, 27 August 2018, https://www.thechicagocouncil.org/publication/record-number-americans-endorse-benefits-trade.