Lowy Institute Poll 2020 Report

Methodology

The 2020 Lowy Institute Poll reports the results of a national survey of 2448 Australian adults between 16 and 29 March 2020. The survey was conducted by the Social Research Centre (SRC), using the Life in Australia™ panel — currently the only probability-based online panel in Australia. Members of the panel were randomly recruited via their landline or mobile telephone (rather than being self-selected volunteers) and agreed to provide their contact details to take part in surveys on a regular basis. SRC uses a mixed-mode approach for the panel, including online surveys (93% of respondents) and computer-assisted telephone interviewing (7% of respondents) to provide coverage of the offline population (households without internet access). SRC boosted the sample for regional and rural populations of the Life in Australia™ panel. The order of questions in the questionnaire was different from the order presented in this report.

On a simple random sample of 2448 responses, the margin of error is 2.0%. Where a complex sample is used, the ‘design effect’ measures the additional variance in comparison with a simple random sample. The design effect for this survey is estimated at 1.93. For the 2020 Lowy Institute Poll survey, a completion rate of 78.5% was achieved. Taking into account the recruitment rate to the panel and attrition from the panel, the cumulative response is 7.9%, which compares favourably with international probability-based panels. Unlike other commercial online panels in Australia, the probability basis of the Life in Australia™ panel sampling method means results are generalisable to the national population and sampling errors and confidence intervals can be calculated.

COVIDpoll by the Lowy Institute Poll reports the results of a nationally representative online and telephone survey conducted by the Social Research Centre between 14 and 27 April 2020 with a sample size of 3036 Australian adults. The survey also used the SRC’s Life in Australia™ panel. A mixed-mode approach was also used for the panel, including online surveys (94% of respondents) and computer-assisted telephone interviewing (6% of respondents) to provide coverage of the offline population (households without internet access).

On a simple random sample of 3036 responses, the margin of error is approximately 1.8%. The design effect for this survey is estimated at 1.86. For the COVIDpoll survey, a completion rate of 76.9% was achieved.

In 2019, the Lowy Institute completed a three-year transition in the methodology for Lowy Institute polling, which until 2017 was conducted solely by telephone. From 2005 to 2011, the Poll was conducted by landline only. From 2012 to 2017, it was conducted using both landline and mobile numbers. In 2017, SRC administered four key questions from the Poll to an online sample of 2513 respondents in parallel with the telephone survey of 1200 respondents, which was reported in the 2017 Lowy Institute Poll. This parallel survey provided valuable comparison information between the two methodologies. In 2018, the Lowy Institute Poll was a combination of a telephone-only sample of 600 respondents and an online sample of 600 respondents drawn from the SRC Life in Australia™ panel. In 2019, the Lowy Institute Poll was administered in the same way as in 2020: with 89% of respondents online and 11% offline.

In order to ensure comparability of the 2018 Poll with the telephone responses of our 2005–17 Polls, the response sets were weighted and blended using the following approach: first, weights for the telephone respondents (50% of the sample) were calculated accounting for the dual chances of being contacted by landline or mobile, and reflecting key population characteristics. Then, for each online respondent, the most similar telephone respondent was found using a range of survey variables, and that person’s telephone weighting was used as the ‘base weight’ for the next step. Finally, the telephone and online responses were then combined into a single data set, and the results then weighted to reflect the demographic profile of the Australian population aged 18 years and over based on Australian Bureau of Statistics population data.

The transition to a predominantly online survey panel for Lowy Institute polls mirrors shifts in survey methods by highly respected polling organisations internationally. The Pew Research Center has moved the majority of its US polling online over the past five years¹, primarily through its American Trends Panel², another probability-based online panel. The Chicago Council on Global Affairs³, which has been conducting nationwide surveys on foreign policy since 1974, moved to online polling in 2004.

The decision to move to an online survey methodology was made because of the declining productivity of telephone surveys. The number of fixed-line telephones is steadily decreasing, and more of the population has moved to mobile only households which facilitates call-screening. A decline in contact rates and, to a lesser extent, cooperation rates, has led to reduced response rates for landline and mobile phones.

Some questions in the 2020 Lowy Institute Poll are new and are not affected by the possibility for ‘mode’ differences (where respondents may answer the same question differently in an in-person telephone interview compared with an online survey which removes the human element and provides more time for consideration of responses). However, the majority of questions in the 2020 survey are ‘tracking’ questions that have been asked in previous Lowy Institute polls, allowing us to compare public opinion on a single issue over time. Footnotes and dashed lines have been used in the charts in the Poll to represent this change of mode, which can elicit slightly different responses.

Data quality checks for the online portion of the sample included checks for ‘speeding’ (completing the survey rapidly), ‘straight-lining’ (providing the same answer to every question of a bank of items with the same response options), and the number of non-substantive responses given (don’t know or refused). There was no single determining factor in deciding whether to exclude a case: factors were considered together and included the degree of speeding and the difficulty of straight-lining a particular bank of items.