Lowy Institute Poll 2022 Report

Methodology

The 2022 Lowy Institute Poll reports the results of a national survey of 2006 Australian adults between 15 and 28 March 2022. The survey was conducted by the Social Research Centre (SRC), using the Life in Australia™ panel — currently the only probability-based online panel in Australia. Members of the panel were randomly recruited via their landline or mobile telephone or via their address (rather than being self-selected volunteers) and agreed to provide their contact details to take part in surveys on a regular basis. SRC uses a mixed-mode approach for the panel, including online surveys (96% of respondents) and computer-assisted telephone interviewing (4% of respondents) to provide coverage of the offline population (households without internet access). The order of questions in the questionnaire was different from the order presented in this report.

On a simple random sample of 2006 responses, the margin of error is 2.2%. Where a complex sample is used, the ‘design effect’ measures the additional variance in comparison with a simple random sample. The design effect for this survey is estimated at 1.77. For the 2022 Lowy Institute Poll survey, a completion rate of 71.7% was achieved. Taking into account the recruitment rate to the panel and attrition from the panel, the cumulative response is 5.4%, which compares favourably with many international probability-based panels. Unlike other commercial online panels in Australia, the probability basis of the Life in Australia™ sampling method means results are generalisable to the national population and sampling errors and confidence intervals can be calculated.

In 2019, the Lowy Institute completed a three-year transition in the methodology for Lowy Institute polling, which until 2017 was conducted solely by telephone. From 2005 to 2011, the Poll was conducted by landline only. From 2012 to 2017, it was conducted using both landline and mobile numbers. In 2017, the SRC administered four key questions from the Poll to an online sample of 2513 respondents in parallel with the telephone survey of 1200 respondents, which was reported in the 2017 Lowy Institute Poll. This parallel survey provided valuable comparison information between the two methodologies. In 2018, the Lowy Institute Poll was a combination of a telephone-only sample of 600 respondents and an online sample of 600 respondents drawn from Life in Australia™. In 2019, the Lowy Institute Poll was administered in the same way as in 2020: with approximately 90% of respondents online and 10% offline.

In order to ensure comparability of the 2018 Poll with the telephone responses of our 2005–17 Polls, the response sets were weighted and blended using the following approach: first, weights for the telephone respondents (50% of the sample) were calculated accounting for the dual chances of being contacted by landline or mobile, and reflecting key population characteristics. Then, for each online respondent, the most similar telephone respondent was found using a range of survey variables, and that person’s telephone weighting was used as the ‘base weight’ for the next step. Finally, the telephone and online responses were then combined into a single data set, and the results then weighted to reflect the demographic profile of the Australian population aged 18 years and over based on Australian Bureau of Statistics population data.

The transition to a predominantly online survey panel for Lowy Institute polls mirrors shifts in survey methods by highly respected polling organisations internationally. The Pew Research Center has moved the majority of its US polling online over the past five years¹, primarily through its American Trends Panel², another probability-based online panel. The Chicago Council on Global Affairs³, which has been conducting nationwide surveys on foreign policy since 1974, moved to online polling in 2004.

The decision to move to an online survey methodology was made because of the declining productivity of telephone surveys. The number of fixed-line telephones is steadily decreasing, and more of the population has moved to mobile-only households, which facilitates call-screening. A decline in contact rates and, to a lesser extent, cooperation rates, has led to reduced response rates for landline and mobile phones.

Some questions in the 2022 Lowy Institute Poll are new and are not affected by the possibility of ‘mode’ differences (where respondents may answer the same question differently in an interviewer-administered telephone survey compared with an online survey, which removes the human element and provides more time for consideration of responses). However, the majority of questions in the 2022 survey are ‘tracking’ questions that have been asked in previous Lowy Institute polls, allowing us to compare public opinion on a single issue over time. Footnotes and dotted lines have been used in the charts in the Poll to represent this change of mode, which can elicit slightly different responses.

The question regarding ‘Confidence in global leaders’ was administered differently in 2021 and 2022. In 2021, online respondents were able to select ‘never heard of the person’ or ‘don’t know’ on the first response screen. In 2022, respondents were only offered these responses if they declined to answer at the first response screen. This resulted in a significant difference in the number of respondents who said they had ‘never heard of the person’ when compared to 2021. The decision was made to re-ask this question in the same format as 2021, to ensure the results were comparable. Respondents were contacted again between 1 and 4 May and 1715 responses to the ‘Confidence in global leaders’ question were completed.

Data quality checks for the online portion of the sample included checks for ‘speeding’ (completing the survey rapidly), ‘straight-lining’ (providing the same answer to every question of a bank of items with the same response options), and the number of non-substantive responses given (don’t know or refused). There was no single determining factor in deciding whether to exclude a case: factors were considered together and included the degree of speeding and the difficulty of straight-lining a particular bank of items.

April 2022 poll

The Lowy Institute fielded additional questions in a nationally representative online and telephone survey conducted by the SRC between 14 and 24 April 2022, with a sample size of 3583 Australian adults. The order of questions in the survey was different from the order presented in this report. The survey was also conducted using the Life in Australia™ panel as explained above.

On a simple random sample of 3583 responses, the margin of error is approximately 1.6%. Where a complex sample is used, the ‘design effect’ measures the additional variance in comparison with a simple random sample. The design effect for this survey is estimated at 1.78.