Lowy Institute Poll 2023 Report

Methodology

The 2023 Lowy Institute Poll reports the results of a national survey of 2077 Australian adults between 14 and 26 March 2023. The survey was conducted by the Social Research Centre (SRC), using the Life in Australia™ panel — currently the only probability-based online panel in Australia. Members of the panel were randomly recruited via their landline or mobile telephone or via their address (rather than being self-selected volunteers) and agreed to provide their contact details to take part in surveys on a regular basis. SRC uses a mixed-mode approach for the panel, including online surveys (97% of respondents) and computer-assisted telephone interviewing (3% of respondents) to provide coverage of the offline population (households without internet access). The order of questions in the questionnaire was different from the order presented in this report.

On a simple random sample of 2077 responses, the margin of error is 2.2%. Where a complex sample is used, the ‘design effect’ measures the additional variance in comparison with a simple random sample. The design effect for this survey is estimated at 1.69. For the 2023 Lowy Institute Poll survey, a completion rate of 72.9% was achieved. Taking into account the recruitment rate to the panel and attrition from the panel, the cumulative response rate is 4%, which is comparable to many international probability-based panels. Unlike other commercial online panels in Australia, the probability basis of the Life in Australia™ sampling method means results are generalisable to the national population and sampling errors and confidence intervals can be calculated.

In 2019, the Lowy Institute completed a three-year transition in the methodology for Lowy Institute polling, which until 2017 was conducted solely by telephone. From 2005 to 2011, the Poll was conducted by landline only. From 2012 to 2017, it was conducted using both landline and mobile numbers. In 2017, the SRC administered four key questions from the Poll to an online sample of 2513 respondents in parallel with the telephone survey of 1200 respondents, which was reported in the 2017 Lowy Institute Poll. This parallel survey provided valuable comparison information between the two methodologies. In 2018, the Lowy Institute Poll was a combination of a telephone-only sample of 600 respondents and an online sample of 600 respondents drawn from Life in Australia™. In 2019, the Lowy Institute Poll was administered in the same way as in 2020: with approximately 90% of respondents online and 10% offline.

In order to ensure comparability of the 2018 Poll with the telephone responses of our 2005–17 Polls, the response sets were weighted and blended using the following approach: first, weights for the telephone respondents (50% of the sample) were calculated accounting for the dual chances of being contacted by landline or mobile, and reflecting key population characteristics. Then, for each online respondent, the most similar telephone respondent was found using a range of survey variables, and that person’s telephone weighting was used as the ‘base weight’ for the next step. Finally, the telephone and online responses were then combined into a single data set, and the results then weighted to reflect the demographic profile of the Australian population aged 18 years and over based on Australian Bureau of Statistics population data.

The transition to a predominantly online survey panel for Lowy Institute polls mirrors shifts in survey methods by highly respected polling organisations internationally. The Pew Research Center has moved the majority of its US polling online over the past five years¹, primarily through its American Trends Panel², another probability-based online panel. The Chicago Council on Global Affairs³, which has been conducting nationwide surveys on foreign policy since 1974, moved to online polling in 2004.

The decision to move to an online survey methodology was made because of the declining productivity of telephone surveys. The number of fixed-line telephones is steadily decreasing, and more of the population has moved to mobile-only households, which facilitates call-screening. A decline in contact rates and, to a lesser extent, cooperation rates, has led to reduced response rates for landline and mobile phones.

Some questions in the 2023 Lowy Institute Poll are new and are not affected by the possibility of ‘mode’ differences (where respondents may answer the same question differently in an interviewer-administered telephone survey compared with an online survey, which removes the human element and provides more time for consideration of responses). However, the majority of questions in the 2023 survey are ‘tracking’ questions that have been asked in previous Lowy Institute polls, allowing us to compare public opinion on a single issue over time. On the Lowy Institute website, footnotes and dotted lines have been used in the charts in the Poll to represent this change of mode, which can elicit slightly different responses.

For all 2023 data points, the ‘income’ variable relates to information respondents provided on household income. For data collected in 2022 and earlier, ‘income’ relates to information provided on personal income.

Data quality checks for the online portion of the sample included checks for ‘speeding’ (completing the survey rapidly), ‘straight-lining’ (providing the same answer to every question of a bank of items with the same response options), and the number of non-substantive responses given (don’t know or refused). There was no single determining factor in deciding whether to exclude a case: factors were considered together and included the degree of speeding and the difficulty of straight-lining a particular bank of items.

Life in Australia™ members are offered a small incentive for joining the panel and another incentive for each survey they complete — a $10 gift card, payment, or donation to a nominated charity.

April 2023 poll

The Lowy Institute fielded three additional questions (on AUKUS submarines and defence) in a nationally representative online and telephone survey conducted by the SRC between 11 and 26 April 2023, with a sample size of 4469 Australian adults. The order of questions in the survey was different from the order presented in this report. The survey was also conducted using the Life in Australia™ panel as explained above.

On a simple random sample of 4469 responses, the margin of error is approximately 1.5%. Where a complex sample is used, the ‘design effect’ measures the additional variance in comparison with a simple random sample. The design effect for this survey is estimated at 1.63.