close× Call Us
Title Reducing Voting Non-Response to Vote Choice in Surveys using Sensitive Survey Item Techniques
Post date 11/06/2018
C1 Background and Explanation of Rationale Survey respondents may censor their answers to public opinion surveys to protect their public image, to avoid social sanction, or evade other repercussions from engaging in “undesirable” behavior (Tourangeau and Yan 2007). Those who have behaved in socially undesirable ways are more likely to censor their responses by refusing to answer questions outright or providing false but socially desirable responses (Kuran 1997, Tourangeau, Rips, and Rasinski 2000), which can result in biased estimates derived from public opinion surveys. In the Americas, non-response is especially acute with respect to retrospective vote reports. Norms of ballot secrecy have taken root across the region, leading voters to refuse to answer standard vote questions at high rates, often insisting, “mi voto es secreto” – my vote is secret. Indeed, the vote choice question is regularly the most sensitive item in cross-national surveys in the region. In 2014, for example, more than 20% of Mexicans who reported participating in a recent presidential election refused to reveal their vote choice in the AmericasBarometer survey, higher than non-response to other known sensitive items in that study, such as income. This study uses audio tools developed to encourage response to sensitive questions (e.g., about drug use) and applies them to the vote choice question. We piloted a similar study in Nicaragua in 2017; the current study includes substantial changes to the protocol in order to test the mechanisms we argue are operating and, as well, tests the audio tools in a significantly more liberal political context.
C2 What are the hypotheses to be tested?

The main outcome of interest is non-response to the vote choice question (don’t know+refusal to answer). Hypotheses to be tested include:

H1: Non-response to the vote choice question will be lower in conditions that employ audio technology (conditions 3-6) compared to the control condition (1).

H1a: Non-response will be lower in condition 4 (the confidentiality reminder administered via audio) rather than condition 2 (where the confidentiality reminder is administered via enumerator) because of the use of novel audio technology.

H2: Non-response to the vote choice question will be lower in conditions that employ either confidentiality reminders (2, 5, 6) and conditions that employ an anonymity guarantee (4, 6) compared to the control condition (1).

H3: Among the treatments administered via audio (3-6), non-response to the vote choice question will be lowest in condition 6 (audio, with a confidentiality reminder and anonymity guarantee) and the highest in condition 3 (audio, with no reminders).

H3a: We do not have strong expectations about the differences between conditions 3 (confidentiality reminder) and 4 (anonymity guarantee); we weakly expect that condition 4 will reduce non-response to a greater extent than condition 3 since the treatment calls for the enumerator to show the respondent a blank screen to demonstrate that the enumerator will not know what the respondent’s answer is (rather than simply verbally reminding them).*

A secondary outcome of interest is how closely results resemble official electoral returns across conditions (DV: candidate selection). We expect the following:

H4: Responses will most closely match official election outcomes in the audio conditions, and especially in condition 6 (audio, with a confidentiality reminder and anonymity guarantee).

Additionally, we will test a series of non-rival hypotheses about the mechanisms that potentially account for observed treatment effects:
First DV: reported beliefs about the anonymity of the vote choice question (asked at end of survey)

H1M (Anonymity Mechanism): Reported beliefs about response anonymity for the vote question will be significantly higher in the audio conditions (3-6) than in the reminder (2) and control (1) conditions.

Second DV: Reported enjoyment of item administered by audio. NOTE: non-voters and a randomly selected subset of those assigned to non-audio conditions will receive an item about soccer later in the survey using the same audio technology.

H2M (Novelty Mechanism): On average, respondents will report that the audio treatment was more engaging compared to the survey as a whole.

H2MA (Novelty Mechanism): There will be no significant differences in reported enjoyment of the audio item by the question administered via audio.

Third DV: Engagement in the survey - non-response and variation in response for 7-10 items following the vote choice item.

H3M (Novelty Mechanism): Engagement in the survey will increase at similar rates following audio treatments, regardless of the content of the audio.

DV: vote choice. Key moderating variable is final election results, gathered from electoral authorities.

H4M (Contextual Mechanism): In electoral districts where the margin of victory between first and second place candidates was very high, respondents in the audio conditions (3-6) will be more likely to report voting for someone other than the first-place candidate than those in the verbal reminder (2) and control (1) conditions.

H5M (Contextual Mechanism): In electoral districts where the margin of victory between first and second place candidates was very high, those in the verbal reminder (2) condition will be more likely to report voting for the first-place candidate than those in the control (1) condition.

DV: reported beliefs about response anonymity
H6M (Anonymity + Contextual Mechanisms): In electoral districts where the margin of victory between first and second place candidates was very high, those in the verbal reminder (2) condition will be less likely to believe their responses to the vote question are confidential than those in the control (1) and audio (3-6) conditions.

* Our ability to test the differences between the audio conditions is limited by the amount of power required; we expect fairly small differences between these conditions, and due to the cell size of each treatment (which is anticipated to be <200), we likely will not be able to separate the relative effect sizes of these conditions in this sample.

C3 How will these hypotheses be tested? *

We will use a series of econometric tests for each hypothesis. Assuming balance on observables, we will run T-tests for H1-H4 and H1M-H3M. If there are any imbalances, we will run regression analyses that control for unbalanced covariates.

We will test the contextual expectations using regression analyses that include an interaction term between the margin of victory and treatment condition.

C4 Country Mexico
C5 Scale (# of Units) 1,800 respondents across 6 treatment conditions; however, respondents will only be treated if they voted, which limits the sample to approximately ~60% of participants.
C6 Was a power analysis conducted prior to data collection? Yes
C7 Has this research received Insitutional Review Board (IRB) or ethics committee approval? Yes
C8 IRB Number 181880 (Vanderbilt University), STUDY00006713 (University of Georgia)
C9 Date of IRB Approval 10/12/2018 (Vanderbilt University, 10/29/2018 (University of Georgia)
C10 Will the intervention be implemented by the researcher or a third party? Data OPM, Mexico City, Mexico
C11 Did any of the research team receive remuneration from the implementing agency for taking part in this research? No
C12 If relevant, is there an advance agreement with the implementation group that all results can be published? not provided by authors
C13 JEL Classification(s) not provided by authors