close× Call Us
close×
Title Survey Non-Response Using Fall 2018 national survey
Post date 10/23/2018
C1 Background and Explanation of Rationale

The main focus is to develop tools to account for non-response on unobservables. This research assesses a survey design approach in which potential respondents are randomized into groups that make response more (or less) likely. Since this factor affecting response propensity is, by design, uncorrelated with any other factors affecting the dependent variable of interest (both observed and unobserved) it can be used in a first stage response equation in a manner that vastly improves the ability of these models to make meaningful predictions.

There are two additional substantive questions that this research will address. I expect these questions to be addressed in separate papers. The first is whether generational identity can be activated to be politically relevant. The second relates to the extent to which policy preferences affect evaluation of political leader versus the extent to which evaluation of political leaders affects policy preferences (Bailey and Wilcox 2016; Lenz 2012).

C2 What are the hypotheses to be tested?

Hypothesis 1: Controlling for non-response on unobservables will yield different results for political questions.
1a: Applied to entire population
This is a foundational hypothesis. In previous work, it was strongly supported for turnout and weakly supported for questions on views of specific politicians and policy.

1b: Applied to Democrats, Republicans and Independents (respectively)
This hypothesis was strongly supported for turnout and for questions on views of specific politicians.

1c: Applied to White working class and Hispanic populations (respectively)
My previous work has not directly addressed this, but analyses of the 2016 presidential election by others indicates potential non-response among white working class (especially in the midwest) and Hispanics (especially in the west).

Hypothesis 2: Activating generational identity will lead to a change in self-reported vote propensity and preferences in the 2018 congressional election.
2a: Young people will be more motivated to vote if the identity is activated (Treatment 1) and more likely to vote and prefer Democrats in the congressional election if the identity is activated and associated with anti-Trump sentiment (Treatment 2), relative to the control treatment.

2b: Older people will be more motivated to vote if the identity is activated (Treatment 1) and more likely to vote and prefer Republicans in the congressional election if the identity is activated and associated with pro-Trump sentiment (Treatment 2), relative to the control treatment

Hypothesis 3: Associating Trump with policies will affect support/opposition to policies.
3a: Associating Trump with specific policies will lead those who like Trump and Republicans to be more favorable to Trump’s positions than control group who were asked questions with no reference to Trump.

3b: Associating Trump with specific policies will lead those who dislike Trump and Democrats to be less favorable to Trump’s positions than control group who were asked questions with no reference to Trump.

C3 How will these hypotheses be tested? *

I will use survey data from the Gfk based on their internet survey panel. The attached pre-analysis plan has additional details and survey questions.

Hypothesis 1 will be tested in two ways.

First, I will use two-stage selection models (building from classical Heckman model structure). The dependent variables will be the political outcomes. I will assess three ways to conceptualize non-response as described in the pre-analysis plan.

Second, I will conduct direct tests (see Bailey 2018 for examples) in which I compare responses conditional on covariates by those who chose politics to those who chose something else as a question topic.

Hypothesis 2: The expectation is that the treatments will have different effect depending on the age category (Young, middle-aged, old) of the respondent.

I will estimate models broken out by age:
Y = \beta_0 + \beta_1 Treatment1 + \beta_2 Treatment2

The dependent variables are political questions that follow the treatment: turnout intention, generic congressional ballot and party id.

Hypothesis 3: There are two directions of influence to assess. The first, which is more likely given results such as Lenz (2012) is that associating a policy with Trump will cause pro-Trump respondents to like the policy more and vice versa.

Y = \beta_0 + \beta_1 TrumpTreatment + \beta_2 TrumpTreatment x Pro-Trump
where Pro-Trump is measured with the Trump feeling thermometer or partisan identification from previous surveys.

The dependent variable will be the four policy questions.

The second is that policy preferences will affect evaluation of political leaders. The model will be
Y = \beta_0 + \beta_1 TrumpTreatment + \beta_2 TrumpTreatment x Pro-Trump
where Pro-Trump is measured with the Trump feeling thermometer or partisan identification from previous surveys.

The dependent variable will be the generic congressional vote and the partisan identification variable. I will use indicators of being pro/anti-Trump and Republican/Democrat as predispositions. The expectation is that anti-Trump Democrats, for example, will become even less favorable toward the Republicans and more likely to vote when Trump and the Republicans are associated with specific policies.

C4 Country United States
C5 Scale (# of Units) 2000 responses are expected. We will also have data on people in the GfK panel who did not respond to the survey request. This number is not known ahead of time.
C6 Was a power analysis conducted prior to data collection? No
C7 Has this research received Insitutional Review Board (IRB) or ethics committee approval? Yes: Exemption Granted
C8 IRB Number Georgetown University IRB: 2018-0876
C9 Date of IRB Approval 09/21/2018
C10 Will the intervention be implemented by the researcher or a third party? The survey will be implemented by Gfk (polling firm)
C11 Did any of the research team receive remuneration from the implementing agency for taking part in this research? No
C12 If relevant, is there an advance agreement with the implementation group that all results can be published? not provided by authors
C13 JEL Classification(s) not provided by authors