--- title: "How do invitations to politics flatten participation? Pre-analysis Plan" author: Authors Omitted abstract: | We describe a design and analysis plan for a simple experiment in Northern Uganda to assess whether more personal invitations to engage in political communication has stronger effects for women than for men. date: "`r format(Sys.time(), '%d %B %Y')`" bibliography: bib.bib output: pdf_document: number_sections: yes toc: yes html_document: number_sections: yes theme: united toc: yes --- ```{r setup, echo=FALSE, include=FALSE} rm(list = ls(all = TRUE)) #set.seed(1) #setwd("c:/Dropbox/pcs/PCS-PLS/15 New Experiment/pap") #setwd("~/Dropbox/PCS-PLS/15 New Experiment/pap") library(foreign) library(car) library(stargazer) ``` \newpage # Motivation Many studies have shown that women engage in political communication at lower rates than men [@gottlieb2015greater], especially in the context of developing countries. However, Grossman, Humphreys, and Sacramone-Lutz [-@grossman2014wld] recently argued that women may be relatively more responsive to engaging in politics using decentralized SMS-based technology. New technology, in short, can have a _flattening_ effect.[^1] This result was generated using a field experiment in Uganda, in which a random sample of individuals were personally invited by enumerators to send messages to politicians via an SMS platform. Take-up rates in that experiment was between 4--6\%. [^1]: In that study 47\% of those sending messages to politicians were women whereas only 26\% of ``highly engaged'' subjects were women (supplementary materials Table 1). A later intervention by the National Democratic Institute, in coordination with the Parliament of Uganda, designed and launched a platform enabling voters nationwide to communicate with politicians. In that intervention, voters were invited using 30 second long radio spots (ads), and not personally. Overall take up rates were much lower and men engaged at much higher rates than women, reflecting more traditional patterns of political communication.[^2] [^2]: Large differences in political participation across social groups are not unique to Uganda; it is also a hallmark of American political life [@brady1995beyond]. The differences between these results are large and striking. Moreover they cannot easily be accounted for by some prominent accounts of differential participation such as different access to resources [@brady1995beyond]. Nor, for this kind of participation, does ``critical mass'' play an obvious role [see @lovenduski2005feminizing for a critique of critical mass accounts]. There are several possible reasons for the differences in rates of engagements between men and women across different interventions. Past work suggests that the relative engagement between men and women can be keenly sensitive to the ways that engagement is structured [@Karpowitz2014]. One likely reason is that women and men respond differentially to a _personal invitation_ to engage in politics. A direct personal invitation may itself have an empowering effect, signaling receptiveness and the possibility that political communication will make a difference. Specifically, an invitation to participate in politics may have a large marginal effect for marginalized populations, such as women and poor, who have lower political efficacy to begin with. Two other closely related possibilities are, first, a _treatment compliance_ effect---that women were less likely to hear (and not less likely to respond, conditional on hearing) appeals issued through mass media invitations, and second, an _outcome compliance_ effect---that women are more likely to respond to personal requests for action, whether or not they expect their engagement to make a difference. To assess the flattening effect of invitations to politics we will exploit an existing SMS platform, UBridge, developed in partnership between UNICEF's [Ureport](http://ureport.ug/) platform and Uganda's Governance, Accountability, Participation and Performance [GAPP](http://www.rti.org/page.cfm?obj=BE95C6D2-B458-6596-7972754060938701) project and operative in Arua district, Uganda. With some loss in external validity, our design aims to keep the _treatment compliance_ effect constant by focusing on respondents in the UBridge system. We hope that parsing the _outcome compliance_ effect will be the focus of a future intervention. ## Data collection Infrastructure: The UBridge System UBridge is an SMS-based (text message) service request system that provides citizens and local government officials with a tool for submitting, tracking, and responding to service delivery requests. The platform has been developed as part of a collaboration between [UNICEF Uganda](http://www.unicef.org/infobycountry/uganda.html) and [RTI International](http://www.rti.org/page.cfm/About_RTI). RTI is the prime contractor for the United States Agency for International Development (USAID)-funded GAPP program. UBridge is an open-source software package that runs on a variety of mobile devices, including tablets and smartphones. UBridge is designed to open a new channel of communication from citizens to local government officials to report public service deficiencies. It is a voluntary program, where citizens actively register to participate in the sending and receiving of messages. Using the UBridge technology, citizens can engage with Arua district government officials in three ways. First, they can send unsolicited _anonymous_ messages to the UBridge short code (8500), at absolutely _no cost_. District officials, in both technical and political positions, are equipped with 3G tablets that enable them to access the messages anywhere, provided they have Internet access. Second, citizens can respond to short weekly polls (usually a single question) solicited from UBridge registered members by the research team. The polls are conducted on weekends using a robocall system operated by [VotoMobile](https://www.votomobile.org/). Third, citizens have an opportunity to attend community meetings, implemented and organized by GAPP, where they received information about national service standards, the performance of their own district and community, and learn about the actions they can take to communicate with local officials. The first round of meetings was held by GAPP in September to October 2014 as part of the service launch. The UBridge system was launched as a pilot study in September 2014 in 90 villages across Arua district, which were clustered into 24 village-clusters. Clusters in the GAPP program are defined as the group of nearby villages that are serviced by the same public health center. Arua has 48 government health centers (i.e. 48 clusters), half of which were randomly sampled as control areas and half as treatment areas. The UBridge system has been introduced only in treatment clusters.[^3] A study evaluating the effect of getting access to the UBridge system is underway and is not the subject of the current pre-analysis plan (PAP). [^3]: The mean number of villages per cluster is 3.75; the minimum number of villages per cluster is 1 and the maximum is 6. At this point there are 4,568 UBridge registered users. The majority of UBridge registered users (3,947) have entered the program through a door-to-door registration campaign that took place between October and November 2014 in the 24 treatment clusters (all such users have provided explicit consent to participate in the program). An additional 373 users registered following GAPP's community meetings, where sign-up sheets were passed between attendees. Finally there are 248 users from Arua that have registered with UBridge independently; for example, by following registration instruction on flyers that were distributed in cluster areas during the registration campaigns and the community meetings. ##Baseline Participation Rate We have information on baseline participation rates among UBridge users. In order to verify which of the UBridge registered numbers are valid, a team of local enumerators was hired by UBridge to personally call registered numbers. The enumerators, who called each number up to three times, were able to positively verify $2,810$ mobile numbers: out of which 2,335 completed the survey, 75 began talking to one of the enumerators but the phone disconnected before the poll was completed, and 400 refused to complete the survey. Additional $1,750$ numbers were not `purged' since they answered the phone in one of UBridge's past polls, for a total of 4,568 UBridge registered users. The verification exercise was completed on June 8, 2012. On Saturday June 13, UBridge conducted a baseline poll using VotoMobile's robocall system. The key outcome of interest is a binary variable that receives the value of 1 if the UBridge user responded to the survey question, and 0 otherwise. The poll included the following question: > Hello, you have been receiving SMS messages and polls from 8500 UBridge as part of an effort to improve communication between citizens and district officials. Results of this week's poll will be shared with Arua district local government officials and with your MP. > Uganda's national budget for 2015/2016 will be read by the Minister of Finance on June 15 and then debated in Parliament. District local governments like the Arua receive money from the national government to fund its administration and salaries and social services like education, health, water, and infrastructure. > Currently district local governments are allocated about 15\% of the national budget. In your opinion, is 15\% for local governments too low, about right, or too high?} 12\% of $2,720$ respondents (verified users whose gender is known to UBridge) agreed to respond to the poll and share their views with UBridge. Consistent with past findings, there is a significant difference in response rates across gender groups: 7.6\% of women responded compared to 12.2\% of men. Thus in this baseline setting in which ICT is used, but invitations are weak and impersonal, there is again evidence of a large and significant gender gap in participation rates. The question that drives this pre-registered experiment is whether the gender gap that is evident in the UBridge panel will be reduced if messages are accompanied by a stronger invitational component. Note that our interest is not simply in whether invitations increase participation but whether they do so disproportionately for women [and, ultimately, whether they do so through an empowering mechanism]. # Pre-analysis Plan To address these questions we are requesting that UBridge run a modified version of their baseline poll but now introducing a variation in treatment. All groups will be able to participate in a poll regarding taxation, similar to the previous UBridge poll. In a randomly selected treatment group however we will precede the call with a set of SMS encouragements that highlight that invite participants to take part and that highlight the importance of individual responses in the message. The two treatments will then be: ## Treatments All messages are sent in the regional language Lugbara. English versions of the messages sent are as follows: > _SMS 1:_ Please look out for Friday's poll! Knowing the opinions of people from [constituency] is very important to ensure politicians make the right choices. > _SMS 2:_ We have just selected you to participate in Friday's poll and we strongly encourage you to respond to it. Let your voice be heard! > _SMS 3:_ Remember we selected you especially to participate in today's poll and we strongly encourage you to respond to it. Let your voice be heard! > _Voter Poll:_ Many people in Arua have been receiving SMS messages and polls from 8500 UBridge as part of an effort to improve communication between citizens and district officials. Results of these polls are shared with Arua district local government officials and with your MP. > Uganda's national budget for 2015/2016 is currently being debated in Parliament. Part of government financing comes from taxes collected from ordinary citizens, through for example taxes on imports and on goods and services. In your opinion, is the tax collected from ordinary citizens too low, about right, or too high? Press one for too low, Press 2 for about right or press 3 for too high. > Ubridge would like to remind you that you can SMS 8500 any time and report service provision issues to your local government. The service is at absolutely free! Thanks for your participation in this survey. All subjects receive the voter poll. In addition, treated subjects receive SMSs 1 and 2, both two days before and one day before the poll and they receive SMS 3 the morning of the poll. Summary results will be shared with MPs from Arua district after the poll is conducted. We highlight that the intervention and poll involve no deceit and are implemented with subjects that are consented into the panel. IRB approval has been obtained from all participating universities. ## Hypotheses We have a simple hypothesis: 1. We expect that the marginal effect of treatment (i.e. receiving the suite of SMS messages) will be stronger for women than for men. ## Units There will be $4,568$ subjects of which half will be in treatment and half in control. The primary analysis focuses on a subset of $2,720$ of these that have been recently verified as active in the SMS system. ## Power Our power analysis uses numbers from preliminary research by Grossman, Rodden and Platas-Izama (_presonal comm_) that estimated response rates to simple messages on the same sample. Our code for the power analysis is below: ```{r cache=TRUE} # Script creates a clean covariate database; this block not printed in output sms.power = function( sims = 20000, pCm = .14, # Prob message by men in Control pTm = .14, # Prob message by men in Treatment pCw = .075, # Prob message by women in Control pTw = .14, # Prob message by women in Treatment NW = 882/2, # N women in each condition NM = 1838/2 # N men in each condition ){ # Basic data structure W <- 1:NW M <- 1:NM G <- rep(c(rep(0, NW), rep(1, NM)),2) T <- c(rep(0, NW+NM), rep(1, NW+NM)) # Random DGP xCw <- rbinom(sims, NW, pCw) # Number of messages received from women in control xTw <- rbinom(sims, NW, pTw) # Number of messages received from women in treatment xCm <- rbinom(sims, NM, pCm) # Number of messages received from men in control xTm <- rbinom(sims, NM, pTm) # Number of messages received from men in treatment YWC <- sapply(1:sims, function(j) W<= xCw[j]) YMC <- sapply(1:sims, function(j) M<= xCm[j]) YWT <- sapply(1:sims, function(j) W<= xTw[j]) YMT <- sapply(1:sims, function(j) M<= xTm[j]) ps <- sapply(1:sims, function(j) summary(lm(c(YWC[,j], YMC[,j], YWT[,j], YMT[,j]) ~ G*T))$coef[2:4,4]) power <- apply(ps<=0.05, 1, mean) power } power <- sms.power() power ``` Under these assumptions, power for the (observational) gender effect is `r round(power[1],2)`, power for the treatment effect for women is `r round(power[2],2)`, and power for the interaction effect is `r round(power[3],2)`. Power may be stronger if the control message, indicating an SMS blast, increases the treatment effect for women. ## Outcome Measurement Our primary measure is the response or non-response by UBridge users. In addition we will report _how_ subjects respond to the item. ## Timeline The intervention will be delivered on 24, 25, and 26 June 2015, with the poll delivered on 26 June. Valid responses will be those recorded within 48 hours of sending messages. Treated subjects receive SMSs 1 and 2, both two days before and one day before the poll, and they receive SMS 3 the morning of the poll. ## Randomization ### Structure We use blocked random assignment, blocking on the following variables: 1. Gender (2 categories) 2. Age (3 categories) 3. Constituency (4 + unknown) 4. Recruitment method (3 categories: individual, group, Ureport) 5. Verified sample (2 categories) 6. Past responses (2 categories). Note that the ``past response stratum'' is nested within the verified respondent sample. Collectively these strata produce $2\times 3 \times 5 \times 3 \times 3 = 270$ possible subgroups. ### Code R code for randomization (using dummy data) is below. Note the randomization scheme was generated, though not applied, before registration. ```{r randomization, include=FALSE, cache=FALSE} as.numeric2 = function(f) as.numeric(levels(f))[f] blocks <- read.dta("deidentified.blocks.dta") B <- data.frame(fakeid = blocks$fakeid) B$verified <- as.numeric2(Recode(blocks$Verified, "'No'=0; 'Yes'=1; NA =2")) table(B$verified, blocks$Verified, useNA="ifany") B$lastpoll <- blocks$CompleteSurvey table(B$lastpoll, blocks$CompleteSurvey, useNA="ifany") B$gender <- as.numeric2(Recode(blocks$Sex, "'Male'=1; 'Female'=0; 'Missing' =2")) table(B$gender, blocks$Sex, useNA="ifany") B$MP <- as.numeric(Recode(blocks$MP_name, "'Atiku Benard'=0; 'Drito Martin'=1; 'Kassiano Wadri'=2; 'Okuonzi Sam'=3; '' =5")) table(B$MP, blocks$MP_name, useNA="ifany") B$age <- as.numeric(Recode(blocks$AgeCat, "NA =5")) table(B$age, blocks$AgeCat, useNA="ifany") B$Source <- as.numeric(Recode(blocks$REGISTRATION, "''=5")) table(B$Source, blocks$REGISTRATION, useNA="ifany") B$block <- (B$verified*100000 + B$gender*10000 + B$MP*1000 + B$age*100 + B$lastpoll*10 + B$Source) B$constituency <- Recode(blocks$county_name, "'' = 'Your Constituency'") table(B$constituency , blocks$MP_name, useNA="ifany") write.dta(B, "blocks.dta") ``` ```{r blockrandomization} #library(foreign) rm(list=ls(all=TRUE)) # Functions for Block Randomization ####################################################### # systematic spreads n units across a set of groups # where each group has a probability of receiving # one unit ####################################################### systematic = function(ps, n=sum(ps), seed=NULL){ if(sum(ps)!=n) print("(sum(ps)!=n)") if(!is.null(seed)) set.seed(seed) k <- length(ps) s <- ((cumsum(ps) +n*runif(1))%%n) e <- s - floor(s) e < c(e[k], e[-k]) } ####################################################### # ra.block does block randomization for a fixed p and # deals with integer problems well ####################################################### ra.block <- function(block, p=.5, seed=NULL){ if(!is.null(seed)) set.seed(seed) # if necessary adjust prob so that there is a whole number target n <- length(block) min <- floor(p*n) sel <- min + (runif(1)< (p*n-min)) q <- sel/n # ensure min within each block blocks <- sort(unique(block)) bsize <- sapply(blocks, function(j) sum(block==j)) target <- floor(q*bsize) # spread remainder across blocks resids <- q*bsize - target if(sum(resids)>0) target <- target + systematic(resids) # sample and merge out<-rep(NA, n) for(b in 1:length(blocks)) { out[block==blocks[b]] <- sample(c(rep(1, target[b]), rep(0, bsize[b]-target[b]))) } out } # Implement B <- read.dta("blocks.dta") # table(is.na(B$MP)) # Show Data Structure head(B) # Check distribution of Block Sizes # sum(is.na(B$block)) # length(unique(B$block)) # table(sapply(unique(B$block), function(j) sum(B$block==j))) # Implement Random Assignment B$Z <- ra.block(B$block, p = .5, seed = 20150617) # Create "content"" variable for treatment messages B$content <- B$constituency B$content[B$Z == 0] <- "No Message" table(B$content) # Show balance table(B$Z, B$gender, B$verified) # Show balance (R2) summary(lm(Z~ as.factor(gender) + as.factor(verified) + as.factor(lastpoll) + as.factor(MP) + as.factor(age) + as.factor(Source), data = B))$r.squared # Check uniform propensities permutations <- sapply(1:2000, function(i) ra.block(B$block, seed = i )) propensities <- apply(permutations, 1, mean) c(mean(propensities), sd(propensities)) # Save write.dta(B, "selected_treatment.dta") ``` ## Analysis Our analysis estimates average treatment effects using linear regression, accounting for block fixed effects. ### Controls Our analysis takes account of the variables used for blocking but introduces no further controls. ### Analysis of Heterogeneous Effects We do not intend to run heterogeneous effects analyses on this data. ### Code for main analysis ```{r analysis, results='asis'} B$flattening <- B$Z*B$gender # Dummy data and Analysis B$dummy.Y <- runif(nrow(B)) < .1*(1 + B$gender + B$Z - B$flattening) M1 <- lm(dummy.Y ~ gender+ Z +flattening + as.factor(block), data = B[B$verified==1 & B$gender <= 1, ]) M2 <- lm(dummy.Y ~ gender+ Z +flattening + as.factor(block), data = B[B$gender <= 1, ]) stargazer(M1, M2, type = "latex", title="Dummy Table: Main results", header=FALSE, column.labels=c("Primary", "Secondary"), covariate.labels=c("Invitation", "Flattening (Male*Invitation)"), omit = c("block", "gender", "Constant")) ``` ```{r echo=FALSE, include=FALSE} #f <- 'pap.Rmd' #knitr::purl(f) ``` ### Additional analyses In addition we will report _how_ respondents respond to the poll, broken down by gender and treatment status. ## Contingencies _Missing data_: By design there is no missing data: responses to the poll are the primary outcome of interest and for all units responses are either received or they are not. Only one response per subject is admitted. For analyses of content the _last_ valid response within the response period will be used in case multiple responses are attempted or recorded. _Non-compliance_: We will have no information about whether messages are received and read or not and so all analyses are "intent-to-treat" analyses. ## Dependencies Code for the preanalysis plan is generated using R version 3.1.1 using the following dependencies: rmarkdown, foreign, car, stargazer # References