Eight Principles of the Metaketa Initiative
The Metaketa Initiative is grounded in eight principles, as follows:
1. Coordination across research teams
The Metaketa Initiative seeks to generate coordination as well as some healthy competition between funded research teams. The next seven pillars of the approach depend vitally on integration and collaboration. The project achieves harmonization through meetings hosted by the steering committees with successful applicants.
2. Predefined themes and comparable interventions
Teams of researchers will work on related questions and study comparable interventions. Themes are pre-defined in two ways: (1) through common theoretical questions, and (2) through the selection of winning proposals, in which we will prioritize comparability across research teams.
3. Comparable measures
Research teams will use consistent outcome measures agreed during post-funding workshops. Researchers will be encouraged to employ mixed methods in measuring outcomes and mediators of the effect of the interventions, including qualitative data collection during the implementation phase.
4. Integrated case selection
Proposals should theorize the channels through which an intervention may affect a given outcome, and provide hypotheses about which channels may be operative in the chosen research context(s). This justifies the selection of research sites and may allow greater ex-ante specification of hypotheses about heterogeneous effects across contexts.
Funded research will be required to hew to EGAP’s standards for analytic transparency (http://egap.org/resources/egap-statement-of-principles). In particular, after revision of research designs at an initial meeting of funded researchers and before initiation of outcome data collection, grantees must post a study protocol that describes a) the study’s purpose; b) the hypotheses it aims to test; c) the main outcome variables; and d) the set of tests and the data analysis that will be performed. In addition, the group of funded studies will itself be pre-registered, with the comparisons and pooled analyses to be conducted from the group of studies made explicit.
6. Third-party analysis
Research teams will be expected to make their data publicly available for independent, third party replication and analysis funded by the metaketa project, with a view to early identification of errors and discrepancies prior to publication. In addition, at publication, all of the data will be archived in a public repository (Dataverse) and provided free of charge.
7. Formal synthesis
Group preregistration will allow the project team and funded researchers to pre-specify a plan for meta-analysis of distinct experiments—and for formal synthesis of experimental results using standard Bayesian, multilevel meta-analysis techniques. In addition, research teams may collaborate on developing a causal model that can integrate results and account for ways in which contexts may condition causal effects. Here, integrated case selection that builds on theory about channels through which interventions affect outcomes assists in stipulating ex-ante expectations of results across experiments.
8. Integrated publication
In addition to individual academic papers and policy briefs, all funded researchers and the steering committee will coauthor one or more books or articles that present results from the distinct studies in an integrated analysis.