Almost Random : Evaluating a Large-scale Randomized Nutrition Program in the Presence of Crossover
Large-scale randomized interventions have the potential to uncover the causal effect of programs applying to a large population, thereby improving on the insights gained from currently dominant smaller randomized studies. However, the external validity gained through larger interventions typically implies less supervision and often comes at the cost of some deviation from the randomization plan. This paper investigates the impact of the Nutrition Enhancement Program, which aims to improve child nutrition in Senegal based on a large-scale randomized community intervention. The analysis explicitly deals with deviation from the planned treatment and suggests approaches for combining ex-post adjustments such as propensity score matching with the randomized treatment plan. The authors do not detect a strong overall program impact on the outcome measure of weight-for-age based on planned treatment status, but do find an impact on the youngest children. Moreover, the project impact is clearer when the analysis considers treatment crossover using alternative estimators of two-stage least-squares and propensity score matching. The findings underscore the importance of addressing the shortcomings of large-scale randomization interventions in a systematic manner in order to understand the selection process that can guide further implementation of such projects, as well as to expose the true, causal effect of such programs.
Summary: | Large-scale randomized interventions
have the potential to uncover the causal effect of programs
applying to a large population, thereby improving on the
insights gained from currently dominant smaller randomized
studies. However, the external validity gained through
larger interventions typically implies less supervision and
often comes at the cost of some deviation from the
randomization plan. This paper investigates the impact of
the Nutrition Enhancement Program, which aims to improve
child nutrition in Senegal based on a large-scale randomized
community intervention. The analysis explicitly deals with
deviation from the planned treatment and suggests approaches
for combining ex-post adjustments such as propensity score
matching with the randomized treatment plan. The authors do
not detect a strong overall program impact on the outcome
measure of weight-for-age based on planned treatment status,
but do find an impact on the youngest children. Moreover,
the project impact is clearer when the analysis considers
treatment crossover using alternative estimators of
two-stage least-squares and propensity score matching. The
findings underscore the importance of addressing the
shortcomings of large-scale randomization interventions in a
systematic manner in order to understand the selection
process that can guide further implementation of such
projects, as well as to expose the true, causal effect of
such programs. |
---|