How Survey-to-Survey Imputation Can Fail
This paper proposes diagnostics to assess the accuracy of survey-to-survey imputation methods and applies them to examine why imputing from the Household Income and Expenditure Survey into the Labor Force Survey fails to accurately project poverty trends in Sri Lanka between 2006 and 2009. Survey-to-survey imputation methods rely on two key assumptions: (i) that the questions in the two surveys are asked in a consistent way and (ii) that common variables of the two surveys explain a large share of the intertemporal change in household expenditure and poverty. In addition, differences in sampling design can lead validation tests to underestimate the accuracy of survey-to-survey predictions. In Sri Lanka, the causes of failure differ across sectors. In the urban sector, the primary culprit is differences between the two surveys in the design of the questionnaire. In the rural and estate sectors, the set of common variables in the prediction model does not adequately capture changes in poverty. The paper concludes that in Sri Lanka, survey-to-survey imputation between the Household Income and Expenditure Survey and the Labor Force Survey cannot produce accurate poverty estimates unless the Labor Force Survey adds additional questions on assets and is redesigned to use a questionnaire that is compatible with the Household Income and Expenditure Survey. Alternatively, a new welfare-tracking survey that satisfies these conditions could be established.
Summary: | This paper proposes diagnostics to
assess the accuracy of survey-to-survey imputation methods
and applies them to examine why imputing from the Household
Income and Expenditure Survey into the Labor Force Survey
fails to accurately project poverty trends in Sri Lanka
between 2006 and 2009. Survey-to-survey imputation methods
rely on two key assumptions: (i) that the questions in the
two surveys are asked in a consistent way and (ii) that
common variables of the two surveys explain a large share of
the intertemporal change in household expenditure and
poverty. In addition, differences in sampling design can
lead validation tests to underestimate the accuracy of
survey-to-survey predictions. In Sri Lanka, the causes of
failure differ across sectors. In the urban sector, the
primary culprit is differences between the two surveys in
the design of the questionnaire. In the rural and estate
sectors, the set of common variables in the prediction model
does not adequately capture changes in poverty. The paper
concludes that in Sri Lanka, survey-to-survey imputation
between the Household Income and Expenditure Survey and the
Labor Force Survey cannot produce accurate poverty estimates
unless the Labor Force Survey adds additional questions on
assets and is redesigned to use a questionnaire that is
compatible with the Household Income and Expenditure Survey.
Alternatively, a new welfare-tracking survey that satisfies
these conditions could be established. |
---|