New deal for communities as a ‘natural policy experiment’

Halliday, Emma and Popay, Jennie (2014) New deal for communities as a ‘natural policy experiment’. In: SAGE research methods cases :. Sage. ISBN 9781446273050

Abstract

In 2010, we embarked on a study evaluating the impact of an area-based regeneration intervention on health inequalities in England. The policy initiative in question was called New Deal for Communities – a major programme implemented in 39 disadvantaged neighbourhoods over 10 years from 1999/2000 to 2010/2011. Due to methodological and practical challenges, it was not possible to use an experimental design to assess impacts. Alternatively, we deployed methods that allowed us to evaluate the initiative as a ‘natural policy experiment’ using innovative approaches to constructing ‘comparators’. In this case study, we use our experience of this study to explain what natural policy experiments are and why there is growing interest in these among public health researchers. We pay attention to some specific features that are important to consider in the conduct of evaluations of a natural policy experiment such as ours. This includes how to deal with variation in the implementation of policy initiatives, as well as in the characteristics of the areas where these policy initiatives are rolled out. We also focus on the benefits and limitations of using existing data to assess impacts within the time frame of an intervention as well as longer term.