Work Package 1. Understand the measurement error mechanisms affecting crime data
The start of the project seeks to build a solid theoretical foundation to inform our subsequent empirical work. Through Work Package (WP) 1 we will provide a comprehensive literature review identifying the different types of systematic and random measurement error mechanisms affecting crime estimates. At least three forms of systematic errors and one form of random errors in reported crime rates can be anticipated. The latter is the least harmful, as it stems from the smoothing process undertaken to anonymise police records at the area level (Ceolin et al., 2014). More prominent sources of systematic error are incomplete ‘clearance rates’ (HMIC, 2014), under-recording (Roberts & Roberts, 2016), and those crimes that are never reported (Skogan, 1977) all of which might realistically be expected to vary across police forces and by crime type. These errors are likely to be substantial. HMIC (2014) warned that over 800,000 reported crimes go unrecorded each year (around 19% of all recorded crime). Skogan’s (1977) comparison of crime rates from police records to those from the first American National Victimization Survey showed that under-reporting was especially evident for less serious incidents (see also Hart & Rennison, 2003; Langton et al., 2012; Lynch & Addington, 2006). Moreover, cooperation with police services and crime reporting rates are conditioned by neighbourhood conditions, meaning police records are likely to be more valid in some areas than others (Bottoms et al, 1986; MacDonald, 2001; Tarling & Morris, 2010; Jackson et al., 2013).
Work Package 2. Combine crime survey estimates and police recorded crime counts
A key ‘innovation’ of this project stems from the conceptualisation of inconsistencies between police data and estimates from the Crime Survey for England and Wales as a measurement error problem. Doing so allows us to capitalise on new methodologies to improve our understanding of the measurement flaws in both police data and the CSEW (Oberski et al., 2017; Law et al., 2014), and to help us adjust for their biasing effect (Carroll et al., 2006; Gustafson, 2003). This is important, the more we understand about the nature and prevalence of measurement error problems in crime figures, the better equipped we are to undertake the necessary adjustments. In WP2 we will build on recent work which has demonstrated how Multi-Trait Multi-Method (MTMM) models can be used to augment administrative records with survey data to isolate the systematic and random forms of error, and estimate the ‘true value’ of an underlying outcome of interest (Oberski et al., 2017; Pyrooz et al., 2019; Yang et al., 2018). Treating the two sources of data as different methodological approaches and identifying a set of comparable offences within each dataset, it is possible to isolate bias (systematic errors) and variance (random errors) associated with the data collection from the shared variance observed in the two data sources across crime types. We can therefore directly quantify the extent and type of measurement error affecting each crime source.
Work Package 3. Generate bespoke adjustments and estimate corrected crime counts
Having estimated the nature and extent of measurement error in police recorded crime and CSEW estimates, WP3 will draw on methodological innovations stemming mostly from the field of biostatistics to develop ex-post adjustments for the ‘errors-in-variables’ (Fuller, 2009) problem affecting studies relying on either of these two forms of crime data. In particular, we will assess the utility of two existing approaches: SIMEX and Bayesian adjustments. Most adjustments assume a simple form of measurement error known as classical error (Novick 1966), X*=X+V, where X* is the observed variable, equal to the true variable X, plus a random measurement error term, V. Since the existing evidence suggests the dominant type of measurement error affecting crime rates calculated from police records is likely to be systematic – under-reporting proportional to the true crime rate – we will employ a modified form of the classical model, X*=XV. Setting the mean of the error within the (0,1) interval we can reflect a type of measurement error that is systematically negatively biased in a rate proportional to the true value . The specific distribution of V will be defined based on the estimations carried out in WP2 and will vary according to area (police force), period and offence. Further elaborations of these adjustments will also aim to account for additional forms of classical additive random errors identified in WP2.
Recounting Crime 