The predictive performance of criminal risk assessment tools used at sentencing : Systematic review of validation studies

Abstract

Although risk assessment tools have been widely used to inform sentencing decisions, there is uncertainty about the extent and quality of evidence of their predictive performance when validated in new samples. Following PRISMA guidelines, we conducted a systematic review of validation studies of 11 commonly used risk assessment tools for sentencing. We identified 36 studies with 597,665 participants, among which were 27 independent validation studies with 177,711 individuals. Overall, the predictive performance of the included risk assessment tools was mixed, and ranged from poor to moderate. Tool performance was typically overestimated in studies with smaller sample sizes or studies in which tool developers were co-authors. Most studies only reported area under the curve (AUC), which ranged from 0.57 to 0.75 in independent studies with more than 500 participants. The majority did not report key performance measures, such as calibration and rates of false positives and negatives. In addition, most validation studies had a high risk of bias, partly due to inappropriate analytical approach used. We conclude that the research priority is for future investigations to address the key methodological shortcomings identified in this review, and policy makers should enable this research. More sufficiently powered independent validation studies are necessary.