I don’t like to ride in airplanes. Part of it is soaring in a tube at 36,000 feet; part of it is being sardined into kindergarten-sized seats. The one other thing that makes me uncomfortable is explaining what I do for a living to the stranger stealing my armrest.
Soon after eye contact and shortly following a standard greeting, the question pops … “So, where do you work?” Then it’s shudder time for me—first, because of the inevitable blank stare that follows my response, “The National Council on Crime and Delinquency and the Children’s Research Center,” and second, because the follow-up question is always, “What do you do?”
Don’t get me wrong. I am very proud of where I work and truly enjoy what I do, but it can be difficult to explain to people who don’t work in my field. Typically, the path of least resistance for me is to talk about risk assessment. Most folks have some understanding of the concept—they have a pension plan or insurance. So I can explain that the agencies I work with have a similar need to identify people to worry about most, and conversely, people to worry about less. It is my job to figure out if the risk assessment they are using is making these classifications correctly, or to use the technical term, determine if a risk assessment is valid.
Of course, the conversation rarely ends here. The next thing people want to know is how I decide which factors end up on a risk assessment. People will usually suggest things like homelessness and family dynamics and, yes, sometimes characteristics such as these can be used to help gauge risk. But my stock response is much simpler: Past behavior is the best predictor of future behavior. The best risk assessments, whether in insurance or social services, are always anchored in prior history. We found as much in a recent examination of eight risk assessments currently used in 10 jurisdictions across the United States.
We examined validity, i.e., we measured how well each increase in risk classification (low to moderate to high) corresponded to a meaningful increase in the likelihood that a person would come back to the system. Simply put, an assessment’s validity is more robust when high-risk people, on average, experience more subsequent involvement with the system than those classified as moderate or low risk.
Risk assessments come in all shapes and sizes. In this study, risk was assessed using as few as eight variables or as many as 87. No matter how simple or lengthy, every assessment had measures of prior history. These often included the age at the first experience with the system, how many times a person had previously been involved, and how serious past involvement had been. In all eight risk assessment systems across 10 jurisdictions we examined, these measures of prior history were shown to have a statistically significant relationship with a return to the system. It is this relationship that gives prior history its predictive clout. In fact, the link between past behavior and subsequent involvement was so strong that in one particular risk assessment, two factors, “age at first adjudication” and “number of prior adjudications,” accounted for two thirds of the possible risk score and virtually all of the predictive power of the instrument.
It is important to note that the relationship between risk and recurrence by no means suggests that people who were previously involved in a system will always relapse, just as folks who pay higher insurance premiums don’t always have accidents. What it really means is that social service agencies should focus more energy on higher-risk people so that they can experience better outcomes in the future.
The idea that past behavior is the best predictor of future behavior rarely shocks any of my seatmates, so my standard explanation usually makes enough sense to folks who don’t have much experience with social service agencies. Thank goodness, too, because it’s hard to talk about these important issues in a casual way. The true story of risk and validity is more complicated and affects people’s lives in meaningful ways. This is a story I love to tell when I am on the clock. But for now, it is time to relax and discuss the degrees of separation between this person’s cousin’s new wife, who used to live in Madison, and my neighbor Dave who knows everyone in the city.
Chris Scharenbroch is a Senior Research Associate at NCCD.