Monthly Archives: February 2015

Early Intervention: You *Might* Get What You Pay For

I find myself caught between the genetics community – which takes it as a given that childhood experiences and education have a very limited role in shaping life outcomes – and the psychiatric community, which takes it as a given that childhood experiences and education are crucial in shaping life outcomes. Both sides have their favorite studies to cite supporting their positions. I’ve already talked about the genetics studies, so I thought I’d bring up a recent particularly good study from the other side.

Dodge et al’s Impact Of Early Intervention On Psychopathology, Crime, And Well-Being At Age 25 is published in last month’s American Journal Of Psychiatry. Gratifyingly, it is a randomized controlled trial. Ten thousand kindergarteners in disadvantaged areas were screened for “conduct problems” until they found about 900 who looked like they were at high risk. 445 were randomly selected for the intervention. Another 446 stayed in the control group. The intervention was a bunch of extra classes and ‘enrichment programs’ from elementary school (age 5) all the way through high school (age 16). The study mentions “social skills friendship groups”, “guided parent child interaction sessions”, “tutoring in reading”, “parent-youth groups on topics of adolescent development, alcohol, tobacco, and drugs”, “youth forums on vocational opportunities”, and “Oysterman’s School-To-Job possible selves intervention aimed at examining emerging identity”.

All of these sound so pretentious that I would have loved to be able to report that they had no effect, but in fact the opposite was true. When they caught up with these kids at age 25, the intervention group was found to have an odds ratio of around 0.6 to 0.7 of having developed various psychiatric disorders the study was testing for, including antisocial personality disorder, ADHD, depression, or anxiety. They had odds ratios around 0.7 of developing drug and alcohol abuse problems by various measures. They reported less risky sexual behavior, less domestic abuse, and fewer violent crimes. All of this was significant at the p < 0.05 level, and some of it was significant at much higher levels like p = 0.001 or below. Subgroup analysis found the data were very similar when you restricted the analysis to various subgroups like boys, girls, whites, blacks, highest-risk, lowest-risk, and by study site (it was a multi-site study). As best I can tell there were not an equal number of anaylses they did that came up negative that they covered up.

The apparent conclusion is that intensive interventions can change children’s outcomes and personalities in important ways ten years down the road, even regarding things believed to be highly genetic like antisocial personality disorder.

A few weak attempts to rebut this. First, there were some things that study didn’t do that one might have expected it to. It didn’t change graduation rates or employment rates. The apparent decrease in domestic violence was mediated entirely by the intervention group being less likely to have relationships (!) – the rate of domestic violence among people in relationships was the same. There was no effect on health. There was no effect on self-reported satisfaction with their parents’ parenting. There were (nonsignificantly) higher death rates and incarceration rates in the intervention group than the control group.

So if I wanted to be maximally mean to the study, I could say that whatever it’s doing to violent crime and drug use has to be compatible with a (nonsignificantly) raised incarceration rate, and whatever it’s doing to drug use and risky sexual behavior and criminality has to be compatible with a (nonsignificantly) raised death rate. This suggests the possibility of an attack based on their endpoints being screwy, though I’m not sure what form such an attack could take. One could argue that since many of their outcomes were based on self-report surveys maybe the kids who had been through all of the enrichment programs had grown to like the study people and had a stronger demand effect to say that they were doing great. But a lot of the survey data was backed up by court records confirming fewer drug and violence convictions. So that doesn’t really work.

If you’re less interested in the pure science of individual differences and more interested in policy, one fact that I forgot to mention was that this program cost $60,000 per kid. The paper points out that this is the same cost as a year or two of incarceration, so if it really changes children’s life outcomes and makes tham less antisocial even that hefty price tag might be justified (although again, remember that it didn’t affect employment or incarceration when checked directly).

If you’re looking for an optimistic spin on that number, they freely admit they have no idea which part of their gigantic ten year intervention program produced the positive effects. It could be that all the youth forums and enrichment programs and friendship groups and so on had zero effect, and the entire benefit came from the “Oysterman’s School-To-Job possible selves intervention aimed at examining emerging identity”. And maybe that’s a piece of paper that can be copied on a copy machine for ten cents a sheet. All this suggests is that at least some part of the ten-year, $60,000 intervention did something.

If you’re looking for a pessimistic spin on that number, consider. Every so often I see things that claim to have completely shifted children from the most high-risk of high-risk groups to upstanding successful members of society by giving them a year or preschool, or a couple of after-school lessons, or something like that. And these studies always boast that they did it with only $1000 or $5000 or some number like that, so it’s nice and cost effective. So far, the studies I have seen like this have been wrong. And so far I have not been surprised, because we already spend between $100,000 to $200,000 per child on education and various social programs. If someone ever found a social program that really worked for $1,000, the first thing we would want to do is tar and feather everyone currently in our bureaucracy of social programs, for being so incompetent that changing their $200,000 in spending to $201,000 in spending (with the extra $1000 going to someone besides them) could completely revolutionize life outcomes.

This study seems more in line with everything else. By going from $200,000 to $260,000, we can slightly push a few things in a positive direction a little bit more, maybe. From a scientific view, it’s pretty interesting. From a policy view, it’s nothing to write home about.