Open threads at the Open Thread tab every Sunday and Wednesday

Race and Justice: Much More Than You Wanted To Know

Previously reviewed: effects of marijuana legalization, health effects of wheat, effectiveness of SSRIs, effectiveness of Alcoholics Anonymous

Does the criminal justice system treat African-Americans fairly?

I always assumed it obviously didn’t. Then a while ago I read this harshly polemical but research-filled article claiming to prove it did. Then I found a huge review paper on the subject, written by a Harvard professor of sociology, which concluded after analyzing sixty pages of exquisitely-researched studies that:

Recognizing that research on criminal justice processing in the United States is complex and fraught with methodological problems, the weight of the evidence reviewed suggests the following. When restricted to index crimes, dozens of individual-level studies have shown that a simple direct influence of race on pretrial release, plea bargaining, conviction, sentence length, and the death penalty among adults is small to nonexistent once legally relevant variables (e.g. prior record) are controlled. For these crimes, racial differentials in sanctioning appear to match the large racial differences in criminal offending. Findings on the processing of adult index crimes therefore generally support the non-discrimination thesis.

Clearly this was more complicated than I thought. I decided to waste my precious free time reading seven zillion contradictory studies to figure out what was going on. Some people on Tumblr have demanded I report back, so here goes:

A. Encounter Rate

There are a lot of tiers to the criminal justice system, each of which will have to be analyzed individual. The first tier is – who does or doesn’t get stopped by the police?

One common point of discussion is traffic stops, leading to the popular joke that you can be stopped for a “DWB” (driving while black). Engel and Calnon (2006) seem to have done the definitive review in this area. Based on a national survey of citizens’ interactions with police, they find that 5% of whites and 11% of blacks have had their cars searched by police, with relatively similar results for other kinds of officer interactions. Therefore, blacks are about twice as likely to be searched as whites. Once you do a multiple regression controlling for other factors, like previous record, income, area stopped, et cetera, half of that difference goes away, leaving an unexplained relative risk of 1.5x.

These data admit to multiple possible interpretations. First, racist police officers could be unfairly targeting blacks. Second, blacks could be acting more suspiciously and police officers correctly picking up on this fact. Third, police officers could be racially profiling based on their past experience of more successful searches of black drivers.

One common method of disentangling these possibilities is search “success rate”. That is, if searching whites usually turns up more real crimes than searching blacks, then innocent blacks are being searched disproportionately often and the police are not just correctly responding to indicators of suspiciousness or past experiences.

Engel and Calnon review sixteen studies investigating this question. If we limit claims of dissimilarity to studies where one race is at least five percentage points higher than the other, there are eight studies with racial parity, six studies with higher white hit rates, and two studies with higher black hit rates.

In other words, in 62% of studies, police are not searching blacks disproportionately to the amount of crimes committed or presumed “indicators of suspiciousness”. In 38% of studies, they are. The differences may reflect either methodological differences (some studies finding effects others missed) or jurisdictionial differences (some studies done in areas where the police were racially biased, others done in areas where they weren’t)

The authors did their own analysis based on a national survey about citizens’ contact with the police, and found that 16% of whites searched and 8% of minorities searched reported that police had discovered contraband, a statistically significant difference. This contradicts the studies above, most of which found no difference and the others of which found much smaller differences.

One possible explanation the authors bring up is that previous research has shown black drivers who have received traffic violations are less likely than whites who have received traffic violations to admit to having received them on anonymous research surveys. For example, among North Carolina drivers known to have received tickets, 75% of whites admitted it on a survey compared to 66% of blacks (Pfaff-Wright, Tomaskovic-Devey, 2000). Comparisons of several different surveys of drug use find that “nonreporting of drug use is twice as common among blacks and Hispanics as among whites” (Mensch and Kandel). Since much of the “contraband” these surveys were asking about was, in fact, drugs, this seems pretty relevant. Overall different studies find different black-white reporting gaps (from the very small one in the traffic ticket study to the very large one on the drug use surveys). Plausibly this is related to severity of offense. Also plausibly, it relates to differential levels of trust in the system and worry about being found out – for poor black people, the possibility of (probably white) researchers being stooges who are going to send their supposedly confidential surveys to the local police station and get them locked up might be much more salient.

There are of course many other forms of police stop. These tend to follow the same pattern as traffic stops – strong data that police more often stop black people, police making the claim that black people do more things that trigger their suspicion instinct (including live in higher-crime neighborhoods), and difficulty figuring out whether this is true or false.

Sampson and Lauritsen review several studies on police stops of pedestrians. I’ll be coming back to and citing sources from this Sampson and Lauritsen article many times during this discussion as it is one of the most rigorous and trustworthy analyses around – Sampson is Professor of Sociology at Harvard and winner of the Stockholm Prize in Criminology and his review is the most cited one on this topic I could find, so I assume he represents something like a mainstream position. After reviewing a few studies, most notably Smith (1986), they conclude these sorts of police stops demonstrate no direct effect of race – in any given neighborhood, black people and white people are treated equally – but that there is an indirect effect from neighborhood – that is, the police are nastier to everybody in black neighborhoods. Although they don’t say so, the most logical explanation to me would be that black neighborhoods are poorer and therefore higher crime, and so the police are more watchful and/or paranoid.

Summary: There is good data that police stop blacks more often, both on the road and in neighborhoods. Studies conflict over whether the extra stops are justifiable; likely this varies by jurisdiction. Extra neighborhood stops are most likely neighborhood-related effects rather than race-related per se, but the neighborhood effects do disproportionately target black people.

B. Arrest Rates For Violent Crimes

Police records consistently show that black people are arrested at disproportionally high rates (compared to their presence in the population) for violent crimes. For example, blacks are arrested eight times more often for homicide and fourteen times more often for robbery. Even less flashy crimes show the same pattern: forgery, fraud, and embezzlement all hover around a relative risk of four.

(White people are arrested at disproportionally high rates for things like driving drunk, and Asians are arrested at disproportionally high rates for things like illegal gambling, but these carry lower sentences and are less likely to lead to incarceration.)

Once again, there are two possible hypotheses here: either police are biased, or black people actually commit these crimes at higher rates than other groups.

The second hypothesis has been strongly supported by crime victimization surveys, which show that the percent of arrestees who are black matches very closely matches the percent of victims who say their assailant was black. This has been constant throughout across thirty years of crime victmization surveys.

While everybody is totally on board with attributing this to structural factors like black people being poorer and living in worse neighborhoods, anyone who tries to analyze higher black arrest and incarceration rates without taking this into account is going to end up extremely confused.

There were some attempts to cross-check police data and victim data against self-reports of criminality among different races, with various weird and wonderful results. Once again, after a while someone had the bright idea to check whether people who said they hadn’t committed any crimes actually hadn’t committed any crimes, and found that a lot of them had well-verified criminal records longer than War And Peace.

Sociologists learned an important lesson that day, which is that criminals sometimes lie about being criminals.

No one has had any better ideas for how to corroborate the crime victimization survey data, so it looks like probably that’s the best we will do.

Summary: Arrests for violent crimes are probably not racially biased.

C. Arrest Rates For Minor Crimes

Usually when people talk about racial disparities in arrest rates for minor crimes, they’re talking about drugs. The basic argument is that black people and white people use drugs at “similar rates”, but black people are four times more likely to get arrested for drug crime. You can find this argument on pretty much every major media outlet: NYT, Slate, Vox, HuffPo, USA Today, et cetera.

The Bureau of Justice has done their own analysis of this issue and finds it’s more complicated. For example, all of these “equally likely to have used drugs” claims turn out to be that blacks and whites are equally likely to have “used drugs in the past year”, but blacks are far more likely to have used drugs in the past week – that is, more whites are only occasional users. That gives blacks many more opportunities to be caught by the cops. Likewise, whites are more likely to use low-penalty drugs like hallucinogens, and blacks are more likely to use high-penalty drugs like crack cocaine. Further, blacks are more likely to live in the cities, where there is a heavy police shadow, and whites in the suburbs or country, where there is a lower one.

When you do the math and control for all those things, you halve the size of the gap to “twice as likely”.

The Bureau of Justice and another source I found in the Washington Post aren’t too sure about the remaining half, either. For example, anecdotal evidence suggests white people typically do their drug deals in the dealer’s private home, and black people typically do them on street corners. My personal discussions with black and white drug users have turned up pretty much the same thing. One of those localities is much more likely to be watched by police than the other.

Finally, all of this is based on self-reported data about drug use. Remember from a couple paragraphs ago how studies showed that black people were twice as likely to fail to self-report their drug use? And you notice here that black people are twice as likely to be arrested for drug use as their self-reports suggest? That’s certainly an interesting coincidence.

The Bureau of Justice takes this possibility very seriously and adds:

Although arrested whites and arrested blacks were about equally likely to be drug use deniers, these results nevertheless have implications for the SAMHSA survey. A larger fraction of the black population than the white population consists of criminally active persons and, therefore, a larger fraction of the black population than the white population would consist of criminally active persons who use drugs but deny it. Consequently, the SAMHSA survey would probably understate the difference between whites and blacks in terms of drug use. Whether the effect of such drug use denial among criminally active persons is large enough to account for the unexplained 13% is not known, but research on the topic should pursue this possibility.

It should be noted that a study investigating this methodology gave random urine drug tests to some of the people who had filled out this survey, and found that half of the actual drug users had reported on the survey that they were squeaky clean. There were no racial data associated with this investigation, which is too bad.

Summary: Blacks appear to be arrested for drug use at a rate four times that of whites. Adjusting for known confounds reduces their rate to twice that of whites. However, other theorized confounders could mean that the real relative risk is anywhere between two and parity. Never trust the media to give you any number more complicated than today’s date..

D. Police Shootings

A topical issue these days. Once again, the same dynamic at play. We know black people are affected disproportionately to their representation in the population, but is a result of police racism or disproportionate criminality?

Mother Jones magazine has an unexpectedly beautiful presentation of the data for us:

The fourth bar seems like what we’re looking for. You could go with the fifth bar, but then you’re just adding noise of who did or didn’t duck out of the way fast enough.

As you can see, a person shot at by a police officer is more than twice as likely to be black as the average member of the general population. But, crucially, they are less likely to be black than the average violent shooter or the average person who shoots at the police.

We assume that the reason an officer shoots a suspect is because that officer believes the suspect is about to shoot or attack the officer. So if the officer were perfectly unbiased, then the racial distribution of people shot by officers would look exactly like the distribution of dangerous attackers. If it’s blacker than the distribution of dangerous attackers, the police are misidentifying blacks as dangerous attackers.

But In fact, the people shot by police are less black than the people shooting police or the violent shooters police are presumably worried about. This provides very strong evidence that, at least in New York, the police are not disproportionately shooting black people and appear to be making a special effort to avoid it.

For some reason most of the studies I could get here were pretty old, but with that caveat, this is also the conclusion of Milton (1977) looking at police departments in general, and Fyfe (1978), who analyzes older New York City data and comes to the same conclusion. However, the same researcher analyzes police shootings in Memphis and finds that these do show clear evidence of anti-minority bias, sometimes up to a 6x greater risk for blacks even after adjusting for likely confounders. The big difference seems to be that NYC officers are trained to fire only to protect their own lives from armed and dangerous suspects, but Memphis officers are (were? the study looks at data from 1970) allowed to shoot property crime suspects attempting to flee. The latter seems a lot more problematic and probably allows more room for officer bias to get through.

[EDIT: A commenter pointed out to me that Tennessee vs. Garner banned this practice in the late 1980s, meaning Memphis’ shooting rate should be lower and possibly less biased now]

The same guy looks at the race of officers involved and finds that “the data do not clearly support the contention that white [officers] had little regard for the lives of minorities”. In fact, most studies find white officers are disproportionately more likely to shoot white suspects, and black officers disproportionately more likely to shoot black suspects. This makes sense since officers are often assigned to race-congruent neighborhoods, but sure screws up the relevant narrative.

Summary: New York City data suggests no bias of officers towards shooting black suspects compared with their representation among dangerous police encounters, and if anything the reverse effect. Data from Memphis in 1970 suggests a strong bias towards shooting black suspects, probably because they shoot fleeing suspects in addition to potentially dangerous suspects, but this practice has since stopped. Older national data skews more toward the New York City side with little evidence of racial bias, but I don’t know of any recent studies which have compared the race of shooting victims to the race of dangerous attackers on a national level. There is no support for the contention that white officers are more likely than officers of other races to shoot black suspects.

E. Prosecution And Conviction Rates

Conviction rates of blacks have generally found to be less than than conviction rates of whites (Burke and Turk 1975, Petersilia 1983, Wilbanks 1987). I don’t know why so many of these studies are from the 70s and 80s, but a more recent Bureau of Justice Statistics finds that 66% of accused blacks get prosecuted compared to 69% of accused whites; 75% of prosecuted blacks get convicted compared to 78% of prosecuted whites.

The 1975 study suggested this was confounded by type of crime – for example, maybe blacks are charged more often with serious crimes for which the burden of proof is higher. The 1993 study isn’t so sure; it breaks crimes down by category and finds that if anything the pro-black bias becomes stronger. For example, 51% of blacks charged with rape are acquitted, compared to only 25% of whites. 24% of blacks charged with drug dealing are acquitted, compared to only 14% of whites. Of fourteen major crime categories, blacks have higher acquittal rates in twelve of them (whites win only in “felony traffic offenses” and “other”).

The optimistic interpretation is that there definitely isn’t any sign of bias against black people here. The pessimistic interpretation is that this would be consistent with more frivolous cases involving black people coming to the courts (ie police arrest blacks at the drop of a hat, and prosecutors and juries end up with a bunch of stupid cases without any evidence that they throw out).

There was a much talked-about study recently that found that “juries were equally likely to convict black and white offenders when there was at least one black in the jury pool, but more likely to convict blacks when there wasn’t.” This is consistent with previous studies. Jury pools contain twenty-seven members; the probability that there will be at least one black jury pool member in the trial of a black subject (who of course is most likely to live in a predominantly black area) is high. The study’s “equally likely to convict black and white offenders” was actually “2% more likely to convict white offenders than black offenders”, which was probably not statistically significant with its small sample size but is consistent with the small pro-black effects found elsewhere.

Summary: Prosecution and conviction rates favor blacks over whites, significance unclear.

F. Sentencing

Older studies of sentencing tend to find no or almost no discrepancies between blacks and whites. This was the conclusion of most of the papers reviewed in Sampson and Lauritsen. The gist here seems to be that there were “four waves” of studies in this area. The first wave, in the 1960s, was naive and poorly controlled and found that there was a lot of racial bias. The second wave, in the 1980s, controlled for more things (especially prior convictions) and found there wasn’t. The third wave was really complicated, and the writers sum it up as saying it represented:

…a shift away from the non-discrimination thesis to the idea that there is some discrimination, some of the time, in some places. These contingencies undermine the broad reach of the thesis, but the damage is not fatal to the basic argument that race discrimination is not pervasive or systemic in criminal justice processing.

The fourth wave expands on this and finds discrimination in some areas that hadn’t been studied before, such as plea bargaining. However, it continues to find that on the whole, and especially in the largest and best-designed studies there is very little evidence of discrimination. The article concludes:

Langan’s interpretation matches those of other scholars such as Petersilia (1985) and Wilbanks (1987) in suggesting that systemic discrimination does not exist. Zatz (1987) is more sympathetic to the thesis of discrimination in the form of indirect effects and subtle racism. But the proponents of this line of reasoning face a considerable burden. If the effects of race are so contingent, interactive, and indirect in a way that to date has not proved replicable, how can one allege that the “system” is discriminatory?

A more recent (fifth wave?) review adds some problems to this generally rosy picture, saying that “Of the [thirty-two studies containing ninety-five different] estimates of the direct effect of race on sentencing at the state level, 43.2% indicated harsher sentences for blacks…at the federal level 68.2% of the [eight studies containing twenty-two different] estimates of the direct effect of race on sentencing indicated harsher sentences for blacks”. The majority of estimates that did not find this were race-neutral, although six did show some bias against whites. They conclude:

Racial discrimination in sentencing in the United States today is neither invariable nor universal, nor is it as overt as it was even thirty years ago. As will be described below, while the situation has improved in some ways, racially discriminatory sentencing today is far more insidious than in the past, and treating a racial or ethnic group as a unitary body can mask the presence of discrimination.

I really like how you can make a large decrease in the level of a bad thing sound like a problem by saying it is becoming “more insidious”.

Even more recent studies have found even larger gaps. A study by the US Sentencing Commission investigating the effect of new guidelines found that blacks’ sentences were 20% longer than those of similar whites; a later methodological update reduced the gap to a still-large 14.5% and a a different recent study says just under 10%. Although the particular effect of these new guidelines is a matter of HORRIBLE SUPER-COMPLICATED DEBATE, neither side seems to deny the disparities themselves – only whether they are getting larger.

It’s not clear to me why there’s such a difference between the earlier studies (which found little evidence of disparity), the middle studies (which were about half-and-half), and these later studies (which show strong evidence of disparity). I guess one side of a HORRIBLE SUPER-COMPLICATED DEBATE would say it has to do with changes in sentencing during that time which replace mandatory sentences with “judicial discretion”. If you’re mandated to give a particular sentence for a particular crime, there’s a lot less opportunity to let bias slip in then if you can do whatever you want. There is some evidence that different judges treat different races differently, although the study has no way of proving whether this is anti-black bias, anti-white bias, or an equal mix of both in different people. Unfortunately, there is also concern that mandatory minimum sentencing is itself racist.

Capital punishment is in its own category, and pretty much all studies, old, new, anything agree it is racist as heck (Sampson and Lauritsen cite Bowers & Pierce 1980; Radelet 1981; Paternoster 1984; Keil and Vito 1989; Aguirre and Baker 1990; Baldus Woodward & Pulaski 1990 – there’s no way I’m reading through all of them so I will trust they say what the review says they say). This seems to consist not only in black suspects being more at risk, but in white victims’ deaths being more likely to get their offenders a death sentence.

Summary: Most recent studies suggest a racial sentencing disparity of about 15%, contradicting previous studies that showed lower or no disparity. Changes in sentencing guidelines are one possible explanation; poorly understood methodological differences are a second. Capital punishment still sucks.

Summary

There seems to be a strong racial bias in capital punishment and a moderate racial bias in sentence length and decision to jail.

There is ambiguity over the level of racial bias, depending on whose studies you want to believe and how strictly you define “racial bias”, in police stops, police shootings in certain jurisdictions, and arrests for minor drug offenses.

There seems to be little or no racial bias in arrests for serious violent crime, police shootings in most jurisdictions, prosecutions, or convictions.

Overall I disagree with the City Journal claim that there is no evidence of racial bias in the justice system.

But I also disagree with the people who say things like “Every part of America’s criminal justice is systemically racist by design” or “White people can get away with murder but black people are constantly persecuted for any minor infraction,” or “Every black person has to live in fear of the police all the time in a way no white person can possibly understand”. The actual level of bias is limited and detectable only through statistical aggregation of hundreds or thousands of cases, is only unambiguously present in sentencing, and there only at a level of 10-20%, and that only if you believe the most damning studies.

(except that you should probably stay out of Memphis)

It would be nice to say that this shows the criminal justice system is not disproportionately harming blacks, but unfortunately it doesn’t come anywhere close to showing anything of the sort. There are still many ways it can indirectly harm blacks without being explicitly racist. Anatole France famously said that “the law, in its majestic equality, forbids rich as well as poor people from begging for bread and sleeping under bridges”, and in the same way that the laws France cites, be they enforced ever so fairly, would still disproportionately target poor people, so other laws can, even when fairly enforced, target black people. The classic example of this is crack cocaine – a predominantly black drug – carrying a higher sentence than other whiter drugs. Even if the police are scrupulously fair in giving the same sentence to black and white cokeheads, the law will still have a disproportionate effect.

There are also entire classes of laws that are much easier on rich people than poor people – for example, any you can get out of by having a good lawyer – and entire classes of police work that are harsher on poor neighborhoods than rich neighborhoods. If the average black is poorer than the average white, then these laws would have disproportionate racial effects.

For more information on this, I would recommend Tonry and Melewski’s Malign Neglect: Race, Crime, and Punishment in America. They begin by saying everything above is true – the system mostly avoids direct racist bias against black people – and go on to say argue quite consistently that we still have a system where (their words) “recent punishment policies have replaced the urban ghetto, Jim Crow laws, and slavery as a mechanism for maintaining white dominance over blacks in the United States”. If you want something that makes the strongest case for the justice system harming blacks, written by real criminologists who know what they’re talking about, there’s your best bet.

(warning: I haven’t read the book. I did read a review article by the same people, which the book is partially based on)

Some police officers say the reason they are harsher in poor urban neighborhoods is that the expectation of high levels of unruly behavior necessitates unusually strong countermeasures. For the same reason, I am screening all comments for the next few days. If you post one, expect it to show up eventually or perhaps disappear into the aether.

295 Responses to Race and Justice: Much More Than You Wanted To Know

  1. Meredith L. Patterson says:

    Regarding arrest rates for violent crimes, has anyone looked at the number of arrests before charges are laid in the case of white suspects vs. black suspects? (I.e., person is arrested but the arraignment leads to no finding of probable cause, so the person is released, eventually someone else is arrested, &c.)

  2. Scott says:

    In part A you base whether police are racially biased by whether they pull over or approach black people more or less than the baseline black crime rate. Which might make sense from a predictive perspective, but is also exactly what most people point to when they say police are biased against blacks. The upset isn’t that the police are overshooting a proportional response to black crime, it’s that they’re applying proportionality at all. The idea being, everyone should get an equally fair chance and not be categorized by their skin color at all.

    This of course doesn’t apply in part C, where police have concrete reasons to shoot back at someone rather than a feeling that they’re ‘acting suspicious’.

    • Montfort says:

      This is a fair point and does seem to conflict with what I intuit “fair” policing would be like.

      But on the other hand, the question is really whether the approach rates are proportional to legitimate suspicious/lawbreaking behavior which may not be constant among people of different races, socio-economic classes, etc. I believe in this case we’re using criminal statistics as a proxy for likeliness to actually be suspicious on non-racial grounds, which should probably follow unless one race produces much-less-suspicious criminals or much more suspicious law-abiding citizens.

      Ideally we could perform a study controlling for suspiciousness, but it seems like a difficult exercise to define what behaviors would count as “legitimate” bases for suspicion. Many people would agree it doesn’t include race, religion, or sexual orientation, but beyond that I could barely hazard a guess at what does and doesn’t count.

      • I agree with commentscott that postscott should note this. The question a lot of people will be interested in is ‘In areas where cops stop people of different races in proportion to how likely members of those races are to commit a crime, do cops become so well-calibrated by simply noticing the same suspicious behaviors in everyone, or do they get a leg up by using people’s race as evidence?’ (Which can mean discounting suspicious signs from white people, or being oversensitive to suspicious signs from black people, or both.)

        And then there’s the factual question of whether racial profiling, if effective, is a good thing for the U.S. to formally or informally practice — whether it contributes to racial inequality and tension, is intrinsically evil, etc.

        The claims I generally see are ‘cops use racial profiling’ and ‘racial profiling contributes to black poverty and crime’, not ‘cops arrest black people too often for how common black people commit crimes.’ Scott undermines a possible method one could use to demonstrate de-facto racial profiling, but it’s not clear how essential that method is for the conclusion, and we certainly haven’t established the conclusion’s falsity. To test this, you might have actors drive around bad parts of town until they get pulled over, and see how often white v. black actors get searched. If it’s a sufficiently well-funded study, you could even use make-up to change the racial appearance of one and the same actor.

        • Anonymous says:

          However, you should note that it is not job of the police to raise more blacks out of poverty. Their job is to catch criminals. Therefore, if racial profiling helps them to catch more criminals, they would (maybe even should) do it. You would have to change the job description of the police if you want to change this situation

          • llamathatducks says:

            The job of police officers is to make society safer. A population that is under special police scrutiny (e.g. a predominantly black neighborhood in NYC) is not very safe, not only because it’s probably relatively high-crime to begin with but also because low police accountability + high police power means people are gonna be pretty damn scared of police. More broadly, I believe part of the job of any public employee is to generally serve society and not make things worse. Yeah, a cop isn’t a social worker, but “do no harm” is important, and the frustration that many black people feel at having been pulled over in suspicious circumstances, the feeling that they are under special surveillance, suggests that harm is being done.

          • Desertopa says:

            The purpose of the justice system may be to make society safer, but that doesn’t mean that this is the job of police officers on an individual basis. While it may be an ideal we’d like to hold them to, it’s not the direction where their incentives lie.

            If we want police to make society safer it’s not enough to say it’s their job, we have to design their jobs so that making society safer will be a natural result of following their job incentives.

          • llamathatducks says:

            I strongly agree with your last statement.

    • Andrew C says:

      And one can of course make the same criticism about police shootings. Several times more people were fired upon by police than fired at police. Even if police are in some sense correct in being scared of black people, that still means that black people who are not in fact to blame are going to suffer for it.

      • Lambert says:

        People carrying handguns are also treated with more suspiscion than those who are not, even when they have no intention of shooting anyone other than in self defense. Is this unfair?
        Of course, it is easier to disarm oneself than stop being black.
        (Please can someone strengthen this argument by finding an analogous behaviour less contriversial than being armed.)

    • [C. Warren] Dale says:

      The idea isn’t that the cops are profiling the correct proportion of people by their skin color, but rather that the cops are correctly identifying suspicious behavior, which is exhibited by certain racial groups in the same proportion as criminal activity.

    • meh says:

      I think this depends on what your model for unsuccessful pull overs are. If a portion are legitimate pull overs that did not meet some threshold for evidence, than you would suspect them to trend toward baseline crime rates; and this would be a reasonable response by police. If they were random events, they would trend toward baseline population, and if they didn’t you could claim bias.

      Probably, there is some combination of both, making it difficult to say what the right percentage should be.

  3. Thad says:

    Why do you assume that police only shoot at those who they believe are about to shoot or attack them? NYPD regulations, at least as of 2004, allowed for officers to shoot if they had “probable cause to believe that he or she must protect self or another person present from imminent death or serious physical injury”

    http://www.nyc.gov/html/nypd/downloads/pdf/public_information/RAND_FirearmEvaluation.pdf

    • John Schilling says:

      It is a fairly common shorthand in legal circles to use “self defense” for the more cumbersome “defense of self or an innocent third party”; I wouldn’t fault Scott for the same.

      And I’m not sure it makes a big difference in the numbers anyhow, particularly in a place like New York City where essentially only the police and the criminals have guns. How often do we really expect an armed criminal, in the presence of an armed police officer, to reason, “I shall shoot this unarmed victim first, and only then turn my attention to the policeman with his gun aimed at me”? If there’s a cop around, he’s probably the number one priority of every criminal in sight.

      • Thad says:

        I am aware of what “self defense” means, but it is not the term that Scott used. I question his assumption because he goes on to infer some things from it

        We assume that the reason an officer shoots a suspect is because that officer believes the suspect is about to shoot or attack the officer. So if the officer were perfectly unbiased, then the racial distribution of people shot by officers would look exactly like the distribution of dangerous attackers. If it’s blacker than the distribution of dangerous attackers, the police are misidentifying blacks as dangerous attackers.

        The only numbers for dangerous attackers we are shown are those who fired on the police. We can argue justifications for using the numbers we have, but the comparison Scott makes doesn’t stand without additional justification.

  4. gattsuru says:

    One aspect that’s topical and reflects your like of Implicit Association Tests is a recent study (pdf warning) showing that flashed images of a black person’s face caused viewers to be more likely to mistake common tools for dangerous weapons than flashed images of a white person’s face, as well as some other interesting implications

    … Memphis officers are (were? the study looks at data from 1970) allowed to shoot property crime suspects attempting to flee.

    It’s /probably/ worth mentioning that the United States courts eventually reviewed the standard for police use of lethal force, resulting in the 1989 case Tennessee v. Garner‘s general prohibition on such barring known risks of physical harm to a person. Many states did allow police to fire on fleeing suspects before that point, but no longer do so (even if the statute remains on the books, juror instructions use the Garner standard).

    Mother Jones magazine has an unexpectedly beautiful presentation of the data for us…

    This is particularly fascinating because Mother Jone’s author pretty clearly intended for the reader to take the exact opposite response from the graph : because African-American individuals were targeted by police more often than their demographic proportion of the populace, that’s direct evidence of Bad Things in the community’s police. I’m actually kinda surprised they bothered to present the entire right side of the chart.

    Caveat: New York’s numbers seem likely to not generalize to the wider populace, both for some structural reasons, and also because New York City had some notorious and neighborhood-structure-focused (that-looked-hella-racist) interventions in the time range.

    • Tarrou says:

      I do psych research for what passes for a living, and Implicit Association Tests are ridiculously fickle, notoriously unreliable and pretty hard to get a decent journal to accept on their own as a sign of much of anything. There’s just too many feedback loops, one of which my team discovered just last year. Apparently vocalization reverses similarity bias in priming tasks (which is basically what IATs are). I’d be very, very skeptical of hanging one’s hat on any study which relies on this sort of experiment. There are huge validity questions.

      • Bob says:

        >Apparently vocalization reverses similarity bias in priming tasks (which is basically what IATs are)

        Can you translate this into something a non-psyche research can understand? And ideally cite studies showing it?

    • Ben Southwood says:

      Association reaction is pretty adaptive given crime differentials.

  5. Bugmaster says:

    If I were a Straw Social Justice Warrior, here’s what I’d say:

    “Of course, the studies show that there is very little bias against blacks in our justice system. This is because the academia is predominantly white, and white people are racist, so naturally they’d produce racist studies. It’s all about privilege distress”.

    So, out of curiosity, is there a way to perform a meta-meta analysis of these studies, to see if the race of a researcher has any significant effect on the outcome of the study ?

    (On an unrelated note, there’s a minor bug in the commenting system: if the article has no visible comments yet, then, when you click the link to the comments, you will see two copies of the “Leave a Reply” form. Most likely, this happens because the code always inserts the top form, followed by the list of comments, followed by the bottom form, without checking if the list is empty).

    • hawkice says:

      A meta-analysis probably isn’t the solution here. If you can’t trust statistics, then make a new study — I’m interested in racism in studies because it gives me insight into academia, but not because it gives me insight into the object-level discussion. If results came back and said black and white researchers disagree, you still don’t know who is right, unless you do another study. So if that’s the objection, just do another study.

    • veronica d says:

      Well, I’m an actual SJW and I say this: even if racial profiling works, insofar as it actually matches the level of suspicion with the level of criminal behavior in a category, we should still hesitate to use it. The reasons are pretty easy to figure out. Stopping one crime today is clearly good, especially for a violent crime. But creating a divided culture is bad. These things are structural. What happens in the aggregate is felt by the individual. They notice. They feel harassed. They increasingly distrust the broader culture. Schisms deepen.

      Which in the end will likely worsen crime in their areas, which is clearly bad. Just as the victim of the crime you fail to stop will pay a cost today, the victims of a racist, unjust culture will pay the costs tomorrow.

      This is an optimization problem.

      • Anonymous says:

        I can be anti-SJW and strongly agree with this. Of course, I’m against a lot of the criminalization of drug use/possession which I would assume is the vast majority of crime prevented by traffic stop (etc) searches.

        • veronica d says:

          Ending the war on drugs would be a win-win-win-win.

          • Matt says:

            Yeah it would. So much of our criminal justice system’s problems flow directly from the war on drugs. It was just a terrible idea.

          • Tongue planted firmly in mouth says:

            But what if you sell weapons to law enforcement and run a private prison? Then it’s lose-lose for you 🙁

      • von Kalifornen says:

        I’d also say there’s a bootstrapping issue, what with cycle of crime and poverty?

      • pneumatik says:

        Why do you assume racial profiling? Whenever Scott mentioned suspicious behavior I assumed that meant some overt action that made the cop think the actor had committed a crime. Do you take it to include cops deciding that being black is suspicious, because I didn’t.

        • Auroch says:

          ‘Suspicious behavior’ is inherently subjective, and assuming you believe that racial bias exists (even if justified), it’s unreasonable to think that bias doesn’t affect this subjective judgment in cases where it’s not actually appropriate.

          • In the discussion of the distinction between a bias in arrests due to racial bias by the police and bias due to blacks engaging in more suspicious behavior than whites (and that correlating with being more likely to be guilty), I’m not sure anyone has thought through the implications of statistical discrimination.

            Suppose black drivers are twice as likely to have drugs in their cars as white drivers, and police know this. Further suppose, to simplify matters, that people with drugs in their cars, black and white, are clever enough not to act suspiciously any more than anyone else. Finally, assume the only objective of the police officer is to catch as many people with drugs as possible.

            Assuming a plentiful supply of both black and white drivers who could be stopped, the police officer will stop only black drivers. The result will be a bias in who is stopped much larger than the bias in who is guilty.

            If, in the real world, twice as large a fraction of blacks are actually guilty and the police stop twice as large a fraction of blacks, that fits the “police are responding to suspicious behavior” model but it doesn’t fit the “police are using their knowledge of relative rates of criminality to make skin color a proxy for likelihood of carrying drugs” model. If both police bias and response to suspicious behavior are happening (and assuming away lots of other possible complications), we would expect relative rates for traffic stops to be more unequal than relative rates of drug possession, not proportional to them.

      • Not THAT anonymous commenter says:

        Thanks for putting this argument forward so elegantly!

        I fear, looking at later comments, that it’s not going to convince people who don’t see a more racially harmonious society as such an important goal versus crime prevention, hiring the right candidate for the job, and so on.

        I really wish I could make a statistical case that the harm caused by having minorities believe the stereotype that white people are generally a bunch of at-least-semi-racists is more than aesthetic or ideological.

        • houseboatonstyx says:

          the harm caused by having minorities believe the stereotype that white people are generally a bunch of at-least-semi-racists

          But how much of that belief comes from personal experience, how much from research sources, and how much from media, politics, and fiction? Practical improvements in, say, police behavior, would take years at least to affect that popular belief, especially if research and reports of improvement were dismissed as coming from white sources.

      • John Schilling says:

        We’ve already got the divided culture. Per Scott’s graph, bars 1-3, blacks and hispanics (henceforth “nonwhites”) make up ~half of New York’s population and commit ~95% of the gun-related crimes. That’s a 20:1 disparity between
        two very different cultures separated by a river of blood.

        Obviously, we ought to do something about that, but if racial profiling is not the right thing to do and if we don’t accept “No we are NOT doing racial profiling; 95% of our stop-and-frisks are against nonwhites because 95% of the people with e.g. suspicious bulges under their jackets are not white”, then it seems as if you are insisting that the metric for doing the right thing must be for ~50% of the stops, searches, and maybe arrests to be of whites.

        How do we plausibly get there? Mathematically, it seems to me that any possible answer has a combination of either A: police giving ~95% of nonwhites who do legitimately suspicious stuff that ought to be investigated a pass on account of we don’t want to look racist, or B: stopping, searching, maybe arresting about twenty innocent white people for every one who is actually guilty or even suspicious just to balance the numbers.

        Plan A leads to endless American Rotherhams, as the police are crippled in their ability to police nonwhite communities. Plan B, to systematic injustice perpetrated against white people to make nonwhites feel better. I can understand why one might thing white Americans deserve this, ought to be subjected to this, ought to put up with this as the price for what they have done to nonwhites. I can even understand the wish or hope that the result will be an increased understanding by whites of the suffering of nonwhtes and thus a path to racial justice.

        That’s wishful thinking. How do you really, truly think that white people are going to respond to this? Who are they going to blame, and what are they going to do about it?Does the answer point anywhere near the direction of a less-divided society?

        And on the subject of wishful thinking, do you really truly believe that either Plan A or Plan B results in the nonwhite community deciding, in anything less than generations, that since the police harassment is now symmetric they will collectively cut down on their violent criminal behavior by 95% or so? Because we’ve still got that little problem driving the division.

      • ATairov says:

        That position is reasonable and nuanced. I wouldn’t consider it an SJW position. (Perhaps an SJA position.)

    • tiny nerd says:

      The less straw objection to all of these results would be to accept them as literally true but claim that blacks are convicted of (and charged with) things they did not do at a disproportionate rate. In that case, all the results that are “fair” when controlled for relative crime rates would actually be racially biased, and one could certainly claim that police target black people excessively, which leads to excessive false convictions which cover up the initial bias. This would also explain why blacks have lower conviction rates in court, if the juries are less biased than the cops (also it takes more evidence to find an innocent person guilty than it does to try one for a crime).

      The problem is, I’m honestly not sure if there’s a way to study this. Like, if your hypothesis is “courts regularly find innocent people guilty and I want to know who and how much”, how do you tell who’s actually innocent and who’s actually guilty? I don’t think overturned convictions would help, since they’re both rare and probably depend more on what kind of resources are going to prove a convict innocent than on the actual truth of the matter.

      • Douglas Knight says:

        There is one study of exonerations that is not subject to such biases. In Virginia a forensic examiner systematically saved evidence which was later systematically given DNA tests. It did not find a racial bias in misidentifications. But it’s only a few dozen exonerations.

    • Nicholas says:

      I do remember one study on paranormal research, which found that even in a double-blind construction, even in a double-blind construction that was actually followed, if the lead researcher so much as met the subject in the lobby there would be demand characteristics (is that what that’s called?).
      In case I got the wording wrong, there is very, very tenuous and orthogonal evidence that racially motivated researchers produce racially motivated results, even if strict double blinds are in place.

  6. Anonymous says:

    To complicate an already complicated issue, how would publication bias play into this?

    • Leonard says:

      I thought this too. I.e.:

      “For some reason most of the studies I could get here were pretty old…”

      ” I don’t know why so many of these studies are from the 70s and 80s…”

      • John Schilling says:

        Crime rates were very much higher in the 70s and 80s than today, so it is not surprising the issue was more extensively studied then than now.

        As it is unlikely anyone is going to simply repeat now the studies that were done then, we’re going to be left with a lot of speculation as to what that means.

      • Steve Sailer says:

        ” I don’t know why so many of these studies are from the 70s and 80s…”

        Because most of these issues were studied in great depth a generation ago. I’ve been a social sciences aficionado since 1972 and I have a fairly good memory of what was being studied back then. The questions were largely the same, as were the findings.

    • gwern says:

      I think you can predict how publication bias comes into play.

      Incidentally, here’s a fun quote I once stumbled across:

      “School Desegregation and Black Achievement: an integrative review” http://deepblue.lib.umich.edu/bitstream/2027.42/68589/2/10.1177_0049124185013003002.pdf , Wortman & Bryant 1985:

      “The 31 studies found to be acceptable [by our statistical criteria for our meta-analysis] contained only two published articles.”

  7. Scott says:

    In case you are interested, another study on sentencing (“Does Unconscious Racial Bias affect trial judges?”):
    http://scholarship.law.cornell.edu/cgi/viewcontent.cgi?article=1691&context=facpub

    On the downside, it uses survey/hypothetical situations. On the upside, it uses real magistrates.

    It finds unconscious racial bias (IAT measured) exists at a similar level among judges as one would expect for their demographic; but when the race of the defendant is made explicit to the judges, the bias does not affect sentencing – at least among white-biased judges.

    When combined with other studies showing (similar, IAT measured) bias does affect the sentence a non-judge would give it may explain some of the discrepancies the third/fourth wave find (eg in plea bargaining the judge would have minimal influence).

  8. Totient says:

    This is an excellent, well written, well researched post that quite literally made me more hopeful that humanity is capable of eventually getting its shit together.

    (I made the mistake of looking at my facebook feed earlier today, which made me a lot less hopeful. So this comes across as a breath of fresh air.)

    Thank you.

  9. Lawful says:

    The Supreme Court put a stop to police departments shooting “fleeing felons” in Tennessee vs Garner (1985), a case that arose out of Memphis where a cop shot an unarmed 15 year old who was running away from a house he robbed for $10 and a purse. Police are only permitted to use deadly force when a suspect poses a significant threat of death or serious injury to the officer or others since that case.

    • Patrick says:

      Yes, but White America responded by collectively deciding that a “criminal” poses a “significant threat of death or serious injury” at all times he cannot be affirmatively determined not to pose such a threat.

      • SplendidBauble says:

        Can you provide evidence of this? My experience in criminal justice statistics and research doesn’t come up with examples, much less sufficient data to justify your conclusion.

  10. thirqual says:

    There is, from the point of view of western Europeans, an additional factor to consider. American policemen use lethal force much more often, for a wider range of reasons, and take, publicly (through their hierarchy, their unions, etc), very risk-adverse positions. Their accountability is also low, compared to the standards of the other side of the Atlantic. The consequences from pissing of a police officer are worse in the US, with smaller chances to defend yourself after the fact. As always, the ones most likely to face police are the poor, leading to, in effect, a lot of dead black Americans at the hand of the police forces, with little to none of the accountability expected.

    • Adam Casey says:

      This is the big thing that I find hard about empathizing with over this whole story. I cannot imagine any civilized people tolerating armed officers shooting people in anything like these numbers. I imagine myself in the officer’s shoes and … find it hard to imagine escalating that much.

      But of course this is also true of America generally. I can’t imagine shooting someone whilst out for a walk in my gated community. Or shooting someone from a rival gang, or… heck … shooting someone.

      It’s most important on the police side naturally. But when we make the comparison Europeans tend to forget that Americans generally are absurdly violent people by our standards.

      • Cliff says:

        That’s easy to say when no one has any guns. You would rather be beaten to death or shot then shoot someone yourself?

      • drethelin says:

        What do you mean by “anything like these numbers”? People seem to treat the absolute number of people shot by the police as Strong Evidence that the police are out of control. But to me, that number is all but meaningless without other information. When all you do is list the names of a bunch of people shot by the police, you are making an appeal to emotion. At least tell me how many people SHOULD be getting shot by the police!

        • Auroch says:

          The same number per capita as in a European country at most; probably less, since there are more people per capita outside the cities in the USA. For a specific country, Belgium would work, as their police are regularly armed (unlike the UK) and still don’t have nearly the shooting rate. (HT yxoque)

          • Matt says:

            Uh, but the people aren’t regularly armed. I’d wager being a cop in the US is quite a lot more dangerous.

          • Anon says:

            Belgium’s per-capita murder rate is a third of that of the US. I tried to find the number of people shot by police in Belgium, but couldn’t; do you have those numbers on hand?

        • Richard says:

          Personally I don’t think anyone SHOULD be shot by anyone else. The word should implies that there is an optimum number to shoot and I have some problems with that whole premise. The optimum number would be zero, and anything above that is a failure on someones part. Possibly the guy being shot, but still a failure that one SHOULD try to avoid.

          • Rowan says:

            The optimum number is the number produced by the optimal system for fallible human civilians, criminals and police officers to be governed by and operating within, which because humans aren’t perfect and cannot be made immune to error, will necessarily be greater than zero.

      • John Schilling says:

        Can you tolerate a civilized society tolerating people shooting at police officers in such numbers? Because even if we assume that American police officers always give armed criminals the first shot and never shoot at someone who is merely trying to stab or club them to death, American policemen are still going to be shooting an order of magnitude or so more people than their European counterparts.

        If the price for being recognized as a civilized nation by Europeans is that American police officers must lay down and die rather than engage in the barbaric act of shooting at armed criminals, then I’m going to count myself among the proudly barbaric Americans.

        If you’ve got the secret sauce that peacefully stops anyone from shooting or stabbing police officers, that works on blacks and hispanics and whites and asians in all their various cultures and subcultures, this might be a good place to share the recipe.

        We have a huge population of criminals of a particularly violent class that you don’t. And you have more violent criminals of your own distinct types than you have really yet acknowledged. Be careful of assuming that the same policing techniques or standards can apply in both places, or of defining your way as the One True Civilized Path.

        • Anonymous says:

          We have a huge population of criminals of a particularly violent class that you don’t. And you have more violent criminals of your own distinct types than you have really yet acknowledged.

          This deserves expounding. An example of a distinct type violent criminal could be general Mladic. Which when he showed up in Srebrenica in 1995, what did the Europeans from UNPROFOR do? The refrained from shooting him. And when a couple years later the world felt that maybe Mladic and his friends had to be stopped from a repeat performance in Kosovo? The Americans had to step in.

      • ” Europeans tend to forget that Americans generally are absurdly violent people by our standards.”

        You oversimplify a little, perhaps because you are thinking of a subset of Europe. Russia (which you may or may not count as European), Lithuania, Estonia, Moldova, Belarus, and Albania all have higher homicide rates than the U.S., Latvia and the Ukraine only slightly lower. Limiting it to western Europe comes closer to justifying your statement, but Norway has a murder rate almost half that of the U.S.

        • Anonymous says:

          You should not compare the murder rates of small countries to the murder rates of large countries. Russian and Ukraine are good examples, but the rest are bad comparisons for this reason. If you break it down by state, America has diversity, too.

          • Anonymous says:

            Why is making a comparisons of rates between small and large countries a bad thing? I don’t understand. It seems to me comparing rates is exactly how you should go about making valid comparisons between countries of different sizes. Now, if we were comparing the Vatican (with a population of 800) with the US, I might agree with you, but Latvia, Estonia, etc. are all large enough for the rates of murder to be meaningful information.

            “If you break it down by state, America has diversity, too.”

            I don’t see how this undermines or qualifies anything that David Friedman said.

          • original says:

            It is true that all of these countries are big enough that the murder rate is stable from year to year. If you really want to compare Norway to America, then, yes, you should use the murder rate and not the number of murders. But why are you comparing Norway to America? If you are Norwegian, maybe it’s a reasonable thing to do. But David Friedman chose Norway specifically because it is an outlier. That is a contrived comparison. It is reasonable to compare America to Western Europe as a whole, or the most violent state to the most violent country. In the case of Eastern Europe, he named a lot of countries, so it is more suggestive, but still a poor methodology.

        • Andrew G. says:

          Norway only has a murder rate approaching half of the USA’s if you look only at one specific year, in which one specific killer accounted for something like 70% of the total victims.

          In other years, the USA’s murder rate is 6 to 8 times higher than Norway’s.

    • TeslaCoil says:

      Having lived in both the U.S. (Boston, MA) and Italy (Torino), and now living in the U.K. (Glasgow), based on my personal experience and news exposure, I would rate police accountability as about the same in the U.S. and the U.K., both of them doing slightly better than Italy.

      I recognize that both the U.S. and Western Europe are huge, so YMMV, but now I wonder if there’s any hard data on police accountability by country.

  11. Were the “There is some evidence that different judges…” link and the “…mandatory minimum sentencing is itself racist” link intended to have the same destination? I think the latter may be going to the wrong place.

  12. Pepe says:

    An article I found very interesting, that I think you will like.

    http://www.mcgilldaily.com/2014/11/everything-problematic/

    • somnicule says:

      My flirtations with the radical left were not as deep, but failing to swallow the anti-intellectualism pill and becoming frustrated with political vaporware were also what broke me out of that particular spiral. Plus the outside view of revolutionary politics in general.

  13. Markus Ramikin says:

    Say you’re playing a game where a light is going to go on 100 times, and each time it’s either going to be green or red, and you have to guess in advance. Every time you guess right, you win a dollar.

    Suppose that you notice that on average, green shows up twice as often as red, but you can’t see a pattern to it. If you want to maximise your winnings, should you on average bet on green twice as often as red to match the frequencies you’re seeing? No, you should strictly bet on green every time.

    (Anyone know where I might have read about this before? Pretty sure it’s somewhere on LW, but I can’t find it.)

    Similarly, if a random black person is statistically more likely to be a criminal than a white person, then a police officer’s or prosecutor’s career incentive is to focus on them.

    Of course this wouldn’t fly as a practical policy. Green and red lights may be independent from your choices, but humans are not. If you completely removed police overwatch from non-black populations, this would encourage crime in those populations. Even if you didn’t care, there’s no way you’d defend yourself from accusations of blatant racism and unfairness.

    Still, the incentive is there. And it’s based on math – racial prejudice not required.

    Because I believe people tend to follow incentives, my current best guess is that police do over-profile (target the higher risk groups more than the actual risk differences would suggest), and they are going to, and the only question is to what extent this can be mitigated.

    …except when the mitigatory incentives become too strong and the balance swings in the other direction, which is where you get the Rotherham fiasco.

    So hm… if it can go either way, I suppose I should admit the possibliity that at least some jurisdictions are getting these things just right, or close enough…

    • Scott Alexander says:

      Good work bringing that up! I’d actually thought about that exact example (you’re probably looking for the reference in Technical Explanation of Technical Explanation) and decided not to bring it up.

      It would apply if police had a quota of, say, one hundred searches per year and had to distribute them in the most crime-catching way.

      But in fact police are probably searching based on some indicator of suspiciousness, combined with race-related effects. So I think that means the light analogy doesn’t hold.

      • Markus Ramikin says:

        Thanks! Yeah, that’s the link.

        Dammit, I was hoping to edit to account for indicators of suspiciousness before you approved my comment. Aren’t you in the States? What are you doing up at this hour? 😛

        Quoting a guy from the “Don’t talk to police” talk:

        “When I was a police officer, a uniform, I could follow a car however long I needed to, and eventually they’re going to do something illegal, and I can pull them over—and justifiably illegal to pull them over.”

        The question then becomes whom the police officer chooses to follow, doesn’t it?

        To the extent that police work involves arbitrary choices, free reign in where to sniff for indicators of suspiciousness in the absence of already visible ones, there’s room for my analogy to work. It’s not the whole picture, noot even close, but I do think it’s enough of a factor that I had a bit of a point.

        • Anonymous says:

          >“When I was a police officer, a uniform, I could follow a car however long I needed to, and eventually they’re going to do something illegal, and I can pull them over—and justifiably illegal to pull them over.”

          I have been followed several times, and pulled over once (when I made the mistake of choosing a route with a stop sign). Normally I will either pull into a friend’s driveway or stop at a store because
          1) fuck off
          2) and turn off your high beams

          I live in 90% white town. The median family income is $140K. I am white and drive a clean car manufactured after 2000. I can’t imagine how harassed people less fortunate than me might feel.

          p.s. I’m not suffering from a persecution complex, I have verified these are police officers every time. A few might be false positives, where they just happened to be following my route. The one who pulled me over after the stop sign admitted he had been following me.

          • Matthew says:

            I recall my (white) cousin telling me that when he was a teenager in North Carolina in the 1990s, the police used to tailgate cars with young drivers, trying to goad them into speeding so they could be pulled over.

            (Eventually, this sufficiently pissed off my cousin or one of his friends that they checked that everyone was buckled up, slammed on the brakes, and claimed an animal had run across the road in front of them. Even a cop can’t really argue that rear-ending someone isn’t his fault. I’m fairly confident this would not have been attempted had the majority of the teens in the car not been white.)

      • vV_Vv says:

        But in fact police are probably searching based on some indicator of suspiciousness, combined with race-related effects. So I think that means the light analogy doesn’t hold.

        Police are more likely to patrol high-crime neighborhoods, and be more “suspicious” in these neighborhoods than in low-crime ones.

      • I made the same point Markus just made before having read this far. But note that if police are acting partly on (rational) statistical bias and partly on suspicious behavior, and suspicious behavior is proportional to guilt, the result will be an arrest ratio (or stopping ratio or whatever) more disproportionate than the guilt ratio.

    • Sniffnoy says:

      The “guess green twice as often as red” strategy is known as probability matching; probably a term you want to look for.

      • Ian says:

        As a litmus test for level of discussion I searched this page for the phrase “probability matching” and I was extremely pleased to see it appear. 🙂

    • Tongue planted firmly in mouth says:

      You’re forgetting that that when the police officer/prosecutor/etc makes the decision to search/prosecute/etc a person, they have much more information than simply that person’s color.

      If you want to frame it as a choice between alternative targets of investigation, the two choice model you presented isn’t really accurate. The implication is that the cop faces a series of decisions to choose between investigating a single red person, and a single green person (with no further information). In this case, the solution of “investigate the red person every time” is optimal.

      A more realistic model is that the cop is forced to choose a limited number of individuals to investigate from a pool, given information about their color as well as other information about the person (for example, are they wielding a blood-stained machete?). The goal is obviously to maximize the number of criminals found. If we can imagine the cop computing a probability function p(crime|Evidence, Color), we note that the optimal solution is to pick the individuals with the N highest probabilities of crime, regardless of color. Picking a red person in place of a green person with a lower p is strictly a suboptimal choice.

      Likewise, when a cop faces each individual case, her decision should only be based on the value of p she calculates. If she investigates a lower p individual when she doesn’t investigate a higher p one, for /whatever/ reason, she is doing strictly worse at catching criminals. Thus the optimal decision is always to have a predetermined p cutoff for each encounter.

      Now, we ask ourselves, suppose the hit rate for reds is lower than the hit rate for greens. Then we could reason that if instead of investigating the least likely red, we could investigate one more green, and we would have a higher success rate overall. Now, to be fair, this is not exactly straightforward mathematically. The hit rate for reds/greens is the average of the true value of p, whereas the important question is true p value of reds/greens on the decision boundary.

      In fact, I began this exercise expecting to show that bias in hit rates necessarily means bias in the estimate of p, but as we can see, this bias could be due to the true distribution of p values in the red population vs the green population. That is, if we imagine that greens tend to be either obviously incriminating, or obviously not in comparison to reds, then the hit rate for greens would be higher even in the case of a perfectly optimal cop.

      Of course, I think you would need evidence of such a disparity of distributions between the populations. For example, if we consider the limit p_cutoff –>1, then the average p and the decision boundary of p are precisely equal. With the current tools at our disposal, I think it would be difficult to measure the distribution and determine how much of the bias was caused by it. On the other hand, once police cameras become standard, I could imagine a study where you paused the video before the encounter, and asked a group of cops for an estimate of the likelyhood of criminality. Cross reference this with the actual hit rate and the actual decisions made by the cop with the camera, and you can find if either group is acting biased…

      (As a side note, why not feed in those p values to a machine learning algorithm… but then such a system could be gamed by exploiting quirks of the classification, so it will need to feature online learning, and since the cop has to carry such a computational system with him anyways, why not attach a robotic exoskeleton… http://d1oi7t5trwfj5d.cloudfront.net/7f/cb/00907f054780ac4776864d164bec/robocop.jpg)

    • meh says:

      So in this game, the light turning on is a crime, and the color is a race. But police don’t always have to find the person that did a particular crime, often they are given a person, and have to determine if they are committing a crime.

    • koreindian says:

      I think this thought experiment is disanalogous to the problem of profiling. In the traffic light betting thought experiment, if the light is going to be red 51% of the time and green 49% of the time, then optimal policy is going to be to bet red all the time. In this example, there is no penalty for for betting green all the time. But, in the case of racial profiling, if 51% of crimes are committed by black and 49% of crimes are committed by whites, and the police decide to profile blacks 100% of the time, there will be major consequences for never guessing white.

      First, white criminals will be able to take advantage of the fact that 0% of the time police are focused on them, making white criminals far more successful. You are ignoring the game theoretic implications of your policy.

      Second, your probabilistic reasoning would require (not enjoin, require) that police never patrol 100% white neighborhoods, even if said neighborhoods had average crime rates.

      Third, how do we deal with predictors that when measured at different scales suggest different policies? Say that in a given jurisdiction, 51% of crimes are committed by blacks and 49% of crimes are committed by whites. But also, in a given neighborhood within said jurisdiction, 99% of crimes are committed by whites and 1% of crimes are committed by blacks. If the police had to make a jurisdiction-wide policy on profiling, they would profile blacks 100% of the time. If the police had to make a neighborhood-wide policy on profiling, they would profile whites 100% of the time. But the neighborhood is contained within the jurisdiction, so our calculus would have us profile blacks 100% of the time and profile whites 100% of the time. This is a contradiction. In order to resolve the contradiction either we have to decide which profiling policy trumps (either we favor the jurisdiction profiling policy, or we favor the neighborhood profiling policy), or we find a different way of reasoning about profiling. I’m all for the latter.

      You say, “[o]f course this wouldn’t fly as a practical policy”. But it doesn’t even fly as a theoretic policy. Note that I haven’t even mentioned the consequences of the racial hostility that would emerge from the police racial profiling 100% of the time. I confined myself to talking about your probabilistic reasoning on the theoretic level.

  14. ishaan says:

    As long as we’re talking about race and discrimination, I’ll take the opportunity to pick the slatestarcodex readership’s brains on a tangential topic:

    There’s a bunch of resume studies where they send out fake resumes and vary things like name of the applicant, by race or gender, or more subtle stuff like address, as well as qualifications to see the impact of each factor.

    Whenever people talk about discrimination that’s always been my go-to “look, this is a completely fair indicator of how much discrimination there is with no confounding factors”.

    I’d really like to know if I’m being wrong or misleading when I do that, because I do it a lot. The amounts of discrimination they’re finding are fairly dramatic and those findings are a central core of my belief that discrimination has rather large economic consequences. It’s also one of the only reasons (in my eyes) that somewhat justifies using affirmative action as a counter, rather than something less heavy-handed.

    So – anyone feel like tearing it down or confirming it?

    • llamathatducks says:

      I’ve been citing this sort of thing too. That said, I think a better solution than (race- or gender-based*) affirmative action, at least for employment, is gender- and race-blind resume review. Of course that breaks down at the interview phase, but hopefully fixing the first step should help. See this analysis of the anti-sexist effects of blind auditions for orchestras.

      *At the college admissions level, I am in favor of affirmative action based on economic status. One’s economic opportunities enormously affect one’s educational opportunities, so the prototypical affirmative action case of “this student is actually brilliant but hasn’t had a chance to prove it very well” can occur a lot among economically disadvantaged kids. A disproportionate number of whom are black.

      (Perhaps it is relevant, though, that black students are also disproportionately likely to be suspended, including at young ages; so perhaps one’s educational opportunities can be shaped by race separately from one’s economic status. So I could be convinced that this actually should be taken into account at college admissions, maybe…?)

      • Those findings are driven mainly by statistical discrimination. The literature is very consistent that racial discrimination in the labour market is primarily statistical, i.e. riding on characteristics about a person which genuinely affect their ability to do the job, not ‘taste-based’ (i.e. racist).

        See the research I’ve gathered together here http://www.adamsmith.org/blog/liberty-justice/markets-dont-like-racism/ (there is a much larger literature besides)

      • ishaan says:

        I like the name-blind application idea, I should have immediately thought of it! Like you said though, the interview… after all the whole reason to resort to resume studies is to get an un-confounded measurement. Presumably the same issues occur constantly in more physical situations.

        (My most compelling reason for NOT supporting affirmative action – other than the heavy-handedness of it as a method – is that I’m just concerned that disadvantaged students will fail out of college at higher rates as well, which might actually end up harming them economically.)

        • llamathatducks says:

          I think that a way to mitigate that potential consequence of affirmative action is to have extra support for disadvantaged students at the university. Or just easily accessible academic/social/etc. support for students in general, where people are trained to deal with the difficulties disadvantaged students face. (I posted a link elsewhere in this thread to a news story about some ways this has successfully been tried.)

          Actually, this would be helpful / should exist regardless of affirmative action, given that based on the statistics in that article poor students with the highest possible SAT scores are less likely to graduate than rich students with the lowest possible SAT scores. So even those poor students who presumably can get in regardless of affirmative action continue to be disadvantaged and would probably benefit from extra support. (Or perhaps just financial aid that lets them not worry about money while they study.)

          • Samuel Skinner says:

            A better plan would be to improve the schools that are feeding into colleges. Study skills and other knowledge you need to fully utilize college isn’t something that can be taught in a year or two which is why blacks under affirmative action had much higher drop out rates than they do know without it.

          • llamathatducks says:

            Samuel Skinner: that’s certainly necessary as well. But some programs at the college level have had some success, which means they’re still a good idea. If I were in charge of all the things, I’d certainly prioritize improving primary/secondary schools. But if I were in charge of a university, I’d prioritize both admitting poor students and giving them support once they arrive.

          • Samuel Skinner says:

            They have to have more than provide results- they need to have better results than being without affirmative action and having more people total pass.

        • Auroch says:

          MIT has found that the best predictor of success in their incoming class was SAT combined score divided by total years of education of the student’s parents. (I think this normalized BA/BS to 4 years, and had fixed numbers to approximate MA/MS, PhD, etc. program, but I wouldn’t swear to that.) There’s obviously some minimum-standard selection effect in ‘who applies to MIT’, but it’s very suggestive that those concerns are ill-founded.

    • blacktrance says:

      While certain names are associated with certain races, there’s also the effect of certain names being associated with certain cultures and/or upbringing, so a minority with an ethnic name is the kind of person who’d have been brought up by the kinds of people who’d give him an ethnic name, which is a cofounder. If you want to isolate the effects of race from the effects of names (I don’t know if studies have already done this), compare James Johnson (black) to DeShawn Johnson (black), and James Johnson (white) to Brayden Johnson (white).

      • nydwracu says:

        The methodology probably can’t separate the effects of race from the effects of class — are there upper-class black names?

        (In a sense, I think, but they’re all also WASP names. Like Charlie Parker.)

        • AJD says:

          I have no direct data on this, but:

          There are traditional common English first names that just happen, at the present time, to be substantially more common among black people than white people—Maurice, Jerome, and Bernard are frequently-cited examples. I’d conjecture these might be more associated with black people of higher social class than more novel-seeming names like Darnell and Tremayne, since in general more educated parents tend to be more conservative in naming. The question is whether Maurice, Jerome, and Bernard would even be recognizedby employers as predominantly black names, though.

          …Or, I could be totally wrong, since Leroy and Tyrone are also traditional English names, and Bertrand & Mullainathan find Leroy and Tyrone to be on the lower end for parental education among the black names they look at.

        • Alan says:

          Not sure what counts as “upper class” but some names used by prominent or educated blacks do not seem to have the same prejudice attached to them.

          Perhaps it has to do with frequency. For example, the names of columnist Ta-Nehisi Coates and actress Quvenzhané Wallis are sufficiently uncommon (if not unique) that they do not immediately invoke the same thoughts as stereotypical “ghetto” names such as Shaniqua or Tyrone.

          • AJD says:

            Is this a fact about the names Ta-Nehisi and Quvenzhané, or is it just due to the fact that you already know who Ta-Nehisi and Quvenzhané are and have already formed mental images of them specifically as talented and accomplished professionals?

            If you heard the name “Qulyndreia Jackson”, say, what kind of prejudice would you attach to the name?

    • lmm says:

      I don’t entirely believe this, but:

      Suppose job candidates wear red or blue hats. And suppose that over months or years an interviewer sees a lot of candidates, and it turns out that for whatever reason – ability, education, something else – people who wear blue hats do significantly better than people who wear red hats. (Maybe this is even an effect that persists after the resume is controlled for – maybe red hats get inflated college grades – but I think the argument goes through either way).

      A Rationalist would say that the colour of someone’s hat is (Bayesian) evidence for their competence or lack thereof. A human would start to judge people by their hats. And it would work; people who took hats into account on hiring would, (even if they did make the wrong decision when faced with a case “outside their training set”, of equal candidates wearing different hats), on average perform better.

      Of course this is still in some sense “unfair”. But it’s harder to condemn people for acting in a way that genuinely will lead them to hire better candidates.

      • Jiro says:

        Because of probability matching, an employer with a list of candidates would want to go through all the blue hats first, and only then move onto the red hats, in order to maximize the probability of competence.

        If the number of candidates (for both hats) is a lot larger than the number of jobs, the result will be that *all* the jobs go to blue hats and *none* to red hats. The slight difference in competence suddenly turned into permanent unemployability for red hats.

        If the hats are glued to their heads and they can’t change color, you’ve created a permanent underclass.

        This is why discrimination can be bad even if it’s based on good Bayseian evidence. This also applies to discrimination based on other unchangeable characteristics.

        (If other characteristics are used as well as the hat color, this becomes less of a problem.)

        • Tongue planted firmly in mouth says:

          If the employer is trying to find the N best candidates, then no, it doesn’t makes sense to throw away half of the applications.

          On the other hand, if the employer is simply trying to find N /satisfactory/ candidates, then it still doesn’t make sense to go through the applicants in that order. The employer has to look at every application to sort them out by hat color, when ze could just as easily sort by any other simple fact instead, such as highest degree, GPA, age, etc. The only reason it would make sense to throw out all red applications would be if hat color was a better prediction than every single other simple bit of data on the resume. Which I think we can agree is not true?

          The optimal employer in such a model would almost certainly sort candidates by some much stronger indicator of IQ, such as degree or GPA, or combination thereof, and would then be perfectly race-blind when controlling for a couple simple factors.

        • RCF says:

          I think that you should think about whether this logic would apply to other attributes, such as college GPA, and if not, why the reason it fails for GPA does cause it to fail in the case of hats.

          • Jiro says:

            Employers would think that whether one GPA is better than another depends on college quality. And no two employers will agree on exactly where all the colleges fall in the quality ranking. Thus, employees as a whole will not all pick people in the same order based on college GPA. Employers will all agree on whether someone has a red or blue hat, or whether someone is black or white. (Note that being mixed-race is not enough to invalidate this argument unless the people *at the lower extremes* are mixed-race enough to matter.)

    • Tarrou says:

      There are huge confounding factors, the most obvious of which is class. Look at the actual “black” and “non-black” names they use. It’s generally stereotypically low-class black names with stereotypically middle-to-upper class white names. You get a Thurston and a JaMarcus applying to a bank, there will be a difference. I haven’t seen research to suggest it’s greater than it would be between a Thurston and a Cletus. Class is the thing that gets lost in so much of the “racial” research. The other is culture. Even when controlled for SES, this puts poor whites from, say, Appalachia or Iowa on the same footing with inner-city minorities when situations actually wildly differ. Bottom line, this sort of research often says more about the researcher than the subject. There’s just too many variables, so the research is pretty well determined by which ones it takes into account.

      • AJD says:

        The classic study on this—Bertrand & Mullainathan (2004), which I don’t have time to look up because my flight is boarding in five minutes, so I hope someone can find the reference—considered that class could be a possible confound that might account for the difference between white names and black names. However, they found *no* correlation of class and interview rate, or a backward correlation, *within* black names or within white names. There are other possible explanations for that, of course—within a race, the class differences are too small to have a noticeable effect—but they did look at it.

        • Montfort says:

          This seems to be the paper you’re referring to.

          The argument against class discrimination in this paper seems reasonable; to the extent that mother’s education represents class, there appears to have been little effect.

          They also very briefly discuss a potential familiarity confounder (e.g. a manager might have no problem hiring a “John” or “Henrietta” regardless of race, but discriminates against names they don’t recognize, like “Latonya” or “Veit”). They argue that as the (black) names they chose are popular among blacks, the hiring manager can be assumed to be familiar with those names. This seems unsatisfying because if you presume hiring managers might be disproportionately common-name types (or just mostly white), they might remain unfamiliar with the selected “black” names. But to back it up, they performed a similar analysis as they did with socioeconomic status and found “no positive correlation” (though they don’t show their result and used raw name frequency for familiarity, presumably without accounting for social clustering?).

    • Emily says:

      The assumption behind the resume studies seems to be that, conditioning on the stuff on the resume, there aren’t systematic differences between candidates from different groups in terms of how they will perform in the job. I thing that’s a big, unwarranted assumption. We have well-documented examples of how using some types of indicators leads to overprediction of one group’s behavior/underprediction of their counterpart’s.

      Example 1 is SAT scores. Using SAT scores to predict college outcomes tends to underpredict the performance of higher-scoring groups (white students, higher-income students) and overpredict the performance of lower-scoring groups. If you are truly just trying to optimize for college performance, you would adjust how you use SAT scores for different groups in way that is discriminatory against lower-scoring groups.

      Example 2 is attrition from PhD programs. Women leave science PhD programs at higher rates. Unless you think there’s pro-female discrimination in the admissions process, this means that whatever set of indicators is being used to admit students, that set of indicators overpredicts female performance/underpredicts male performance. Again, if you were optimizing for outcomes, you should strongly prefer the application package from the male candidate over the one from the female candidate. They look equal, but they aren’t equal in the sense of what they predict about eventual performance.

      So, in both of these examples, doing “blind” admissions would actually be a worse way to optimize your outcomes than doing discriminatory admissions. Is that possible for hiring for jobs in general? Could be.

      Of course, it’s totally illegal to do hiring this way. And I think it probably should be totally illegal to do hiring this way.

      • Princess Stargirl says:

        Not to state the obvious but there is well known and rather strong affirmative action for female applicants to science/math/engineering phds (not as much in biology). So the second example is not mysterious.

        • llamathatducks says:

          Another way of looking at it is that high-performing people from underrepresented/disadvantaged groups have trouble in college and such despite being high-performing. e.g. through stereotype threat, self-doubt, a biased culture, etc. If this is the case, the more just way to deal with this issue is to try to combat the factors that make certain environments inhospitable even to well-prepared members of disadvantaged groups.

          See e.g. this article. There’s a graph showing the likelihood of graduation for rich students and poor students given SAT scores. Rich students with quite low SAT scores are more likely to graduate than poor students with the highest possible SAT scores. And there are ways to reduce that gap by working with the high-performing but poor students and building their confidence.

          • Rebecca Friedman says:

            Another possibility is that the rich kids aren’t trying as hard – that from their perspective, getting good SATs isn’t going to make or break them, or that they’ve always been good at things and aren’t used to having to try hard. When they get to college, find things are harder than they expected and get a dose of reality, they have the option of focusing in more, devoting fewer resources to computer games and more to homework, and rising to meet the challenge. Whereas a marginal black student may be more likely to be someone who’s been working hard her whole life, put everything else on hold to study for the SATs, and as such has scores reflecting her peak performance… and when it turns out that succeeding at the top school she’s gotten into requires more than that, just doesn’t have any more to give.

            I have no idea how accurate a portrayal the second of these examples is, but I know the first applies to at least one person (me – I had a lot of growing up to do at that age). If this pattern does hold true you would expect to see things like SAT scores on average underrepresenting the academic ability of people who have less incentive to make them as high as possible. Again, I don’t know if you actually do, but it was the immediate assumption that sprung to mind when I saw the original claim.

          • RCF says:

            “Another way of looking at it is that high-performing people from underrepresented/disadvantaged groups have trouble in college and such despite being high-performing.”

            That’s simply a mysterious answer to a mysterious question. The issue is not why black people do poorly in college, the issue is why black people do worse in college than they do on the SAT. Invoking “stereotype threat” does nothing to explain that, unless you provide an explanation for why stereotype threat is affecting their college performance, but not their SAT results.

            Also, stereotype threat does not explain the discrepancies between high-performing groups and low-performing ones.

          • Silva says:

            Sometimes the problem is less esoteric than anything involving fighting stereotype threat. I now rate my chance of ever finishing a computer science course at less than 50%, and this happened because I simply had 2 choices: living with my parents with, among other problems, 0 personal space (I started sleeping in the living room when I stopped sharing a bedroom with my mother), or working and studying with an inflexible course schedule and a large number of working hours. Had I had the opportunity of living in a tiny bedroom+bathroom away from my parents and getting fed just enough, I’d have had no problems, and even modestly moneyed students have much better options than that: either their parents represent no problem to them, or they have vast personal space, or they end up living away from them in pretty good conditions. And not only the probability of lacking personal space is greater for poor people, so is that of having asshole parents (no personal space with sufficiently non-assholes might work better).

      • ishaan says:

        >SAT scores. Using SAT scores to predict college outcomes tends to underpredict the performance of higher-scoring groups (white students, higher-income students) and overpredict the performance of lower-scoring groups.

        Ah yes, this is the sort of information I was looking for, thanks! The resume is a scenario where a lot of the stuff that one might use race/class/gender as a proxy for is laid out in right front of you, but this indicates that it might still be rational to discriminate even with all that info (although, I still doubt that names provide the equivalent of 8 years of experience or a criminal record’s worth of information, so I’m still tending towards “the monkey mind understands group dynamics better than it does work experience and so weighs the former disproportionately more”)


        This next section doesn’t pertain to you specifically and you seem to be on the same page as me on this, but many of the other replies are all saying the same thing and I want to respond to them in one place:

        I do recognize the distinction between rational discrimination and irrational discrimination – but I’d actually consider both of those to fall under the category of “racism” and “stuff that creates economic impact” and am puzzled that so many people seem to think that just because a behavior is instrumentally rational makes it not racism.

        Most things humans do should be assumed to be at least kinda instrumentally rational in some scenarios. We wouldn’t even have evolved a gender-appearance-and-tribal-markers discrimination module in our brain if it didn’t usually work for most purposes. But we’re still talking about gut-level stuff biasing judgements (which is practically the definition of implicit racism), and those biases are obviously still going to influencing markets and resource distribution even if everyone is acting in their own interests.

        (The distinction between implicit racism leading to rational vs. irrational behavior is still interesting, since the latter probably distorts the market even more and the distinction may help for identifying how the causal arrows flow – but that’s not an *ethical* distinction. From a virtue ethicist’s standpoint, the gut feeling driving the effect is racism regardless of the outcome’s rationality, and from a consequentialist perspective it still contributes to marginalization).

        • haishan says:

          I do recognize the distinction between rational discrimination and irrational discrimination – but I’d actually consider both of those to fall under the category of “racism” and “stuff that creates economic impact” and am puzzled that so many people seem to think that just because a behavior is instrumentally rational makes it not racism.

          The other way to go here is to argue that it is racism, but that in fact some racism is justified.

          (join the dark side, we have long inscrutable blog posts and citations to Google Books)

      • Nicholas says:

        I think this is confounded by a report earlier this year (referenced here http://www.usnews.com/news/blogs/at-the-edge/2014/09/03/sexism-and-science-go-hand-in-hand ) that suggests a timeline where promising female PhD’s are admitted into programs, and then drummed out of those programs by biased peers, professors, and administrators.

      • haishan says:

        I can think of at least two kinds of reasons why job performance would not be conditionally independent of race, conditional on all the other information in resume studies. The first class of reasons should be immediately obvious; if you need more information, feel free to talk to your friendly neighborhood Reactionary.

        The second class of reasons is I think more palatable to the Left, and to the sort of people who do these studies, and so I wonder if there might have been studies trying to control for it. The idea is that a Black job applicant might be equally able to do the work as a white applicant, but if the employees are majority-white — as is likely in most jobs — a marginal Black employee would face all sorts of disadvantages, ranging from stereotype threat to social alienation to increased likelihood of racial harassment. So it would be rational to hire the white guy, even if the resumes are unbiased measures of job-related skills and there are no racial population differences. One way to test this might be to send resumes to majority-black employers and see if there’s still a bias in favor of the white names.

        • AJD says:

          Bertrand & Mullainathan (2004) (again) find that employers located in areas with higher black populations had a very slightly better callback rate for résumés with black-sounding names.

    • Alexander Stanislaw says:

      Simple example of why statistical discrimination is still justified given identical resumes:

      Suppose we have two populations that vary a lot in mean height. We have some way of measuring height but it isn’t perfect and there is a random error. So our measured height is H (actual height) + E (error term).

      If we measure the height of two people from either population and find that they are the same, we should predict that the person from the taller population will still be taller on average because the expected error is different.

      I expand on this here in a less abstract fashion.

      I will freely admit though, that I haven’t done the math, and figured out whether this effect is sufficient to reconcile the resume studies with studies such as Steve Levitt’s ( he showed that a blacker sounding name does not affect your life prospects).

      • ishaan says:

        Levitt is interesting, considering that a lot of the other commentators seem to think the issue is “ghetto/unusual black names” as opposed to “traditional black names”.

        Levitt interprets his findings to say that the distinction only matters insofar as “ghetto black names” are more obviously black, and that black people with traditional names get their dosage of discrimination in the face-to-face interview, thereby equalizing the outcomes.

        Which is highly relevant given the other comments, but I think I’m missing out on some of your inferences because I’m not quite sure how that connects to the
        regression-to-the-mean issue that you outlined?

        • Alexander Stanislaw says:

          Yeah, that comment was low quality and confused. But is the core idea at least clear? Namely that discrimination based on demographic information is still epistemically justified even given identical resumes. (I’m not advocating specific policy right now, nor am I claiming how much discrimination is justified).

          • ishaan says:

            Oh, yes that part was clear. I thought there was a connection between those two ideas which I wasn’t getting.

      • I did some math on this out (though with abstract data, not real data) here: https://plus.google.com/+AndrewHunter/posts/3Nn22d4S1ev TLDR you’re right, your posterior after identical resumes should be different, and the more towards a tail you want to select the less you care about resumes.

        SJWs were not pleased by this post.

      • Alexander Stanislaw says:

        Wow, height was an horrible example (because we _can_ measure height but we can’t ask someone what their job competency is). Basically my earlier comment sucks – the one I linked to was better.

    • jeorgun says:

      I don’t know all that much about the experiment, but the one cofounder I’ve heard is that the test doesn’t discriminate against black names so much as it does against unusual ones— someone named “Geoff” or “Cemil” would show the same effects, even though neither of those are names with any obvious racial associations.

      To which the obvious response is that “standard” names are all white, so it’s a distinction without a difference. And it has no bearing on gender or address. But I still think it’s an interesting point if true.

      • AJD says:

        “Geoff”?

        There’s actually two different ways I can go with this comment.

        1) Geoff is an unusual name? It’s somewhat more unusual than “Jeff”, but it’s solidly within the inventory of customary English names. If there’s a bias against unfamiliar names, I’d be very surprised if “Geoff” is subject to it.

        2) Among Americans born in 1992 (a cohort that might be looking for entry-level jobs) there are about the same number of people named “Geoffrey” as “Terrell” and “Ernesto”. I would be very surprised to see the same amount of name discrimination against Geoff as against Terrell and Ernesto. (The reason for which could be (1) above, of course.)

        …Actually, now that I look it up, Geoffrey specifically was one of the white-sounding names used in Bertrand & Mullainathan’s (2004) résumé study. It was judged as white-sounding by 73% of the sanity-check survey (“are the names we selected as white-sounding actually white-sounding?”)—more than Neil but less than Matthew. It did, however, get a lower callback rate than the majority of white names, lower than the black names Leroy and Jermaine, and on par with the white names Brett and Neil and the black name Jamal.

        Also, I would guess “Cemil” is Turkish. Even if I’m not right, that’s not no racial association.

        • Mary says:

          Ah, but the question is whether it’s a common name in America.

          I assure you “Geoff” is odd here.

          • Matthew says:

            My freshman roommate was Geoff. It’s not that uncommon in the parts of the US that might have had immigration from Quebec in the past. (He was from Vermont.)

          • AJD says:

            What? “Geoffrey” was among the top 200 most common names for boys born in the US in the 1970s and 1980s, and top 500 in the 1990s; more than 1000 American boys were named Geoffrey each year from 1969 to 1990 and more than 100 each year from 1940 to the present. So it’s slightly out of fashion now relative to its peak, but far from rare. It certainly doesn’t make sense to classify it with “Cemil”, which doesn’t appear on the lists of American baby names for any year at all. (…Which means it’s never been given to 5 or more boys or 5 or more girls with known place of birth in the US who applied for a Social Security card.)

            When I said “customary English names”, I meant English as in the language, not as in the country.

      • RCF says:

        “To which the obvious response is that “standard” names are all white, so it’s a distinction without a difference.”

        No, it’s a very important difference. There’s a big difference between discriminating against black people, and discriminating against something is more common among black people.

    • gwern says:

      So – anyone feel like tearing it down or confirming it?

      I feel like you missed the point of most of this post: that race is an observable proxy for a lot of things which are hard to observe, and is providing information which may influence choices. The resume studies, reliable fodder for outrage that they are, do not distinguish between statistical discrimination and actual bias, in the same way that a cop pulling over black people because he finds guns or drugs on them more often is engaged in statistical discrimination but not racial bias.

      So when you see a female name on a resume rather than a male one, your decisions may be different. For example, if you run a lab and want slaves^Wgrad students to work for you, you may well statistically discriminate against a resume with a female name because smart women are more likely to want things which don’t contribute to you getting grants & publishing papers with your name on it, such as ‘work-life balance’ or ‘a decent income’ or ‘kids’ or ‘time with family and friends’ (see for example the recent “Life Paths and Accomplishments of Mathematically Precocious Males and Females Four Decades Later “, Lubinski et al 2014 http://pss.sagepub.com/content/early/2014/11/10/0956797614551371.full.pdf+html – note that while each survey item may seem to show a small gender difference, overall there’s a big consistent difference). Or, they might get pregnant. Or, even if you think they will work hard for you and be celibate, they may be more likely to do the sensible thing and drop out of academia for better jobs and now you’ve lost your investment in a potential ally/acolyte. And so on.

      (What really needs to be done is to show that the subjects in these experiments are passing up on a free lunch of women or blacks, and are reliably hurting themselves with bias, rather than rationally coping with incomplete information as best as they can. But gosh, that sounds like work!)

      • Chuck says:

        “So when you see a female name on a resume rather than a male one, your decisions may be different. ”

        Sociologists take these audit studies as providing smoking gun proof of market racial bias. For a discussion of findings, see, for example: Pager, D. (2007). “The use of field experiments for studies of employment discrimination: Contributions, critiques, and directions for the future”. I don’t think that they can easily be dismissed. That is, they show something and show it across numerous methodological designs. That said, I don’t think that they demonstrate racial bias in the sense of discriminating on account of race independent of outcomes. One problem, as you noted, is that at best experimenters can only try to match confederates or resumes on some relevant traits (i.e., year of education and height); it’s unlikely that a full matching can be done; and in practice important traits like FSIQ are neglected. Even if one were to match e.g., confederates on all relevant traits and if differences were still found it would not be clear if the employees would recognize that the individuals were so matched. They know that e.g., Blacks or immigrants are generally worse employees; they don’t necessarily know why. Perhaps because of education, perhaps because of factor X. I suppose it depends on the question of interest. One problem is that sociologists/economists tend to confuse these e.g., Do employees discriminate on the basis of race as a proxy for trait Y matching for education versus Do employees discriminate on the basis of race per se?

        Also, I notice that few of these researchers present proper effects sizes. Instead, they present discrimination ratios (which then have to be converted into odds ratios and then d-values or percentile points) which tend to make the effect appear larger than in reality.

    • Ryan says:

      I’d be curious to see a study that tries to account for “first name bias” in general. In the US there is a long history of immigrant groups keeping their ethnic surnames but giving their children traditional American first names. They did it because they thought it secured an advantage (or negated a disadvantage) for their children. When such large numbers of people all have the same idea in their head and all act on it (especially with something as personal as their child’s name), I pretty much expect them to be right about what’s going on.

      So, maybe send out identical resumes where one person is Jason Nang and the other is Kyung-Hu Maeng, etc. and see if you get the same disparity in call backs.

    • Cliff says:

      I think there are a couple of problems with such studies. One is that the factor meant to indicate race (classically, name) goes far beyond signaling the race of the applicant. Rather, a “stereotypical” black name may indicate all kinds of other information about the applicant (family cultural traits, SES, parental genetics). There are plenty of black people with “normal” names that would be unremarkable for blacks or whites and such names obviously are not used in the studies.

      Second, discrimination may be rational. Perhaps blacks are more likely to put fake information on their resumes and so the reviewers assign some lower reliability to the claimed skills/education/experience on the resume. I have heard that studies show similarly experienced/credentialed blacks have worse job performance than whites (perhaps an IQ or, if you prefer, cultural effect).

    • disciplinaryarbitrage says:

      I recall reading some discussion on this elsewhere that suggested, basically, that the method of signaling race via the fictitious applicant’s name (e.g. “Craig” vs. “LeShawn”) was unintentionally confounding the study based on class markers. Under this model, the HR managers might be perfectly color-blind, but biased as hell against applicants from lower-class backgrounds, on the (questionable) assumption that smart, conscientious, culturally assimilated parents of all races name their kids “normal” (read: white) sounding names and lower-class parents choose “weird” ones (read: with strong ethnic or regional associations, non-standard spellings, and so on).

      (Perhaps a related assumption is that children who grow up with a non-standard name will, if they’re smart, conscientious, etc., adopt a more standard alias professionally; DeShawn goes by Sean, for example.)

      The proposed methodological fix is to shift from using white names with an upper-class white association to those with a lower-class association–the proffered example was Appalachian-sounding names (think any character on Justified.)

      My evaluation of the critique: I strongly doubt that controlling for class markers would eliminate the effect size (though it might still be worth doing to see if it explains some of the difference), and to the extent that it does I’m not sure how much to herald it as an improvement in the reducing-invidious-discrimination department. Saying “We’re not racist, we just don’t like people who look like they grew up with poor!” can look a lot like a distinction without a difference.

      All of that speculation notwithstanding, I’m in agreement with you that this line of research is cleverly designed and is very good evidence of real, ongoing discrimination.

    • Troy says:

      On resume studies:

      – Employers most likely do engage in statistical discrimination; other studies bear this out. Employers use race as a proxy for such characteristics as IQ, work ethic, and criminality.

      – They do this largely because more direct tests for these are either illegal or of questionable legality. One study found that employers allowed to look at employees’ criminal records were more likely to hire blacks, because they could now hire blacks who they could see did not have a criminal record.

      – Another contributing cause to employers hiring fewer underrepresented minorities is fear of lawsuits. Employers have a harder time firing an incompetent black employee than an incompetent white employee because they are more likely to be sued for discrimination in the former case. So they figure, in effect, better to not take a chance in the first place.

      – The solution is not affirmative action or legally mandated race-blind hiring, both of which add more government coercion; these would stifle employers’ abilities to hire rationally and likely have various bad unintended side effects. The solution is to get rid of the coercive existing state practices that are leading employers to (rationally) hire fewer blacks and other disadvantaged minorities. (While you’re at it, get rid of minimum wage laws too, because employers will then hire more lower-skilled black workers who under the current system are chronically unemployed.)

      • Toggle says:

        “Another contributing cause to employers hiring fewer underrepresented minorities is fear of lawsuits. Employers have a harder time firing an incompetent black employee than an incompetent white employee because they are more likely to be sued for discrimination in the former case. So they figure, in effect, better to not take a chance in the first place.”

        Can this be supported?

        • Troy says:

          Are you asking whether I have evidence for my claim, or whether the practice is ethical? The practice is rational from a self-interested perspective. I’m not taking a position on its morality, just pointing out that the way to eliminate the practice is to take away the disincentives businesses have: in this case, by making it harder to sue businesses for discrimination.

          If you’re asking what evidence I have, my understanding is that it’s a fairly common view among economists that the ability of disadvantaged minorities to sue their employers for discrimination discourages employers from hiring them in the first place. Here, for example, is a quote from Bryan Caplan’s The Myth of the Rational Voter: “Economists know about the negative efficiency consequences of affirmative action. Giving special categories of employees the right to sue their employers makes them less likely to be hired in the first place” (p. 61).

          I have not seen studies testing for this in particular (unlike the earlier mechanisms I mentioned, which I have seen studies for), and if anyone knows of any (for or against), I’m open to correction on this point. But given that the proposed mechanism is plausible (that is, hiring fewer members of protected groups is in businesses’ economic self-interest), I’m inclined to trust economists on this one.

    • ” that discrimination has rather large economic consequences.”

      I haven’t read the studies. Do they distinguish between “otherwise identical applicants are more likely to be hired if white/male/whatever” and “being w/m/w has as enough weight in the decision to cancel or outweigh obviously important characteristics, such as work experience in the field”? You might get a large effect between otherwise identical applicants even if the weight of the bias was quite small.

      • AJD says:

        In the Bertrand & Mullainathan (2004) study, they found that the effect of having a black-sounding name on a résumé was approximately eight times the magnitude of the effect of having one less year of work experience.

        Also, they found that better credentials overall (education, work experience, special skills, etc.) improved the callback rate for résumés with white-sounding names but not for black-sounding names (or at least, not to a statistically significant extent).

  15. Harald K says:

    “One common method of disentangling these possibilities is search “success rate”. That is, if searching whites usually turns up more real crimes than searching blacks, then innocent blacks are being searched disproportionately often and the police are not just correctly responding to indicators of suspiciousness or past experiences.”

    No, this is not sufficient. The key point is that you’re not allowed to discriminate on just any easily identifiable characteristic. Some information may be useful, but off limits. If you have a long nose, and people with long noses statistically do worse on the sports team, you still have the right to be judged as an individual based on your personal sports performance, not the average performance of people who look like you.

    This is really basic. It’s part of why the NSA spying is a big deal, it’s why you would have a right to be angry if 23andme sold your genetic data to insurance companies. Some information is off limits to discriminate by.

    • Where do we draw the line? (This isn’t a trick question, and I’m fine with the line being fuzzy; right now I’m not even seeing where we could draw a fuzzy line in any principled way.)

      You say it’s off-limits to discriminate based on genes. What about based on brain scans? (Including using old brain scans to judge a likely causal product, your present-day character — much as we’d use genes to judge the same likely causal product.)

      If brain scans are off-limits, then it seems like mental states of all kinds are off-limits, since there’s no obvious reason it’s fair to use ‘he seemed reckless in that interview’ as evidence, but not ‘his brain looked reckless in that image’. What makes long noses any different? (Especially in a world where long noses are causally linked, by some complicated route, to the dispositional state we’re interested in.)

    • lmm says:

      I think this gets to the core of many people’s position, so thank you for stating it this clearly. That said:

      Why? Presumably you can discriminate on *some* grounds when hiring – if only skills and experience. If so, why not any other factor that’s correlated with job performance?

      Because people don’t get a choice about it? Maybe, but a lot of e.g. intelligence is heritable too – should it be illegal to try to hire more intelligent people?

      • Jiro says:

        Intelligence isn’t just *correlated* with job performance; it *directly affects* job performance. You use your intelligence in performing job tasks. You don’t use your skin color in performing job tasks (except in unusual cases like an actor playing a part that requires a specific skin color).

        • lmm says:

          Most people’s job isn’t taking IQ tests, so you can’t really say that IQ score is something people use directly to perform their job.

          • Jiro says:

            The question was about hiring people based on intelligence, not based on IQ score. Whether it is okay to hire based on IQ score would depend on how accurate IQ is as a measure of intelligence.

          • lmm says:

            Which presumably cashes out as the strength of the correlation between test score and job performance? Is there some magical value for the correlation coefficient where it becomes ok, or what?

    • suntzuanime says:

      The problem is it’s hard to distinguish between “discriminating based on race in a manner which is correct but contra to public policy” and “discriminating correctly based on neutral indicators of criminality, which correlate with race but are ok vis-a-vis public policy”. One way to deal with this problem, which gets used in employment law, is to say that neutral indicators which correlate with race are actually not ok vis-a-vis public policy, and anything which causes “disparate impact” is impermissible. The main problem with this, of course, is that there are an awful lot of things that correlate with race.

    • Jos says:

      Has anyone done the math on this? My intuition is that Scott is right.

      If people with long noses statistically do worse on sports teams, and if coaches have some ability to predict success that doesn’t rely on long nosedness, then I would expect longnosed people to be picked statistically less often than shortnosed people, even in the absense of discrimination.

      I think that if the coaches are using longnosedness as a proxy for swim team success, then you would expect the distributions of picked to actual success to skew from their expected distributions, but the math is hard and I haven’t taken the time to work it through.

      • Harald K says:

        Of course, with direct measurements of all sorts being noisy, you could probably get better results by including nose length in your model. Likewise, you could probably reduce crime – at least short term – by, say, decreeing that Roma aren’t allowed to enter convenience stores in groups.

        But we are not, and should not be willing to optimize that one result (less crime) at all costs. It is more important that those Roma who would never steal from a shop are treated with respect. Not just as a means, but as an end in itself.

        It’s tempting to resort to consequentialist arguments – for instance, saying that unless we start treating Roma with respect, the problems surrounding their culture and interactions will never improve. And such arguments may well be sound. The danger is that in focusing on outcomes we lose sight of principles, such as the principle that you are entitled to be treated as an individual.

        • lmm says:

          The problem is that our principles conflict with each other and we’re unwilling to tradeoff between them.

          Imagine a future where we know all the correlations. An interview candidate has a particular criminal record, IQ score, performance on an interview question, and race. All of these things have a particular correlation with job performance. What principle says it’s ok to judge on one but not another?

          That it’s a matter of choice or not? But intelligence is somewhat heritable, and I’d bet that job performance is as well. And on the other end does that make it ok to discriminate by e.g. martial status?

          Do you go the whole hog and say employers have to assign jobs at random? Do you allow them to judge by anything? AIUI the current US policy is that if you want to discriminate based on something that correlates with race then you have to fund a study that demonstrates it’s correlated with performance on this particular job. But that fails at both ends: everything correlates with race (and with everything else). And you could do one of those studies using race as your factor, and you’d find a correlation.

        • Jos says:

          I guess the question is whether the success rate per individual is informative as to whether a racist proxy is in use.

          Let’s say that long-nosed people are arrested twice as frequently per capita than short-nosed people, but that in each case, 50% of the people turn out to be guilty.

          Does the fact that the guilt rate turns out to be the same tell us anything?
          (Yes, the guilt test could itself be racist, but let’s assume we’re confident in it for purposes of this example, because I want to know about the math.)

    • 27chaos says:

      They’re not necessarily using the strategy you criticize. Perhaps police simply judge people by whether or not they seem dangerous, and the correctness of their judgement is borne out by the fact that their choices match the correct racial proportion.

      Also, I don’t understand why you think that it’s not allowed to use some sorts of information. I would understand if you claimed that police place too much emphasis on the information, but you seem to be making a categorical objection to its use.

    • Jos says:

      Sorry for spamming this, but I’ve been thinking about this question and it’s (1) very interesting and (2) hurts my head. If anyone has a good source I could read on this question, let me know.

      Let’s say that of the 1000 people who approach a bank on a given day, we know that 900 will have hair, and of them, 10 will rob the bank and 100 will be bald, and of them, 10 will rob the bank. (All bank robbers and guests approach individually). All bank robbers carry a “give me your money note,” so once we stop them, we know if they were a bank robber or not. The bank has to come up with a test to decide who to stop and question, and it does not want to discriminate against bald or hirsute people unfairly.

      It turns out that you can search for guns with a metal detector, and that 50% of bank robbers carry guns, and 1% of non-bank robbers. So the bank stops everyone with a gun and checks them for a note. They therefore stop 13.9 hirsute people per day of whom 5 are robbers, and 5.9 bald people, of whom 5 are robbers.

      If you’re bald, your chances of being stopped are 5.9%, versus 1.5% if you are hirsute, but if you are bald and innocent, your chances of being stopped are 1%, exactly the same as if you are hirsute and innocent. (Similarly, the chances of being stopped if you are bald and guilty are also exactly the same as if you are hirsute and guilty.)

      I guess there are two questions: (1) how do the results change if we change the assumptions, and (2) what do we consider to be discrimination? I mean, if the bank had a 100% accurate test (e.g., all bank robbers enter wearing a domino mask and holding a sack with a dollar sign, and no non bank robbers do,) that would produce a disparate result, but I’m not sure we would find it to be discriminatory in an objectionable way on the bank’s part.

    • Cliff says:

      You are allowed to discriminate on the basis of any easily identifiable characteristic unless there is a law against it. Of course the classic example is insurance, they discriminate against all kinds of risk factors that have nothing to do with your actual driving ability or etc.

    • “you still have the right to be judged as an individual based on your personal sports performance, not the average performance of people who look like you.”

      I’m not sure if I agree in the context of law enforcement, but I strongly disagree in the more general context. I don’t think I have a right to have other people behave reasonably towards me. It’s nice if they do, but there is a wide range of decisions which ought to be up to the individual to make. If someone decides not to be friends with me for a bad reason that’s unfortunate, it is in some sense unjust, but it does not violate any right I have.

    • Null Hypothesis says:

      Just a General comment, because I’ve thought about the same thing a lot.

      Take that same argument into a different situation – say Healthcare. You meantioned 23andme. In the current situation, we’re talking about race being off-limits because it would let other people treat you too generally – too much as part of a group.

      Giving genetic information, conversely, is very specific information. On the contrary, for an insurance company you try to blend in with other people who seem the same as you, but actually will have better health outcomes. If we could look into a crystal ball and determine everyone’s total health expenditures for their life, insurance itself would dispensar because that would take the luck and the chance out of it. Which I would tend to agree – would not be the best thing.

      So why in one situation, is it “unfair” to treat you as part of a statistical group, but in another situation its “unfair” to treat you as an individual instead of a statistical group. There may be just some “it depends on the situation” answer here. But two diametrically opposite assertions just based upon which one is advantageous to the individual, to me at least, is just begging for hypocrisy to be pointed out somewhere.

      And as a side thought, is it possible that, to some degree, the inability to know about relevant things over which we have no control is socially optimal? And not just at a digital total-knowledge/no-knowledge scale either. Would Gattica-level accuracy of disease and dysfunctional potential derivable from our genomes be sub-optimal for running a society?

      Curious on your thoughts.

      • llamathatducks says:

        Here’s how I see the difference.

        I’m one of those people who consider healthcare to be a right, something that everyone is entitled to. I consider it just plain wrong to deny someone affordable healthcare because of, well, any fact about them. So I consider it wrong to discriminate in this arena at all, whether by a general category like race or by individual genetic information.

        Hiring is very different because it necessarily involves discrimination, in the more general sense of the word. Generally there are fewer job openings than job applicants, and moreover, an employer has a right to choose only those applicants who will most likely do well at the job. So people do actually need to be excluded. Since I care about trying to promote equality of opportunity, I believe that this choice/exclusion should be made in as fair a way as possible, which means zeroing in on the specific attributes of the people involved.

        (Why is that fair? Well, because it’s more accurate.)

        • dhill says:

          I don’t see how “I’m one of those people who consider healthcare to be a right, something that everyone is entitled to.” answers the question here.

          What if you have (just like we do) more patients than you can serve, which is exactly the same as more applicants than openings for the job?

  16. llamathatducks says:

    I wish there were stats on rates at which police shoot unarmed people of various races. I can’t seem to find any. These are the cases that get the most media outrage (and which are indeed the most outrageous), and they seem to happen fairly often to black men and not often result in severe consequences to the shooter. But of course that’s a “seem to” based on media-filtered anecdotes.

    I recently came across this analysis of federal data on police shootings. The data are incomplete, but still. Your post made me wonder how this compares with your findings.

    The authors unfortunately present only a few of the numbers I want, so these are some incomplete conclusions.
    – From 1980 to 2012, 44% of victims of (reported) police shootings were white. According to the Wikipedia article on race and crime, white people commited 45.3% of murders from 1980-2008. This seems proportional, then. (Sadly it doesn’t say what proportion of all the people killed in these years were black.)
    – From 1980 to 2012, 41 kids age 14 or under were killed, of which 27 were black (65.8%). I have no idea what percent of crime committed by kids 14 or younger is committed by black kids. According to that same Wikipedia article, 58.5% of youth homicide is committed by black youths, but I don’t think that’s really the same age group.
    – The statistic this article focuses on is that in the three most recent years with available data, 2010-2012, among boys age 15-19, black boys are 21 times more likely than white boys to be shot. According to Wikipedia, black people in general have a homicide rate 8x the homicide rate of white people. This may differ from the rate among youths, but presumably not that much, since black people make up 52.5% of murderers and black youth make up 58.5% of youth murderers, which is pretty similar (unfortunately Wikipedia doesn’t give the proportion of youth criminals who are white people, only the proportion of youth gang members, which is different). This does seem pretty egregiously disproportionate. Of course, this is only one bit of data from an analysis of much more data, much of which went unreported.

    • FBI tables says:

      “unfortunately Wikipedia doesn’t give the proportion of youth criminals who are white people, only the proportion of youth gang members, which is different”

      Black homicide offenders skew a lot younger than white homicide offenders, giving a factor of 1.5-2 depending on the age group. White homicides have a larger share of ‘crimes of passion’ that skew older, and less in the way of homicide associated with other crime.

      http://www.fbi.gov/about-us/cjis/ucr/crime-in-the-u.s/2011/crime-in-the-u.s.-2011/tables/expanded-homicide-data-table-3

      In offender statistics Hispanics are counted as white, but in victim statistics they are separated out. Since Hispanic Americans have substantially higher crime rates per capita, this tends to distort comparisons between the homicide rate of non-Hispanic and Hispanic whites with the victimization rate of non-Hispanic whites alone. The asymmetric counting creates the appearance of disparities where none exist.

      Combining that knowledge with the FBI table, a 21:1 ratio between black and non-Hispanic white males among boys in some age category could be explained entirely by offending rates if one used the same definitions for offenders and victims.

      • Anonymous says:

        I think that the only reason black homicides skew younger is that blacks skew younger. Since the article takes teenage population into account, you should not further correct for population. According to page 15 blacks and whites have 2x the homicide rate at age 18-24 as at age 14-17*. These age buckets are coarse and I’m just eyeballing the graph, but it’s the right number to look at.

        *2x in the relevant, recent years; of course the crack epidemic skewed young, 1.5x for males, both black and white. 2x is also the ratio for females of both races, both then and now.

        • Anonymous says:

          According to the graph Ben Southwood posted, “FBI tables” is correct and disparity is entirely due to black homicides skewing young. While they don’t skew young in the sense of 14-17 vs 18-25, they do skew young in the sense of 14-25 vs 25-35.

    • Anonymous says:

      That article splits out hispanics killed by police as not white, while the homicide rate you used includes homicides by hispanics as homicides by whites. Consistently including hispanics as white reduces the ratio from 2.6x to 1.8x. These two definitions of white are extremely consistent in crime reporting and exist solely for the purpose of making you make that error. Yes, this is just one number, cherry-picked for the largest discrepancy. If you don’t restrict to teenagers, non-blacks are killed by police out of proportion to their homicide rate.

    • Toggle says:

      “I wish there were stats on rates at which police shoot unarmed people of various races. I can’t seem to find any. These are the cases that get the most media outrage (and which are indeed the most outrageous), and they seem to happen fairly often to black men and not often result in severe consequences to the shooter. But of course that’s a “seem to” based on media-filtered anecdotes.”

      I also couldn’t find any study like this, but I found Corell et. al.. It set up a study involving a shooting gallery/video game played by a number of Standard Affluent Undergraduates, with targets of different races that were holding guns, and non-targets of different races that were holding non-weapons. It found that participants were rather more likely to accidentally shoot an unarmed black figure than an unarmed white figure- and also that they were less likely to mistake an armed target for an unarmed one.

      (I was going to suggest that perhaps unarmed shootings are too rare for a valid statistical sampling, but then I found a newspaper analysis demonstrating that one third of all shootings in Houston from 1999-2004 were against unarmed suspects, at a rate of about one per month. So the data set should be large enough.)

      • JK says:

        Another study used cops as test subjects in a rather more realistic lab experiment and found that they were less likely to erroneously shoot unarmed black suspects and hesitated longer when shooting armed black suspects: http://www.policeone.com/use-of-force/articles/7653755-Cops-hesitate-more-err-less-when-shooting-black-suspects-study-finds/

        • Toggle says:

          (Incidentally, the study proper can be accessed as a pdf download with a quick Google Scholar search for ‘Results from experimental trials testing participant responses to White, Hispanic and Black suspects in high-fidelity deadly force judgment and decision-making simulations’)

          The article you link points out that it didn’t just find a bias- it found an error rate 25(!) times lower for black suspects than for white, in the case of experienced officers. Six times lower for untrained people off the street. That kind of extreme result is probably worth a methodological sanity check.

          This study involved live actors in fairly complicated scenarios, which may produce some weird skew. Also, their subjects were volunteer, not random, which means that you want to be really careful treating these kinds of studies like they were a poll or something. “[O]ur police samples over-represent White males, making the generalizability of our results to law enforcement populations tentative. We do not imply that these findings can be generalized to the U.S. law enforcement population. However, our samples are representative of Eastern Washington law enforcement.” It’s also true that their civilian subjects were the usual students, whereas their officer subjects were necessarily drawn from a different pool; the two groups aren’t likely to be directly comparable due to divergent selection effects.

          That said, this is still the most informative study I’ve seen so far, and in the absence of other data it’s definitely moving my beliefs away from the conventional narrative.

      • pylon shadow says:

        “Houston Lawman” by Culturcide
        https://www.youtube.com/watch?v=G0ZUUwvwD8w

    • Jaskologist says:

      An additional statistic I would like to know: how many of these shootings (hopefully broken down by race) occur in raids rather than spontaneous confrontations on the streets? How common are raids, even?

      I would assume that raids are much more likely to harm children and unarmed bystanders, and the main correlation there would not be individual guilt but simply living near likely offenders.

      I have absolutely no clue what the numbers are for any of this.

  17. Sonata Green says:

    The unlabeled section G. Legislation is conspicuously lacking in concrete statistics. It would be unreasonable to ask Scott to expand further on his already heroic research effort, but perhaps some enterprising commenter could turn up some hard data.

  18. chaosmage says:

    The slightly higher conviction and sentencing rates for whites are exactly what you’d expect to result from the fact that whites, as you showed, fail to hide their crimes. After all, lots of convictions and sentences are mostly based on confessions.

  19. pineapple says:

    Steve Sailer on the racial disparity in the death penalty:

    “The simplified model of how it really works is that white-dominated jurisdictions tend to be more conservative and more pro-death penalty, while black-dominated jurisdictions tend to be more liberal and less pro-death penalty. So, people who commit murders in white-dominated jurisdictions (a mix of white and black perps with mostly white victims) are more likely to get the death penalty than people who commit murders in black-dominated jurisdictions (overwhelmingly black perps with mostly black victims).”

  20. Richard says:

    Although they don’t say so, the most logical explanation to me would be that black neighborhoods are poorer and therefore higher crime

    how do you know this is the cause and effect and not the other way around; higher crime -> poorer?
    This is a genuine question btw and I can’t seem to find a sensible answer anywhere.

    • llamathatducks says:

      “how do you know this is the cause and effect and not the other way around; higher crime -> poorer?”

      I don’t have time to track down historical crime & poverty numbers right now, but I think it’s important that we know some ways black poverty was created, like slavery (which means most black people couldn’t save money) and more recently and most relevantly federally enforced housing discrimination through e.g. redlining. (This article explains how this actually affected the lives of black people who ended up having to pay exorbitant prices for houses compared to white homebuyers, and how it gave us segregated poor black neighborhoods.) So at the very least black poverty has causes that have nothing to do with criminality.

      • Richard says:

        Right, I was not attempting to claim that crime is the only possible source of poverty. It seems fairly obvious that for instance drought will lead to poorer farmers. It seems that very few of the farmers who were displaced in the 20s and 30s due to drought in the dust-bowl states became criminals however, so it is not clear to me that poverty leads to significant increases in crime.

        I live in a country where poverty is not something that is forced upon you. I know people who own their own house in the suburbs or country side, raise kids as single parents and can still afford a weeks vacation on the Mediterranean every other year or so when their only income is the minimum dole.

        Still, there are criminals who are also receiving the same dole and are flinging dope on corners and poking holes in each other with knives and sometimes guns. Yes, our crime rate is a lot lower than the US, but then again we don’t have a broken justice system and prison-industrial complex, which is a major confounder.

        If I’m to judge from first impressions without any studies to back me up because I can’t find anyone who has actually looked into this, my current prior would look like this:

        Low IQ -> bad decisions == poverty == crime == drugs == the entire misery complex.

        where poverty is one effect among many, not a cause.

        Since everyone else seems to take for granted that poverty is a major cause of crime, I suspect that I am missing something fundamental, but I can’t seem to figure out what, hence the question.

        • llamathatducks says:

          Crime can be a result of lack of opportunity. This country IS one where poverty is forced onto people. If one lives in a neighborhood where the local school is not good and the best kind of job one can get without crime is a minimum-wage one (note that here minimum wage is insufficient for a relatively unstressful life), and one sees drug dealers making much more money, that becomes appealing. Obviously theft is similar. And then there are things that are typically called “criminalization of poverty”, like fines and court fees for having a broken car.

          [Edit] Also, if we’re still looking at the history of black poverty, we know that the genesis of the black condition in this country has nothing to do with IQ and everything to do with slavery.

          • Richard says:

            This seems to be the standard argument, but according to Sudhir Venkatesh who seems to be the only one who has actually looked into the economics of crime, a drug dealer makes $3.30 an hour while the minimum wage is $8.25. Follow the Amazon affiliate link and then look for “Off the books” or “Gang Leader for a Day”

            So, if poverty causes crime by the mechanism you say it does, it must be because the criminals don’t know the numbers and I find that hard to understand given that the same people work both dealing drugs and at McDonalds. They should notice that the one pays better than the other.
            Also, they might notice that crime is a singularly bad solution for solving the poverty problem given its side effects.

            [Yes, I realize that this post is essentially about colour, but it’s not directly relevant to whether or not poverty causes crime which is my particular hangup at the moment. Slavery == bad, no problem]

            Anyway, thanks for replying

          • llamathatducks says:

            But people higher up in the criminal organization do make more money. And moving up that organization may seem more accessible than moving up any other organization’s chain of management.

          • Troy says:

            Granting for the sake of argument that being poor could lead people to do things like deal drugs or rob houses out of desperation, what’s the plausible mechanism for things like murder, assault, and petty theft? Stealing a pack of gum from the grocery store isn’t going to help you out of poverty; nor is a violent crime that doesn’t aim at getting money.

          • llamathatducks says:

            Troy:

            Petty theft is petty, but it still gets you something (and maybe you’re hungry). Assault can be a mugging. Murder is part of the business model of cartels that sell illegal drugs.

          • Troy says:

            But not all violent crimes aim at getting money: e.g., assault/murder of one’s domestic partner or someone who one gets into a fight with on the street or in a bar.

            I will grant though that I don’t know what percentage of violent crimes fall into categories like the above and what percentage are, e.g., related to drug trafficking.

          • llamathatducks says:

            Right. You asked what mechanism there could be for increased assault, petty theft, and murder due to poverty; the answer is that sometimes these crimes are economic in nature. I didn’t claim that they always are, and like you, I don’t know which kind is more common.

            But actually, there’s also a plausible mechanism for poverty increasing non-explicitly-economic crime like street fights and domestic abuse, which is that poverty breeds frustration and anger, which are sometimes expressed through violence. (Relatedly, racism also breeds frustration and anger.)

  21. JT says:

    Great post.

    As regards traffic stops: suppose police officers stopped people completely at random. If there’s a higher crime rate in the general minority population, they’d have a higher average success rate with minorities! They would have to search with an anti-minority bias to get an equal success rate.

    More generally, we’d want a fair system to have an equal *marginal* chance of search success by race. If one race has a higher crime rate in the general population, then we could reasonably expect that this fair system has unequal *average* rates.

    Consider a histogram of police-observed probability of crime “c” (x-axis) against density p(c) (y-axis) for different races. If the police (correctly) stop everyone with c >= x, we might still find E(c | c >= x) is different for different races if the unconditional p(c) curves differ by race.

  22. ckp says:

    The classic example of this is crack cocaine – a predominantly black drug – carrying a higher sentence than other whiter drugs. Even if the police are scrupulously fair in giving the same sentence to black and white cokeheads, the law will still have a disproportionate effect.

    Not exactly – “Those Who Can See” (a HBD/race realist blog) did a post about this kind of thing:

    …Black leaders were the first to sound the alarm about the drug, as Harvard law professor Randall Kennedy documents in Race, Crime, and the Law. Harlem congressman Charles Rangel initiated the federal response to the epidemic, warning the House of Representatives in March 1986 that crack had made cocaine “frightening[ly]” accessible to youth. …
    Queens congressman Alton Waldon then called on his colleagues to act: “For those of us who are black this self-inflicted pain is the worst oppression we have known since slavery. . . . Let us . . . pledge to crack down on crack.” The bill that eventually passed, containing the crack/powder distinction [5 years minimum for 5 g. of crack], won majority support among black congressmen, none of whom, as Kennedy points out, objected to it as racist.

    Later on, once the disparity in sentencing manifested, there was a complete reversal in the narrative:

    The disparity in penalty triggers between crack and powder cocaine is one of the most notorious illustrations of racism in the criminal justice system. … The National Conference of Black Lawyers helped to convene a national symposium in 1993 … The near unanimous consensus from those assembled was that the sentences for crack cocaine are not medically, scientifically or socially supportable, are highly inequitable against African Americans and, thus, represent a racially discriminatory national drug policy.
    The Coalition also sponsored a legislative briefing which culminated in Representative Charles Rangel (D-NY) introducing a bill to eliminate the disparity and make the sentences of those convicted of crack cocaine offenses equivalent to the current sentences for powder cocaine.

    It takes shameless sleight of hand to turn an effort to protect blacks into a conspiracy against them. If Congress had ignored black legislators’ calls to increase [crack] cocaine-trafficking penalties, the outcry among the groups now crying racism would have been deafening.

    • Anon says:

      This doesn’t contradict Scott’s point: no matter what the reasons for the law, it will still disproportionately affect black people. It is worth noting that the intent of the law was an honest effort to help black people, but that doesn’t change the effect.

  23. Richard Gadsden says:

    A hypothesis.

    There are a significant number of racist cops who hassle and arrest black people for things they’d let white people do.

    There are also a significant number of anti-racist (reverse racist?) cops who let black people alone for things they’d hassle and arrest white people for.

    This would explain both the felt experience of black people that they’re being hassled and discriminated against, and the statistical evidence that they aren’t: you don’t notice when you’re silently let alone, but you do notice when you get unfairly hassled.

    • 27chaos says:

      Ooh. This one is gorgeous. Smart idea.

    • RCF says:

      Doesn’t this reduce to “Sometimes people get away with things they shouldn’t, and sometimes they are hassled when they shouldn’t, and one’s race doesn’t affect the frequency of either”?

  24. Deiseach says:

    I’m not qualfied to comment on American police shootings, so I’ll just confine myself to this:

    the possibility of (probably white) researchers being stooges who are going to send their supposedly confidential surveys to the local police station and get them locked up might be much more salient

    Based on recent direct experience, I’d advise anyone – even if you’re concerned about drug dealing in your neighbourhood – to tell nothing to anyone who isn’t the police, and even then to be careful about the cops.

    8:15 a.m. one workday, I’m the only one in the office, I get a call from a very distressed tenant who is looking for a transfer out of her neighbourhood to anywhere else. She’s been targetted for abuse by locals involved in drug-dealing, and is in genuine fear for her and her children’s physical safety. She’s crying, she’s afraid to leave her house in case she’ll be attacked in the street, she’s also afraid to stay in her house for fear of attack (and, given things like this, her fears are not groundless).

    All because an anti-drugs campaigner* came to town a few months back, got a campaign (with the involvement of some local politicans) going and interviewed people willing to talk about the drugs scene in the town.

    This woman is involved through her ex-partner, she’s trying to get out of that situation for the sake of her kids, she goes and gives her testimony to this guy. He’s recording everyone, and the assumption on the part of those who talk to him is (a) anonymity (b) he’s only going to the police with it. Instead, what does he do?

    He goes to the alleged dealers named by various people, challenges them, tells them X, Y and Z said they’re involved in drug-dealingk and backs it up by playing the recordings.

    He goes back to his own place (at quite literally the opposite end of the country), there are no arrests made, and instead a lot of very angry and violent people now have a list of names of alleged informers.

    Our tenant spoke to the police about her concerns, and the only advice they gave her was clear out of town for about six months until “things cooled down”.

    So yes – no matter who is asking, no matter how civic-minded you are, the moral of this little story is: tell nobody anything because it could be your neck on the line.

    *There’s a messy story behind why the guy turned up in town on an anti-drugs campaign, but that’s for another day.

    • chaosmage says:

      Horrible. Has that campaigner been stopped from doing the same thing again elsewhere?

      Your advice to not talk to anyone is overshooting, though. Of course that incompetent stranger was not to be trusted, but I don’t think that’s evidence that talking to competent trusted people is a bad idea. As a heuristic, “ask for help” may sometimes lead to bad outcomes, but “don’t talk about it to anyone” is surely worse.

  25. Radley Balko has said that he started out working on his police beat thinking he’d find a severe racist bias in the police and justice system and ended up finding mostly an anti-poor bias. I’ll take this post as further corroboration of that idea.

  26. Deiseach says:

    Their [American police] accountability is also low, compared to the standards of the other side of the Atlantic.

    The Abbeylara shooting in 2000 was a case where the Emergency Response Unit of the Irish police force fatally shot a man armed with a shotgun (after a two-day ‘siege’ of his house).

    Public disquiet was such that the Gardaí (Irish police) commissioned an investigation by the FBI (yes, the American one) and their only criticism was that the police hadn’t shot soon enough; American police practice would have taken him down much earlier:

    In the words of the FBI report, “Garda personnel repeatedly and emphatically ordered Mr. Carthy to halt and throw down his weapon. Despite these warnings, Mr. Carthy was allowed to continue undeterred beyond the wall which served as one side of the inner perimeter and walk toward the outer perimeter which was manned in part by unarmed Garda officers….to allow Mr. Carthy to cross the inner perimeter armed after he had repeatedly ignored warnings and had previously aimed and fired his shotgun at Garda officers was inconsistent with accepted law enforcement practices in the United States”

    Did this calm the public? Not at all! In fact, continuing public outrage forced the setting up of a tribunal of enquiry which lasted four years, resulting in a public apology by the government, which the family (and public) still deemed unsatisfactory as the police were exonerated of any legal culpability for the death.

    Contrast this with the view as expressed by the FBI that it is commonplace for the police to be prepared going in to use lethal force and to do so on a regular basis, and you see the difference in attitude.

    And don’t say it’s because crime in Ireland and America is different – we have drug crime, gangland crime, and the IRA committed armed robbery of banks and post offices, kidnappings, and killed six Gardaí and one Irish Army soldier, mostly during robberies, in the Republic during the 70s-80s.

  27. C.B. says:

    This is (relatively) good news! I had assumed that the narrative of “every part of the criminal justice system is biased against black people” was true, and I’m glad to see that it’s mostly not.

    On the other hand, it’s kind of like your causes of ADHD dilemma. If the disproportionate rates of imprisonment for black Americans had been caused by bias in the criminal justice system, that would probably be a lot easier to fix than whatever we’ll actually have to do.

  28. Daniel Speyer says:

    The success rate metric seems a little flawed. Unless I’m missing something.

    Suppose (i.e. make up numbers for easy math) the community has 16 black people and 16 white people. 4 black and 2 white are carrying drugs. 8 black and 4 white are searched, including all the guilty ones. Both searches have a 1/2 success rate. This means that 4/12=1/3 of innocent blacks but only 2/14=1/7 of innocent whites get searched.

    This model is agnostic over whether the police are applying evidentiary standards fairly.

    • llamathatducks says:

      Whoa, this is important.

      I’ve been wondering how the data in this post reconciles with the many stories I’ve read in which black men are unfairly targeted in situations that for me would be perfectly ordinary. Maybe this is one way how.

      I wonder, also, if there could be a difference in how people of different races act around the police. Lots of the stories black men tell of unwarranted police contact also mention that they have had to learn to be extra-respectful of police to avoid making them suspicious. “The talk” that black parents must have with kids about how some police officers are racist is something I’ve often seen mentioned; there doesn’t seem to be anything analogous among parents of white kids. So I wonder if black people might be actually acting better around police officers than white people are? In which case equal rates of arrest/brutality might actually hide bias.

      • llamathatducks says:

        I’m trying to figure out whether Daniel’s logic applies to police shootings as well. On the one hand, there are fewer people shot by police than there are shooters, so it would be consistent with statistics if police officers only ever shot people who were themselves shooters. On the other hand, we know that police officers do sometimes (often?) shoot unarmed people. So it’s still possible that police officers more often shoot innocent black people. (For instance, if police officers are particularly unlikely to be around when actual shooting happens among black people.) But maybe this is an entirely separate issue from Daniel’s point.

        • William O. B'Livion says:

          According to Wikipedia in 2008 there were 765,000 police officers with arrest powers in the US.

          That’s a lot of people with guns.

          There’s going to be a certain error rate, and yes we should work to get that error rate down, but these days (and I realize this is tangential) in big police departments when given a choice between sexual harassment & diversity training, and actual exercises in use of force (not class room, out on a range/shoot house moving and de-escalating and making choices and getting stressed) which do you think the city and the higher-ups in the PD will go for?

          Uh huh. Then you get a 12 year old shot for playing with a air-soft gun with the orange tip removed. You get a guy in a tenement shot for opening a door because two rookies were patrolling together (in violation of procedures) and one had his gun out (in violation of procedures) and flinched. That’s just within the last 2 weeks.

          In both cases proper tactics could have preserved a (relatively) innocent life. But cops are human, cities have budget pressures and life isn’t perfect.

          • llamathatducks says:

            I strongly agree that police departments need to get much more serious on training and safety procedures. I don’t see why you’d blame the unseriousness specifically on sexual harrassment/diversity training, though; that seems like an example chosen at quasi-random of a thing you think is not useful. I haven’t looked into whether/how much such training helps, but in general I believe that training police officers (people with a lot of power!) to work respectfully with diverse populations is super important as well.

            I suspect that, as in many situations involving a governmental agency failing to behave well, part of the issue is just a lack of money. We know that e.g. in St. Louis County there are lots of tiny, poor towns that get their budgets from fines and court fees; obviously there are terrible incentives involved, partly due to insufficient city budgets. More broadly, I think a lot of competency can be bought by paying people better.

      • William O. B'Livion says:

        My father never sat me down and talked to me about cops being racially motivated to put my lily white ass in jail, however he, on several occasions, discussed how to behave when a police officer focused his attention on me, and why.

        Traffic stops–turn the car *off* and put your hands on the windshield. Do not reach for *ANYTHING* in the car other than your wallet and do that BEFORE the officer gets out of his car, otherwise wait.

        On the street–be polite and do EXACTLY what they tell you. Don’t talk back, don’t argue. You won’t win, and you might get hit.

    • 27chaos says:

      Thank you!

  29. Pingback: The Discrimination Bias | Miscetcetra

  30. Handle says:

    “But I also disagree with the people who say things like “Every part of America’s criminal justice is systemically racist by design” or “White people can get away with murder but black people are constantly persecuted for any minor infraction,” or “Every black person has to live in fear of the police all the time in a way no white person can possibly understand”.

    Freddie DeBoer – usually must more level-headed, provides an almost textbook example of this:

    I guess the essential thing that has to be repeated, again and again, is that this outcome, and so many like it, are the result of a system functioning the way it is intended to function. Racism is baked right into the foundation of the system. When racist outcomes happen they happen not because of the evil in the hearts of individuals but because our social, economic, and legal systems have been designed to deliver those racist outcomes.

    Every one of those grand jurors might have hearts of purest gold. The outcome was predetermined precisely because the outcome did not rely on the individual character of the jurors. … None of that can be solved through having pure hearts and pure minds. Racism is not a problem of mind. Racism cannot be combated by individuals not being racist. A pure heart makes no difference. In response to systemic injustice, you’ve got to change the systems themselves. It’s the only thing that will ever work.

  31. stillnotking says:

    The classic example of this is crack cocaine – a predominantly black drug – carrying a higher sentence than other whiter drugs. Even if the police are scrupulously fair in giving the same sentence to black and white cokeheads, the law will still have a disproportionate effect.

    I thought the City Journal article did an admirable job of explaining the genesis and evolution of draconian crack cocaine laws. I was a teenager in the 1980s, and my memory of the crack epidemic corresponds exactly to Mac Donald’s account. Crack was seen, with no little justification, as an existential threat to black communities in the United States. Harsh punishments for crack dealing were pushed by black politicians following the loud and organized demands of their constituents — the people whose lives really were adversely affected by the appearance of a new, cheaper form of a deadly addictive drug. If there was race-based criticism of crack laws, it was entirely in the other direction! “White people don’t care what happens in the inner city” was the Blue party line at the time. (Well, Red in those days, but you know what I mean.)

    Those 1980s politicians who saw themselves as championing a cause of the black community by coming down hard on crack dealers must be pretty surprised to hear themselves described as racists today.

  32. Alexander Stanislaw says:

    This is excellent work. I wish I had the patience and skill needed to do an analysis this thorough.

    I wish I could link it IRL, but I don’t think that would fly well.

  33. Robbbbbb says:

    There’s an elephant in the room here that needs to be considered: At what rate are various races victims of crime? If you are poor and black, and live in a poor, black neighborhood then you are more likely to be a crime victim, and it is likely that the perpetrator is also poor and black.

    Section B talks about that explicitly: There are structural factors at play that lead to higher arrest rates for poor blacks. This sucks. But failure to prosecute crimes like this is even more racist, because you are denying justice to poor, black people.

    • Nornagest says:

      Don’t think it’s quite that simple.

      Let’s say for example that it’s 1911, and you’re a second-generation Polish-American in an East Coast city split evenly between Polish immigrants and their children, Irish immigrants and their children, and Americans not of recent immigrant descent. All these groups mostly stay in their own neighborhoods, and yours is the poorest and most criminal of all.

      Because of the general lack of ethnic mixing, most crime is intra-ethnic: if you’re Polish and your house gets broken into, the perp is almost certainly also Polish. But because the police recruit city-wide, most of the cops you interact with aren’t Polish, and so when they come to arrest someone who’s been accused of a crime, it usually looks like a Pole being hauled off by Irish or American cops — not the sort of thing that reduces ethnic tension!

      This seems to be a double bind. If the cops take a hands-off approach, it looks like Poles are being left to their own devices (which has consequences even worse than you’d naively expect; most criminal gangs originate in neighborhood self-defense efforts). If they don’t, it looks like they’re being persecuted by more powerful ethnic groups. You could send cops to neighborhoods of their own ethnicity, but that’s going to be a tough political sell when one of those beats is a lot more dangerous than another.

      • Robbbbbb says:

        Yeah. This is kinda what I was driving at, although you come in with a good analogy to drive the point home.

        Of course, the Poles and Irish (and Italians, and …) managed to integrate successfully, and the immigrant communities stopped needing their local benevolent associations.

        I’ll leave further elaborations on the subject as an exercise for the reader.

    • Tarrou says:

      When the police cannot deliver justice to a community, it is most common for organized crime to pick up the slack. A hundred years ago, it was common for certain neighborhoods to be ceded entirely to the various ethnic mafias. The fact that this almost never happens today should be a good indicator that the police do a good enough job to at least make them a better option than the gangsters, an admittedly low bar.

      • At only a slight tangent … . The inside of American prisons mostly is ceded to ethnic mafias. See David Skarbek’s very interesting _Social Order of the Underworld: How Prison Gangs Govern the American Penal System_. And they are largely “picking up the slack,” providing the sort of services within the prison that governments are supposed to provide elsewhere.

  34. Jeremy says:

    Excellent post, thank you.

    I don’t know how much you care about tiny typos. I personally don’t care much, but they leap out at me. It’s in the wiring.

    “If you’re mandated to give a particular sentence for a particular crime, there’s a lot less opportunity to let bias slip in then if you can do whatever you want.”

    should be “to let bias slip in *than* if you can do whatever you want.”

    That is all. Please feel free to delete this comment instead of posting it.

    • Anonymous says:

      I don’t care much either, but Never trust the media to give you any number more complicated than today’s date.. has an extra dot.

  35. If criminal record is a confounding factor, and, some people are more likely to have criminal records, then any unfairness in the system will get amplified.

    Ta Nehisi Coates has written about it being harder for black people to get homes outside of poor neighborhoods even though when they have as much money as white people living in better neighborhoods. If this is true (I haven’t checked out the arguments), then being in a poor neighborhood disproportionately adds to the risk of a bad interaction with the justice system.

    Possibly pointing in the opposite direction: In her introduction to The New Jim Crow, Michelle Alexander talks about how long it took her (a black civil rights lawyer) to find out how serious the incarceration problem was for poor blacks, and that major black civil rights organization didn’t have incarceration as part of their agenda.

    • Tarrou says:

      Chicken and egg problem. Blues look at it and say that poverty causes crime, reds do and say crime causes poverty, and they’re both partially right. At the baseline, it’s a cultural problem, and the mistrust of the black community for the nation as a whole makes it very hard to have a policy response. Note the crack law evolution several other commenters mention. Unintended consequences are rampaging around the black community. Almost everything the country has tried has backfired. I’m not hugely optimistic.

      • 27chaos says:

        What evidence is there that argues it’s a cultural problem? (I don’t want forwards from grandma about rap music and the devil, I want actual data.)

        • “What evidence is there that argues it’s a cultural problem?”

          One piece of evidence that the primary cause of black poverty is neither genetics nor racial discrimination is offered by Tom Sowell in _Ethnic America_. By his account, West Indian immigrants make it up to the median income in about one generation. The are “blacker” than the average Afro-American, so should do worse if the problem is either genetics or racial discrimination.

          Sowell thinks it is culture, and attributes the difference to differences between slavery in the American South and slavery in the West Indies.

          • However, if the prejudice is against African Americans specifically, and West Indians look and sound different, then there can still be unfair bias against African Americans.

            I believe that race is an artificial category, and people are much more affected by ethnicity (their own and what they believe about other people’s).

            Has there been any research on to what extent people overreact to real differences between groups?

    • ckp says:

      Regarding difficulty in getting houses outside of bad neighborhoods, this might have something to do with it.

    • William O. B'Livion says:

      Given Coates writings on subjects I know, I wouldn’t trust a thing that came out of his pen.

      I’ve lived in poor neighborhoods in Chicago, both mixed, predominately black and predominately white. A seriously large part of it is, to quote Ice-T from something like a million years ago:

      Escape from the Killing Fields

      “Yo man, it sounds like you’re selling out to me, cause I’m from the ghetto, I lived in the ghetto all my life
      We ain’t supposed to leave here, we’re black… we’re supposed to be poor.”

      Shut up do you know
      How dumb you sound?
      That mentality
      What keeps my people down
      No one wants to
      Live in an urban war
      You live there cause
      Your parents were poor
      They live there because
      Theirs were also
      Get yourself together
      Hit the gates bro!

      I will also grant that Ice-T isn’t the most reliable witness in these events given his background, but it rhymes and fits with what I’ve been paying attention.

      People *like* to live in tribes, and over the last 60 years there has been enough cultural divergence between “urban blacks” and “urban whites” that they walk the same streets and shop in the same stores[1], but increasingly consume entirely different media (music, television and internet). Even the “hip hop” that white kids glom onto is largely different than that the black community calls their own.

      And this isn’t limited to Blacks. Take “The Hill” in Saint Louis, historically referred to “Dago Hill” in less PC times. To quote from Wikipedia:

      That heritage remains evident today. As of May 2003, about three-quarters of the residents are Italian-Americans, helped perhaps by the practice of rarely listing homes on the open market.[3] The neighborhood is home to a large number of locally renowned Italian-American restaurants, bakeries, grocery stores, salons, and two bocce gardens.

      http://tiny.cc/un1zpx is the area they’re talking about. I’ve got family (Italian-American) that lives just outside of that area. They’re on their 3rd generation living on the same block IN THE SAME HOUSES.

      Some people move in and put down roots and you’ve got to kill them to move them, even if the neighborhood changes.

      So it’s not just economic pressures, and it’s not significantly racism. Even the nicer lower class neighborhoods in Chicago were 1/3 to 1/2 black. I lived at 4900 N. Wolcott, and most of my neighbors were not white. It was a pretty nice place for the money.

      [1]Both Cabrini Green residents and Gold Coast residents used to shop at the Jewel/Osco @Division and Clark. Well, the Cabrini Green residents did. The Gold Coast folks sent The Help to do it).

  36. RES says:

    Quick technical note:

    “Once again, there are two possible hypotheses here: either police are biased, or black people actually commit these crimes at higher rates than other groups.”

    These two hypotheses are not mutually exclusive. Blacks may commit more crime, and the police may be biased against them as well.

    I’d also be curious if these can factor in cumulative bias. Imagine there are four steps in the process from investigation to conviction with a 50% chance of moving on to the next, and each involves a bias of 2%. That’d be an ~8% difference in convictions per suspect, and that’s quite significant.

    One way or another, it’s heartening to see a substantial weight of evidence points against major racial biases in the justice system.

  37. Airgap says:

    1. I read somewhere–I swear it was a fairly reliable source!–that the sentencing disparity between blacks and whites was largely due to black judges punishing black criminals more harshly. I think this was from one of Tom Sowell’s books, but I could be wrong. It’s still racial bias, but it makes ya think.

    2. Similarly, the coke-crack sentencing disparity, although still an instance of racial bias, was originally put into place by black community activists frustrated that crack dealers ready and willing to shoot witnesses kept getting off with light sentences. You can argue that it’s since redounded to the benefit of racist oppressors, but I’ve personally never met any white people with the kind of vicious cunning to say “I know: we’ll make watching more that 2 Tyler Perry movies a felony!” You’re a lot more likely to have it in for Tyler Perry if you see what he’s doing to your community every day.

    • Ryan says:

      On a similar note I’ve seen several opinion polls which show that black people living in poor neighborhoods are strong supporters of gun control. It could be because they’re liberal idealists with no respect for the second amendment, though I lean toward the “they don’t want to get shot” explanation.

      • von Kalifornen says:

        I’d say a little more complex: Law-abiding black people have no use for guns because it would (it least in impression) make them targets to police and make racists mark them as criminals. Meanwhile, gun control does help suppress black criminals.

        White people (at least in their impression) don’t have that problem, so they decide to suppress criminal armaments or arm themselves at their leisure.

        • William O. B'Livion says:

          Meanwhile, gun control does help suppress black criminals.

          To quote Wikipedia: [citation needed]

  38. haishan says:

    So in part A, we have a lot of data about the rate of “Type I errors” in policing — i.e., police stops of (presumably) law-abiding citizens. Has anyone ever tried to look for evidence of a racial bias in “Type II errors?” Do cops fail to stop white people carrying contraband more often than they do black people carrying contraband? Obvious problem: How do you measure this?

  39. Ryan says:

    My good friend is an ADA and has given me a few insights into the criminal justice system.

    So, for example, say you are guilty of a crime. Do you want a jury trial or a bench trial?

    Wrong answer. You want a bench trial. The jury doesn’t give a crap about a key piece of evidence being ruled inadmissible and the state therefore not proving the case beyond a reasonable doubt. The judge will let you off on the technicality almost every time.

    Guilty or not guilty, should you pay a bunch of money to a hot shot lawyer to contest the admissibility of every piece of evidence? Nope. The prosecutor is just as lazy as you are. Worst yet, they’re not getting paid anywhere near what they think they deserve. And if you make them work harder than they’re “supposed” to, it’s going to seriously piss them off. You just sank any chance you had at a good plea deal.

    Speaking of lazy. Holy crap cops are even lazier than you are. The best way to get away with a crime is for the police officer to conduct an unconstitutional search because he was too damn lazy to spend 3 hours getting a warrant which would be rubber stamped. In the case of serious enough crimes my friend will actually be called out to the crime scene to make sure the cops don’t screw up. “Yes, you can search that. No, get a warrant for that.” “But I don’t see how-” “Dammit, I said get a warrant.”

    And on that same note. The cops don’t frame people for crimes they know they didn’t commit. It’s not because they’re angels, it’s just too much fucking work. These people are not geniuses and they know they can’t keep a web of deceit straight. That said, they will lie with impunity if they think you’re guilty. “When I approached the car door I could see the marijuana visible under the driver’s seat.” Total bullshit, but they don’t care.

    My general impression is the people in the system function more or less as intended. The cops arrest someone because they genuinely believe they’re guilty. They may be factually incorrect, but they’re sincere in their incorrect believe. The prosecutor won’t prosecute you unless they think you’re guilty. And they’re only going to refuse to offer standard plea deals if your lawyer pisses them off somehow. The last issue is partially my personal experience with judges: smart, hard working, complete and total true believers in the justice system.

    So on the issue of the post, my initial guess would be that any racial bias which exists is almost completely unintentional.

    • JRM says:

      Your friend is mistaken, I think.

      You (the criminal) almost always want a jury trial. (There’s a reason most people elect for one.) Convince one person, and you hang it. Retrials are more work. If the evidence is excluded, the jury never sees it. The jury often cares a lot and tries to get it right.

      Possible exception: You seem to be a very strange person. You have a defense-sympathetic judge.

      Should you a hire a hotshot lawyer? Depends. What are your options? You should have a good lawyer. Whether to go to full battle or surrender early is a complicated tactical decision. (It may also be a moral decision if the defendant just wants to take responsibility for what he did. Doesn’t happen often; does happen.)

      A “hotshot” lawyer knows when to throw up the white flag and when to grind it. A prosecutor who says you should get El Foldo the defense lawyer is… not other-optimizing. (Public defenders in some jurisdictions are very good; in my jurisdiction, you’d rather have Random Public Defender than Random Private Attorney.) You want a great lawyer as much as you want a great doctor.

      And sometimes there’s no other choice. If you’re charged with first degree murder and the prosecution is offering: “Plead. Die in prison,” and you either didn’t murder anyone or you don’t want to take responsibility for murdering that jerk, it’s in your own interest to have a diligent, smart lawyer by your side.

      I personally don’t get irritated at defense attorneys who try. Sure, there are early resolution discounts for most cases. If you’re just dragging it on, that’s going to make the early resolution go away.

      Prosecutors as lazy slugs who don’t care, who think they should get paid more? I should get paid more. I know lazy slug prosecutors exist. But bet against that being your prosecutor. Especially if you’re accused of something serious. There are a lot of driven people in the business.

      Cops as liars: Not my experience. Does happen, doesn’t happen often. I’d suggest that the interested go to the local courthouse and watch cops testify. Evaluate.

      Cops as lazy: Some are. Most aren’t.

      Those are just my opinions. I could be wrong.

    • Patrick says:

      This matches my experience.

      As for cops lying… In my experience cops, don’t sit there and think, “I know [X] happened, but I’m going to say that [Y] happened because I think this guy’s guilty and that will put him in jail.” Its more like… “I pulled this guy over for drunk driving three months ago. Since then I’ve pulled over sixty other people while cruising for drunks. I don’t remember anything specific about this guy. But I wrote down all the usual stuff in my notebook, so I’ll just mouth all the magic words that say I remember everything in detail and that the usual stuff happened.”

      The judge isn’t going to notice or care, and the prosecutor is just happy that the cop said the magic words.

      The defendant is enraged, of course, because he knows that the cop said things that aren’t true, and he understandably would prefer to focus on that instead of on the trivial fact that he is guilty as sin. Sucks for him I guess. The defense attorney just thinks to himself, “this is why I told the guy to take the standard deal,” then makes a few arguments he knows won’t work.

      Things are worse in civil court, particularly in collections. Big Company comes in and testifies from business records that all the usual stuff happened. The defendant says the business records are wrong. Big Company won’t/can’t produce the person who made the business record entry, and they don’t have to. The Magistrate notices that the alleged error isn’t substantive (it doesn’t change the fact of default or the amount owed, doesn’t meaningfully affect notice, etc) and decides that life is too short to care. And how would you resolve it anyways? The business only has its records. The defendant either won’t testify, or, if he does, will screw himself by admitting every substantive aspect of Big Company’s case, or else by transparently lying about matters that are documented beyond dispute (“Honest yer ‘onner, I have no idea why all this paperwork says I borrowed $250k on the same day I bought a $250k house! I don’t remember any of this!”). Plus the magistrate knows that in other case exactly like this one, if the defendant asked for a deal, he almost always got one. So the magistrate, as gently as possible, pushes the defendant in that direction out of pity.

  40. JRM says:

    First off, what Totient said. The general level of discourse on Scottified topics is terrible. Scott is not only less terrible than Facebook Politics, but less terrible than basically all media. [I edited before posting; originally accidentally said “not less terrible than Facebook Politics, which I think would have merited the Achieving the Impossible badge.]

    Nitpicks:

    1. The others upthread are right; police departments generally shoot to protect themselves from serious harm or to protect others from serious harm. If you’re on a beer run and stole a 12-pack, they’re not shooting at you. If you just did an armed home invasion and they tell you stop, they can shoot you if you don’t.

    2. You talk about crimes for which “the burden of proof is higher.” This is not the right technical phraseology; virtually all crimes in the US have the same burden of proof, which is “beyond a reasonable doubt.” More serious crimes may or may not be harder to prove; you can have a multi-defendant armed bank robbery that is very, very easy to prove (the biggest slam dunk I ever had was one of these based entirely on circumstantial evidence) and a simple possession of drugs that isn’t. But the burden of proof is the same.

    3. “Jury pools contain twenty-seven members.” Well, the ones in the study did, but that’s not an actual general rule. In my jurisdiction, jury pools range from 45 to hundreds of people. (High press long cases need a lot of people to filter through.) The implication that this is a fixed number normally is mistaken.

    (I read the original study. I very much question the theory that the selection pool matters as much or more than the actual jury. Their theories as to why don’t make sense to me.)

    Finally, it’s posts like this that want me to set up the Foundation for Excellence in Everything with Scott as the Chief Right About Stuff Officer. In my dream world, the media doesn’t just want screaming tribal allegiance and Scott’s views are conveyed to millions of humans. I’ll go pet my unicorn now.

    • DrBeat says:

      you can have a multi-defendant armed bank robbery that is very, very easy to prove (the biggest slam dunk I ever had was one of these based entirely on circumstantial evidence)

      Can we get some more details on this?

      Did they all wear clown masks?

    • ” In my dream world, the media doesn’t just want screaming tribal allegiance and Scott’s views are conveyed to millions of humans.”

      The existence of the web has gotten us a little closer to that world, as this thread demonstrates.

  41. Matthew says:

    I can’t find where I saw it in convenient chart form, but a 2012 report by the NYC Public Advocate’s office did find that the Stop and Frisk program there

    * The likelihood a stop of an African American New Yorker yielded a weapon was half that of white New Yorkers stopped. The NYPD uncovered a weapon in one out every 49 stops of white New Yorkers. By contrast, it took the Department 71 stops of Latinos and 93 stops of African Americans to find a weapon.

    * The likelihood a stop of an African American New Yorker yielded contraband was one-third less than that of white New Yorkers stopped. The NYPD uncovered contraband in one out every 43 stops of white New Yorkers. By contrast, it took the Department 57 stops of Latinos and 61 stops of African Americans to find contraband.

    (It’s unclear to me if the data is based on reports coming from the police or from the stop-ees.)

  42. Jay Shooster says:

    I think there’s a lot of studies on implicit racial bias that haven’t been taken into account here, including studies that don’t rely on the IAT. Furthermore, a defense of the IAT is probably warranted as well. Finally, I think that the evidence of a disparity in punishment between black and white drug offenders is significant and convincing.

    I highly recommended checking out these links.

    http://kirwaninstitute.osu.edu/wp-content/uploads/2014/03/2014-implicit-bias.pdf (For a defense of the IAT and lots of studies on implicit racial bias)

    http://courses2.cit.cornell.edu/sociallaw/student_projects/Definingbias.html
    (for lots more studies on implicit racial bias, including a study showing disparities in bail amounts for black and white defendants)

    http://www.hrw.org/news/2009/06/19/race-drugs-and-law-enforcement-united-states
    (arguing that blacks are disproportionately affected by drug enforcement)

    • 27chaos says:

      What if the test just measures how nervous you are about taking a test intended to guess whether you’re racist? Intrusive thoughts exist, it seems like they could apply here.

  43. caryatis says:

    “Every black person has to live in fear of the police all the time in a way no white person can possibly understand”.

    I think it’s quite plausible that black people have an exaggerated fear of the police. With the media constantly highlighting the exceptional cases to tell us that the system is biased, that innocents are convicted, whites get away with murdering blacks…I would expect the irrational fear to be greater when it comes to middle- and upper-class blacks, who have less contact with the police but read more media.

    “There was a much talked-about study recently that found that “juries were equally likely to convict black and white offenders when there was at least one black in the jury pool, but more likely to convict blacks when there wasn’t.””

    I’ve never understood why this story got so much attention given the impossibility of determining whether the result is because adding blacks to juries leads to more acquittals of innocent black defendants or more acquittals of guilty black defendants (or differences with white defendants).

  44. Protagoras says:

    As usual, it wasn’t more than I wanted to know. In fact, while it was a lot of very helpful information, there is one major element that continues to worry me, though it’s not something Scott could really have done anything about. It is my impression that the data supplied by law enforcement about crime and about their own activities is extremely inconsistent from place to place and on the whole quite unreliable. A lot of the studies Scott mentions use surveys instead (which of course have problems of their own, some of which Scott discusses), but sometimes official crime statistics are used, either on their own or as a comparison or contrast with the surveys. I don’t know if the poor quality of the data biases the results in any particular direction, but it’s one more reason to be uncertain about the studies.

  45. JK says:

    Regarding traffic stops, you don’t mention the “veil-of-darkness” method. It’s a natural experiment where the researchers utilize the fact that it’s much easier to identify someone by race in daylight than in the dark. If the police are racially biased, the racial discrepancy in traffic stops should be higher in daylight. Four studies have used this method to study black-white differences: two found that light conditions did not affect racial disparities in stops, one found that blacks were relatively more likely to be stopped in the dark, and one found that whites were relatively more likely to be stopped in the dark. See this paper for more.

    While the veil-of-darkness method has its limitations (e.g., the police or motorists may behave differently depending on time of day), it is superior to the multiple regression approach that is used to explain racial differences in most of the research Scott cites. For example, he writes that killing a white person is associated with a much higher likelihood of capital punishment, but this conclusion is based on the assumption that all the relevant covariates were entered into regression models. It is a given that this is usually not the case, and if you use a very large number of apparently relevant variables, the “white victim effect” vanishes. Of course, you can criticize the latter study for controlling for too many variables. Regression is not a good method for causal analysis in the absence of very strong theory.

  46. Douglas Knight says:

    The Vox article on police homicides says that the SHR reports 426 in 2012, all shootings, with one exception. No tasers at all? Here is a list of 38 taser deaths in 2012. I doubt all of them fit in the appropriate category of killing by police at apprehension, but I checked the last 4 and they were all arrests by police. Since tasers deaths usually take a day, these might be counted as deaths in custody, rather than deaths at apprehension. That would be silly, but not nefarious. (This comment has nothing to do with race, though the Vox article does.)

    • CaptainBooshi says:

      My bet is this has to do with what officially counts as a ‘justifiable homicide,’ which is where that 426 number comes from. As the article notes, this number is best viewed as a minimum, rather than the actual count. Maybe they don’t consider death by taser to be a homicide, but label it in some other manner, which then doesn’t get reported?

      • John Schilling says:

        I believe that in most jurisdictions, “I meant to kill him and here’s why it’s good that I did that” is justifiable homicide, whereas “I didn’t mean to kill him and I was being reasonably careful about it but, freak accident” is classified as excusable homicide.

        A cop shoots someone with a firearm, he’s going to look pretty silly arguing he didn’t mean to kill them. Or at least hurt them real bad such that death was a likely outcome, which is about the same thing for legal purposes. So, justifiable homicide if the claimed justification holds up, manslaughter or murder if it doesn’t.

        Using a Taser when there’s a perfectly good pistol on your belt, it’s pretty clear that you are trying not to kill someone. So, excusable homicide unless the jury
        believes you were being careless or malicious, in which case manslaughter or murder. It would be a very rare thing to have a justifiable homicide with a Taser.

        • Douglas Knight says:

          That makes sense, but I think it is just not how SHR works. The word “justifiable” is slapped on all homicides by police and doesn’t mean anything.

          I suppose it is possible that taser deaths are classified as accidents, but this says that accidental deaths at arrest are 10% as big as officer homicides, which is big enough to contain taser deaths, but just barely.

  47. maxikov says:

    I see a possible explanation for the case if both little overall racial bias and high racial bias reported by black people are true. Namely, there might be high dispersion of racism levels to both sides. It could be the case that left-leaning police officers have less anti-black bias that right-leaning ones, and may even have pro-black bias. That is also consistent with the data from NYC, which known for the high percentage of left-leaning people. If these effects compensate each other, on average there’s gonna be little bias, but in every particular case it be high, low, or even negative. And then there’s huge selection bias – both media and regular people are more likely to report something outrageous than the cases where everything expectedly goes fine.

    This hypothesis is hard to control for, but I would suggest comparing the level of racial bias with the percentage of democrat and republican voters, hoping that it correlates with the racial attitudes

    • llamathatducks says:

      Re: New York, see Matthew’s post elsewhere in this thread about disproportional stopping of black people (and lower hit rates than among white people). The data Scott quoted is somewhat positive for the NYPD, but it’s not overall a paragon of fairness.

  48. JP says:

    The shooting data from Mother Jones was probably the most striking thing in the post. However, the number of subjects hit in the chart is only 28, so there’s a lot of randomness.
    There are two charts shown in the article you link:
    http://www.motherjones.com/politics/2014/08/police-shootings-michael-brown-ferguson-black-men
    In the first one, there are injuries/kills for a 12-year period, instead of just 1 year. White injuries/kills are around 13% of the black ones, which is very far from the close to 50% that your chart shows. However, if you do mouse hover over the chart, you see huge numbers, like thousands of kills, which cannot be right.

    In any case, I don’t think it’s a good idea to use data from some news media without checking the original sources.

    I after all checked numbers directly from the NYPD firearms discharge report cited in Mother Jones:
    http://www.nyc.gov/html/nypd/downloads/pdf/analysis_and_planning/nypd_annual_firearms_discharge_report_2011.pdf
    Page 76 has the 12-year data. Here white injuries/kills are 33, and black ones 244. I.e. the ratio of 13.5% is about correct in the first Mother Jones figure. (Incidentally, in this table the 2011 numbers don’t exactly agree with the second MJ chart, which however is straight from the police report.)

    The longer term numbers still may not show racial discrimination, but anyway are more useful than the one year numbers.

    • llamathatducks says:

      Wow, I’m finding it really hard to read that first chart. Why are the numbers not explained?? From what I can tell it seems like they’re saying “13.667 black people are shot per year on average”, but I just went through page 76 of that report and counted these up and those figures seem off.

      Speaking of which, I got different numbers than you did, too. I got 176 shootings of black people over those 12 years (60.7% of all shootings) and 25 shootings of white people (8.6%). 58 black people were killed by police and 13 white people; since the total number killed was 99, those are just about percentages as well. I really don’t feel like chasing down violent crime rates by race for those same twelve years, but if we go with the 70ish percent from that chart about 2011, this still seems to show that police shootings are not targeting black people disproportionately to their violent crime rate.

      On the other hand, we know the NYPD absolutely do make fatal mistakes in killing black people. At most one of those was armed (the Wiki article about the security guard doesn’t say if he had a weapon), and note that in the Sean Bell case the police fired fifty times. Note also this case where the police raped a black man, though they didn’t kill him.

      I’m having a whole lot of trouble finding any examples of the NYPD killing an unarmed non-Hispanic white person. (I found these two cases where the names of the people involved sound probably Hispanic.) It could be that the victim’s race is just not mentioned when they’re white, or that cases like that don’t incite racial tension so they’re less well known. But basically I feel like I have a good reason to suspect that the NYPD’s killings of unarmed people are racially biased.

    • llamathatducks says:

      I also tried and failed to track down more details where this bit in the MJ article came from:

      “In Oakland, California, the NAACP reported that out of 45 officer-involved shootings in the city between 2004 and 2008, 37 of those shot were black. None were white. One-third of the shootings resulted in fatalities. Although weapons were not found in 40 percent of cases, the NAACP found, no officers were charged. (These numbers don’t include 22-year-old Oscar Grant, who was shot and killed by a transit authority officer at the Fruitvale BART station on New Year’s Day of 2009.)”

      That does sound pretty terrible. (There are about as many black people as white people in Oakland.) The best I could find for a source was this, which doesn’t have any additional info.

      If we compare this to homicide rates:
      – Black people are 82.2% of victims of police shootings. (This includes armed and unarmed, wounded and killed. Grrr that I can’t separate those out.)
      – Of all homicides, 64.7% have black suspects. (Btw, according to these stats, there were no homicides by white people at all in 2005. Really??)
      – But 24.4% of homicides are “suspect’s race unknown”. Of those cases where suspect’s race is known, 85.7% have black suspects.

      So the police’s shooting rates are not out of proportion to homicide rates.

      But this part is definitely important: “Although weapons were not found in 40 percent of cases, the NAACP found, no officers were charged.” ?!??

      And I do still think that it’s really important and terrible that this is a burden borne primarily by black people. Though the shooting rates may be in line with murder rates, still innocent people are being shot by police and they are exclusively people of color, mostly black. That is still terrible no matter how you slice it.

  49. William O. B'Livion says:

    Regarding the paragraph about Malign Neglect, the problem with their argument (as presented) is that *if* the system (as your analysis seems to indicate on first reading) is fair (to noise or nearly noise levels) regarding arrest and conviction, then all you have to do is *avoid the sorts of crimes cops look for*. Don’t, to reference the Chris Rock video, jump a turn style while strapped and smoking a blunt.

    Very, very few whites (and yes, fewer blacks) are rich. One of my buddies in HS spent a few years in prison because he stole a car. His parents got a *good* lawyer and got him off on most of the charges, but he was a dumb ass and violated probation. You don’t see me whining about the system being stacked against dumbass rednecks (the only reason, as you sort of note above, that trailer dwellers don’t get incarcerated like inner city blacks is that they are spread out, not concentrated.)

  50. Nick T says:

    This relies heavily on correlational studies with control variables. It’s hard to see how it could do otherwise, but, generically, how skeptical should that make us? I’m not familiar with the relevant meta-literature.

    • Anonymous says:

      Extremely skeptical, as with every other time Scott has done this.

    • Anonymous says:

      Some of the control variables are causal and some are not.

      If judges are supposed to take prior convictions into account, of course you should control for that.

      Black neighborhoods have lots of crime. That may or may not be a good reason to send lots of police there, but once that decision has been made, those police should not be accused of individual racism for spending all their time encountering blacks.

      In contrast, the success rate of searches is used as a proxy for things that the police observe about the suspect that the researcher cannot. This is much more statistically problematic.

  51. CaptainBooshi says:

    I am consistently impressed with the amount of work you put into these reviews, Scott.

    An additional factor to consider is juveniles being tried as adults. As far as I can tell, there is some evidence that black juveniles are tried as adults at an disproportionately high rate, which can end up having all kinds of knock-on effects for the youths in question even beyond the harsher punishments it would involve.

    One other note I had. You say:

    In fact, most studies find white officers are disproportionately more likely to shoot white suspects, and black officers disproportionately more likely to shoot black suspects. This makes sense since officers are often assigned to race-congruent neighborhoods, but sure screws up the relevant narrative.

    I’m not sure it does mess up the narrative, since as far as I can tell, the narrative isn’t that white cops shoot black people too much, it’s that cops in general shoot black people too much. Being black isn’t going to stop someone from having unconscious biases that affect split-second decisions. Of course, as you say, there really isn’t much recent data figuring out whether this is true or not, and it doesn’t appear to be true in New York in 2011 (although this might have something to do with multicultural training put in place specifically to deal with this issue after Sean Bell was shot in 2006. I have no idea if the training actually made a difference, just wanted to note it as a confounding factor). I was just noting that the narrative I’ve seen going around lately doesn’t focus on the idea of white cops specifically doing the shooting.

  52. Steve Sailer says:

    Here’s James Q. Wilson’s summary in 2002:

    A central problem—perhaps the central problem—in improving the relationship between white and black Americans is the difference in racial crime rates. No matter how innocent or guilty a stranger may be, he carries with him in public the burdens or benefits of his group identity…

    Estimating the crime rates of racial groups is, of course, difficult because we only know the arrest rate. If police are more (or less) likely to arrest a criminal of a given race, the arrest rate will overstate (or understate) the true crime rate. To examine this problem, researchers have compared the rate at which criminal victims report (in the National Crime Victimization Survey, or NCVS) the racial identity of whoever robbed or assaulted them with the rate at which the police arrest robbers or assaulters of different races. Regardless of whether the victim is black or white, there are no significant differences between victim reports and police arrests. This suggests that, though racism may exist in policing (as in all other aspects of American life), racism cannot explain the overall black arrest rate. The arrest rate, thus, is a reasonably good proxy for the crime rate.

    Black men commit murders at a rate about eight times greater than that for white men. This disparity is not new; it has existed for well over a century. When historian Roger Lane studied murder rates in Philadelphia, he found that since 1839 the black rate has been much higher than the white rate. This gap existed long before the invention of television, the wide distribution of hand guns, or access to dangerous drugs (except for alcohol).

    America is a violent nation. The estimated homicide rate in this country, excluding all those committed by blacks, is over three times higher than the homicide rate for the other six major industrial nations. But whatever causes white Americans to kill other people, it causes black Americans to kill others at a much higher rate.

    Of course the average African American male is not likely to kill anybody. During the 1980s and early 1990s, fewer than one out of every 2,000 black men would kill a person in any year, and most of their victims were other blacks. Though for young black men homicide is the leading cause of death, the chances of the average white person’s being killed by a black are very small. But the chances of being hit by lightning are also very small, and yet we leave high ground during a thunderstorm. However low the absolute risk, the relative risk—relative, that is, to the chances of being killed by a white—is high, and this fact changes everything.

    When whites walk down the street, they are more nervous when they encounter a black man than when they encounter a white one. When blacks walk down the street, they are more likely than whites to be stopped and questioned by a police officer…

    The differences in the racial rates for property crimes, though smaller than those for violent offenses, are still substantial. The estimated rate at which black men commit burglary is three times higher than it is for white men; for rape, it is five times higher. The difference between blacks and whites with respect to crime, and especially violent crime, has, I think, done more to impede racial amity than any other factor. Pure racism—that is, a visceral dislike of another person because of his skin color—has always existed. It is less common today than it once was, but it persists and no doubt explains part of our racial standoff. But pure racism once stigmatized other racial minorities who have today largely overcome that burden. When I grew up in California, the Chinese and Japanese were not only physically distinctive, but they were also viewed with deep suspicion by whites. For many decades, Chinese testimony was not accepted in California courts, an Alien Land Law discouraged Asian land purchases, the Chinese Exclusion Act (not repealed until 1943) prevented Chinese immigration, and a Gentlemen’s Agreement, signed in 1907, required Japan to cut back sharply on passports issued to Japanese who wished to emigrate to California. When World War II began, the Japanese were sent to relocation camps at great personal cost to them. Yet today Californians of Asian ancestry are viewed by Caucasians with comfort and even pride. In spite of their distinctive physical features, no one crosses the street to avoid a Chinese or Japanese youth. One obvious reason is that they have remarkably low crime rates.

    The black murder rate, though it is much higher than the rate for whites or Asians, does not always change in the same way as the white rate. Between 1976 and 1991, the murder arrest rate for black males aged twenty- five and older fell dramatically even though the murder arrest rate for the nation as a whole did not change at all. Apparently, adult black men were becoming less violent. But in some years, such as 1965 to the early 1970s, the black murder rate increased much faster than the white rate. By the late 1960s the black rate was over eighteen times higher than the white one. Then, beginning around 1975, the black rate declined while the white rate continued to increase, so that the ratio of black arrests to white arrests fell to around six to one. From 1980 until the present, the rate at which adult blacks and whites are arrested for murder dropped more or less steadily. By contrast, the rate at which black and white juveniles are arrested for murder increased sharply from 1985 to the early 1990s, with the white rate almost doubling and the black rate more than tripling. Starting in the mid- 1990s, the juvenile rate fell again, almost down to the level it was at in 1985.

    In short, though the gap sometimes widens and sometimes narrows, white and black homicide rates tend to remain different.

  53. http://nymag.com/news/features/pedro-serrano-2013-5/

    Excessive stop and frisk searches in NYC, racial profiling, gender profiling.

  54. Martin says:

    > I don’t know of any recent studies which have compared the race of shooting victims to the race of dangerous attackers on a national level.

    The Sociological Images blog just added this post:
    http://thesocietypages.org/socimages/2014/11/28/chart-of-the-week-63-of-white-people-are-wrong-about-ferguson/

    It seems that blacks are more likely to be killed than whites when not attacking the police.

    I quote: “This question was also studied by sociologist Lance Hannon. With an analysis of over 950 non-justifiable homicides from police files, he tested whether black people were more likely to take actions that triggered their own murder. The answer was no. He found no evidence that blacks were more likely than whites to engage in verbal or physical antecedents that explained their death.”

    Do the articles and statistics linked from that page change your analysis regarding police shootings?

    • Anonymous says:

      That article is very confused. It has some good points and some bad points, but the author doesn’t understand any of it.

      The first point you mention about blacks being somewhat more likely than whites to be killed when fleeing is drawn from the good Vox article, and I strongly urge you to cite that article instead.

      The quote by Hannon has nothing to do police killings. It is about civilians killing civilians, something about which the police have detailed records.

  55. Anthony says:

    This analysis says that in general, the death penalty in the U.S. is either racially neutral, or applied disproportionately to whites. Except in Pennsylvania. The analysis does not look at bias based on the race of the victim, though almost all the numbers are there in the article; but I’d guess that a murderer of any race is more likely to get the death penalty when the victim is white than when the victim is black. Since only 1% of black murder victims are killed by whites, this significantly lowers the proportion of black murderers not getting the death penalty.

    • Anonymous says:

      The percentage of black murder victims killed by whites is 7% (page 13), not 1%. When La Griffe’s Table 1 says “per capita” it means per offender population.

  56. Sun says:

    There is more crime among the African population and there is more focus by law enforcement on the African population. In many cases, a report is filed by the victim who tells the police that an African actually committed a crime. We can see that differnt neighborhoods have higher crime rates. Even anti-racist admit, priori, when they argue that poverty, education, institutionalized racism (and even culture), is the main reason for the high crime rates (whether true or not is another story).

    Men commit more crime. Police focus on men more than women. Women receive less harsh prison sentences for the exact same crime than men. This is sex disparity is wider than racial disparities. The prison population and arrest are disproportionately male over women. Again, way wider than racial disparity. A) No one cares because it isn’t the population people care about B) Biological and cultural arguments are made to why men commit more crimes. Never do you hear the screams of “sexism.” Again, no one cares. C) Everyone is completely fine with more focus given to men due to these realities that men commit more crime than women DESPITE all the discriminatory bias within the justice system (that men receiving harsher prison sentences than women, etc).

    Basically, people aren’t able to apply their stupid line of reasoning to other instances with other population groups to see if its consistent (with their rationale) and, more importantly their morality. Hypocrisy is a valid concern if you’re an egalitarian.

    The fact is that the two don’t butt heads. It rather builds on top of each other. We generalize as a survival mechanism (each one of you do this in your daily life). This generalization does feed into the social consciousness of society and how society targets/focuses on certain groups more than others. But these generalizations are not false. They’re real. Generalizations get falsely called a fallacy because people assume it means everyone when it doesn’t. It’s a broad categorization. Men are taller than women. That’s a generalization. It’s true and accurate. Yes, I have met taller women and there’s women that are taller than men. It doesn’t make it any less true about the generalizations that men are taller than women as a general rule. We account this as a society in our daily lives (in ways you can’t imagine.). Liberals have a very difficult time understanding generalizations because they’re so individualistic. A generalization equals a fallacy to them.

    I just wanted to add my two cents. Thanks.

    • Princess Stargirl says:

      The Men’s right activists are very NOT ok with sentencing disparities. And this is one area they are clearly in the right imo.

      (The sex profiling stuff is more arguable though I oppose it)

  57. Art says:

    You may be missing one important category – how police generally treat people they do not charge with crimes.
    Back in the 90s I drove a cab in Boston for several years.
    Based on that experience my impression is that the officers at Boston police and MA State police enjoy being mean to people when they can get away with it. They will target for mistreatment those who they feel have no means to retaliate. I do not believe this targeting is race-based but it does have disproportionate effect on blacks and immigrants.

  58. Pingback: Outside in - Involvements with reality » Blog Archive » Chaos Patch (#38)

  59. Michael Ejercito says:

    This does beg a very good question.

    If there is substantial racial bias in policing, why would California Attorney General Kamala Harris say the following:

    “Local law enforcement must be able to use their discretion to determine who can carry a concealed weapon”

    If local law enforcement were prohibited from preventing certain people from carrying concealed weapons (such as those not on sex offender registries), then racial and ethnic bias would not come into play. But allowing discretion would mean that racial and ethnic biases can determine whether or not someone could carry a concealed weapon.

    • Nornagest says:

      California’s may-issue stance on CCW means in practice that carrying concealed is banned in urban areas and wealthy suburban counties unless you are a politician or a current or former police officer, but that the upstate redneck counties that sane, civilized urbanites don’t care about can do what they want (which generally resolves to shall-issue). It’s a way of adopting California leftism’s preferred stance on guns in the places that California leftists live, without evoking too much protest from the vast and still-fairly-powerful swaths of inland Western culture that hold sway on the other side of the Coast Range. Harris’s comment should be seen in that light.

  60. Pingback: The Politics of Voting: Four Suggestions | Policy of Truth

  61. Pingback: Let’s See What’s In the News Today (Nov. 30, 2014) | Shaun Miller's Ideas

  62. Pingback: The Discrimination Bias | Battle of the Forms

  63. ontray says:

    Great post, as usual. One question that came to mind while reading the section about traffic stops:

    Suppose for the sake of argument that police are completely terrible at being able to spot people who have drugs in their cars, to the extent that whether a person gets stopped by cops has zero correlation with whether a person has drugs in their car. Suppose also that in this purely hypothetical world, black people and white people drive around with drugs in their car at an equal rate, but the cops are racist and are more likely to stop black people. Then you’d expect to find that black people are stopped at a higher rate than white people, but that when the police search the car for drugs, they find drugs with equal probability. But this outcome seems to match the findings that Scott reported.

    So isn’t one possible (partial) explanation for the observed data just that cops are racially biased, but not good at reliably figuring out which cars have drugs in them? Sorry if I’m missing something obvious here.

  64. Pingback: Lightning Round – 2014/12/03 | Free Northerner

  65. Chuck says:

    “There seems to be a strong racial bias in capital punishment and a moderate racial bias in sentence length and decision to jail.”

    A lot of this depends on how you crunch the numbers. See, for example, La Griffe’s “THE COLOR OF DEATH ROW” http://www.lagriffedulion.f2s.com/DP.htm

    Also, it’s noticeable that many of these studies neglect to control for IQ, a prominent driver of group differences.

  66. Ralph Holder says:

    In predominantly Black or poor communities, policing is the product of the Broken Windows Theory, which is flawed. Police scrutinze every suspected crimnal offense no matter how trivial it may be, thus inviting officers to stop, frisk, interrogate and/or even arrest for alleged misdemeanor crimes. This level of law enforcement scrutiny does not exist in predominantly White middle class neighborhoods. By default, Broken Windows is a thinly veiled racist law enforcement policy, custom, pattern and practice which leads to animosity and resentment by the community. Most Chiefs of Police have suspect hiring practices. They tend to hire someone who looks, acts and even thinks like they do, which is why most departments have predominantly young White males ex-jocks from the burbs to patrol the poor black communities. The police are run by mostly White males who have grown up with racially biased policing practices. Young White cops bring with them the team mentality or attitue of Us v. Them or the learn it from others already on the job. This country needs to drop Broken Windows theory of policing because its broken and flawed policing practics.

  67. Pingback: LINKing you to the best of the web | taylortoowen

  68. Pingback: The Practice of Protest | Andrew Glidden

  69. Pingback: yummyfajitas comments on “Einstein: The Negro Question (1946)” – On Advertise.com Blog

  70. Pingback: yummyfajitas comments on “Einstein: The Negro Question (1946)” – Exploding Ads

  71. Pingback: yummyfajitas comments on “Einstein: The Negro Question (1946)” – Official Offeryour.com Blog

  72. Pingback: Police and community roundup - Overlawyered

  73. Jenna says:

    The most flawed part of this article is the assumption that the data stating blacks are more predominantly represented in crimes is fair & biased. Who are the majority of the people creating this data which was reached by profiling, targeting blacks to begin with. The same data gained from profiling of blacks, from white officers, white prosecutors, white judges, the same white justice system that lets white killers of blacks off fro centuries, the same federal system that agreed blacks were property and not even a human being, the same government that agreed Jim Crow, lynching, segregation was legal, so do you see where I’m going with this. The data is prejudiced from jump at the source so please do not use the same biased data to prove lack of bias. Really. We know what numerous studies show, studies from Harvard, and numerous other sources not just one possible racist with an agenda who wants the injustice against blacks and white privilege to continue. Your argument is as reliable as the source of your data and sorry to say the source of your data is the same anti black friendly source that sanctioned enslaving blacks and making them property to be abused and owned and without rights. So tell me what’s changed. That source is NOT unbiased that’s for sure so your supporting evidence is from a biased, anti black source from the jump so?
    http://www.newrepublic.com/article/119060/michael-brown-studies-show-racial-bias-police-shootings

    http://www.huffingtonpost.com/keith-rushing/the-reasons-why-so-many-b_b_883310.html

  74. Vidur Kapur says:

    Nice post! I’ve done my own research on this, utilising many of the same studies that you did and most of it concurs with what you were saying – sentencing and capital punishment still need improvements.

    Nevertheless, on the minor crimes issue, particularly with respect to marijuana, your confidence level in the belief that blacks are discriminated against when it comes to marijuana appeared to decrease when you looked at that Bureau of Justice report. But, firstly, the report does look slightly outdated. And, secondly, it appears to show that blacks are more likely to have used drugs in the past week, but is this necessarily the case for marijuana? Similarly, the fact that blacks are more likely to use drugs considered to be worse may not necessarily explain the disparity in marijuana arrests.