January 18, 2018

On That Paper Showing Bias in Police Killings

Several people have requested that I respond to the paper “A Multi-Level Bayesian Analysis of Racial Bias in Police Shootings at the County-Level in the United States, 2011–2014” (Here after Ross (2015)). This paper has only ever been cited by 1 academic paper, but in the last few days it has gone viral, receiving over 150,000 views.

The paper is being cited as evidence for bias against Blacks in police shootings. The paper claims to have found that Blacks, armed or unarmed, are significantly more likely than Whites to be shot by police.

Moreover, the relative rate at which Blacks are shot by police within a given county does not correlate with Black crime rates in that county suggesting that racial crime differences do not explain differences in police shooting rates.

This paper has several levels of problems. First of all, it does not tell us why Blacks are more likely than Whites to be shot by police. People citing the paper seem to think that it provides of evidence for racism as an explanation, but it actually does no such thing.

It does not, for instance, do anything to rule out the possibility that Blacks, armed or unarmed, are more likely to attack police officers or to engage in dangerous behavior within the context of a police encounter.

In fact, there is good reason to think that Blacks are more likely than Whites to attack police. After all, Blacks make up a mere 13% of the US population but account for 40% of people who kill police. Thus, Blacks are roughly 3 times as likely as non-Blacks to kill police. This can be taken as a proxy measure of how much more likely Blacks generally are to act aggressively towards police.

This cohere’s nicely with the finding of Ross (2015), which is that Blacks are roughly 3 times more likely than Whites to be shot by police. Police are acting exactly as we would expect them to if they were reacting to the higher rates of aggression against police exhibited by Blacks.

But wait, some may say, didn’t Ross (2015) also find that the rate of violent crime among Blacks within a given county did not correlate with the relative risk of Blacks being shot by police within that county? He did, and, on virtually any narrative, this finding makes absolutely no sense.

If Blacks within a given county are getting arrested more often than average for various kinds of violent crimes than they must surely be interacting with police more often than average as well. Thus, this study is suggesting that the frequency with which Blacks interact with police has no impact on the frequency with which they are shot by police. This obviously does not make sense.

Another way of looking at this is as follows, shooting by police can be divided into two categories: just responses to violent behavior and unjust responses to non violent behavior. Even if you think that police engage in the latter an awful lot of the time, and even if you think that Blacks are disproportionately the victims of it, shooting as a response to violent crime obviously occur as well. Given this, places with more violent crime on the part of Blacks should obviously have more police killings. The only way for this to not be true is if police officers only shoot Blacks for unjust reasons and don’t shoot Blacks when they are actually engaged in dangerous violent crime. This is clearly not plausible.

So, why would a study produce a result that doesn’t make sense? Normally, this implies that one of the measures involved is flawed in an important way. In this case, the relevant measures we should look at were supposed to measure violent crime and police shootings.

Either one of these measures could have problems. Ross (2015) chose to only include rates of assault and weapons related arrests for his measure of violent crime. Ross gives no explanation for this choice, and there is no obvious reason why other kinds of crime, such as homicide, property crime, and, probably most importantly, resisting arrest (!), should have been excluded.

Ross’s measure of police shooting could also be problematic. The database of police shooting that Ross used is crowd-sourced, meaning that the database includes whatever killings random people on the internet happen to add to it. There is no reason to think that this is a representative sample of police shooting, especially not at the county by county level.

Ross uses this database, as opposed to statistics provided by the government, because he thinks that government databases may, themselves, be biased. However, he provides no actual evidence that such bias exists.

In fact, the database that Ross used and the FBI’s paint a very similar macro level picture about police shooting and race. In the database Ross relied on, Blacks accounted for 35% of people shot by police. In the FBI’s data base, they accounted for 32% of people killed by police. Thus, neither seems to be particularly biased at this macro level. (This does not, however, imply that either would be accurate for comparing counties, which is how Ross uses his data-set.)

Thus, the first part of Ross’s analysis fit well with the narrative that police kill Blacks more than Whites because Blacks are more aggressive, against police, than Whites are. His analysis of county level predictors of the relative risk for Blacks vs Whites getting shot by police is not consistent with any plausible narrative and this is likely because both of the measures involved with this analysis are suspect.

It is also worth mentioning that many people are taking Ross’s paper to be a study on police killings. It’s not. It’s a study on police shootings, and that includes shootings which were not fatal. This has two important implications. First, it makes that non-correlation with violent crime even more obviously absurd. Secondly, it means that racial differences in the fatality of being shot by police are not accounted for.

Finally, I would like to leave you with something that Ross does not address: Blacks account for a greater proportion of violent criminals than they do people killed by police. Given this, the more that it is true that Blacks are being killed not because of violence on their part but, rather, because of racism, it is also, necessarily, more true that Blacks who are violent criminals are less likely than Whites who are violent criminals to be killed by police.


Facebook Comments
  • He also said this:

    What I’ll say is that the objection in point
    #1 would have to show that this results is a bias against blacks. Of
    course there will be methodological problems with any data collection
    program, and merely pointing this out doesn’t do anything. We also have
    the NCVS to corroborate the arrest data that the FBI has, and this is
    evidence that no systematic bias occurs in the methodology by which the
    FBI counts homicides.

    Afaik the NCVS is data for arrest, not for police homicides. My argument
    was that the FBI database does not account for all police homicides and
    thus its data minimizes the number of blacks killed by police. If you
    are referring to the CDC data sometimes used in complement or as a
    substitute to that of the FBI, it is likewise only partial, merely
    tracking a handful of states. And even by FBI/CDC data, unarmed blacks
    are shown to be killed in a disproportionate number. As for the relation
    with the arrest rate, I discuss that further below.

    Also, [citation needed] for blacks being targeted by the police FOR being black.


    It is well documented that police intervene disproportionately with
    racial and ethnic minority individuals. Studies have confirmed, for
    instance, the disproportionate representation of minorities among
    subjects who are arrested or ticketed,1 searched,2 stopped as
    pedestrians or drivers,3 or otherwise surveilled.4 Disproportionate
    intervention with minorities has also been documented with regard to
    police use of force.5


    1. Samuel Walker, Cassia Spohn, and Miriam DeLone, The Color of Justice:
    Race, Ethnicity and Crime in America, 5th ed. (Belmont, CA: Wadsworth,

    Tammy Rinehart Kochel, David B. Wilson, and Stephen D. Mastrofski,
    “Effect of Suspect Race on Officers’ Arrest Decisions,” Criminology 49,
    no. 2 (2011): 473–511; Lynne Langton and Matthew Durose, Police Behavior
    during Traffic and Street Stops, 2011,Bureau of Justice Statistics
    Special Report (Washington, D.C.: U.S.

    Department of Justice, September 2013).

    2. Christine Eith and Matthew R. Durose, Contacts between Police and the
    Public, 2008, Bureau of Justice Statistics Special Report ( Washington,
    D.C.: U.S. Department of Justice, October 2011); Andrew Gelman, Jeffrey
    Fagan, and Alex Kiss, “An Analysis of the New York City Police
    Department’s “Stop-and-Frisk” Policy in the Context of Claims of Racial
    Bias,” Journal of the American Statistical Association 102, no. 479
    (2007): 813–823.

    3. James E. Lange, Mark B. Johnson, and Robert B. Voas, “Testing the
    Racial Profiling Hypothesis for Seemingly Disparate Traffic Stops on the
    New Jersey Turnpike,” Justice Quarterly 22, no. 2 (2005): 193–223;
    Richard L. Lundman and Robert L. Kaufman, “Driving While Black: Effects
    of Race, Ethnicity, and Gender on Citizen Self-Reports of Traffic Stops
    and Police Actions,” Criminology 41, no. 1 (2003): 195–220.

    4. Albert J. Meehan and Michael C. Ponder,“Race and Place: The Ecology
    of Racial Profiling African American Motorists,” Justice Quarterly 19,
    no. 3(2002): 399–430.

    5. Douglas A. Smith, “The Neighborhood Context of Police Behavior,”
    Crime and Justice, vol. 8, Communities and Crime, eds. Albert J. Reiss,
    Jr., and Michael Tonry, (Chicago, IL: University of Chicago Press,
    1986), 313–314; William Terrill and Stephen D. Mastrofski, “Situational
    and Officer-Based Determinants of Police Coercio

    For #2, well, we can see whether police
    psychology has any effect on the studies that were done on reaction time
    in dummy tests. Reaction times were slower to shoot black perpetrators
    in controlled tests and this is likely to be because blacks are
    perceived as part of a protected group. This suggests that his just-so
    theory for a cyclical runaway mechanism is incorrect.

    These results are discussed in the link above. It is from one recent
    study that come after several others (including one by the same author
    Correll from 2 years earlier) that showed just the contrary. That is not
    to say that one is more wrong or right than the other. What the author
    found is that while police officers still held stereotypes about blacks
    being more dangerous and aggressive, the current climate and the
    pressure put on officers in the case of shooting unarmed blacks made
    them less “trigger happy”.

    These results prove that movements such as Black Lives Matter can make a
    difference in reducing police shootings since media attention and
    public outrage influence officers – regardless of their personal bias
    towards black – to be more thoughtful before shooting a suspect.

    It also suggests that while such a phenomenon is thankfully likely to
    become more and more widespread as the public become more sensitive
    about police shootings of unarmed civilians, it is unlikely to be
    replicated in police departments where racism and a sense of impunity
    run high. If you read the report on the Ferguson case I have linked
    before, I am sure you will agree that this particular department fits
    this description. While this particular department may not be
    representative of all American police officers, there are certainly
    still issues of widespread racism in departments accross the country
    including in major cities, as the recent investigation of the San Francisco Police Department has shown.

    Regardless, according to the FBI’s statistics which might not reflect
    the entirety of police homicides, the fact remains that unarmed blacks
    are killed much more often by police by than any other ethnicity (last year, 30.5% of black people killed were unarmed, while only 16.1% of white people killed had no weapon).
    If you believe that blacks’ higher crime rates can explain this there
    still is a problem here. A higher crime rate would only explain a higher
    chance of encountering the police, not being killed by it. What this
    suggests is the police is effectively more likely to kill an unarmed
    black suspect. Which brings us to our next point:

    #4 ignores the fact that “unarmed” =/=
    “non-violent” or “compliant”. For instance, Mike Brown was unarmed, but
    the officer was justified in shooting Mike Brown in this case. And the
    implication that BLM is somehow some scholarly movement that used this
    peer reviewed finding (from NOVEMBER 2015) when their inception was
    based clearly off of visceral reaction to Trayvon Martin and Mike Brown
    is silly. This is a new finding, being of course stretched and
    misinterpreted as if “unarmed” invariably means correctly compliant or
    peaceful, and is now being pushed to the forefront as a means of

    I would say two things about this.

    The first is that the distinction you make between “unarmed” and
    “uncompliant” is hardly ever relevant. A non-homicidal criminal does not
    deserve to be killed unless he or she represents a clear threat to an
    officer or other civilians’ lives. Police is the United States is incredibly trigger-happy:
    In 2014, U.S. police officers killed 100 times more people than in
    China, a country with a population 4 times larger. This is a pressing
    issue regardless of whether you believe blacks are unfairly targeted or
    not. Movements such as Black Live Matter bring visibility to this issue
    and have made it one of the most discussed topics in the past couple of
    years. It wouldn’t be the first time that a movement started by blacks
    and for blacks ended up benefiting the whole country (for example the
    civil rights movement, which started social reforms that benefited both
    minorities and whites).

    Dismissing the “visceral reaction” to Trayvon Martin and Mike Brown as
    “silly” ignores the background against which these events took place.
    Even if you argue that the officer was justified in shooting Brown and
    that the rumors that Brown had his hands in the air were unfounded, the
    event served as a catalyst for years of power abuse and racism which are
    document in the report on the Ferguson PD. The episode of the French
    Revolution that led to the abolition of feudalism, also started because
    of false rumors.
    This merely provided an occasion to expose and fight an unjust system
    of exploitation. There is a real problem of police brutality (killing of
    unarmed civilians, regardless of race, although it disproportionately
    affect blacks) and bias in policing practices in the United States, and
    every event that can lead to a more thorough investigation and
    discussion of these problems should be publicized.

    • Jeff

      I’ll only address the few points that have substance.

      “Studies have confirmed, for instance, the disproportionate representation of minorities among subjects who are arrested or ticketed,1 searched,2 stopped as pedestrians or drivers,3 or otherwise surveilled.4 Disproportionate intervention with minorities has also been documented with regard to police use of force.5”

      This is because blacks do more things which will get them stopped by the police.

      “What the author found is that while police officers still held stereotypes about blacks being more dangerous and aggressive”

      This is because they are. Stereotype accuracy is one of the most robust findings in social psychology.


      “the current climate and the pressure put on officers in the case of shooting unarmed blacks made them less ‘trigger happy'”

      This is entirely consistent with the original study that was reported.

      There are plenty of homicidal unarmed people. Unarmed people can also be extremely dangerous. Someone who is unarmed can easily become armed by taking the officers weapon.

      Police in America being more violent than other countries is an interesting point. If that is the case, then something should be done about that. However, this is completely irrelevant to whether the American police system is systemically racist. Also, I wouldn’t be surprised if racial diversity in America caused the American police to become more militaristic. Massive amounts of evidence demonstrate that racial diversity does have many other undesirable effects. Police militarization could just be an outgrowth of that.


      In fact, here is an article from a left wing site that seems to indicate that is the case.


      I know that the article doesn’t say “racial diversity caused police militarization” and they ultimately would not want any readers to come to that conclusion. However, that is the obvious conclusion that I came to.

  • This is a good news article about resisting arrest rates in San Francisco.

    “From January 2010 to April 24 of this year, law enforcement officers cited suspects with resisting arrest 9,633 times in cases where the suspect was not charged with a felony. African Americans accounted for 45 percent of those cited, even though they make up just 6 percent of the city’s population.”

    “Whites, who make up roughly half of San Francisco’s population, made up 39 percent of those cited for resisting arrest. Asian Americans, who make up roughly a third of the population, accounted for just 3 percent of those cited for resisting arrest. Latinos are not broken out as a separate demographic and instead are generally included among whites.”


  • Bayesian Dissident

    I have reproduced this study using the author’s data and R code. In addition to (and reinforcing) what’s already been said, its most fundamental problem is its lack of validity – the regression coefficients for for the predictor variables don’t mean what people have inferred. The trick is in the dependent variable – ‘racial bias in police shooting’ – which is operationalized as a ratio of the percentage of blacks who are shot/killed & unarmed to the percentage of whites who are shot/killed & unarmed. (Technically, the ratio of the binomial probability of being black & unarmed & shot be police to the binominal probability of being white & unarmed & shot by police). So, when the author states, “there is no consistent relationship between the race-specific crime proxies (neither assault-related nor weapons-related arrest rates) and racial bias in police shootings”, this does NOT mean that there is no correlation between the frequency of blacks being shot by police (whether armed, unarmed, or both) and crime across counties or across the US as a whole – THERE IS.

    If you regress not a Black/White ratio but rather the frequency of blacks who are shot (armed, unarmed, or both) on predictor variables that include black-specific crime rates such as assault (which is used in this study), the result is a strong (in the sense of frequentist p-values) correlation. So, the intuition that the rates at which blacks get shot, (even unarmed) has something to do with black crime is confiromed using the authors own data. To take just one quick example, using the author’s own data. I regressed the total frequency (not the B/W probability ratio) of unarmed blacks shot or killed by police per county on just 3 independent variables: log(population), log(Percent of Pop. Black), and Black Assaults. Here are the results:

    Estimate Std. Error t value Pr(>|t|)
    log(TOT_POP) -7.359e-04 3.418e-02 -0.022 0.983
    log(BlackPrcnt) 7.281e-02 3.228e-02 2.256 0.025 *
    AssaultsBlack.sum 7.209e-04 8.802e-05 8.190 1.55e-14 ***

    But this study doesn’t address that question. Instead, it focuses on how
    Running simple general linear regression, where the dependent variable is NOT the ratio, but rather, the frequency of blacks shot (armed, unarmed, or both)

    • Kris Polk

      Thanks a lot for this, very interesting.
      I’m not very good with statistics; how does this argument:
      connect to what you’ve discussed regarding the correlation between police killings and violent crime rates?
      Originally I saw it and, knowing that killings of unarmed people are so rare, it could be indicative that some police departments are better at preventing justified killings than others, with better tactics. Of course it doesn’t prove anything about racial bias.
      But it was interesting to see that, at least according to their data, in the 60 largest police departments there appears to be no correlation with violent crime rates.
      What am I missing?

      • Bayesian Dissident

        Hi Kris. As you suggest, this HuffPo article/study you mention doesn’t pertain to racial bias at all, nor does it address police shooting of civilians who are specifically unarmed. But it is still interesting in its own right. I recently downloaded and cleaned up the FBI’s crime data for metropolitan statistical areas, but haven’t combined it yet with police shooting data (a very tedious process given discrepancies in how the cases/towns are labelled). But I do plan to run the regression myself once I have a couple hours of free time. Without knowing much about this data set, one obvious reason why you don’t see a correlation there is THE LACK OF VARIATION IN THE SAMPLE. Again, just speculating, but this might be a reason. As an analogy, think of NBA basketball players and height. If you restricted your sample to people who are say, over 6’5, you wouldn’t see much correlation there between height and whether or not somebody plays pro-basketball. After a certain minimum height is reached, being taller doesn’t help you that much (diminishing returns and what not). If you expanded your sample, however, you’d see a clear correlation – basketball players are a lot taller than the total pop. average. There might be some threshold effect, such that, if you are just sampling on one side of the threshold, the co-variance seems to disappear. But then again, what would that threshold be in this case? I’d like to see the actual data and the correlation coefficient. I’d like to rerun the correlations with a larger sample size, including more cities/counties, etc.

        The infographic/scatterplot is misleading because of the two superimposed y-axes, and because they aren’t being measured in standard deviations. But, aside from all that, it might be that there isn’t any correlation after all. Maybe its all noise, or there’s a significant outlier that throws off the correlation. I try not to be dogmatic. Maybe some police departments do have better tactics, or they got lucky, or they retreated, or … ? I certainly wouldn’t be opposed to the police shooting fewer people if that can be accomplished without undermining their other objectives of public safety.

        • Kris Polk

          Thanks for that explanation. One reason I asked is that Project Zero, basically a much more polished version of Black Lives Matter, that at least claims to focus more on police shootings in general rather than just racial inequities, and this was probably the most convincing talking point I ever saw either one make.
          It also had some resonance with my own personal experience. Albuquerque and tucson are largely mirror images of each other in terms of demographics, politics, race, crime levels, etc., and even economy, but Albuquerque has far, far more fatal shootings, including of White people, and that and some anecdotal cases certainly made me *open* to the possibility that some police departments really are rotten and less effective at responding non-lethally to erratic and outrageous behavior, but NO ONE has suggested to me with any real argument that there are serious differences in how this process would diferentially affect racial groups.

  • Bayesian Dissident

    Readers may have come across this recent NTY article: “Study Supports Suspicion That Police Are More Likely to Use Force on Blacks”. The original report can be found here: http://policingequity.org/research/1687-2/

    The article reports that: “African-Americans are far more likely than whites and other groups to be the victims of use of force by the police, even when racial disparities in crime are taken into account.”

    The original report is interesting. The Tables speak for themselves. The report states: “… benchmarking to arrests reduces Black-White disparities, but does not eliminate them. **Table 5 shows that benchmarking to violent Part I arrests reverses the direction of the Black-White gap.**”

    1. The Achilles Heel of this study is mentioned in the introduction: it does not take into account the dynamic sequence of events during the encounter that lead to the use of force. The obvious confounding variable here is the disparity in rates of resisting arrest. Running away from cops is stupid. Attacking cops is stupid. Blacks do both more than whites.

    2. The authors actually report a disparity in the use of lethal force that disadvantages whites, after benchmarking for population size and rates of violent crime. Needless to say, this is not the headline that the NYT chose to run with, and is just barely acknowledged by the authors of the report, who are not doing scholarship, but rather activism – i.e. pushing the data to support a preconceived ‘social justice’ narrative.

  • blackacidlizzard

    ” Ross (2015) chose to only include rates of assault and weapons related arrests for his measure of violent crime.”

    That alone seals it. Something that ridiculous and dishonest clearly has intent behind it.

  • Bayesian Dissident

    This paper reported in NYT received a lot of attention this week: “Surprising New Evidence Shows Bias in Police Use of Force but Not in Shootings”
    The NBER working paper is here:

  • waffleironmarch

    This poisonous article from Vox has been making the rounds recently again. It’s so melodramatic I can’t believe that it is taken so seriously

  • Bayesian Dissident

    For the sake of posterity, I will mention what I regard as the BIGGEST PROBLEM with this study (that I forgot to mention in my earlier posts): it commits the ECOLOGICAL FALLACY (see https://en.wikipedia.org/wiki/Ecological_fallacy). Put simply, correlations **or lack thereof** between aggregate-level variables (counties in this case) cannot be inferred from correlations **or lack thereof** between individual-level variables. This alone completely invalidates any attempt to conclude, on the basis of this study, that crime and police shootings are uncorrelated. The study shows no such thing. So when Sean Last writes:
    But wait, some may say, didn’t Ross (2015) also find that the rate of violent crime among Blacks within a given county did not correlate with the relative risk of Blacks being shot by police within that county?