June 29, 2017

Cultural Bias on IQ Tests

One frequent lay argument against IQ test results is that they are culturally biased in favor of whites. This has some superficial plausibility, especially with some “obviously” biased questions. But intelligence researchers generally no longer treat bias as a serious argument, and for fairly straightforward reasons.

Views of the Field

A 2008 paper entitled “Survey of opinions on the primacy of g and social consequences of ability testing: A comparison of expert and non-expert views” detailed the results of researchers Charles Reeve and Jennifer Charles contacting 99 intelligence “experts” and “non-expert” applied psychologists’ views on multiple questions about cognitive abilities tests.

Subjects Not Biased Biased
“Experts” 73.3% 13.3%
“Non Experts” 51% 29.2%

They found that 73.3% of the “experts” believed that Cognitive Ability Tests (IQ tests) did not have any racial bias, as did 51% of the “non-experts”, compared to 6.7% of “experts” and 29.2% of “non-experts” who believed the tests were racially biased.

As far back as 1982, a panel from The National Research Council and The National Academy of Sciences conducted an investigation of bias in mental testing, and concluded that cultural bias explained virtually none of the racial variance in IQ scores.

In 1996, the American Psychological Association formed a panel, and it resulted in the publication entitled “Intelligence: Knowns and Unknowns” which concluded “Considered as predictors of future performance, the tests do not seem to be biased against African Americans.”

In 1994, following the release of The Bell Curve, 52 intelligence researchers authored a document entitled “Mainstream Science on Intelligence” laying out 25 postulates that they believed to represent mainstream science on intelligence.

The 5th postulate was “Intelligence tests are not culturally biased against American blacks or other native-born, English-speaking peoples in the U.S. Rather, IQ scores predict equally accurately for all such Americans, regardless of race and social class. Individuals who do not understand English well can be given either a nonverbal test or one in their native language.”

All of this doesn’t necessarily mean IQ tests aren’t culturally biased, but it should give one pause when people from the Universities say that something that everyone wants to be racially biased – isn’t racially biased.

East Asians

One natural experiment on the “cultural bias” of IQ tests involves East Asians. East Asians actually score higher than Europeans and European Americans in all sorts of environments.

This is seen in 14 studies around the world where East Asians live in European-majority countries, including adoption studies, a whole host of IQ test data and IQ test proxies in the United States, international IQ tests and international test scores, and the idea that this is down to genetics is supported by the frequency of specific genes associated with intelligence, and in rates of nearsightedness which is associated with nonverbal IQ and has a heritability of 0.42.

Subjective Analysis of “Cultural Bias”

A straightforward way to analyze test bias is to assemble a bunch of “experts” in various fields, show them questions, have them rate which ones they think are more or less biased, and then compare these ratings to the actual results.

A paper in 1987 by Frank McGurk and Arthur Jensen documented just that, and described their method thusly:

“A panel of 78 judges, including professors of psychology and sociology, educators, professional workers in counseling and guidance, and graduate students in these fields, were asked to classify each of the 226 test items into one of three categories: I, least cultural; II, neutral; III, most cultural.”

In the results, they found that the black and white test score difference was significantly larger for questions which were judged to be the least cultural and the lowest for questions judged to be the most cultural.

Predictive Validity

This is a “pinch test” for the validity of these tests – do these tests show higher white scores disproportionate to life outcomes? Well, the College Board (which is in charge of the SAT) tracked test-takers and college GPA, and found that the SAT scores actually slightly overpredicted black and hispanic SAT scores relative to whites.

Similarly, in the paper “Role of mental abilities and mental tests in explaining high-school grades”, Jeffrey Cucina found that IQ tests administered in high school slightly overpredicted black and hispanic GPAs relative to whites.

But in the differences in both cases were not substantial.

Rank-order difficulty

The McGuirk study mentioned above also looked at the questions both groups got wrong. In it they found that blacks and whites had exactly the same order of hardest to least difficult questions.

Moreover, the correlation between how often whites got a question wrong to how often blacks got a question wrong as a function of total score was 0.95.

Earlier studies from the 1970s found scores ranging from .94 to .99.

In terms of which questions blacks and whites find the most difficult, it’s almost exactly the same. If there were biased questions against blacks, you would expect those questions to be much harder for Blacks than for Whites. But this is not what we see.

 

Sources

National Research Council on Test Bias:

http://openpsych.net/forum/attachment.php?aid=440

Intelligence researchers on bias: (Table 1, question 36, which is the 10th question down)

http://www.sciencedirect.com.sci-hub.io/science/article/pii/S0160289608000305

Intelligence: Knowns and Unknowns

http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.134.1282&rep=rep1&type=pdf

IQ tests in high school slightly overpredict black and hispanic GPA:

http://www.sciencedirect.com/science/article/pii/S016028961500166X

SAT to GPA correlations by race (Table 9):

https://research.collegeboard.org/sites/default/files/publications/2012/8/researchreport-2012-6-validity-sat-predicting-cumulative-gpa-major.pdf

SAT slightly overpredicts black and hispanic college GPA, relative to a white SAT to GPA prediction:

https://research.collegeboard.org/sites/default/files/publications/2012/7/researchreport-2001-6-differential-validity-prediction-college-admission-testing-review.pdf

Table 4 covers multiple test analyses from the 1970s:

http://lp.wileypub.com/HandbookPsychology/SampleChapters/Volume10.pdf

The McGuirk Study:

http://arthurjensen.net/wp-content/uploads/2014/06/Black-White-Bias-in-Cultural-and-Noncultural-Test-Items-1987-by-Arthur-Robert-Jensen-Frank-C.-J.-McGurk.pdf

Mainstream Science on Intelligence:

https://www.udel.edu/educ/gottfredson/reprints/1997mainstream.pdf

Facebook Comments