Monday, 9 November 2015

Chisala and Powerpoint Publishing


Just because research hasn’t been published doesn’t mean it’s right.

Chandra Chisala has continued a line of argument based on results from some schools-based research in the UK and USA. I commented on his first post here:

In brief, it included unpublished work on particular schools chosen because African students were doing particularly well, with hard to track down references and incomplete methods sections.

Chisala has now posted a Part 2 which gives unpublished results from Seattle schools. Compared to Part 1 (which included national UK exam results) there is even less detailed material available. Everything said may be true, but the exhibits are not available for inspection, so it repeats the style of his first article, in which it is very difficult to trace references back to the published source. Call it “Powerpoint Publishing”.

Chisala summarises his argument thus: Remember our goal. We only need to show that blacks in Africa would have a higher average IQ than native black Americans if they were moved from Africa to America since the African environment clearly depresses IQ, as both environmentalists and hereditarians agree in principle. This result would mean that whatever “problem” the black Americans have that result in such a large and intractable IQ gap with whites and other groups, has nothing whatsoever to do with the genetic evolution of races, especially since they even have more white genes than Africans. It is not their sub-Saharan African (black) genes that are responsible for their chronic academic under-achievement; it has to be a factor that is endemic to African American history.

There are many assumptions in this line of reasoning. The main one is the assumption that recent African migrants to the USA are a representative sample of Africans in Africa. This is not unreasonable, but one would need a better understanding of the family backgrounds to be confident about it. The second assumption is that the African environment is universally bad. In fact, Africa is propelling more of its people into better standards of life, by the open-market business methods which work everywhere. Health is improving, as is school attendance. Many African countries are showing a Flynn effect. Kenya has gained 8 IQ points per decade, and the global picture is positive, though convergence in maths will be a long time coming (Meisenberg and Woodley. Intelligence 41, 2013 808-816). The third assumption is that we can sort out the relative contributions of nature and nurture by looking only at Africans. Asians also suffered poverty in the post colonial period, so that recently enriched nations like Vietnam are also relevant. Poor malnourished Vietnamese refugees fled Communism decades ago (unclear whether they were from cognitive elites). The fourth assumption is that “factors endemic to African American history” are unique. As I keep explaining, most African slaves ended up in Brazil, so the achievements of Africans in Brazil are also relevant, and a particularly interesting test case, since there was a much more relaxed attitude to intermingling.

There central element in this article is a Powerpoint presentation of Seattle schools results, which begins with a disclaimer that using “language spoken at home” to infer race does not necessarily map onto genetic groups. The education authority says Please note: this is an important and critical limitation of this study. The slides make it clear that:  The “Admission Form” also collects specific primary race data for Asians and Native‐Americans Americans and ethnicity for Hispanics Hispanics per state regulations. This data is not collected in specific detail for Whites or Blacks / African‐ Americans.  So, we don’t have specific ethnic details on the key groups being compared.  A supplementary table in a published paper might provide further data to assist in estimating this potential error term. It would have been helpful to have included this disclaimer in the article.

Chisala says: The fact that these are only group pass rates (on mathematics and reading) does not matter for purposes of ethnic IQ comparison since the pass rate positions correlate perfectly with expected mean IQ score ranks of the groups before disaggregation (that’s the same logic we were using for GCSE pass rate comparisons, especially when mathematics is included).

The problem with using pass rates rather than actual scores is that if pass rates are raised by making tests easier, then it will appear that group differences are reduced. If any test gives generous marks to students who simply attempt answers to questions, and indicate knowledge of very basic terms, then real differences in competence are obscured. The Seattle data are based on overall district pass rates of up to 70%, which is understandable for an education authority, but loses a lot of detail. Nonetheless, most findings impart some information, and taken at face value the results are interesting. In the Bayesian spirit of “hunt the submarine” one should try to work out the truth of an important matter using the best available data, however slight. Chisala has potentially found something interesting, something which opens the door to new hypotheses. Looking at language differences gives additional information. However, there is also a potential distortion (apart from not knowing how language maps on to race) which is that brighter children pick up new languages more quickly, such that those in the “not needing to go to English classes” are very probably brighter, or conversely, have had much longer to learn the language (another detail it would be good to have).

For example, first including and then excluding those with poor English, pass rates for Maths are:

Asians and Pacific Islanders: Chinese 87-91%, Japanese 88-89%, Korean 88-87%, Vietnamese 75-82%, Filipino 67-72%,  Indian 66-74%, Samoan 39-44%

Hispanic and Latinos: Hispanic English Speakers 58-58%   Hispanic Spanish Speakers 39-54%

Black and African Americans: Amharic 51-62%,  Tigrigna 46-58%, Oromo 39-53%, English 36-36%, Somali 28-47%

Comment: Overall, those who need English language teaching are less able to do maths. African American (Black English-speakers in this classification) do badly on Maths, but in the same sort of range as Samoans and Somalis. Sample sizes are generally reasonable, but rather low for Amharic (143), Tigrigna (106) and Oromo (94)  and their representativeness is unknown.  Chinese and Vietnamese do very well, despite coming from previously very poor countries. Personally, I do not see a clear pattern of environmentally deleterious effects here.

The presentation also gives a measure which includes attendance and discipline, where 3 is the district average and 10 means not likely to complete high school. By this measure the following are at risk: Somali 5.5, Samoan 5.5, African Americans 5.4, Spanish Speaking Hispanics 5.3, Oromo 5.1 which suggests that a mixture of genetic and cultural elements are involved. African descent is still a partial but plausible contributing factor to poor school progress.

Chisala agrees that proper representative samples would be the most informative, but in the absence of those is using some available results on individual school districts, and extrapolating from those. Although this is a weaker method than proper sampling, it can sometimes achieve informative results. For example, if high achievement were found in disproportionate numbers among African students, relative to the number of Africans in the world then one could estimate, from that extreme high-performing group, the likely average intelligence from the population from which they are drawn. This is a particularly useful method if there is no other ability data, or you  want to trust only data based on open competition, like chess tournaments where players get Elo rankings based on win/lose scores.

If the number of very bright Africans is higher than would be expected from those, say, 2 standard deviation above a mean IQ of 80, then the mean of 80 is called into question. The real mean is likely to be higher, thus accounting for the larger than expected number of those achieving +2 sigma performance. Therefore, get a good estimate of peak African achievement, divide by African population, consult normal curve statistics, work out the implied mean.

Chisala has found some high performing Africans in the UK and the US. Good. What does this tell us about the population from which they are drawn? At the moment, not enough to conclude that the mean of 80 is wrong.

Perhaps I need to spell out my concerns more clearly. I don’t think one can draw firm conclusions from this sort of reporting of results. It is simply not good enough to say the research was not intended for publication. It cannot be evaluated until we have been able to read it properly. (Just seeing the full report would be enough: it does not need to be in a journal. I have emailed the Seattle School dept asking if they have any reports they can send me).  It is premature to rush to conclusions about what this means for various hypotheses when we haven’t got sufficient detail on the basic results.


  1. "Chisala agrees that proper representative samples would be the most informative, "

    I'd say. Too bad Chisala ignores the issue of selective migration in all of his arguments (not to mention it appears he hasn't the foggiest idea about matters like regression to the mean).

    Without knowing how representative these samples are of their home countries, there's simply no case made.

    1. We need to get him to read your explanations of regression to the mean!

  2. Much research is rubbish. Almost all research in schools is rubbish.

    Still, it's rather brave of him to hypothesise that something may have happened to African-Americans or their ancestors to make them a peculiarly cognitively ungifted lot.

  3. Human cognitive long term development is the same or similar than other animals***

    I mean, animals have long term development,

    - baby
    - kid
    -adult (because adolescence is absent among them, seems, i see in the documentary a long... long time)
    and older

    Seems cognitive development of all nonhuman animals to obey a long term stable and predictable behavioral pattern that correlates with their evolutive development, life cycle.

    In part may could.. could (now in echoes) explain the disruption of schizophrenia for example (as well later creativity), when brain, generally, become mature, in the 20's years old.

  4. Khan sleek looking all kinds of stuff I could see if they got that you I write with the spelling it 70-243 Exam Questions will be here to be really paranoid you know there'd be a learning curve getting used to it but again I mean so many people have multi-touch devices .

  5. for you one of the things I really like about nature box the fact that once you sign up for the service 70-246 Exam Questions you get the triumphant a different things that I didn't know I would like flight for example my personal favorite now are these Santa Fe corn statistics .

  6. Hi, There is one great publishing site I have come across. Visit it here