On best understanding Nisbett and co.

 

Occasionally, curiosity leads one into byways, and thus disinters shards of memory lost in the grass of our neglect, letting ghosts speak. Here is the chain: a few days ago Greg Cochran posted a link to a 2007 opinion piece by Eric Turkheimer in which the latter said that “The important questions about the role of genetics in the explanation of racial differences in ability are not empirical, but theoretical and philosophical”. This suggested that neither a positive nor a negative empirical result would have an important effect. On the contrary, if no genetic investigation can explain intellectual differences between racial groups then that destroys the genetic hypothesis. It will be wrong, end of story. I also noted that his essay was written in 2007 so it was possible he had changed his mind.

Later that evening I realised I ought to check up on this possibility, and looked at Turkheimer’s personal website, and in which he lists a recent publication in the September 2012 American Psychologist DOI: 10.1037/a0029772 Nisbett, Aronson, Blair, Dickens, Flynn, Halpern, Turkheimer “Group Differences in IQ Are Best Understood as Environmental in Origin”.

It is a minor gripe, but “best understood” strikes me as a curious phrase. What is wrong with “Group differences are environmental in origin”? That might be true, or partly true, but at least determining it depends on fact. Even “environmental explanation is best fit with all the data” would have been clearer.

Anyway, there is much to comment on in this paper, which is written by distinguished authors in the field, but my attention was drawn to one line of text: Gains in sub-Saharan African countries of 0.50 to 0.70 SD in response to a few months of Western-style education have been reported for heavily g-loaded fluid intelligence tests (McFie, 1961). This dramatic claim is supported by far the oldest reference in the paper. Apart from another reference from 1993 all the rest are from the present century. Nothing wrong with a reference being old, but I decided to look it up. Here is the abstract:

SUMMARY Twenty-six African boys entering technical school were given a series of intellectual tests involving verbal, numerical, pictorial and constructional material, and also an ' abstraction ' test (Weigl) and ' memory for designs ' (Terman-Merrill) . None showed qualitative differences from English subjects in their performance on verbal, numerical and abstraction tests. On the non-language tests their performance was slower than would be expected of English subjects, and they showed differences in their innaccurate orientation of drawn and constructed designs. At the conclusion of two years' technical training, the subjects were retested on the same material. Significant changes were increases in scores on the non-language tests. These were associated with increased speed and accuracy of orientation, and also apparently with a more ' synthetic ' approach toward visual material. It is suggested that an ability which may be poorly developed under these cultural conditions, and which may be increased by appropriate educational methods, is that of perceiving visual material as a whole (or ' Gestalt ' perception).

From this abstract we see: 1) the sample size is small 2) the study was carried out in one country 3) the sample is specific, in that it is students entering a technical school who may well be above the local norm, and 4) the extent of technical training was two years and four months, not “a few months”.

When one reads the paper, it is apparent that the boys are between 16 and 19 and are entering what in Uganda would be tertiary training prior to taking up technical occupations like carpentry, motor maintenance and machine fitting. Retesting took place after 2 years and about 4 months. The 7 tests were in fact McFie’s adaption of standard tests, including putting in some new material and utilising some new scoring methods, and significantly increasing the time taken to complete items. This makes interpretation difficult. One can talk about change after education, but not locate the result properly in conventional intelligence testing. The paper is an indication of an effect, no more, carried out by a thoughtful researcher exploring some possible effects. Despite the very small sample, he includes some factor analytic results. These will not be stable when n=26 and tests=7, but they are a welcome addition as a statement of intent for later work.

Here are his descriptions of the tests:

The main group of tests corresponded to subtests of Wechsler's (1944) scale, with questions omitted which were obviously related to European cultural experience, and some added from other scales (e.g., the Terman-Merrill 1937) :

A.--Comprehension : Six questions : maximum two points each.

B.-Similarities : Seven pairs ; maximum two points each.

C-Arithmetic : Six questions, one point each ; two questions, two points each.

D.-Picture Description : Four photographs of scenes from African life ; 1 point for enumeration or description, 2 for synthetic interpretation.

E.-Picture Arrangement : One example and four test stories, taken from a popular African cartoon strip : maximum three points each.

F.-Block Designs : Wechsler's designs, 1-6, with time limits extended to 2' for designs 1-3, and 3' for designs 4-6.

G.-Memory for Designs : As Terman-Merrill IX, 3, but scoring two, four or six points according to quality of reproduction of each design.

H.-Weigl's Sorting Test : As described by Weigl in 1927 : pass or fail recorded according to ability to sort both ways (cf. McFie and Piercy, 1952).

Criticisms are often made about the testing of intelligence in Africa. McFie was clearly giving his subjects every chance to score well by making the material culturally appropriate, which is good. He severely cut down the number of test items, which reduces reliability and range significantly; he made his own decisions on cultural bias which may or may not have been correct; and he extended the time limits, which damages the interpretation of results. The better procedure would have been to have recorded times up to some generous limit, and then shown how the results were affected by using the generous as opposed to the official time limits. McFie also changed the marking system. This is regrettable. Better to have kept the original and then compared it with his more generous system. The tests have lost their integrity, and can only serve as being broadly indicative.

By the way, in the heated atmosphere of debate about African intelligence, if such a paper was being put forward as a proof of low African intelligence it would rightly be rejected as flawed. The tests are not complete, include other material, have been altered by the examiner, and have had their scoring system and timings altered. This still allows before and after comparisons, but the results only weakly relate to the g loaded originals.

Here are the main results in a screen grab:

It is hard to make a judgment about what the results signify in terms of overall intelligence. Even if one looks only at the tests which most resemble full Wechsler subtests: Comprehension, Similarities, Arithmetic and Block Designs; the original test results are probably equivalent to Full Scale IQ 85 and they rise after over two years of technical education to FSIQ 89. This change is within the usual 4 point retest difference, but it is certainly suggestive of a training effect. If we knew more about the selection of the sample we could judge how this compared to the local norm. For example, if they are the brighter students, chosen for tertiary training, one might expect them to be one standard deviation above the national mean. That would correspond to being drawn from a population mean of IQ 70. If they are run of the mill students then the local average is IQ 85 which is very much better than average results for sub-saharan Africans. As always, the representativeness of samples is crucial when trying to estimate the mean of a bell curve.

Once these IQ results have been spelt out, which McFie did not do, his comment: “None showed qualitative differences from English subjects in their performance on verbal, numerical and abstraction tests” becomes particularly interesting and informative. Right from the start, the students knew what was expected of them, and understood the concept of the tests. They did not have an operating system incompatibility, although there was a power difference. (I make this point because some ill-advised commentators try to suggest that there is an African way of thinking which is fundamentally different from the European way of thinking, as profound as the difference between Microsoft and Apple operating systems). These students may have had a problem with the unfamiliar blocks, but they did not have a problem in realising that they had to make a copy of the block design.

Although the tests have been much altered, in terms of the original standard deviations the statistically significant gains are Picture Description 0.67 Block Designs 0.65 Memory for Designs 0.63 and the total of all tests is 0.73. If you look at the narrow standard deviation for Picture Description, this made-up test lacks discriminative ability as does Arithmetic and Comprehension. Either there weren’t enough items and not enough hard items, or the group were already highly selected and homogenous in ability on these particular skills. Fuller testing might have shown different results. My own reading is that this small study suggests that 2 years of technical education probably improves Block Design and Memory for Designs, but without control groups we cannot be sure.

In my view, the Nisbett et al. account is not a good representation of the paper. Here it is again:

Gains in sub-Saharan African countries of 0.50 to 0.70 SD in response to a few months of Western-style education have been reported for heavily g-loaded fluid intelligence tests (McFie, 1961).

It makes the results sound extensive, broadly effective across countries, and quickly and easily attained. How should they have reported the results? There are various options, but here is one suggestion:

McFie (1961) tested 26 Ugandan adolescent boys on 7 adapted Wechsler type tests after over two years of technical education, finding gains of 0.6 sd on 3 non-verbal tests.

It is 29 words to their 30 (and 143 versus 163 characters) and I think it captures the main results more accurately. What do you think?

Postscript

So, where is the ghost to whom I referred earlier? John McFie was my first boss, a kind man who left a promising career at The National Hospital, Queen Square, London to work as a rural doctor in Africa. When he eventually returned, to England with his African wife, and to brain research at Guy’s Hospital, he hired me in 1968 and mentored me as we studied the cognitive effects of cortical injuries sustained in childhood. When Arthur Jensen’s paper came out in 1969 we both decided we would attack what we thought was Jensen’s suggestion of a genetic cause of African deficits on Block Designs by extending the work John had done in his 1961 paper. He was the senior author of the first paper I published, broadly arguing that Jensen’s findings were an artefact of cultural restrictions regarding constructional toys. Did we prove our case? Was Jensen convinced when I presented the results to him at a conference? More of that later.