Flynn effect as a retesting, rule-based gain

It is very good to see a paper which takes a large scale effect, the secular rise in intelligence test results, and links it to an intriguing large scale explanation. A new contribution to understanding the Flynn Effect is to be found in the journal Learning and Individual Differences, which became available 30th October.  Elijah Armstrong and Michael Woodley “The rule-dependence model explains the commonalities between the Flynn effect and IQ gains via retesting” http://www.sciencedirect.com.libproxy.ucl.ac.uk/science/article/pii/S1041608013001556

Woodley you already know, but more about Armstrong later. Armstrong and Woodley argue that the Flynn effect is partly driven by the retest effect, whereby familiarity with the test material means that if you can learn a rule of thumb you can solve those particular sorts of problems when you see them again, without having to use much intelligence. In very simple terms, the test wears out quickly once you get to learn how it works. Using implicit learning and working memory, test takers learn how to solve rule dependent problems, which leads to apparent IQ gains which are partly independent of general intelligence.

As readers of this blog will know, the ultimate IQ test is the one for which no-one knows the answers at the moment. Intelligence tests in the real world are more modest affairs. Raven’s Matrices is a test based on progressions: you need to find the rule which underlies the visible changes in the problem arrays, and a good enough memory to hold in mind how those changes are progressing, so that you can correctly choose the final missing picture. For example, if the figures are growing as they move from left to right, and getting paler as they move from top to bottom, you still have to remember exactly how large and pale the missing item should be when you check through the proffered options. Carpenter et al. (1990) found that 5 rules covered all the items in the test. Once you know that, it is less of a test.

Another aspect of being test savvy is the capacity to de-contextualise, that is, to be able to generalise about types of problem, without being confused by the particular context in which the specific example is presented to you.

Armstrong and Woodley assert that, from the point of view of intelligence, education amounts to a vast re-testing enterprise. There are modest gains from rules of thumb, mnemonics and being “taught to the test”. Indeed, the reliance on exam results makes teachers and pupils confederates in ensuring that nothing is taught which is not taught to be examined. Incidentally, this view does not exclude what James Flynn calls “scientific spectacles” which more people now adopt when solving problems.

Armstrong and Woodley rank tests according to how much “cognitive scaffolding” they have. Raven’s Matrices is level IV: rules are very helpful, only a few of them are required; Catell Culture Fair is Level III: rules help, but will not help you on many items; the majority of IQ tests are Level II: very many rules are required, but working out which to use is difficult, (and selecting the rule is what requires general intelligence); and Draw A Man test is Level I: no rule is of much help.

They then simply correlate the vector of the position of any particular test in the rule dependence typology with the vector of the size of the Flynn effect on that test. A positive correlation would indicate that tests that were more dependent upon rules were yielding the larger Flynn effects. They tested it on 14 data sets, and found a correlation of 0.6

 

 

The authors say: “It is proposed that tests like the Raven’s are only highly g loaded when encountered initially — even basic familiarity with the rules and heuristics on a test, or familiarity with inductive reasoning itself, has the potential to radically diminish the g loading of this test over time, both under controlled conditions (such as in a retesting scenario) and over larger societal time scales (i.e., across generations in the case of the Flynn effect).”

They continue: “The increasing capacity of societies to detect and explicitly utilize rules as a function of the Flynn effect may be related to increasing rule exposure via mass education and to ‘ways of thinking’ endemic to cognitive modernity (Flynn, 2009). It may also be caused by increased rule sensitivity as a consequence of changes in brain functioning that might relate to the prevalence of slower life history strategies in the modern era. These in turn might be in part a change in life history strategy due to dramatic improvements in environmental quality.”

This is a good paper. It contains lots of ideas, proposes a theory and then tests it, and draws out the conclusions in a thoughtful way. Not content with linking the observed phenomenon with the Flynn Effect and life speed theory, it also includes 5 testable predictions, to encourage other researchers to test whether their proposal has merit. It is a notable debut for the first author, whose first paper this is, and whose ideas formed the basis for the eventual publication.

Postscript

Elijah lives in Marin County, California, and is interested in philosophy and intelligence research. He originated the rule-dependence model in early 2012 and worked on it for eighteen months thereafter. He claims his conscientiousness is below the 10th percentile. He is also prone to end all his emails saying “Excuse typos, I typed this with my feet”. If you imagine that he is a sad old man gathering up a lifetime of scholarship into a well-honed rant, your imagination would be wrong. Elijah is 15. Welcome to a lifetime of academic labour!