You want it good, or you want it Tuesday?

All of us deal with speed/accuracy trade-offs. At some point a decision has to be reached: is it good enough now, given the urgency of the demand, or should it be kept back for further improvement? In the case of a bowl of soup, urgency usually triumphs. In the case of a commercial airliner, getting every component working properly generally predominates.

A rule of thumb is the principle that if it is 80% right then it is worth shipping in cases of urgency, because delay is more dangerous than using something which is partially effective. For example, a reasonably effective vaccine or treatment for Zika might be rushed through in affected areas, so long as the side-effects were not worse than the terrible effect on some pregnant women.

Therefore, it is particularly interesting to reflect on the relationship between speed, accuracy and intelligence. Which way will bright people jump when it comes to balancing speed and accuracy?

Phillip L. Ackerman and Victor J. Ellingsen. Speed and accuracy indicators of test performance under different instructional conditions: Intelligence correlates. Intelligence Volume 56, May–June 2016, Pages 1–9

https://drive.google.com/file/d/0B3c4TxciNeJZanBYWXk5SmZ5NkU/view?usp=sharing

Abstract

Performance on speeded ability tests, in contrast to power tests, reflects an individual's ability to get answers correct and to do so rapidly. However, with speeded tests, overall performance scores represent some unknown combination of the individual's strategy toward greater speed or higher accuracy. Scoring methods, such as penalties for wrong answers, are often imposed to either encourage examinees to adopt a specific speed-accuracy tradeoff strategy, or to attempt to derive performance scores that ‘factor out’ such strategic differences. In the current study, baseline assessments of four perceptual speed and psychomotor ability tests were administered, along with three different instructional conditions (accuracy-emphasis, speed-emphasis, and balanced accuracy and speed). A general ability composite was derived from a battery of intellectual ability tests. Changes in speed-accuracy tradeoff emphasis resulted in a consistent pattern of changes in the g correlates of latency/completion time performance indicators and number of errors. Increasing emphasis on accuracy resulted in increasing g correlates with latency/completion time, and decreases in g correlates with error rates. Implications for construct validity of ability tests and for further consideration of the conditions of testing are presented.

 

One take-away from such results is that performance (especially latency) on these tests is consistently associated with general intellectual ability. Historically, estimates of the g-loadings for perceptual speed and psychomotor tests have been modest, but these may be underestimates because previous researchers used perceptual speed and psychomotor tests that are too brief to obtain adequate reliability (for a discussion of related issues, see Ackerman & Cianciolo, 1999).

Nonetheless, under explicit instructional emphasis toward accuracy, speed, or an equal emphasis on both accuracy and speed, a remarkably consistent set of results was found, which complicates the above conclusion about the relationship between test performance and g. Under accuracy-emphasis instructions, correlations between test item latency and g were diminished, in comparison to both baseline measures and speed-emphasis instructional conditions. In contrast, error rate measures were most highly related to g under accuracy-emphasis conditions, again both in comparison to the baseline assessments, and in comparison to the speed-emphasis conditions.

Although Woodworth (1899) was perhaps the first to empirically identify individual differences in reactivity to conditions which demanded changes to speed and accuracy emphasis, there has been inadequate study of the issue in the subsequent decades. Nonetheless, there are some hints in the broader literature (e.g., Salthouse, 2000), that as adults age, there is a more substantial decline in speed of processing, in comparison to accuracy of responding, under traditional task instructions emphasizing both speed and accuracy. Whether this is represents an increase in rigidity associated with aging or a change in personal ‘tempo’ is not clear, nor is it known what specific physiological/neurological changes that might be responsible for such changes. In addition, there is evidence that lower-IQ individuals have more difficulty in regulating speed-accuracy tradeoffs, compared to normal-IQ individuals (e.g., see Brewer & Smith, 1984), suggesting that there may be a more executive processing limitation associated with flexibility in strategic reactions to differential speed and accuracy demands.

Ultimately, the consideration of speed-accuracy tradeoff functions, both imposed (e.g., by instruction) and self-generated (through an individual's preferential strategy or through an individual's understanding of the costs and benefits associated with the penalties for incorrect answers), comes full circle to Thorndike et al. (1926) assertion, which is in essence a ceteris paribus argument—that is, if examinees produce the same number of correct answers on a test, those that produce the answers faster are considered to be more intelligent. However, as noted extensively by Lohman and others, because examinees respond in such a way that they ordinarily are at different positions on their own speed-accuracy tradeoff curve, the ceteris paribus condition rarely occurs naturally during testing. Thus, speed and level of an individual's intellectual ability may be inextricably intertwined in many testing situations. We believe that the current study illustrates that there may be much to be learned about the nature of intellectual abilities, by focusing efforts on attempting to separate these two important components of test performance from a construct validity perspective (e.g., when investigating the relationship between speeded measures on one hand and general intelligence on the other hand), and also by implication, for criterion-related validity purposes.

This is a good paper, which opens up interesting questions about the way an individual (and individuals in particular circumstances) may chose to alter the balance of their speed/accuracy preferences. The substantive matter of this particular trade-off was much researched by W.D. Furneaux from the 1950s onwards, and his Nufferno tests were based on those insights.

http://psycentre.apps01.yorku.ca/drpl/commercial-test/nufferno-speed-tests

Speed, accuracy and persistence were the core concepts which Furneaux used to discuss mental performance. A very useful triad when considering real life intellectual achievements. While it is sad to see previous work mostly forgotten, it is good to see the topic alive and well, progressing speedily to an accurate conclusion.