Tuesday, 4 June 2013

The Motion Quotient and other distractions


Galton believed that cleverness would be associated with the capacity to make fine sensory discriminations. As usual, he was way ahead of his time, but he did not have the statistics available to analyse his results with sufficient power. Once Binet had taken the broad-brush educational approach by using a non-theoretical selection of mental tasks to develop intelligence quotients (skills expressed by comparison with peers of the same age) research on sensory intelligence lapsed. Only when under pressure to explain the underlying basis of intelligence in the 1960’s did researchers return with renewed interest to sensory measures like inspection time, choice reaction time, brain waves, and sensory nerve transmission times. They put together a reasonable case that whatever intelligence is, it can be measured weakly by physiological surrogates with little intellectual content, and minimal cultural influences.

Now a new test of sensation has entered the lists. Melnick et al. A Strong Interactive Link between Sensory Discriminations and Intelligence
Current Biology 23, 1013–1017, June 3, 2013 ª2013 Elsevier Ltd All rights reserved http://dx.doi.org/10.1016/j.cub.2013.04.053

Already dubbed the “Motion Quotient”, this procedure looks at the sensation of movement caused by displays of lines in either a small or large visual field. This is better seen in action than described in words, although the sequence of test examples below is not itself very well explained. The test items come too quickly for you to be able to record your answers, but they ask two questions: do the lines appear to move to the right or the left, and is it easier to make that judgement with a small focussed display or a larger screen version?


My somewhat confused answer was that I had little idea what was going on, but of course it was much easier to see the lines moving on a big display than a small central display. It was “bloody obvious”, to use a British colloquialism.

However, my impressions turn out to be of little consequence. As you would expect from any inspection task, the experimenters adaptively adjusted the stimulus duration to estimate the shortest exposure durations sufficient for threshold level performance. In other words, the experiment has to be calibrated to each individual person, and you cannot really work anything out about yourself by looking at the above demonstration.

There is a lot to like in this study. They drew their volunteers from the general population, and tested them face to face on the Wechsler individual intelligence test, which is the best validated. That must have been time consuming. They know exactly how their result fits in with inspection time research, namely that they are getting a higher correlation than is usual in this often disappointing line of research.  They have convincing explanation as to why their somewhat more complex “grating stimulus” is a better, and more ecologically valid test of sensory discrimination, namely that it tests the subject’s ability to detect the important signal from the unimportant but distracting background noise. They say: “Rapid processing is of limited utility unless it is restricted to the most relevant information” Finally, they can show a correlation between their test and intelligence which is probably around 0.7 which is as good as the correlation between a subtest and the overall IQ figure.

So, why am I not celebrating their impressive result? First, we have been here before. The early results on inspection time looked almost as good. Second, one should be slightly on guard when the result is in line with what one wants. Third, one should be particularly on guard when the explanation for the results are given by clever researchers who have thought through their experiment carefully.

Nope, I have only one real gripe.

N=67


9 comments:

  1. One should be on guard, should one not, because Psychology in particular is littered with claims that prove to be non-reproducible?

    ReplyDelete
  2. 67 people drawn from the general population: do you know whether you, as an experienced IQ psychologist, can make reasonable estimates of the IQ of such people by talking to each for, say, ten minutes? I'll define "reasonable" as "correlating with test-based IQs with a correlation coefficient higher than (say) 0.6".

    ReplyDelete
  3. The point is that the subjects were given a full IQ test.

    ReplyDelete
    Replies
    1. Indeed, but my question was whether you think that over the years you have yourself developed an ability to estimate IQ before you know the subject's score on a test. I assume there is a literature on comparing non-test estimates of IQs with measurements, I just wondered whether ten minute interviews (say) are any good.

      Delete
  4. I do not have an ability to quickly estimate IQ, other than having discussions on difficult subjects, which in fact constitutes a vocabulary and sometimes a numeracy test. Even then, that will tap verbal ability, and leave out non-verbal skills. I will look up whether there is any significant literature on the subject. At the anecdotal level, given a free choice of difficult subjects, then 3 or 4 minutes should suffice to get estimates at the 0.6 level.

    ReplyDelete
    Replies
    1. What would you estimate are the IQs of fellow psychometricians as compared to social-scientists in general? (I've been curious about this.) What about the IQs of political elites; are your estimates consonant with Wai's? Or those of top scientists? (I always thought Roe's and Cox's estimates were rather low.)

      Delete
    2. Psychometrics is objectively more demanding, because you have to be statistically adept just to follow the discussions. In social science generally researchers can do lots of potentially interesting work, and then go to a statistician to get their magic input, without fully understanding how it was done. So, while there are bright people in all callings, in much of social science there is too much reliance on bright statisticians who "correct" the data without the authors understanding fundamental problems of covariance. Earl Hunt's "Human Intelligence" is good at explaining these issues. As to politicians, I'd like to pass on that at the moment. They tend to be heavily scripted by brighter backroom assistants.

      Delete
  5. It's rather annoying that the video wasn't normed - and 1 minute long.

    ReplyDelete
  6. Duje Tadin, who did the study, explains that you have to calibrate the setup for each person. The only way to do it on a home screen is to compare yourself with another person using the same setup, and all you would get it a comparative result. Will post on this now.

    ReplyDelete