Friday, 9 May 2014

So you think you’re intelligent?

I have never been particularly interested in people’s self-assessments of intelligence. It seemed likely they would be wrong, and so what? Years ago Furnham and Gasson (1998) had a look at what families thought about their intelligence, and found that men rated themselves at IQ 108 and women at IQ 104, their boys at 109 and their girls at 102.

Of course, this might be true. Mother chooses a good father by finding a man brighter than herself, and then their bright children grow up even brighter because of the Flynn Effect, with boys brighter than girls, because any fool knows men are cleverer. Never sneer at folk wisdom.

Or, all parties could be deluded. Modest Mum downplays her abilities to make her fool of a husband feel important, and dotes on her average children, seeing abilities no one else can detect, and favouring the male of species, as well she should, so the next generation of women have a bright man to look up to, or a fool who thinks he is bright.

Into this mush of self-perceptions steps Sophie von Stumm, who has the temerity to actually test people’s intelligence before asking them how well they did on the test, and then where they put themselves on the intelligence bell curve.

“Intelligence, gender, and assessment method affect the accuracy of self-estimated intelligence”. Sophie von Stumm, British Journal of Psychology (2014) 105, 243-253. (This arrived in paper form, so you will have to get the link yourself).

200 is a good number of subjects for a psychology paper, but a little low for intelligence estimations. However, the human subjects seem to be the wild sort, not the usual laboratory rats, so that is a welcome change. The tests used are reasonable but only the Matrices test is well known, and she used a 12 item short form, and Lettersets and Nonsense Syllogisms are less easily compared with other psychometric findings. Perhaps it is not important: a general factor derived from the three tests accounted for 60% of the variance. However, the loadings are highest for Raven’s at .93, Lettersets lower at .78 and Syllogisms only .50 which is not very good. Wechsler Coding would have only taken 2 minutes and loaded strongly on g!

The correlations between the actual IQ measure and the self rating are pretty pathetic at very roughly .20 which suggests general inability to self-assess intellectual ability. Figure 2 shows what the problem is: Dull people over-rate their intelligence. This is interesting, and begins to explain a few things. Dunning Kruger were on to this, but it is nice to see it confirmed for intelligence self-estimation. Brighter people somewhat under-estimate their intelligence, but to a much lesser extent.

In a nutshell, the dull have delusions of adequacy.

This interesting paper was submitted in January 2013, revised in March 2013 and achieved publication in May 2014.  This is the academic cost of publishing in a good journal: very delayed scholarly feedback. Journals with faster and more complete review processes should, with any luck, rise to the top. Reviewing papers must become an accepted fast-track to promotion in academia.

It may just be me, but the British Journal of Psychology rarely excites me. This paper is a good exception. The journal is worthy, no doubt, but it usually lacks human interest. Too much to expect from Psychology, perhaps.

12 comments:

  1. "Reviewing papers must become an accepted fast-track to promotion in academia." Couldn't agree more. However, review should be open, because traditional peer review is hidden from public view and this leads to all sorts of abuse and stupidity. For an example of an innovative peer review system, check out the journal that Davide Piffer and Emil Kirkegaard have created. This is what journals in 2020 should be like: http://openpsych.net/about-the-journals/

    ReplyDelete
    Replies
    1. Hopefully, it will work out well. So far the success has been much higher than expected. We will see how well it does in the future. The review form seems to work very well. Quick replies from reviewers and new drafts from authors, as well as in depth review of the analysis that results from the data sharing policy.

      Delete
  2. As a young psychology student, we all had to test each other on the WAIS. I went into another career originally, then back to university and into psychology, where I have administered cognitive assessments routinely for 20 years. When the internet started to become something I spent a lot of time on, I was intrigued by an online IQ test, and decided to give it a go. Items were mainly comprehension, vocabulary, matrix reasoning and picture concept type tasks.

    Memory is a fickle thing, and it was @ 15 years between my taking the WAIS at uni (administered by a peer as his first go at it, so could have been some non-standardised administration quirks in there), but the number seemed to be exactly the same, and consistent with what I would expect my IQ to be (competent student at university, but not outstanding, and definitely less gifted in this way than peers who have done well in research).

    This doesn't say much about self-assessment, but I was intrigued by the accuracy(??) of a free, five-minute online IQ test, at least in this instance. Probably not something for psychologists who enjoy performing cognitive testing for a living to trumpet too loudly!

    ReplyDelete
    Replies
    1. Paolo, I too was tested on the WAIS by a fellow psychology student, and then working with him on the results corrected one of the items which was culturally loaded (British light bulbs use a bayonet fitting, South American light bulbs the Edison screw fitting). Score was 130. I was also given the AH5 test, which gave finer discrimination for university student of that time, who were mostly at the 98th percentile and that was a B, which is roughly top 15% of university students of that time. Years later I took an internet test which was a version of the WAIS (by Tickle, I think) and that score was 135. Recently I did a matrix test (which much irritation at the boredom of it) and got 139. However, those latter test providers obviously did not know how to calculate percentile ranks, so no credence can be given to the result.
      I think that, if anything, tests are getting a little more flattering, rather like clothes sizes. Even the WAIS has been re-normed, and many items dropped even though they were good discriminators.
      However, my results over 50 years are all much of a muchness, and allowing for different test material and different standardisations, very much of a stable characteristic. Of course, it might have been simpler to do digit-symbol for two minutes!

      Delete
    2. My collection of online tests:

      http://sifter.org/iqtest/
      http://mensa.dk/testiq.xml
      http://mensa.se/bli-medlem/provtest/
      http://www.mensa.no/olavtesten/index_2.html

      Delete
    3. Oy' vey, 129-144 in sifter.org-iqtest and 107 in mensa.se-provtest



      Gottlieb

      Delete
    4. 119 on mensa.dk and 117 on mensa.no

      Gottlieb

      Delete
  3. One thing about teaching intelligence, is that the literature is locked down behind paywalls. Anyone who wants to teach intelligence or any other academic field, must come to terms with copyright policy.

    So here: http://emilkirkegaard.dk/en/wp-content/uploads/Intelligence-gender-and-assessment-method-affect-the-accuracy-of-self-estimated-intelligence.pdf

    ReplyDelete
  4. Hi James.
    I'm an average person reading and commenting on content that was never meant for me and is beyond my comprehension.

    However I'm curious about something. Raven's Progress Matrices is apparently highly "g-loaded".
    However would you say that RPM is more about fluid intelligence or crystallized intelligence?

    I ask about this because on an online MENSA RPM test I've consistently scored between 117 and 120, but I suspect my actual IQ is below 110. I'm not able to process information very quickly, and my verbal "quickness" is...very slow.

    ReplyDelete
  5. Dear Martin, As you wisely show, being doubtful about one's mental powers is associated (somewhat) with higher intelligence, so if the scores on Mensa RPM come out higher than you imagine on self assessment, then you are probably on to a good thing. Matrices would be seen as a test of fluid intelligence. Try some of the other tests for variety, and please remember that although verbal quickness is indeed a sign of intelligence, some people are stronger on performance type tasks. Better a thoughtful engineer than a glib politician (this is a value judgment on my part). Actual test results are a balance between speed and accuracy, and you may value accuracy more, which is no bad thing. Lastly, if you are reading this blog you get an immediate 4 IQ point boost (I made that up, but it corresponds to the usual re-test practice effect improvement, so why not enjoy that anyway).

    ReplyDelete
  6. So the paper says that dumb people fail at estimating their intelligence; smart people are somewhat better, but not perfect.

    Well, estimating one's intelligence is a mental task. Intelligent people are better at mental tasks than less intelligent people. So far nothing new.

    Peculiar is that intelligent people seem to underestimate. So there is a bias. Or perhaps simply that a character trait, e.g. modesty, is correlated with intelligence.

    Seriously, I doubt that intelligent people will (and others) will give an honest answer when they are asked how intelligent they are. But I'm no psychologist and I assume that those working in field are aware of the problem that people might give less than completely truthful answers.

    Karl

    ReplyDelete
  7. Perhaps people would be more accurate in self-rating using percentiles rather than IQ / deviation scores? That's how school standardized tests are scored, so more familiar. If there were a reward which depended on accuracy, that might improve performance, too.

    ReplyDelete