At intelligence conference there are two sorts of participants: pure researchers in intelligence (the vast majority) and researchers who continue to actually test people’s intelligence in face to face assessments (a very small number of individuals). Aristocrats of intellect and peasants of psychometry. I am in the latter category, but I cannot tell you anything about it at the moment, because I am busy giving some tests.
Except that…… it never ceases to amaze me how an individual responds to test problems, and exactly how, and at what stage, and with what sorts of materials, they run into difficulties. You watch a person sail through the easy items (separate post needed on what makes an item easy) and then suddenly you find them pausing, struggling, and with any luck overcoming the difficulties they encounter with more difficult items (separate but related post needed on what makes an item difficult). Watched closely, you can see when people run out of old solutions and then, perhaps a little later, you can see when they run out of the capacity to generate new solutions (intelligence). Encountering more problem than intelligence is a humbling moment for all of us.
Kohs blocks are the best (in its modern incarnation of Block Design), because you can sometimes see subjects battling with a schemata which is wrong in scale, or orientation, or in internal logic, and then having to take it down and try again.
With every item you see fascinating issues about the level of complexity, the types of answer required in that minority of subtests which require scoring guidelines, the types of errors generated (subtle or not so subtle misunderstandings or ambiguities) and when person was raised in another culture, speaking another language, there is also a host of interesting questions about the comparability of translation and standardisation. Ceiling effects on skills like mechanical reading, and much higher ceiling effects on reading comprehension raise the tantalising question as to where harder and harder comprehension tests morph into tests of verbal intelligence.
Of course, giving a face to face intelligence test also tests the intelligence of the tester. Sorry I can’t talk at the moment.