At intelligence conference there are two sorts of participants: pure researchers in intelligence (the vast majority) and researchers who continue to actually test people’s intelligence in face to face assessments (a very small number of individuals). Aristocrats of intellect and peasants of psychometry. I am in the latter category, but I cannot tell you anything about it at the moment, because I am busy giving some tests.
Except that…… it never ceases to amaze me how an individual responds to test problems, and exactly how, and at what stage, and with what sorts of materials, they run into difficulties. You watch a person sail through the easy items (separate post needed on what makes an item easy) and then suddenly you find them pausing, struggling, and with any luck overcoming the difficulties they encounter with more difficult items (separate but related post needed on what makes an item difficult). Watched closely, you can see when people run out of old solutions and then, perhaps a little later, you can see when they run out of the capacity to generate new solutions (intelligence). Encountering more problem than intelligence is a humbling moment for all of us.
Kohs blocks are the best (in its modern incarnation of Block Design), because you can sometimes see subjects battling with a schemata which is wrong in scale, or orientation, or in internal logic, and then having to take it down and try again.
With every item you see fascinating issues about the level of complexity, the types of answer required in that minority of subtests which require scoring guidelines, the types of errors generated (subtle or not so subtle misunderstandings or ambiguities) and when person was raised in another culture, speaking another language, there is also a host of interesting questions about the comparability of translation and standardisation. Ceiling effects on skills like mechanical reading, and much higher ceiling effects on reading comprehension raise the tantalising question as to where harder and harder comprehension tests morph into tests of verbal intelligence.
Of course, giving a face to face intelligence test also tests the intelligence of the tester. Sorry I can’t talk at the moment.
in the testing "trenches" one notices things:) (right before becoming exhausted by all the testing:) the blocks of Koh (who never copyrighted, hence copied by free for most IQ tests:) measure spatial ability for most people (except - some very high verbal's can talk their way thru & look higher spatially than they are - AND sometimes high spatial's have trouble processing too much 2D symbol/line/angle info (which also messes them up on matrices items with a lot of lines & angles, but not on the "aha" immediately visual matrices items) when in reality they are very good at spatial.
ReplyDeletei always give the blocks first - most folks i see that's their best - some folks struggle a bit (due to the 2D aspects of it) & a few are totally stymied by it (nonverbal learning disorder - low spatial/high verbal). i'm lucky, b/c i mainly test triple digit IQ smarties:) so if they're bad at blocks, it's usually the rare nonverbal LD & they're also bad at high-level math (math with X's & Y's:)
some tasks don't always measure the same thing for everybody, e.g., for folks with weak processing speed/language retrieval deficits - timed achievement tasks measure how well they process text/symbols/language quickly & accurately - rather than comprehension, etc. or some kids with excellent vocabs but language retrieval problems can do well generating lists if things in a category, but can't name #'s/lettters fast (most with that latter problem struggle with the former, but not all, & some vice-versa!)
you are right to be wary of rascally ceiling & floor effects:) some tests have pile-ups where the item gradient jumps. some are normed easier at certain levels & harder at other levels. i hate when low-g tests are normed too easily (e.g., DAS-II speed of info) b/c i need certain processing tasks to be low & some high-g tasks to be high for me to better explain it all to kids/parents/etc. selfish, i know, but usually i get folks who are good at high-level thinking, but pesky automatic processing deficits pull down parts of reading).
random observation from 15 years of testing (for 15 years before that i published peer reviewed research articles - about 1 a year - my backwards career - tho i tested part time during those 15 years, too) stimulants improve scores on low-g processing speed & auditory memory (hey say these #'s) tasks - stimulants raise their scores more than regression to the mean predicts:) stimulants help on the low-g aspects of high-g tests (sustaining intense attention to Matrices beyond 5 seconds in order to uncover all the rules) so some people think stimulants raise "g" but really they just let you use the "g" you have b/c "g" no longer has to run over & help bale out weak processing:) stimulants don't help you know more words than you already know (but you can find/say the words you do know faster:) sorry - hope my comment isn't a boring "thread killer!" test-on, dr. thompson! may it yield researchable hypotheses!
Intriguing. I would like to have a career where I get to do both theoretical/research & clinical work; I think there are insights to be gleaned from the latter that the former alone cannot provide (plus, I like working with kids). Not sure what the best way to go about this is, though. Any ideas, everyone?
ReplyDeletetake my job... please! :) an academic job in "school psych" can do that - many universities have clinics for the prof. to supervise grad student testing, & that tenure track PhD position carries with it the usual responsibilities of research, teaching & service.
ReplyDeletebeware, school psych is sort of the low rung on the psych pole (&/or housed in college of education sometimes, which is a lower rung than liberal arts & sciences, where psych depts usually are) it's all psychometrics, applied psychometrics & research in psychometrics:) also, the test publishing companies provide the opportunity to play with data (but not test kids much), however nowadays there are plenty of data to be had without resorting to working for the lean & mean test publishing industry!
you are clearly plenty bright enough to be a tenured full professor somewhere, & then testing in one's spare time (& using that data for research:) can be helpful.
I like school psych. The work is interesting, and I believe (hope?) that it will become less of a backwater, in part because of the improvements in testing tools. There has been a huge change in how testing is seen in my area (Australia). Used to be that teachers held up the garlic (as in, to ward off vampires) when 'IQ' was mentioned - not now, even those who are uncomfortable with the concept are less likely to bring up Steven Jay Gould anymore.
Deletei'm glad to hear it's better there -- being a school psych in the rural US in the 1980s was actually fun:) now it's paperwork + an egalitarian magical thinking multicultural agenda since 1990 - actively shunning good science (psychometrics/g/heredity/twin studies/test fairness in prediction) to promote happy alternatives - held hostage by PC enthusiasts running government/public schools - there's no hybrid vigor from cross breeding the (left-leaning) fields of psychology & education!
Deleteschool psychs should lead the charge & say: "hey, education: achievement gaps come from intelligence gaps, wild ducks come from wild duck eggs! stop expecting teachers to eliminate achievement gaps - come, let us consider the reality of twin studies & normal curves & heredity!" but the field of school psych here doesn't dare. it's been neutered (domesticated? please insert unrefined euphemism for emasculated here:)
"a schemata"?
ReplyDeleteI thought schemata was plural?
Yep. Should have said "a scheme"
Delete