Can tests predict academic outcomes?

http://www.isironline.org/isir-2015-invited-address-paul-sackett-nathan-kuncel/

Paul Sackett and Nathan Kuncel cover a lot of ground in this talk, and slay a lot of dragons as well. Test bias, critical thinking, tests of rationality, the idea that once you have basic intelligence higher scores don’t contribute anything more: all those sorts of beasts.

A distinguished member of the audience commented to me at the time: Paul Sackett and Nathan Kuncel utterly destroyed the idea that SAT tests do not predict college performance. Their "ginormous" dataset comprised over a million students. A droll and data rich talk, they left myths about the non-utility of standardised tests lying like road-kill on the highway of evidence.

The link is above, but here are their conclusions:

The final point is somewhat obscured by their modesty: they have ginormous data sets (1.2 million students), which they analyse very carefully, looking at many confounding variables. Clever sillies who want to muddy the water can always find a small sample to prove anything they want. A more dependable picture emerges from the collation and evaluation of the available literature, graded for quality.

Here is a typical example of a clever silly being “lawyerly” rather than scholarly:

Psychology will never get anywhere if this sort of courtroom science is given any credence.

Colleges and schools are prone to the “rubber ruler” distortion: academic scores for easy subjects boost apparent ability, so corrections have to be made for the difficulty of courses. Academic performance measures often conflate exam results with attendance scores (and even credits for taking part in experiments). The purer the measure of academic ability, the more powerfully it can be predicted by good admission tests.

Sackett and Kuncel are good grinders and polishers, flicking away the dust to get to the polished steel. They know their data, and handle it with aplomb. By doing so they demolish lots of popular delusions.

Their talk is worth watching in its own right but also because they only take 45 minutes, which gives time for the audience to ask questions, and this gives you a look at some of the key researchers in the field, and an insight into their interests and opinions.