What do college admission tests predict?

 

Put the brightest people onto the hardest problems and everyone benefits.

The first step in that process is to use very well validated tests in order to find the brightest people, and then give the best students the best education so that they then go on to solve the hardest problems. The rest of us, speaking for myself, then try to understand and implement their findings. When calculating my location, my satnav implements a time correction based on Einstein’s theory of relativity (satellites move at very fast speeds, such that their atomic clocks seem slightly wrong when their time signals are received by the cheap quartz clocks in the satnav).

Paul Sackett identifies an issue which can prevent the brightest people solving the hardest problems. He says: One recurring theme in my work is the tension between designing selection systems to maximize job performance vs. to maximize ethnic, racial, and gender diversity. The current controversies over the future of affirmative action attest to the prominence of this concern.

You know what I think. I think it is wrong to confuse competence with demographic representativeness. Competence is needed so as to obtain the best outcomes of skilled behaviour. Representativeness is needed to establish that a sample represents a population. Confusing the two will lead to bad decisions and sub-standard performance.

 

 

I am sure Paul Sackett knows all this, because he has been instrumental in long-term, high quality work on admissions tests, and is particularly hot on the methodological issues which bedevil the field (and most of psychology). His work shows the continuing relevance and power of testing. Sackett and Kuncel have a sample a million students which, as they modestly say, “delivers robust answers”.

Sackett and Kuncel found that SAT and high school grades contributed to predicting academic performance in college. Taking parents’ education and family income into account had little effect on the relationship between SAT scores and college performance. The SES of students actually enrolled in college was very similar to the SES of students who were applying to college. When they examined the data more closely and looked at the entire applicant pool, they found that fewer low-SES students were entering the college admissions process.

Based on these findings, it seems that low-SES students are not underrepresented in colleges because low SAT scores prevent them from gaining admission, but rather because fewer low-SES students apply to college in the first place.

“We view this as broadly relevant,” says Sackett. “Entrance tests such as the SAT receive a great deal of public scrutiny and it is important for all involved—students, parents, college officials—that accurate information about how the test functions be available.”

This duo should receive a medal for doing educational research in a minefield. In fact, that is what ISIR is offering them, in the form of the President’s Invited Symposium. We will listen carefully to these scholars.

 

What do college admissions tests predict, and for whom? Insights from a large scale research program. Paul Sackett & Nathan Kuncel (Introduced by Michael A. McDaniel)

Student success in higher education is a critical and fascinating topic. Student success is multifaceted; stakeholders emphasize different outcomes such as degree completion, work skills and critical thinking. Multiple individual characteristics are associated with different facets of success. College admissions use numerous measures ranging from interviews to standardized tests. Here we will discuss decades of our research on these issues including many studies that have come out of our joint research laboratory. We make use of a still-growing primary database now following over 1,000,000 students through college, combined with meta-analyses on hundreds of thousands of students. Higher education policymakers need to know about test validity, the role of SES on student admissions and performance, linearity of test-performance relationship, and predictive bias by race and gender. Our very large N delivers robust answers to these and other key questions.

http://www.psych.umn.edu/people/profile.php?UID=psackett