Saturday 19 September 2015

Correlated vectors

 

Jensen developed the “method of correlated vectors”  in which a positive correlation between group differences and g-loadings strongly supported the hypothesis that the group differences were largely due to general mental ability. I first met Jelte at the Amsterdam ISIR conference in 2007, when he had just finished his PhD and was discussing the need for confirmatory factor analyses to establish measurement invariance. This is his critique of the method of correlated vectors.

Your photograph

http://wicherts.socsci.uva.nl/CVJMW.pdf

MORE PSYCHOMETRIC PROBLEMS WITH THE METHOD OF CORRELATED VECTORS Jelte M Wicherts1 1 Tilburg University, j.m.wicherts@uvt.nl.

The method of correlated vectors (MCV) was developed by Jensen (1980, 1998) to study the hypothesis that the relation between cognitive measures and an extraneous variable (e.g., ethnicity) is fully explained by g. Despite criticism, MCV continuous to be widely used in the study of group differences in intelligence test performance, most often to study Spearman’s hypothesis stating that ethnic group differences in cognitive performance are most pronounced on the most highly g loaded tests or items. In addition, recent studies have submitted MCV results to psychometric meta-analytic techniques in which MCV results are corrected for particular psychometric artifacts.

In this talk, I critically evaluate the psychometric assumptions underlying MCV. I focus on meta-analytic corrections applied to vectors of g loadings and on the application of MCV to item-level data. I use both formal arguments and empirical data to illustrate the drawbacks of MCV. I particularly address studies that have applied MCV to study group differences on items of Raven’s Standard Progressive Matrices (SPM). SPM data have yielded strong MCV correlations (i.e., Jensen effects) that have been interpreted as showing that the SPM measures g similarly across ethnic groups and is not subject to item bias. Using formal arguments, I show that failures of metric measurement invariance (group differences in g loadings) actually lead to higher MCV estimates in meta-analyses despite the fact that such violations clearly contradict Spearman’s hypothesis that group differences are due to g. Moreover, I show that MCV applied to item-level data does not provide accurate information about the comparative psychometric properties of a cognitive test and the role of g.

The empirical results show that MCV applied to SPM items in one group yields substantial Jensen Effects even when the items in the second group (N=252) are not from the SPM but rather from a test composed of items from the State-Trait Anxiety Inventory and the State-Trait Anger Scale. Combined, these results highlight problems with meta-analytic corrections to MCV results and show that MCV applied to item level data does not accurately refect the degree to which item bias or g plays a role in the ethnic group differences. Although MCV may be useful in some circumstances, it is better to use model-based approaches like multi-group confirmatory factor analysis or Different Item Functioning (DIF) analyses whenever these are feasible.

2 comments:

  1. It seems to me that MCV usually gives results that are concordant with more sophisticated techniques.

    ReplyDelete
  2. Interesting. I find the topic too arcane for me, and I worry that the data sets required for these tests are large and hard to find. However, I can follow it much of the time.

    ReplyDelete