Wednesday, 17 December 2014

Are women easily discouraged?


I cannot pry into your domestic arrangements, not without a larger budget anyway, but I wonder if you would agree that women are easily discouraged? Personally, I have not found it to be the case. On the contrary, to me women seem to have a well-developed ability to triumph in argument by the vigorous exposition of their cause, and a photographic recollection of such errors and transgressions committed by any man who would dare oppose them. This may be a unique observation. You will tell me. Methodologically, I understand that your perspective may differ if you are a woman or a man, but I trust you will follow the usual calm and detached empirical procedures to ensure an objective evaluation of this contentious matter.

However, my personal impressions may be due to restrictions in sample size. On the basis of studying larger samples of women with social psychology experiments on priming some researchers assert that if you communicate to a woman that she is about to take a test on which women do badly, then she will do badly on that test. That is, worse than she would have done if you had not communicated that negative expectation. This is called “stereotype threat” and is said to be the reason that women and some racial groups do badly on some tasks.

First, let us see if this is true. Paulette Flore works with Jelte Wicherts, who likes nothing more than taking a chainsaw to other people’s statistics, often with good effect. Will stereotype threat survive the investigations of this dynamic duo? Paulette gave an enthusiastic presentation on this subject at ISIR2014 in Graz.

Does stereotype threat influence test performance of girls in stereotyped domains? A metaanalysis. Flore, P.C., Wicherts, J.M.;

The effect of gender stereotype threat on math, science or spatial skills tests performance of girls has been studied in numerous experiments. Although theory predicts that girls in stereotype threat conditions will underperform compared to girls in control conditions, outcomes of the experiments strongly vary. In order to
understand the effect of stereotype threat in school-aged groups, we conducted a meta-analysis of stereotype threat experiments on test performance with schoolgirls as participants. In addition we considered the following theoretically relevant moderators: test difficulty, presence of boys, gender equality within countries, and the type of control group used in the experiments. We also studied the possibility of publication bias.

We carried out an extensive (grey) literature search. Selected papers included a sample of girls in their study, had a sample with mean age younger than 18 years, used a (quasi)-experimental design, administered the stereotype threat between-subjects, and used a math, science or spatial skills test as dependent variable. We used Hedges’ g as the effect size and fitted random and mixed effects meta-regressions with restricted maximum likelihood estimation. We carried out several tests for publication bias and excess of significant results.




We obtained 26 relevant papers or unpublished reports, out of which we selected 47
independent effect sizes for the analysis (total N= 3760). The estimated average effect size was small, but significant (Hedges’ g = -0.22). None of the theoretical moderators significantly predicted the stereotype threat effect. The tests for publication bias were all significant. Furthermore, the test of the excess of significance results indicated there were more significant studies included in the meta-analysis than to be expected based on the observed power. A consequence of publication bias, or an excess of significant results, is that the average estimated effect size is probably inflated.

Although the theory of stereotype threat has been well established, based on our results we question the robustness of the effect for the population of school-aged girls. To obtain a more reliable estimate of the average effect of stereotype threat we need new cross culturally administered experiments with sufficient power, which would enable us to disentangle the bias from the actual effects of stereotype threat.

Why did papers which showed an apparent effect of stereotype threat have a better chance of getting published? Of course, papers reporting any sort of effect are more attractive than papers which fail to find an effect, though it is important to know about both.  Do editors and reviewers want to find that women are underperforming because of an unfair performance expectation? Have they thought that through?

Let us do a thought experiment. Say you are an employer. (I know this is unlikely, and that probably like so many of us you are now or have been in the past a hireling of the State or their cosmetically enhanced front organisations and running dogs). You are told that if you employ a woman (or an ethnic minority person) they are likely to underperform when given a task on which they are told they will do badly. I am not an employer, but I have listened to employers, and the last thing they want is to hire someone who is “high maintenance”. Imagine you were an employer and one day, coming across a rival product being offered for sale, you rushed into your own office and said to your employees: “Our competitors have come up with a brilliant innovation. We have to use our brains to work out how to improve our own offering, and we have to do so quickly so as to outwit them with our intelligent counter-offer”. At this point you notice all the women are in tears, and all the ethnic minority people are mute with anxiety. From the employers’ point of view, hiring such fragile minds is a dreadful selection error. Never employ people who collapse under pressure and, most certainly, never let them fly an aeroplane in a thunderstorm.

It many be too much to expect, even after this thorough dismissal by Flore and Wicherts, that the notion of stereotype threat will not be pursued further, but brace yourself for further publications purporting to show that girls simply can’t take the pressure of negative expectations, and need to be guided through their fragility by sensitive condescension.


  1. "You are told that if you employ a woman (or an ethnic minority person) they are likely to underperform when given a task on which they are told they will do badly ... At this point you notice all the women are in tears, and all the ethnic minority people are mute with anxiety. From the employers’ point of view, hiring such fragile minds is a dreadful selection error."

    Henry Harpending once touched on this. In short, even when two sets of individuals are equalized in some set of traits (IQ + whatever else that helps worker competence) – let's call the Competence Quotient (CQ) – those from the populations with the lower mean will tend to do worse overall, because they are more likely to have erroneously high scores due to measurement error (luck, basically). This includes women (who are less intelligent than men on the high end of the bell curve) and individuals from ethnic groups with lower means. An individual with a CQ +2σ from his mean is more likely to have scored accurately than an individual +3σ from his/her mean.

    Of course, theoretically, taking into account accomplishments and various forms of repeat testing should attenuate this and give you an accurate idea of an individual's ability, but for that to work, said credentials themselves need to be equalized. Two graduates from Harvard are probably comparable if their scores are similar. Not so much if one is a from Harvard and the other from some backwoods State U.

    All that said, this effect on the whole is fairly small – test scores overwhelmingly predominate, and the fact that stereotype threat is apparently rubbish strengthens the idea that women perform fairly, as expected.

  2. I wonder how many years or decades back stereotype threat will hold social / differential psychology?

    Cognitive priming requires multiple trials and very-well controlled / counterbalanced stimulus presentation, and the effects are only in milliseconds. Yet thinking of grandpa is going to make you walk slower to the elevator?

    Social priming research sure seems like a house of cards...

    p.s. I believe Wicherts presented a meta-analysis on this (or perhaps ST in general) at ISIR about 4 years ago. It seemed like it never got published-- is this it?

    1. Don't know. Jelte does quite a bit of this type of work, so this paper is one of a suite of methodological critiques. If, as I expect, Paulette takes over these tasks, then we can expect an even greater output. Jelte will eventually return to the sub Saharan intelligence debate, presumably looking at all the studies in detail. I am encouraging him in this.

  3. I can only tell you my experience of teaching many hundreds of boys and girls over more than a decade. Girls are much more risk averse, even at the first grade level, than boys. I believe this is a logical result of evolutionary pressures on the two sexes, with boys having to take a lot of risks to gain access to the most desirable women.

    1. pyrrhus - risk aversion - I'm sure it's not a unique insight, but this is pretty much also the view of Prof Roy Beaumeister

      Not to mention that great evolutionary psychologist Thomas Hardy in The Return of the Native:

      "You are just like all women. They are ever content to build their lives on any incidental position that offers itself; whilst men would fain make a globe to suit them."

  4. It depends what is discouraging. Contmept of familarity applies here. Closer the person, less the discourage.

    Issue related to math or physics principle are very discouraging to most females.

  5. "women seem to have a well-developed ability to triumph in argument by the vigorous exposition of their cause

    Didn't Kipling have something to say on this ?

    a photographic recollection of such errors and transgressions"

    I have what I used to call a theory but is now apparently a hypothesis, being unsupported by evidence - that no man likes to admit a possibly mistaken or less than perfect choice of mate, any more than he likes to admit to being lost when driving, and so quickly consigns his mate's errors and transgressions to offline storage, to be retrieved only with effort, if at all. Anything else might call his judgement into question.

    A woman will keep errors and transgressions in high-speed random access memory for instant recall, be they years or decades old.

  6. Stereotype threat has more to do with initiating anxiety which can hinder working memory and cognition. This may ultimately cause women to avoid activities that trigger stereotype threats.