Nature, the journal, and the Nature of Samples

 

As you will know by now, I lead a quiet life, avoiding trouble wherever possible. The natural calm of my afternoon was shattered by a Nature News piece with the startling and quite specific headline: “Stress alters children's genomes”. As if the drama of the finding was not enough, Jyoti Madhusoodanan’s article added further details: “Poverty and unstable family environments shorten chromosome-protecting telomeres in nine-year-olds”.

http://www.nature.com/news/stress-alters-children-s-genomes-1.14997

They shorten the genome. Not, they are associated with shorter genomes. They take a healthy long genome and shorten it. Imagine what the shock of the headline must have done to my genome. On your behalf, I read on, undaunted. Nature had posted up a nice picture of telomeres, which is usually enough to win over the unconvinced. Ever the sceptic, I read on, and came across these prize paragraphs:

When researchers examined the DNA of 40 boys from major US cities at age 9, they found that the telomeres of children from harsh home environments were 19% shorter than those of children from advantaged backgrounds. The length of telomeres is often considered to be a biomarker of chronic stress.

The telomeres of boys whose mothers had a high-school diploma were 32% longer compared with those of boys whose mothers had not finished high school. Children who came from stable families had telomeres that were 40% longer than those of children who had experienced many changes in family structure, such as a parent with multiple partners.

At this point I wondered who would be silly enough to imagine that n=40 was a suitable basis for concluding anything, and whether anyone would be even more silly to imagine that if there were observed differences between bright and dull mothers that it followed that those differences were caused by independently existing stressful environments, rather than being due to prior differences in the genetics and the behaviour of brighter mothers.

To spell it out: dull mothers are at risk of all sorts of things, from their genes upwards and outwards; brighter mothers might be spared those risks for reasons which range from their genes upwards and outwards.

However, I knew that I must have got it wrong, because such foolish errors would never be tolerated by Nature. So I had a look at the paper.

http://www.pnas.org/content/early/2014/04/02/1404293111.full.pdf+html

Here is the method statement:

Initially, we identified 40 families based on a three-step process. In the first step, the sample was constrained to boys meeting the following conditions: (i) boys provided saliva at age 9 (wave 5) in-home interview, (ii) whose mother self-identified her race as black or African American, (iii) for whom no data were missing on the criterion variables described below, and (iv) who were male.

Next, we arrayed the subsample on an index of advantage–disadvantage from birth to age 9 based on an equally weighted combination of (i) family economic conditions, (ii) parenting practices, and (iii) family structure/stability.

Finally, we took children with the 20 highest scores in the disadvantaged index whose mothers had experienced at least one depressive episode and the children with the 20 lowest scores on the index whose mothers had never experienced depression. Thus, boys who scored highest on this index (n = 20) lived in homes with high levels of poverty, high levels of family instability, harsh parenting, and maternal depression. Boys who scored lowest (n = 20) lived in affluent, stable families and were not exposed to either harsh parenting or maternal depression. We then assayed the children for Telomere Length.

So, from one perspective we can say they chose a bad genetic group and compared it with a good genetic group. From another perspective we can say they chose a bad luck group and compared it with a good luck group.

What steps did the authors take to compare these two perspectives? What steps did they take to distinguish, for example, between between bad luck on the one hand and bad decisions on the other; between the slings and arrows of blind fate on the one hand and the natural consequences of damn fool decisions on the other?

As far as I can tell, none. They assume that all this bad stuff rains down on one group and that the other group is spared, but that in genetic terms both groups are identical, or close enough to warrant a comparison of the effects of these extraneous life events.

I looked up the telomere lengths and did a t test. For the harsh environment 9.6 (4.2) and for the nurturing environment 10.3 (2.5). 20 boys in each sample. Non-significant. Nowhere close. So, I presume it is only significant if you put together a model of variables to be controlled for, but otherwise not.

To my primitive eyes the sample size seems far too small to conclude anything much, and far too small for comparisons of individual gene effects. In addition, there is a highly questionable assumption that all that these boys inherited from their parents was an “environment”. Incidentally, so many fathers were untraceable in the “unlucky” group that fathers had to be left out of the whole study. A look at the genetics of the mothers would be a start. Looking at the telomere length of the mother’s DNA might also be worth a look.

Have I got it entirely wrong, and is there some innocent explanation I have missed?

I would like some assistance from anyone who can help me understand why this paper, which I think flawed, is considered suitable for publication by the editors of Nature.