tag:blogger.com,1999:blog-4624586630299165335.post2117911195730189575..comments2024-03-14T09:50:44.315+00:00Comments on Psychological comments: Digit Span bombshellAnonymoushttp://www.blogger.com/profile/09320614837348759094noreply@blogger.comBlogger29125tag:blogger.com,1999:blog-4624586630299165335.post-62372416861227437392020-11-25T12:08:55.062+00:002020-11-25T12:08:55.062+00:00great issues altogether, you simply gained a brand...great issues altogether, you simply gained a brand new reader. What might you recommend in regards to your post that you just made some days ago? Any sure?ed medicationhttps://erectiledysfunctionpillscvs.com/noreply@blogger.comtag:blogger.com,1999:blog-4624586630299165335.post-53004630565045432952020-11-25T11:49:56.539+00:002020-11-25T11:49:56.539+00:00Hello, i read your blog from time to time and i ow...Hello, i read your blog from time to time and i own a similar one and i was just curious if you get a lot of spam comments? If so how do you prevent it, any plugin or anything you can recommend? I get so much lately it's driving me crazy so any support is very much appreciated.herbal erection pillshttps://erectiledysfunctionpillscvs.com/noreply@blogger.comtag:blogger.com,1999:blog-4624586630299165335.post-53292956102104179122015-05-05T17:04:54.809+01:002015-05-05T17:04:54.809+01:00From Michael A Woodley of Menie
[This is a repost ...From Michael A Woodley of Menie<br />[This is a repost of a comment dated 06/04/2014. Chronologically, it is a response to Gwern 06/04/2014, and Gwern 07/04/2014 is in turn a response to this comment] <br /><br />This is my final commentary on the matter of the digit span paper.<br /><br />Let’s cut to the quick.<br /><br />Comment 1: This seems to be ignoring…<br /><br />Response 1: Why are you weighting by the standard deviation? Normally one weights by the standard error of the mean (SD/√N), inverse of the variance (1/SD^2), or, as in the case of the study under discussion, N, if the samples are being evaluated use an equivalently scaled measure of the same construct. If we are going to consider the effects of standard deviation let us at least do it properly – by factoring it into the model as a predictor on the basis that perhaps the trend is being driven by decreasing range restriction over time. Running an N-weighted least squares multiple regression (SPSS v.21) using year to predict backwards digit span scores, whilst controlling for study standard deviation yields a β value of -.647. This is in fact larger than the N-weighted temporal correlation for this subset of the data (-.41). Note that only the seven studies reporting standard deviation values were used here. The 1923 study is not driving this trend, nor are standard deviation differentials between studies. If anything, the latter is supressing the secular trend to some extent.<br /><br />Comment 2: Where is this ‘significant’ slope….<br /><br />Response: 2: Once again – the correct value of N for determining the significance of the coefficients is the sum of the study N’s, as this gives you the correct degrees of freedom for N weighted analysis. I suggest that you update your code to reflect this meta-analytic reality.<br /><br />Comment 3: So in other words…<br /><br />Response 3: The fact that you don’t see the relevance of nomological validity reinforces my impression that you need to brush up on it.<br /><br />Comment 4: The fact that you explain away…<br /><br />Response 4: Contrary to your representation, Meehl always strongly advocated for the use of theory in deriving scientific predictions – that is what is being done in the paper under discussion. I do not know what you mean by “inferential bankruptcy”, but in modern philosophy of science, theory is not inferred from data but tested against it. That is precisely what we are doing in the paper under discussion.<br /><br />Comment 5: So when digit spans increase…<br /><br />Response 5: No, the decrease of one and increase of the other is explicitly predicted by the co-occurrence model.<br /><br />Comment 6: Really? I’ve already provided the code…<br /><br />Response 6: The code for executing N-weighted correlations in R can be found at the following url:<br /><br />http://web.mit.edu/~r/current/arch/amd64_linux26/lib/R/library/boot/html/corr.html<br /><br />Alternatively, just run the analysis in SPSS or Statistica, the latter will automatically compute the correct significances for you also.<br /><br /><br /><br /><br /><br />Anonymoushttps://www.blogger.com/profile/09320614837348759094noreply@blogger.comtag:blogger.com,1999:blog-4624586630299165335.post-32421004094913784942015-04-07T09:05:31.775+01:002015-04-07T09:05:31.775+01:00It seems that a comment made by Michael has been r...It seems that a comment made by Michael has been responded to by gwern, but it is not on screen. If any of you can retrieve that comment (some time after 6 apr 1.26 I presume) can you please copy it and post it up? I cannot find anything on my admin section of the blog, but we have had problems with comments being lost from time to time.Anonymoushttps://www.blogger.com/profile/09320614837348759094noreply@blogger.comtag:blogger.com,1999:blog-4624586630299165335.post-40060258647164929152015-04-07T03:23:55.931+01:002015-04-07T03:23:55.931+01:00> These are my final remarks on the matter of t...> These are my final remarks on the matter of the digit span paper:<br /><br />This is unfortunate, as I find your responses are deeply unsatisfactory, as well as your strange reluctance to reveal anything whatsoever of your code and how you are actually calculating the numbers you claim.<br /><br />> Response 1: Why are you weighting by the standard deviation?<br /><br />Because the standard deviation is the relevant number for the particular datapoint being meta-analyzed, and the weight of studies is indicated by the N, both of which provide what one needs to know. Standard error is calculated from *them*, why are you precalculating it and running it through a weird correlation process? (I hate to stand on personal anecdotes here but of all the meta-analyses I've read in psychology, I can't say I've seen any where first studies' standard deviations were regressed on their Ns to yield something new; such a procedure seems to be non-standard.)<br /><br />> If we are going to consider the effects of standard deviation let us at least do it properly – by factoring it into the model as a predictor on the basis that perhaps the trend is being driven by decreasing range restriction over time.<br /><br />Range restriction... For data where the mean values are 4 SDs from the lower bound?<br /><br />> Running an N-weighted least squares multiple regression (SPSS v.21) using year to predict backwards digit span scores, whilst controlling for study standard deviation yields a β value of -.647.<br /><br />Using what SPSS code? Please provide it.<br /><br />> I suggest that you update your code to reflect this meta-analytic reality.<br /><br />How?<br /><br />> Response 4: Contrary to your representation, Meehl always strongly advocated for the use of theory in deriving scientific predictions – that is what is being done in the paper under discussion.<br /><br />He also advocated for the things I mentioned.<br /><br />> Response 5: No, the decrease in one and increase in the other is explicitly predicted by the co-occurrence model.<br /><br />Cite? I've read most of your earlier papers, and I do not recall any advance predictions to the effect that 'forward digit span will increase and backwards will decrease'.<br /><br />> Response 6: Some code for executing N-weighted correlations in R can be found at the following url:<br /><br />Irrelevant. I know perfectly well how to run weighted linear regression, and have already done so. A corr function is not a replacement for a mature meta-analysis library.<br /><br />> Alternatively, just run the analysis in SPSS or Statistica, the latter will automatically compute the correct significances for you also.<br /><br />I would be happy to run it in SPSS. If you would provide the code which yields the numbers and claims you have made here.gwernhttps://www.blogger.com/profile/18349479103216755952noreply@blogger.comtag:blogger.com,1999:blog-4624586630299165335.post-72305060831315803032015-04-06T01:26:04.165+01:002015-04-06T01:26:04.165+01:00> Find some solid evidence that Flynn effects a...> Find some solid evidence that Flynn effects are occurring on g and I’ll update my priors accordingly.<br /><br />So when digit spans increase, then it can be on non-g factors (which ones, you don't know, but you feel free to write it off), but when it decreases, it cannot? You want to have your cake and eat it too.<br /><br />> Response 4: False – excluding the 1923 sample yields an N-weighted β of -.41 (df=6789, p<.01).<br /><br />Really? I've already provided the code for my analysis using metafor where excluding 1932 does not yield a negative coefficient. Again, feel free to point out where I coded up the analysis wrong (did I mistranscribe a datapoint? misuse metafor? misinterpret a parameter?) and provide your own corrected code, instead of continued assertion.<br /><br />> Why not exclude all of the smaller, older studies (N<100)?<br /><br />Sure. Now you've restored the sign... but managed to halve the effect:<br /><br />fe3 <- rma(measure="MN", method="FE", yi=LDSB, vi=LDSB.SD, ni=N, mods=Year, data = digit[digit$N>=100,]); fe3<br />...mods -0.0009 0.0606 -0.0142 0.9887 -0.1197 0.1180<br /><br />> Thus the overall trend is not being driven by an outlying value, but comprises two directionally consistent secular trends, the earlier of which has an exaggerated slope relative to the later one.<br /><br />When the sign reverses when you drop the smallest study, it's not consistent. Any consideration of sensitivity would find this a big problem, which is why I brought it up in the first place!<br /><br />> Response 5: Then you are not going to like what is about to hit the pages of Frontiers in Psychology.<br /><br />Not sure what you mean there, but I hope whatever it is is better than this.gwernhttps://www.blogger.com/profile/18349479103216755952noreply@blogger.comtag:blogger.com,1999:blog-4624586630299165335.post-60471851419512470242015-04-06T01:09:29.505+01:002015-04-06T01:09:29.505+01:00> On Bayesian reasoning, and its proper role in...> On Bayesian reasoning, and its proper role in interpreting the current finding, I suggest you brush up on the concept of nomological validity (start with Cronbach & Meehl, 1955, Psychol. Bull, 52, 218).<br /><br />So in other words, despite the fact that the data is so scanty it cannot provide any sort of meaningful update of one's priors, you believe so strongly in dysgenics that you're going to ignore the fact of sampling error and just repeat your priors. So, in less fancy terms: GIGO.<br /><br />> The fact that the trends detected in the present paper are consistent with expectations from the model and with the behavior of other elements in the co-occurrence nomological network increases the nomological validity of this finding<br /><br />The fact that you explain away the trends you don't like, such as the forward digit span increase, and ignore trends you have no explanation for such as the increase in backwards digit span in recent decades (what is sauce for the goose is sauce for the gander), and claim it all confirms your model, shows the inferential bankruptcy of your model. One doesn't need to read Meehl to know that (although I'm surprised that a fellow Meehl fan puts such uncritical reliance on such weak arguments; Meehl was among the most cogent critics in psychology of assumptions like linearity & normality, the homogenous of testing procedures, the constant low-grade background noise of changes in people and the 'crud factor', the questionable causal inferences derived from 'controlling', the abuse of p-values, and - particularly relevant in this context - how psychologists pretended to falsification while actually tendentiously winding their way through a series of underpowered tests of hypotheses and creating ad hoc explanations as necessary to preserve their theories).gwernhttps://www.blogger.com/profile/18349479103216755952noreply@blogger.comtag:blogger.com,1999:blog-4624586630299165335.post-1289130360922563682015-04-06T01:08:31.429+01:002015-04-06T01:08:31.429+01:00> Response 1: If you want to weight by standard...> Response 1: If you want to weight by standard error of the mean you either use sample-specific standard deviations, or you fix the parameters using the largest standard deviation value as the reference value. You can choose one or the other, but not both. You are using the sample-specific standard deviations for each study where these data are available, but you then go and assign the largest value to the smallest study! If you are going to use the largest value as a reference value you apply it to all SE calculations, not just to one data point, as all data points with N’s smaller than the reference population are going to suffer from some degree of sampling error.<br /><br />This seems to be ignoring my point by equiocating on standard error when the question is about the standard deviation. If I do not know a parameter like a mean in a small sample, but I know the parameter exactly in the population, the best unbiased estimator of the sample's mean is going to be the population mean (possibly with a small-sample correction); of course sampling error means the sample's actual mean could be smaller or larger, but it will vary around the population mean, since the sample is drawn from said population. The standard deviation will not change; if I sample 100 people from a population with a true standard deviation of 10, then the sample standard deviations will dance around 10; if I sample 1000 people, then the sample standard deviation will be somewhere around... 10; if I sample a million people, the sample standard deviation will be somewhere around... 10. Because the generating population standard deviation is 10, so of course any sample will be yoked to that. If you run some simulations on a normal distribution comparing the mean of a bunch of sample SDs for various n with the true SD, they're going to look very similar. (eg something like `for (n in 2:6) { print(mean(replicate(10000, sd(rnorm(10^n, sd = 10))))) }`). If one has the SD value for a specific sample, of course it's better, but the most precisely estimated SD is the best if you're forced to impute.<br /><br />Not sure this point is worth the space - if the real SD is larger like 1.49, it only means Wells & Martin 1923 (which is entirely driving the supposed decline) is even less informative than before. But moving onwards.<br /><br />> Response 2: False - the slope is significant at <.0001. How could a Beta value of -.55 with df=6839 NOT be significant?<br /><br />Where is this 'significant' slope? The p-value is not significant at <0.0001 because it's more like 0.99. Feel free to present code showing otherwise. As I said, I sure can't figure out how to get my meta-analysis code to spit out p<.0001 for a small decline in digit span based entirely on a n=50 sample.gwernhttps://www.blogger.com/profile/18349479103216755952noreply@blogger.comtag:blogger.com,1999:blog-4624586630299165335.post-18359363744894464772015-04-06T00:38:11.828+01:002015-04-06T00:38:11.828+01:00Comment 3: while claiming that the *other* variabl...Comment 3: while claiming that the *other* variable's changes are driven by non-g factors in the subjects<br /><br />Response 3: Find some solid evidence that Flynn effects are occurring on g and I’ll update my priors accordingly. <br /><br />Comment 4: In other words, this supposed decline in backwards digit span is due *solely and completely* due to the one small early Wells & Martin 1923 estimate of 5.10 (n=50) compared to all the other estimates of ~4.8; without Wells & Martin, we would instead estimate a raw rise of 0.001 digits per decade. That is, the trend is being driven by not just the smaller samples, but by the smallest sample.<br /><br />Response 4: False – excluding the 1923 sample yields an N-weighted β of -.41 (df=6789, p<.01). But why stop with excluding the 1923 data point? If you are going to do a robustness analysis of this sort, you really should employ proper exclusion rules. Why not exclude all of the smaller, older studies (N<100)? When this is done the strength of trend with respect the remaining large-sample cases is clear (β=-.74, df=6623, p<.01). What happens if we just look at the smaller, older studies separately? We get another strong indication of a trend (β =-.98, df=214, p<.01). Thus the overall trend is not being driven by an outlying value, but comprises two directionally consistent secular trends, the earlier of which has an exaggerated slope relative to the later one. The stronger slope across the earlier samples could well be due to sampling error. Simply regressing all of the sample scores against time, weighting each by sample size is therefore a perfectly sound way of deriving a valid point estimate of the overall temporal trend. <br /><br />Comment 5: I am not sure why one would want to write a paper on implications derived from noise and a tower of questionable assumptions.<br /><br />Response 5: Then you are not going to like what is about to hit the pages of Frontiers in Psychology. The co-occurrence model has now been directly and exhaustively tested and validated. My priors have never looked so good!<br />Michael A. Woodley of Menienoreply@blogger.comtag:blogger.com,1999:blog-4624586630299165335.post-15698937002760989972015-04-06T00:31:29.606+01:002015-04-06T00:31:29.606+01:00Comment 1: I don't follow. The argument in you...Comment 1: I don't follow. The argument in your paper was that the SD of the large samples, with n=thousands and hence minuscule sampling-error, can be considered to be equivalent to the true population SD; given this assumption, then in smaller draws from that population, such as n=50, the best estimate of the unknown sample SD will be the same as the population SD, no? I don't understand this business about correlations.<br /><br />Response 1: If you want to weight by standard error of the mean you either use sample-specific standard deviations, or you fix the parameters using the largest standard deviation value as the reference value. You can choose one or the other, but not both. You are using the sample-specific standard deviations for each study where these data are available, but you then go and assign the largest value to the smallest study! If you are going to use the largest value as a reference value you apply it to all SE calculations, not just to one data point, as all data points with N’s smaller than the reference population are going to suffer from some degree of sampling error. <br /><br />Comment 2: So, you're writing a paper about an estimated slope which is:<br /><br />not remotely statistically-significant evidence for it being non-zero, and almost zero Bayesian evidence for non-zero values - on a set of data which has no estimated heterogeneity beyond sampling error which, for the point-estimate to mean anything at all, requires that one pretend sampling error is not driving the estimate, that the measured populations are identical, that the change reflect g-related changes in the subjects rather than the large variance due to all other factors <br /><br />Response 2: False - the slope is significant at <.0001. How could a Beta value of -.55 with df=6839 NOT be significant?<br /><br />On Bayesian reasoning, and its proper role in interpreting the current finding, I suggest you brush up on the concept of nomological validity (start with Cronbach & Meehl, 1955, Psychol. Bull, 52, 218). There is now a large nomological network of supporting evidences for the co-occurrence model. The fact that the trends detected in the present paper are consistent with expectations from the model and with the behavior of other elements in the co-occurrence nomological network increases the nomological validity of this finding – in addition to the prior probability of this latest result indicating something rather than nothing. <br />Michael A. Woodley of Menienoreply@blogger.comtag:blogger.com,1999:blog-4624586630299165335.post-56348371679593899852015-04-04T23:57:01.152+01:002015-04-04T23:57:01.152+01:00> With a combined N of several thousand, the re...> With a combined N of several thousand, the results of most N-weighted regressions will be statistically significant.<br /><br />In this case, it seems it was not. As one would expect on a time-series where the most distant (and hence, most informative) datapoints are tiny like n=50.<br /><br />> For this he should not use the SD value for the largest sample, but should instead use the correlation between SD and N to impute an SD value for the missing case based on its sample size (50).<br /><br />I don't follow. The argument in your paper was that the SD of the large samples, with n=thousands and hence minuscule sampling-error, can be considered to be equivalent to the true population SD; given this assumption, then in smaller draws from that population, such as n=50, the best estimate of the unknown sample SD will be the same as the population SD, no? I don't understand this business about correlations.<br /><br />> In the present case however the digit span measures are scaled equivalently, therefore weighting by SE is not necessary and simple N weighting is therefore appropriate.<br /><br />It may be a simplification to drop the standard deviations, but it is useful information about how large population variability is, and so I am loathe to omit it.<br /><br />> ii) The robustness of the particular result (i.e. the degree to which it withstands the application of alternative and more rigorous statistical procedures).<br /><br />So, you're writing a paper about an estimated slope which is:<br /><br />- not remotely statistically-significant evidence for it being non-zero, and almost zero Bayesian evidence for non-zero values<br />- on a set of data which has no estimated heterogeneity beyond sampling error<br />- which, for the point-estimate to mean anything at all, requires that one pretend sampling error is not driving the estimate, that the measured populations are identical, that the change reflect g-related changes in the subjects rather than the large variance due to all other factors <br />- while claiming that the *other* variable's changes are driven by non-g factors in the subjects<br /><br />I am not sure why one would want to write a paper on implications derived from noise and a tower of questionable assumptions.<br /><br />> The trends presented in the paper are not being driven by the smaller samples, but by the bigger ones.<br /><br />That is impossible. The early samples are also the smallest; estimates of a linear trend are dominated by the earliest and latest datapoints, because the predicted difference from the intercept is maximized at them, hence, the possible temporal trend is thanks in part to the smaller/earlier samples. (This is well understood for the theory of optimal experiment design, where one maximizes the power of experiments intended to detect a linear trend by allocating samples to the 2 extrema, and spending no samples on any intermediate points. See for example "Optimal Design in Psychological Research", McClelland 1997, pg4, which discusses the most efficient design for linear effects.) And we can see how much the early data points matter by simply dropping the *first* study and noting that the sign becomes *positive*:<br /><br />R> fe2 <- rma(measure="MN", method="FE", yi=LDSB, vi=LDSB.SD, ni=N, mods=Year, data = digit[-1,]); fe2<br />...<br />mods 0.0001 0.0171 0.0038 0.9969 -0.0334 0.0335<br /><br />In other words, this supposed decline in backwards digit span is due *solely and completely* due to the one small early Wells & Martin 1923 estimate of 5.10 (n=50) compared to all the other estimates of ~4.8; without Wells & Martin, we would instead estimate a raw rise of 0.001 digits per decade. That is, the trend is being driven by not just the smaller samples, but by the smallest sample.gwernhttps://www.blogger.com/profile/18349479103216755952noreply@blogger.comtag:blogger.com,1999:blog-4624586630299165335.post-91925658427690199152015-04-04T22:04:26.883+01:002015-04-04T22:04:26.883+01:00@JayMan
N weighted regression deals with the prob...@JayMan<br /><br />N weighted regression deals with the problem of sampling error by assigning larger weighting terms to samples with bigger values of N (and commensurately lower sampling error), thus bringing out the "true" relationship amongst the variables. The trends presented in the paper are not being driven by the smaller samples, but by the bigger ones. <br /><br />To understand this issue better, I suggest reading Chapter 3, specifically pages 95-96 in Schmidt and Hunter (2014).<br /><br />Ref.<br /><br />Schmidt, F. L., & Hunter, J. E. (2015). Methods of meta-analysis. Correcting error and bias in research findings (3rd ed.). Thousand Oaks: Sage.Michael A. Woodley of Menienoreply@blogger.comtag:blogger.com,1999:blog-4624586630299165335.post-86116941183596908822015-04-04T21:37:24.695+01:002015-04-04T21:37:24.695+01:00[Second part]
Alternatively, he could assign to e...[Second part]<br /><br />Alternatively, he could assign to each mean a fixed standard deviation value, using the largest value (i.e. the one that is least attenuated by sampling error; 1.35 for digits backwards and 1.49 for digits forwards), which is meta-analytically sound as it is a synthetic correction for sampling error.<br /><br />The commenter has a good point - weighting by SE is very often useful, especially in meta-analyses where different indicators of the same construct, scaled in different metrics, are being compared (i.e. GRE and SAT scores), thus making them comparable. In the present case however the digit span measures are scaled equivalently, therefore weighting by SE is not necessary and simple N weighting is therefore appropriate.<br /><br />My co-author and I are fully aware of just how arbitrary significance is in the case of N weighted regression. In point of fact, in the first version of the paper, my co-author and I did not even report significance. We actually ended up doing to so on the suggestion of an expert on meta-analysis. <br /><br />The commenter will notice also that we don’t make a big deal out of the significance of the results. Instead, we explicitly state that more attention should be paid to the following: <br /><br />i) The nomological validity of a given result (i.e. the degree to which it accords with theory and with the pattern of other results). <br />ii) The robustness of the particular result (i.e. the degree to which it withstands the application of alternative and more rigorous statistical procedures). <br />iii) The magnitude of the result (modular coefficients of >.5 are considered to be large magnitude by many polytomous effect size division schemata). <br /><br />We are simply building on Gignac's findings and incorporating them into the broader theoretical framework of the Flynn effect, of which the co-occurrence model is now an established part. <br />Michael A. Woodley of Menienoreply@blogger.comtag:blogger.com,1999:blog-4624586630299165335.post-15042289063701673892015-04-04T21:29:08.546+01:002015-04-04T21:29:08.546+01:00[First part]
In computing significance and 95% CI...[First part]<br /><br />In computing significance and 95% CI’s for the results of N weighted regression you use the combined N of participants - this is because weighting by N simulates the results that would be obtained in performing a secondary analysis of the pooled raw data. Computing an N-weighted correlation is equivalent to simply duplicating each observation by the number of participants per observation. So, correlating (simulated) paired values of 1930, 0.3 (N=4), 1940, 0.2 (N=5), and 1960, 0.7 (N=6) is equivalent to correlating the following columns:<br /><br />X Y<br />1930, 0.2<br />1930, 0.2<br />1930, 0.2<br />1930, 0.2<br />1940, 0.3<br />1940, 0.3<br />1940, 0.3<br />1940, 0.3<br />1940, 0.3<br />1960, 0.7<br />1960, 0.7<br />1960, 0.7<br />1960, 0.7<br />1960, 0.7<br />1960, 0.7<br /><br />The degrees of freedom in the above are not 3-2 (the number of observations), but 15-2 (the sum of the weights). With a combined N of several thousand, the results of most N-weighted regressions will be statistically significant. Some analytical software (e.g. certain versions of SPSS and Statistica) will calculate the correct significances and CIs automatically, some do not however, and require that these parameters be manually computed instead. <br /><br />There are some errors in the commenter’s replication attempt. I assume that he is attempting to weight by standard error of the mean (henceforth SE), in which case there are two ways of doing this. Firstly, he can use sample-specific standard deviation values, which necessitates that he impute the missing standard deviation value for the Wells and Martin datapoint. For this he should not use the SD value for the largest sample, but should instead use the correlation between SD and N to impute an SD value for the missing case based on its sample size (50). The correlations for both backwards and forwards digits are virtually perfect (r=.97 and .93 respectively), therefore he would assign the values of 1.15 (SE=.16, i.e. 1.15/√50) and 1.10 (SE=.16) to the backwards and forwards digit span means from Wells and Martin. <br />Michael A. Woodley of Menienoreply@blogger.comtag:blogger.com,1999:blog-4624586630299165335.post-30867279124136189242015-04-04T04:27:29.042+01:002015-04-04T04:27:29.042+01:00You're very welcome, James! Unfortunately, I t...You're very welcome, James! Unfortunately, I think physics is difficult even for a physicist to break into. This is part of what I love about the soft sciences - they're still in their golden age, when low hanging fruit is ripe for the picking. To make breakthroughs in physics, one either needs billions of dollars in laboratory equipment, or else a focus so narrow and sophisticated that most laypeople won't understand one's research at all.Markhttp://awesomescience.us/noreply@blogger.comtag:blogger.com,1999:blog-4624586630299165335.post-18884259328053148232015-04-04T00:13:10.814+01:002015-04-04T00:13:10.814+01:00with an estimated mean of roughly 5 sd 1.5 or 2 (f...with an estimated mean of roughly 5 sd 1.5 or 2 (from memory) I doubt it is normal, more likely to have negative skew.Anonymoushttps://www.blogger.com/profile/09320614837348759094noreply@blogger.comtag:blogger.com,1999:blog-4624586630299165335.post-66315649602099574542015-04-03T22:33:26.351+01:002015-04-03T22:33:26.351+01:00Has anybody attempted to see if backwards digit sp...Has anybody attempted to see if backwards digit span in fact follows a strictly normal curve?<br /><br />Insofar as it expresses a fundamental brain mechanism or process that varies according to additive genes, one would expect that it would adhere pretty strictly to a normal curve. <br /><br />It would be pretty valuable to know that a measure such as backward digit span is both a measure of real abilities and also follows a strictly normal curve.candid_observerhttp://www.excite.comnoreply@blogger.comtag:blogger.com,1999:blog-4624586630299165335.post-40973039328643185182015-04-03T22:32:55.425+01:002015-04-03T22:32:55.425+01:00Has anyone reproduced Woodley's analysis here?...Has anyone reproduced Woodley's analysis here? I can't figure out why he's putting out this paper; whenever I try meta-analyzing Gignac's dataset the obvious way (fixed-effects meta-analysis weighted by N & SD, Year as moderator variable), the year variable isn't even *remotely* statistically-significant (nor are the backwards digit spans heterogeneous in the first place), but Woodley's paper claims 95% CIs which exclude 0...<br /><br />My data & R code: http://pastebin.com/L2MDM1WR<br /><br />Or inline:<br /><br /># "DST = Digit Span Total; W-B = Wechsler-Bellevue; WMS = Wechsler Memory Scale; WAIS = Wechsler Adult Intelligence Scale; MAS = Memory Assessment Scales; NA = not available."<br /># standard deviations split out as separate columns<br />digit <- read.csv(stdin(),header=TRUE)<br />Source,Year,N,Ages,LDSF,LDSF.SD,LDSB,LDSB.SD,DST<br />Wells and Martin,1923,50,Adults,6.3,NA,5.1,NA,11.40<br />Wechsler,1933,236,Adults,6.60,1.13,NA,NA,NA<br />Weisenburg et al,1936,70,18-59,6.69,1.02,4.87,1.16,11.56<br />W-B,1939,1081,17-70,NA,NA,NA,NA,12<br />WMS,1945,96,20-49,6.53,1.17,4.80,1.12,11.23<br />WAIS,1955,1785,16-75,NA,NA,NA,NA,11<br />WAIS-R,1981,1880,16-74,6.45,1.33,4.87,1.43,11.32<br />MAS,1991,845,18-90,6.63,1.22,4.83,1.30,11.46<br />WAIS-III,1997,2000,16-74,6.59,1.35,4.85,1.49,11.44<br />WAIS-IV,2008,1900,16-74,6.72,1.31,4.84,1.39,11.4<br /><br /># 'Wells and Martin 1923' is missing SDs; following Woodley, impute an SD:<br /># "...1.49 for DSB and 1.35 for DSF from the 1997 standardization of the WAIS- III, N = 2000, Wechsler, 1997"<br />digit[is.na(digit$LDSB.SD),]$LDSB.SD <- 1.49<br />digit[is.na(digit$LDSF.SD),]$LDSF.SD <- 1.35<br /><br />library(metafor)<br /><br />fe1 <- rma(measure="MN", method="FE", yi=LDSB, vi=LDSB.SD, ni=N, mods=Year, data = digit); fe1<br /># Fixed-Effects with Moderators Model (k = 7)<br /># Test for Residual Heterogeneity: <br /># QE(df = 5) = 0.0307, p-val = 1.0000<br /># <br /># Test of Moderators (coefficient(s) 2): <br /># QM(df = 1) = 0.0112, p-val = 0.9157<br /># <br /># Model Results:<br /># <br /># estimate se zval pval ci.lb ci.ub<br /># intrcpt 7.8354 27.9588 0.2802 0.7793 -46.9629 62.6337<br /># mods -0.0015 0.0142 -0.1059 0.9157 -0.0294 0.0263<br />plot(fe1)<br /># https://i.imgur.com/P7rMfNQ.png<br />gwernhttps://www.blogger.com/profile/18349479103216755952noreply@blogger.comtag:blogger.com,1999:blog-4624586630299165335.post-31832017200844778712015-04-03T22:02:21.981+01:002015-04-03T22:02:21.981+01:00it could be, but throughout all the telephone year...it could be, but throughout all the telephone years up to 1966 UK cities had a 3 letter 4 number structure. Guy's Hospital was HOP7600 because hop pickers came from Borough and went hop picking in Summer. Scotland Yard was WHI1212 for Whitehall, and so on. So you just said your neighbourhood, and then four digits. Now, as you say, numbers are no longer required, but passwords are, so unless we use the same one everywhere, that should keep us just as nimble.<br />Anonymoushttps://www.blogger.com/profile/09320614837348759094noreply@blogger.comtag:blogger.com,1999:blog-4624586630299165335.post-38346559173975438122015-04-03T21:49:30.530+01:002015-04-03T21:49:30.530+01:00Jayman, it is not a big effect, but there's so...Jayman, it is not a big effect, but there's something there. Make some allowances for data archaeology Anonymoushttps://www.blogger.com/profile/09320614837348759094noreply@blogger.comtag:blogger.com,1999:blog-4624586630299165335.post-6350502985121314512015-04-03T21:14:53.662+01:002015-04-03T21:14:53.662+01:00I wonder if mobile phones have a larger storage ca...I wonder if mobile phones have a larger storage capacity shows up on these tests/has an effect? I remember when land lines were predominant and mobiles came with stands and a slot for an extra battery-charger (One in the phone and one in the charger). <br /><br />In my memory it was a point of pride among the youth to be able to remember and recall mobile numbers of the social hierarchy. Granted they didn't look at it as a mental exercise but rather a source of status signaling. Do you think that the current high storage capacity of the mobile's contact list may show up on tests as an effect among the younger (say 1985 to 1995) millenials versus those born later?CowardlyAnonymousnoreply@blogger.comtag:blogger.com,1999:blog-4624586630299165335.post-60262342088674777832015-04-03T19:48:13.331+01:002015-04-03T19:48:13.331+01:00Digit span backwards looks pretty dang constant to...Digit span backwards looks pretty dang constant to me, especially when you factor in random sampling error from the earlier smaller samples.JayManhttp://jaymans.wordpress.com/noreply@blogger.comtag:blogger.com,1999:blog-4624586630299165335.post-76949721534153438912015-04-03T19:45:01.985+01:002015-04-03T19:45:01.985+01:00On a quick search the first reference I can find i...On a quick search the first reference I can find is Humstone, J.(1919). Memory Span Tests. Psychol. Clin., 12, 196-200, so I think that it is merely that at that stage memory span testing took on the digit span form. However, learning long numbers probably followed the rise in use of the telephone, as you suggest.Anonymoushttps://www.blogger.com/profile/09320614837348759094noreply@blogger.comtag:blogger.com,1999:blog-4624586630299165335.post-30886038516884891292015-04-03T19:38:12.041+01:002015-04-03T19:38:12.041+01:00No kitchens here. Sculleries and larders!
No kitchens here. Sculleries and larders!<br />Anonymoushttps://www.blogger.com/profile/09320614837348759094noreply@blogger.comtag:blogger.com,1999:blog-4624586630299165335.post-80424079616792275122015-04-03T19:35:17.101+01:002015-04-03T19:35:17.101+01:00Mark, thank you for linking me to your informative...Mark, thank you for linking me to your informative blog. It is always good when a physicist gets interested in psychology (Dick Feynman did). As far as I know there are few instances of psychologists breaking into Physics. If you come across any, let me know.Anonymoushttps://www.blogger.com/profile/09320614837348759094noreply@blogger.com