Comments on: Genetic Testing and Self-Fulfilling Prophecies

By: More Links For January | Slate Star Codex

More Links For January | Slate Star Codex — Sun, 26 Jan 2014 08:59:08 +0000

[…] Genetic Testing and Self-Fulfilling Prophecies […]

By: Nancy Lebovitz

Nancy Lebovitz — Thu, 23 Jan 2014 16:47:53 +0000

http://www.independent.co.uk/news/science/older-people-dont-lose-cognitive-power-they-just-know-too-much-say-scientists-9074205.html

This might imply a way to test– see whether older people’s memories improve after reading this sort of article. It’s at least possible that “too much data” would lead to looking for better strategies to remember, while “irreversible brain deterioration” doesn’t.

Report comment

By: Douglas Knight

Douglas Knight — Thu, 23 Jan 2014 13:01:08 +0000

Also, the meta-analysis Chris cited compared published studies to unpublished ones and found massive publication bias. The study you cited is half unpublished studies, but I don’t see them making the comparison, or making a funnel for just published studies. In particular, the second largest study wasn’t published.

Report comment

By: Kibber

Kibber — Thu, 23 Jan 2014 08:28:49 +0000

Just a data point: for this precise reason, 23andme was “locking” such information by default and required some deliberate actions on user’s part to unlock it. Those who don’t want to know – don’t have to know.

Report comment

By: Douglas Knight

Douglas Knight — Thu, 23 Jan 2014 06:11:49 +0000

Yes, looking at the top of the funnel plot amounts to looking only at large studies. Yes, insofar as those counterfactuals hold, that is how you should interpret them. But it is hugely skewed. The effect size for studies under 100 people is a full standard deviation different from the effect size for studies over 100 people.

Report comment

By: Scott Alexander

Scott Alexander — Thu, 23 Jan 2014 00:55:22 +0000

Okay, it’s been a while since I’ve had to read a funnel plot, so let me see if I understand what you’re saying.

Because the top of the funnel is close to zero, that means the most likely effect size value for stereotype threat is close to zero. Insofar as the funnel is slightly asymmetrical skewed to the negative direction, that means there is probably some publication bias pushing results to be more negative. Insofar as the funnel isn’t too asymmetrical, that means there’s probably not too much publication bias. Is that right?

Report comment

By: Scott Alexander

Scott Alexander — Thu, 23 Jan 2014 00:52:34 +0000

Yes, I’ve seen those posts. Unfortunately, they all rely on word of mouth from people in the field and unpublished papers (the only link is to a presentation at a conference that doesn’t have a transcript). One of the researchers involved apparently submitted his result for publication, but it’s been years since then and no word of it.

Report comment

By: pwyll

pwyll — Wed, 22 Jan 2014 23:32:22 +0000

Scott, glad to hear you’ve already started delving into the question of whether stereotype threat exists. Here are a couple of links referring to studies that you’re probably already aware of, but just in case you’re not:
http://isteve.blogspot.com/2010/01/stereotype-threat-scientific-scandal.html
http://isteve.blogspot.com/2011/11/high-stakes-stereotype-threat.html
http://isteve.blogspot.com/2012/10/john-list-on-virtual-nonexistence-of.html
All of these argue against its existence; unfortunately I’m not familiar enough with the topic to know what the best arguments *in favor* of the existence of stereotype threat are.

Report comment

By: Douglas Knight

Douglas Knight — Wed, 22 Jan 2014 04:53:03 +0000

Because the funnel plot you cite sure looks like the studies with large sample size have effect sizes less than 0.2. Come on, look at it.