Comments on: Two Dark Side Statistics Papers

By: rich

rich — Sun, 02 Mar 2014 15:42:21 +0000

How about manufacturing a false negative? They used to have two major categories of autism, late onset and early onset, but when they saw the need to censor the vaccination connection they started lumping them together because the late onset is so suggestive of standard vaccination schedules. Now that statistic is buried in noise.

Report comment

By: Somewhere else, part 110 | Freakonometrics

Somewhere else, part 110 | Freakonometrics — Tue, 21 Jan 2014 00:52:22 +0000

[…] “Undisclosed Flexibility In Data Collection And Analysis Allows Presenting Anything As Significant” http://socio.mta.hu/dynamic… and ” How To Have A High Success Rate In Treatment: Advice For Evaluators Of Alcoholism Programs” http://slatestarcodex.com/S… via ”Two Dark Side Statistics Papers” http://slatestarcodex.com/2014/01/… […]

Report comment

By: Chronic Pain and Addiction « Econstudentlog

Chronic Pain and Addiction « Econstudentlog — Fri, 17 Jan 2014 11:07:16 +0000

[…] may be reasonably said to be engaging in medical malpractice.” (on a related point, see this post by Scott Alexander (section […]

Report comment

By: Y. V.

Y. V. — Fri, 10 Jan 2014 23:56:31 +0000

That seems like an awfully stupid reason to discard a whole plan of research. What about survey data prohibits the scientific method from being implemented?

Report comment

By: False-Positive Psychology

False-Positive Psychology — Fri, 10 Jan 2014 20:01:21 +0000

[…] post over at Slate Star Codex on how people use statistics to cheat at science. I already gave my take […]

Report comment

By: 7 Quick Takes (1/10/14)

7 Quick Takes (1/10/14) — Fri, 10 Jan 2014 06:04:25 +0000

[…] And finally, if you want to warp our understanding of reality (or be on your guard against people who do), you want to read Scott’s summary of Dark Side Statistics. […]

Report comment

By: Julia

Julia — Fri, 03 Jan 2014 00:44:50 +0000

>”One of them will be significant 20% of the time, so you can just report that one: “People who eat a moderate amount of vegetables are less likely to get depression than people who eat excess vegetables” sounds like a perfectly reasonable result.””

This sounds a lot like the “Women who have more pre-marital sexual partners are more likely to fail at first marriage” graph from your anti-Reactionary FAQ.

Report comment

By: John Salvatier

John Salvatier — Thu, 02 Jan 2014 19:29:48 +0000

That strategy actually seems like it could work, but almost seems like something referees do now, pressure authors to tell all the metrics they collected and tested.

Report comment

By: Nancy Lebovitz

Nancy Lebovitz — Thu, 02 Jan 2014 15:31:18 +0000

Thanks– it’s good to have a compendium (incomplete?) of methods for cherry-picking data.

Report comment

By: Cyan

Cyan — Thu, 02 Jan 2014 15:07:36 +0000

The problem with the discussion (paragraph, really) of Bayesian approaches is that they don’t discuss the actual performance of the default WWAGD (what would Andrew Gelman do?) model as applied to their researcher-degrees-of-freedom scenarios. Researcher-degrees-of-freedom of types 1, 3, and 4 are all based on permitting one comparison to to stand out from the crowd entirely thanks to chance, and multilevel modelling (of any type; distinctions between Bayesian and frequentist approaches get a bit blurry) specifically mitigates that. That is, if you use such a model, comparisons that stand out from the crowd are far less likely to do so entirely thanks to chance.

I’m still thinking hard about optional stopping (type 2 researcher-degree-of-freedom).

Report comment