Comments on: Effects Of Vertical Acceleration On Wrongness

By: Tab Atkins

Tab Atkins — Thu, 06 Mar 2014 10:13:45 +0000

Focus on the “poorly-understood” part. If you read a study on some drug and assume it to be fully true and accurate, you might not ever hear about the rest of the literature that disproves that study or shows it’s results only apply in some limited circumstances.

This happens constantly in real life, and is fairly scary; doctors learn something at some point in their career (often medical school) and then don’t keep up on the progress of their field, and end up giving decades-old medical advice that would be grounds for disciplining a medical student of today.

Report comment

By: Multiheaded

Multiheaded — Wed, 26 Feb 2014 11:43:45 +0000

I swear, Scott’s blog is so good that even the spambots manage to sound wise and thoughtful instead of, say, appealing to base greed.

Report comment

By: Felix Benner

Felix Benner — Fri, 21 Feb 2014 06:30:24 +0000

A double blind study is a statistical method that we use because we have no underlying model to engineer a solution. We do not have a model how different chemicals work in the human body, but we do have a model how a parachute works.

Report comment

By: jsalvatier

jsalvatier — Fri, 21 Feb 2014 05:07:45 +0000

What kinds of things do you have in mind for “actively anti-empirical”?

Report comment

By: Decius

Decius — Fri, 21 Feb 2014 02:19:28 +0000

Yeah, if you wanted to study the effects of parachutes on crash test dummies.

Report comment

By: Athrelon

Athrelon — Thu, 20 Feb 2014 22:25:58 +0000

A very important point, and one that is often lost when this stuff becomes popularized. The cargo cult of the RCT with p > 0.05 is initially a step up from the cargo cult of argument from authority, but with rampant gaming, that advantage has been eroding for some time.

For related reasons, I have serious doubts that the spread of “evidence-based medicine” is currently improving clinical decision-making on the margin; there are many instances when poorly understood EBM, despite using the buzzwords of rationality, ends up being actively anti-empirical.

First-rate truth finding ultimately requires first-rate reasoners; there’s no stable institutional shortcut.

Report comment

By: Sieben

Sieben — Thu, 20 Feb 2014 21:43:38 +0000

Because people will know when they pull the cord whether something activates and comes out. You’d have to blind them, numb their entire bodies so they couldn’t feel the chute pull (or not pull), and probably deafen them with earplugs for good measure.

… You could also use crash test dummies as a proxy for injury outcomes 😛

Report comment

By: suntzuanime

suntzuanime — Thu, 20 Feb 2014 20:24:15 +0000

Parachutes *are* evidence-based medicine, there are other sorts of evidence than the double-blind RCT. The double-blind RCT is the *best* form of evidence (short of the actual Word of God Spoken From On High), but, as in the parachute case, sometimes it’s prohibitively expensive to run one.

Report comment

By: Scott Alexander

Scott Alexander — Thu, 20 Feb 2014 19:55:17 +0000

Yes, it’s a classic joke and widely-referenced. Problem is, when people invoke it to defend not testing their favorite intervention, the intervention is never quite on the same footing as parachutes.

Report comment

By: Scott Alexander

Scott Alexander — Thu, 20 Feb 2014 19:53:56 +0000

Why would a double-blind trial be tricky? Just give backpacks that, when the cord is pulled, either release a parachute or release cords not attached to anything, then blindfold the participants. The skydiving instructors hand out backpacks without knowing which is which.

Report comment