(In the spirit of GiveWell, only less obsessive.)
This lists major mistakes I’ve made in my posts that somewhat affected my conclusions about things. Most are factual; a few are cases where I said things in a misleading way or a way I later regretted. This doesn’t list times other people thought I was wrong but after reading all the arguments I still stand by my original position, or minor corrections of typos/details.
1. (10/20/13) In The Anti-Reactionary FAQ, I claimed that there was likely not much difference in crime between the distant past (especially Victorian England) and today, because although the reported burglary rate was up, the reported murder rate stayed the same, and murder is the most accurately recorded crime. Michael Anissimov points out that medical care has improved since that time, so that many things that would have been murders in the past are now only attempted murders, lowering the apparent murder rate by as much as five times. Discovering the mistake caused me to reverse my conclusion that crime has not been increasing since the Victorian age.
2. (6/29/14) In Invisible Women, I pointed out a paradox – how come women are joining the workforce, but GDP has not gone up proportionally to the increased number of workers? Pseudoerasmus explained that men were working less often and working fewer hours in a way that counterbalanced increased female employment. Discovering the mistake did not affect any “conclusion” since I was just asking the question, but the question turned out to be much easier and less weird than I had expected.
3. (4/22/15) In Growth Mindset 3: A Pox On Growth Your Houses, I claimed that a graph showed that most conditions of an educational experiment deteriorated over time, and that since this was very strange the study probably couldn’t be trusted. In fact, the graph was standardized in a way I didn’t notice, and showed only that those conditions did worse than the other conditions, without deteriorating outright. This made the study much more believable than I had thought. The author of the original study corrected me and I explained the correction in detail here. Discovering the mistake lessened my confidence in my conclusion (growth mindset isn’t very impressive and often fails outright) without entirely reversing it.
4. (12/1/15) In College And Critical Thinking, I claimed that a graph showed a u-shaped relationship between time spent in college and critical thinking, which suggested that the relationship between the two was too confusing and unpredictable to be very strong. In fact, commenter PSJ pointed out that this was only true of a small sample of two-year college students, and that most college students showed the expected linear relationship. Discovering the mistake strengthened my conclusion (that college probably does improve critical thinking skills at least in the short term).
5. (1/1/15) In my January 2016 links post, I noted that according to an article in Mother Jones, OxyContin abuse kills three times more people than homicide. Although the article was about OxyContin abuse, the specific statistic cited was about all fatal drug overdoses. This wouldn’t have been such a big deal except that it was linked by Marginal Revolution. Sorry, Marginal Revolution.
6. (3/26/16) In my March 2016 links post, I linked to a Wikipedia page about a radar detector detector detector detector. Rational Conspiracy has looked into it further and believes that was a hoax and there was no such thing. The hierarchy of radar detection most likely ends at radar detector detector detectors.
8. (12/1/16) I mentioned a few times (eg here) a theory that increased carbon dioxide in poorly-ventilated areas might hurt cognition, especially in the context of global warming which we expect to cause increased carbon dioxide everywhere. Commenters pointed out that submarines had carbon dioxide levels double or triple those of anywhere else studied, but submariners seem pretty with-it and don’t show noticeable cognitive declines. My guess (75% confidence) is now that there’s no cognitive penalty for carbon dioxide within the levels humans encounter in normal situations, but this could change if I see more good studies on it.
9. (12/30/16) In Vegetarianism For Meat-Eaters, part 2 suggested that donating to animal welfare charities could save 3 – 11 animal lives per dollar. Based on critiques like those in this essay, I now think those numbers are heavily exaggerated, maybe by several orders of magnitude. I don’t know what the right numbers are or whether the point is still somewhat valid.
10. (12/31/16) In Contra NYT On Economists On Education, I said that an NYT article was so deceptive that it constituted journalistic malpractice. Some people commented that they interpreted the article in a different, non-deceptive way. Although I still think most people would interpret the article incorrectly, I think it’s sufficiently possible to interpret it as intended that it was probably just an honest disagreement in interpretations and not a deliberate attempt to mislead. I deleted the “journalistic malpractice” phrase after about an hour, and I apologize to the Times and to the author for imputing motive. I still think they dropped the ball, though.
11. (4/9/17) In Some Groups Of People Who May Not 100% Deserve Our Eternal Scorn, I said that I thought Vox was mostly behaving responsibly, insofar as everyone knows its biases, and looking beyond them it has good articles on a variety of subjects. Several commenters argued that they had an irresponsible tendency to publish informal “meta-analyses” of the arguments for and against a specific position, while exaggerating the arguments on their side and leaving out the best arguments on the other. They said the “review of evidence” format has an implicit claim to objectivity that can’t be sidestepped just by saying “everyone knows their biases”, and gave some examples of this being used really egregiously. After some thought, I agree. I continue to personally respect many of the people at Vox and enjoy their articles, but saying that they “may not 100% deserve our eternal scorn” was probably going too far.
12. (4/9/17) G.K Chesterton On AI Risk was an April Fools’ joke and not meant seriously. But it did criticize Maciej Ceglowski’s piece where he accused singularitarians of not caring enough about the poor. The fake Chesterton of the piece said that if Ceglowski himself gave to charity at the same level as the people he was criticizing, he would “eat his hat”. Ceglowski pointed out that he does indeed give a lot of money to charity, including a $15,000 donation last year. Under the circumstances, I felt honor-bound to eat my hat and post a video of it on Twitter.
13. (7/10/17) In Change Minds Or Drive Turnout, I repeated the claim that turnout was down 2% between 2012 and 2016, and tried to determine what that meant for political strategy. In fact, this was based on early numbers before all ballots were in, and turnout probably rose during that time. Some of my analysis in Part III of the post is invalid or obsolete, though it probably doesn’t affect the overall conclusion.
14. (9/24/17) In Hungarian Education III, I mention that it’s implausible that the same family could give birth to three geniuses – regardless of the IQ of the parents – due to regression to the mean. Several commenters pointed out this is true only if the parents had high IQ for non-genetic reasons. If the parents had high IQ for genetic reasons, and performed neither better nor worse than their genes, then regression to the mean would not be expected and the family could give birth to three geniuses.
15. (9/24/17) In Mental Disorders As Networks, I tentatively endorsed a theory that there may not be a single underlying cause to all the symptoms of psychiatric disorders, but that instead they might all cause each other by spreading activation through a symptom network. While I still think there might be some element of that, learning more about predictive coding has made me think that psychiatric disorders might come from imbalances in various really-high-level features of the kind of processing the brain does, like “too much bottom-up compared to top-down processing” or “overly high confidence in neural predictions”. See Towards A Predictive Theory Of Depression for an example. This would explain the previously inexplicable tendency of psychiatric disorders to be caused by practically anything but still have the same symptoms, and it would mean that network theory was at least mostly a dead end.
16. (10/22/17) In Book Review: Age of Em, I said we could calculate where hardware would be in fifty years by assuming Moore’s Law held. But Moore’s Law doesn’t seem to be holding anymore, or at least is on the verge of not holding, and that calculation would have been very wrong.
17. (11/19/2017) In Depression Is Not A Proxy For Social Dysfunction, I cited results that happier states had higher suicide rates. A more recent study using a finer level of analysis is not able to replicate those results. This strengthens the case that the European results are just an artifact of more vs. less industrialized, strengthens a possible case that western states and high-altitude states are correlated and western states have higher happiness while high-altitude states have more suicide, and weakens a case that high altitude directly causes higher suicide rates. Overall it weakens a case that happiness and suicide are correlated in a meaningful way.
18. (12/17/2017) In my posts about tax policy, I misunderstood the rates at which growth to the economy compounds; as a result, economy-expanding policies look better compared to direct-redistribution policies than I’d originally calculated. I still think the existing tax bill probably doesn’t expand the economy that much. There were a few other errors mentioned in the same post.
19. (12/17/2017) In Right Is The New Left, I predicted that liberalism was becoming so mainstream and busybodyish that rebellious young people would start leaning right just to avoid it. Instead, it looks like rebellious young people have moved to a democratic socialist Bernie-Sanders-esque position that constantly attacks mainstream liberals for being sellouts and shills and morons. In retrospect this makes a lot of sense, but it’s the opposite of what I predicted.
20. (4/10/2018) Despite some efforts otherwise, my post on adult neurogenesis came off sounding like it was settled science that it didn’t happen. A few days later, this study came out, providing some more evidence for the “it does happen” side. See also “this comment. Although the subject is still in doubt, and although there’s still a really interesting contrast between the certainty with which people discuss the intricacies of neurogenesis and the uncertainty about whether it happens at all, it would have been more accurate to frame the post in these terms instead of as “it definitely doesn’t happen”.
21. (4/12/2018) The post Why DC’s Low Graduation Rates found that DC has very low graduation rates relative to their test scores (relative to other cities and states) and concluded that the most likely explanation was DC had stricter standards. This was framed as other states and school districts likely had borderline-fraudulent practices of passing students who hadn’t learned very much. Although there’s independent evidence for this and it definitely happens to some degree, based on some of the evidence in Highlights From The Comments On DC Graduation Rates it looks like another major (more important?) factor is DC’s strict policies on student absences, and possible screwy incentives that make student absences difficult to avoid. If this is true, focusing so much on standards was a mistake of emphasis.
22. (6/16/18) In HPPD And The Specter Of Permanent Side Effects, I focused on cell death as the likely explanation for occasional permanent side effects from psychedelics. Some commenters brought up an alternate model based on aberrant learning. Although the specifics don’t make a lot of sense, this would present a better explanation for occasional LSD flashbacks and for seemingly-similar conditions like mal de debarquement. It was a mistake of emphasis to focus on cell death rather than on this possibility.
23. (8/28/18) In Elegy For John McCain, I tried to write a poem that balanced my various conflicting feelings about him, including as a high-minded and noble person, a martial hero who refused to tolerate evil, and somebody who when you got down to the consequentialist brass tacks of it spent a big part of his life promoting war and death in a disastrous way. I apparently failed terribly at this and everyone thought it was just attacking McCain in a way that was cruel to do in a period of mourning after his death. I don’t want to claim this is just them being wrong, because I think the poem does have an element of that and I misjudged its appropriateness at least as much as I misjudged other people’s reaction, and because enough smart people interpreted the poem that way that I probably have to accept that’s the way the poem is. I also should hold myself to higher standards since I’ve come out as against this kind of thing before.
25. (11/20/18) My post The Economic Perspective On Moral Standards started with a discussion of the phrase “there is no ethical consumption under late capitalism”. Some people brought up that this phrase may usually be used in a way opposite to the way I was describing it. See the comments for discussion, but given the potential error I excised it from the post.
26. (3/28/19) I reported on a survey I did that showed people’s intuitive moral valuation of animals matched their number of cortical neurons spookily well. A commenter replicated the survey with a larger sample and got different results. I partially retracted the post on the survey based on these concerns, but then we found a way to sort of reconcile the two studies, so I updated the partial retraction. It’s complicated.
27. (4/24/19) A key piece of evidence in Why Were Early Psychedelicists So Weird? was a study finding psychedelics caused a permanent increase in openness. A more recent study failed to replicate this finding. I’m not sure how much to discount the post based on this.
28. (8/21/19) In It’s Bayes All The Way Up, I suggested that psychedelics might work by strengthening the brain’s top-down priors. A more recent paper by Carhart-Harris and Friston provides compelling evidence that psychedelics probably work by weakening the brain’s top-down priors, meaning I got this one maximally wrong. This probably suggests a few other related posts, like Mysticism and Pattern-Matching, have serious flaws or are looking at things in unproductive ways.
29. (9/8/19) In Age Gaps And Birth Order Effects, I found that birth order effects stop mattering suddenly after a seven year gap between siblings. A Less Wrong user tried to replicate and reanalyze the results, and found a less sudden decrease around seven years. They concluded that birth order effects likely decreased more gradually after “4 – 8” years. I agree their reanalysis is better and have edited my post to link to it.
30. (11/19/19) In Promising The Moon (written 2014), I argued that technological progress had not slowed from 1970 to the present (compared to its pre-1970 speed). Since then, research by a bunch of people (summarized very well here by Tyler Cowen) finds strong evidence that it has. Despite its confidence and snark, my 2014 post was wrong.
31. (2/9/20) In Adderall Risks: Much More Than You Wanted To Know (written 2017) I stated that although Adderall might increase risk of Parkinson’s Disease, Ritalin didn’t. Since then I’ve been informed of several studies showing Ritalin probably does this too (see eg here). I’m not sure what went wrong and I can’t retrace my steps to find what made me think Ritalin was safe before; I think it was personal communication with some people who I thought would have known. In any case, don’t switch to Ritalin because you’re worried about Parkinson’s; it won’t help.
Probably many other mistakes, but these are the ones I remembered to record. If you know of an objective mistake (not subjective disagreement!) that is not listed here, please let me know.