Book Review: Human Compatible

Posted on January 30, 2020 by Scott Alexander

I.

Clarke’s First Law goes: When a distinguished but elderly scientist states that something is possible, he is almost certainly right. When he states that something is impossible, he is very probably wrong.

Stuart Russell is only 58. But what he lacks in age, he makes up in distinction: he’s a computer science professor at Berkeley, neurosurgery professor at UCSF, DARPA advisor, and author of the leading textbook on AI. His new book Human Compatible states that superintelligent AI is possible; Clarke would recommend we listen.

I’m only half-joking: in addition to its contents, Human Compatible is important as an artifact, a crystallized proof that top scientists now think AI safety is worth writing books about. Nick Bostrom’s Superintelligence: Paths, Dangers, Strategies previously filled this role. But Superintelligence was in 2014, and by a philosophy professor. From the artifactual point of view, HC is just better – more recent, and by a more domain-relevant expert. But if you also open up the books to see what’s inside, the two defy easy comparison.

S:PDS was unabashedly a weird book. It explored various outrageous scenarios (what if the AI destroyed humanity to prevent us from turning it off? what if it put us all in cryostasis so it didn’t count as destroying us? what if it converted the entire Earth into computronium?) with no excuse beyond that, outrageous or not, they might come true. Bostrom was going out on a very shaky limb to broadcast a crazy-sounding warning about what might be the most important problem humanity has ever faced, and the book made this absolutely clear.

HC somehow makes risk from superintelligence not sound weird. I can imagine my mother reading this book, nodding along, feeling better educated at the end of it, agreeing with most of what it says (it’s by a famous professor! I’m sure he knows his stuff!) and never having a moment where she sits bolt upright and goes what? It’s just a bizarrely normal, respectable book. It’s not that it’s dry and technical – HC is much more accessible than S:PDS, with funny anecdotes from Russell’s life, cute vignettes about hypothetical robots, and the occasional dad joke. It’s not hiding any of the weird superintelligence parts. Rereading it carefully, they’re all in there – when I leaf through it for examples, I come across a quote from Moravec about how “the immensities of cyberspace will be teeming with unhuman superminds, engaged in affairs that are to human concerns as ours are to those of bacteria”. But somehow it all sounds normal. If aliens landed on the White House lawn tomorrow, I believe Stuart Russell could report on it in a way that had people agreeing it was an interesting story, then turning to the sports page. As such, it fulfills its artifact role with flying colors.

How does it manage this? Although it mentions the weird scenarios, it doesn’t dwell on them. Instead, it focuses on the present and the plausible near-future, uses those to build up concepts like “AI is important” and “poorly aligned AI could be dangerous”. Then it addresses those abstractly, sallying into the far future only when absolutely necessary. Russell goes over all the recent debates in AI – Facebook, algorithmic bias, self-driving cars. Then he shows how these are caused by systems doing what we tell them to do (ie optimizing for one easily-described quantity) rather than what we really want them to do (capture the full range of human values). Then he talks about how future superintelligent systems will have the same problem.

His usual go-to for a superintelligent system is Robbie the Robot, a sort of Jetsons-esque butler for his master Harriet the Human. The two of them have all sorts of interesting adventures together where Harriet asks Robbie for something and Robbie uses better or worse algorithms to interpret her request. Usually these requests are things like shopping for food or booking appointments. It all feels very Jetsons-esque. There’s no mention of the word “singleton” in the book’s index (not that I’m complaining – in the missing spot between simulated evolution of programs, 171 and slaughterbot, 111, you instead find Slate Star Codex blog, 146, 169-70). But even from this limited framework, he manages to explore some of the same extreme questions Bostrom does, and present some of the answers he’s spent the last few years coming up with.

If you’ve been paying attention, much of the book will be retreading old material. There’s a history of AI, an attempt to define intelligence, an exploration of morality from the perspective of someone trying to make AIs have it, some introductions to the idea of superintelligence and “intelligence explosions”. But I want to focus on three chapters: the debate on AI risk, the explanation of Russell’s own research program, and the section on misuse of existing AI.

II.

Chapter 6, “The Not-So-Great Debate”, is the highlight of the book-as-artifact. Russell gets on his cathedra as top AI scientist, surveys the world of other top AI scientists saying AI safety isn’t worth worrying about yet, and pronounces them super wrong:

I don’t mean to suggest that there cannot be any reasonable objections to the view that poorly designed superintelligent machines would present a serious risk to humanity. It’s just that I have yet to see such an objection.

He doesn’t pull punches here, collecting a group of what he considers the stupidest arguments into a section called “Instantly Regrettable Remarks”, with the connotation that the their authors (“all of whom are well-known AI researchers”), should have been embarrassed to have been seen with such bad points. Others get their own sections, slightly less aggressively titled, but it doesn’t seem like he’s exactly oozing respect for those either. For example:

Kevin Kelly, founding editor of Wired magazine and a remarkably perceptive technology commentator, takes this argument one step further. In “The Myth of a Superhuman AI,” he writes, “Intelligence is not a single dimension, so ‘smarter than humans’ is a meaningless concept.” In a single stroke, all concerns about superintelligence are wiped away.

Now, one obvious response is that a machine could exceed human capabilities in all relevant dimensions of intelligence. In that case, even by Kelly’s strict standards, the machine would be smarter than a human. But this rather strong assumption is not necessary to refute Kelly’s argument.

Consider the chimpanzee. Chimpanzees probably have better short-term memory than humans, even on human-oriented tasks such as recalling sequences of digits. Short-term memory is an important dimension of intelligence. By Kelly’s argument, then, humans are not smarter than chimpanzees; indeed, he would claim that “smarter than a chimpanzee” is a meaningless concept.

This is cold comfort to the chimpanzees and other species that survive only because we deign to allow it, and to all those species that we have already wiped out. It’s also cold comfort to humans who might be worried about being wiped out by machines.

Or:

The risks of superintelligence can also be dismissed by arguing that superintelligence cannot be achieved. These claims are not new, but it is surprising now to see AI researchers themselves claiming that such AI is impossible. For example, a major report from the AI100 organization, Artificial Intelligence and Life in 2030, includes the following claim: “Unlike in the movies, there is no race of superhuman robots on the horizon or probably even possible.”

To my knowledge, this is the first time that serious AI researchers have publicly espoused the view that human-level or superhuman AI is impossible—and this in the middle of a period of extremely rapid progress in AI research, when barrier after barrier is being breached. It’s as if a group of leading cancer biologists announced that they had been fooling us all along: They’ve always known that there will never be a cure for cancer.

What could have motivated such a volte-face? The report provides no arguments or evidence whatever. (Indeed, what evidence could there be that no physically possible arrangement of atoms outperforms the human brain?) I suspect that the main reason is tribalism — the instinct to circle the wagons against what are perceived to be “attacks” on AI. It seems odd, however, to perceive the claim that superintelligent AI is possible as an attack on AI, and even odder to defend AI by saying that AI will never succeed in its goals. We cannot insure against future catastrophe simply by betting against human ingenuity.

If superhuman AI is not strictly impossible, perhaps it’s too far off to worry about? This is the gist of Andrew Ng’s assertion that it’s like worrying about “overpopulation on the planet Mars.” Unfortunately, a long-term risk can still be cause for immediate concern. The right time to worry about a potentially serious problem for humanity depends not just on when the problem will occur but also on how long it will take to prepare and implement a solution. For example, if we were to detect a large asteroid on course to collide with Earth in 2069, would we wait until 2068 to start working on a solution? Far from it! There would be a worldwide emergency project to develop the means to counter the threat, because we can’t say in advance how much time is needed.

Russell displays master-level competence at the proving too much technique, neatly dispatching sophisticated arguments with a well-placed metaphor. Some expert claims it’s meaningless to say one thing is smarter than another thing, and Russell notes that for all practical purposes it’s meaningful to say humans are smarter than chimps. Some other expert says nobody can control research anyway, and Russell brings up various obvious examples of people controlling research, like the ethical agreements already in place on the use of gene editing.

I’m a big fan of Luke Muehlhauser’s definition of common sense – making sure your thoughts about hard problems make use of the good intuitions you have built for thinking about easy problems. His example was people who would correctly say “I see no evidence for the Loch Ness monster, so I don’t believe it” but then screw up and say “You can’t disprove the existence of God, so you have to believe in Him”. Just use the same kind of logic for the God question you use for every other question, and you’ll be fine! Russell does great work applying common sense to the AI debate, reminding us that if we stop trying to out-sophist ourselves into coming up with incredibly clever reasons why this thing cannot possibly happen, we will be left with the common-sense proposition that it might.

My only complaint about this section of the book – the one thing that would have added a cherry to the slightly troll-ish cake – is that it missed a chance to include a reference to On The Impossibility Of Supersized Machines.

Is Russell (or am I) going too far here? I don’t think so. Russell is arguing for a much weaker proposition than the ones Bostrom focuses on. He’s not assuming super-fast takeoffs, or nanobot swarms, or anything like that. All he’s trying to do is argue that if technology keeps advancing, then at some point AIs will become smarter than humans and maybe we should worry about this. You’ve really got to bend over backwards to find counterarguments to this, those counterarguments tend to sound like “but maybe there’s no such thing as intelligence so this claim is meaningless”, and I think Russell treats these with the contempt they deserve.

He is more understanding of – but equally good at dispatching – arguments for why the problem will really be easy. Can’t We Just Switch It Off? No; if an AI is truly malicious, it will try to hide its malice and prevent you from disabling it. Can’t We Just Put It In A Box? No, if it were smart enough it could probably find ways to affect the world anyway (this answer was good as far as it goes, but I think Russell’s threat model also allows a better one: he imagines thousands of AIs being used by pretty much everybody to do everything, from self-driving cars to curating social media, and keeping them all in boxes is no more plausible than keeping transportation or electricity in a box). Can’t We Just Merge With The Machines? Sounds hard. Russell does a good job with this section as well, and I think a hefty dose of common sense helps here too.

He concludes with a quote:

The “skeptic” position seems to be that, although we should probably get a couple of bright people to start working on preliminary aspects of the problem, we shouldn’t panic or start trying to ban AI research. The “believers”, meanwhile, insist that although we shouldn’t panic or start trying to ban AI research, we should probably get a couple of bright people to start working on preliminary aspects of the problem.

I couldn’t have put it better myself.

III.

If it’s important to control AI, and easy solutions like “put it in a box” aren’t going to work, what do you do?

Chapters 7 and 8, “AI: A Different Approach” and “Provably Beneficial AI” will be the most exciting for people who read Bostrom but haven’t been paying attention since. Bostrom ends by saying we need people to start working on the control problem, and explaining why this will be very hard. Russell is reporting all of the good work his lab at UC Berkeley has been doing on the control problem in the interim – and arguing that their approach, Cooperative Inverse Reinforcement Learning, succeeds at doing some of the very hard things. If you haven’t spent long nights fretting over whether this problem was possible, it’s hard to convey how encouraging and inspiring it is to see people gradually chip away at it. Just believe me when I say you may want to be really grateful for the existence of Stuart Russell and people like him.

Previous stabs at this problem foundered on inevitable problems of interpretation, scope, or altered preferences. In Yudkowsky and Bostrom’s classic “paperclip maximizer” scenario, a human orders an AI to make paperclips. If the AI becomes powerful enough, it does whatever is necessary to make as many paperclips as possible – bulldozing virgin forests to create new paperclip mines, maliciously misinterpreting “paperclip” to mean uselessly tiny paperclips so it can make more of them, even attacking people who try to change its programming or deactivate it (since deactivating it would cause fewer paperclips to exist). You can try adding epicycles in, like “make as many paperclips as possible, unless it kills someone, and also don’t prevent me from turning you off”, but a big chunk of Bostrom’s S:PDS was just example after example of why that wouldn’t work.

Russell argues you can shift the AI’s goal from “follow your master’s commands” to “use your master’s commands as evidence to try to figure out what they actually want, a mysterious true goal which you can only ever estimate with some probability”. Or as he puts it:

The problem comes from confusing two distinct things: reward signals and actual rewards. In the standard approach to reinforcement learning, these are one and the same. That seems to be a mistake. Instead, they should be treated separately…reward signals provide information about the accumulation of actual reward, which is the thing to be maximized.

So suppose I wanted an AI to make paperclips for me, and I tell it “Make paperclips!” The AI already has some basic contextual knowledge about the world that it can use to figure out what I mean, and my utterance “Make paperclips!” further narrows down its guess about what I want. If it’s not sure – if most of its probability mass is on “convert this metal rod here to paperclips” but a little bit is on “take over the entire world and convert it to paperclips”, it will ask me rather than proceed, worried that if it makes the wrong choice it will actually be moving further away from its goal (satisfying my mysterious mind-state) rather than towards it.

Or: suppose the AI starts trying to convert my dog into paperclips. I shout “No, wait, not like that!” and lunge to turn it off. The AI interprets my desperate attempt to deactivate it as further evidence about its hidden goal – apparently its current course of action is moving away from my preference rather than towards it. It doesn’t know exactly which of its actions is decreasing its utility function or why, but it knows that continuing to act must be decreasing its utility somehow – I’ve given it evidence of that. So it stays still, happy to be turned off, knowing that being turned off is serving its goal (to achieve my goals, whatever they are) better than staying on.

This also solves the wireheading problem. Suppose you have a reinforcement learner whose reward is you saying “Thank you, you successfully completed that task”. A sufficiently weak robot may have no better way of getting reward than actually performing the task for you; a stronger one will threaten you at gunpoint until you say that sentence a million times, which will provide it with much more reward much faster than taking out your trash or whatever. Russell’s shift in priorities ensures that won’t work. You can still reinforce the robot by saying “Thank you” – that will give it evidence that it succeeded at its real goal of fulfilling your mysterious preference – but the words are only a signpost to the deeper reality; making you say “thank you” again and again will no longer count as success.

All of this sounds almost trivial written out like this, but number one, everything is trivial after someone thinks about it, and number two, there turns out to be a lot of controversial math involved in making it work out (all of which I skipped over). There are also some big remaining implementation hurdles. For example, the section above describes a Bayesian process – start with a prior on what the human wants, then update. But how do you generate the prior? How complicated do you want to make things? Russell walks us through an example where a robot gets great information that a human values paperclips at 80 cents – but the real preference was valuing them at 80 cents on weekends and 12 cents on weekdays. If the robot didn’t consider that a possibility, it would never be able to get there by updating. But if it did consider every single possibility, it would never be able to learn anything beyond “this particular human values paperclips at 80 cents on 12:08 AM on January 14th when she’s standing in her bedroom.” Russell says that there is “no working example” of AIs that can solve this kind of problem, but “the general idea is encompassed within current thinking about machine learning”, which sounds half-meaningless and half-reassuring.

People with a more technical bent than I have might want to look into some deeper criticisms of CIRL, including Eliezer Yudkowsky’s article here and some discussion in the AI Alignment Newsletter.

IV.

I want to end by discussing what was probably supposed to be an irrelevant middle chapter of the book, Misuses of AI.

Russell writes:

A compassionate and jubilant use of humanity’s cosmic endowment sounds wonderful, but we also have to reckon with the rapid rate of innovation in the malfeasance sector. Ill-intentioned people are thinking up new ways to misuse AI so quickly that this chapter is likely to be outdated even before it attains printed form. Think of it not as depressing reading, however, but as a call to act before it is too late.

…and then we get a tour of all the ways AIs are going wrong today: surveillance, drones, deepfakes, algorithmic bias, job loss to automation, social media algorithms, etc.

Some of these are pretty worrying. But not all of them.

Google “deepfakes” and you will find a host of articles claiming that we are about to lose the very concept of truth itself. Brookings calls deepfakes “a threat to truth in politics” and comes up with a scenario where deepfakes “could trigger a nuclear war.” The Guardian asks “You Thought Fake News Was Bad? Deepfakes Are Where Truth Goes To Die”. And these aren’t even the alarmist ones! The Irish Times calls it an “information apocalypse” and literally titles their article “Be Afraid”; Good Times just writes “Welcome To Deepfake Hell”. Meanwhile, deepfakes have been available for a couple of years now, with no consequences worse than a few teenagers using them to make pornography, ie the expected outcome of every technology ever. Also, it’s hard to see why forging videos should be so much worse than forging images through Photoshop, forging documents through whatever document-forgers do, or forging text through lying. Brookings explains that deepfakes might cause nuclear war because someone might forge a video of the President ordering a nuclear strike and then commanders might believe it. But it’s unclear why this is so much more plausible than someone writing a memo saying “Please launch a nuclear strike, sincerely, the President” and commanders believing that. Other papers have highlighted the danger of creating a fake sex tape with a politician in order to discredit them, but you can already convincingly Photoshop an explicit photo of your least favorite politician, and everyone will just laugh at you.

Algorithmic bias has also been getting colossal unstoppable neverending near-infinite unbelievable amounts of press lately, but the most popular examples basically boil down to “it’s impossible to satisfy several conflicting definitions of ‘unbiased’ simultaneously, and algorithms do not do this impossible thing”. Humans also do not do the impossible thing. Occasionally someone is able to dig up an example which actually seems slightly worrying, but I have never seen anyone prove (or even seriously argue) that algorithms are in general more biased than humans (see also Principles For The Application Of Human Intelligence – no, seriously, see it). Overall I am not sure this deserves all the attention it gets any time someone brings up AI, tech, science, matter, energy, space, time, or the universe.

Or: with all the discussion about how social media algorithms are radicalizing the youth, it was refreshing to read a study investigating whether this was actually true, which found that social media use did not increase support for right-wing populism, and online media use (including social media use) and right-wing populism actually seem to be negatively correlated (remember, correlational studies are always bad). Recent studies of YouTube’s algorithms find they do not naturally tend to radicalize, and may deradicalize, viewers, although I’ve heard some people say this is only true of the current algorithm and the old ones (which were not included in these studies) were much worse.

Or: is automation destroying jobs? Although it seems like it should, the evidence continues to suggest that it isn’t. There are various theories for why this should be, most of which suggest it may not destroy jobs in the near future either. See my review of technological unemployment for details.

A careful reading reveals Russell appreciates most of these objections. A less careful reading does not reveal this. The general structure is “HERE IS A TERRIFYING WAY THAT AI COULD BE KILLING YOU AND YOUR FAMILY although studies do show that this is probably not literally happening in exactly this way AND YOUR LEADERS ARE POWERLESS TO STOP IT!”

I understand the impulse. This book ends up doing an amazing job of talking about AI safety without sounding weird. And part of how it accomplishes this is building on a foundation of “AI is causing problems now”. The media has already prepared the way; all Russell has to do is vaguely gesture at deepfakes and algorithmic radicalization, and everyone says “Oh yeah, that stuff!” and realizes that they already believe AI is dangerous and needs aligning. And then you can add “and future AI will be the same way but even more”, and you’re home free.

But the whole thing makes me nervous. Lots of right-wingers say “climatologists used to worry about global cooling, why should we believe them now about global warming?” They’re wrong – global cooling was never really a big thing. But in 2040, might the same people say “AI scientists used to worry about deepfakes, why should we believe them now about the Singularity?” And might they actually have a point this time? If we get a reputation as the people who fall for every panic about AI, including the ones that in retrospect turn out to be kind of silly, will we eventually cry wolf one too many times and lose our credibility before crunch time?

I think the actual answer to this question is “Haha, as if our society actually punished people for being wrong”. The next US presidential election is all set to be Socialists vs. Right-Wing Authoritarians – and I’m still saying with a straight face that the public notices when movements were wrong before and lowers their status? Have the people who said there were WMDs in Iraq lost status? The people who said sanctions on Iraq were killing thousands of children? The people who said Trump was definitely for sure colluding with Russia? The people who said global warming wasn’t real? The people who pushed growth mindset as a panacea for twenty years?

So probably this is a brilliant rhetorical strategy with no downsides. But it still gives me a visceral “ick” reaction to associate with something that might not be accurate.

And there’s a sense in which this is all obviously ridiculous. The people who think superintelligent robots will destroy humanity – these people should worry about associating with the people who believe fake videos might fool people on YouTube, because the latter group is going beyond what the evidence will support? Really? But yes. Really. It’s more likely that catastrophic runaway global warming will boil the world a hundred years from now than that it will reach 75 degrees in San Francisco tomorrow (predicted high: 59); extreme scenarios about the far future are more defensible than even weak claims about the present that are ruled out by the evidence.

There’s been some discussion in effective altruism recently about public relations. The movement has many convincing hooks (you can save a live for $3000, donating bednets is very effective, think about how you would save a drowning child) and many things its leading intellectuals are actually thinking about (how to stop existential risks, how to make people change careers, how to promote plant-based meat), and the Venn diagram between the hooks and the real topics has only partial overlap. What to do about this? It’s a hard question, and I have no strong opinion besides a deep respect for everyone on both sides of it and appreciation for the work they do trying to balance different considerations in creating a better world.

HC’s relevance to this debate is as an extraordinary example. If you try to optimize for being good at public relations and convincingness, you can be really, really good at public relations and convincingness, even when you’re trying to explain a really difficult idea to a potentially hostile audience. You can do it while still being more accurate, page for page, than a New York Times article on the same topic. There are no obvious disadvantages to doing this. It still makes me nervous.

V.

My reaction to this book is probably weird. I got interested in AI safety by hanging out with transhumanists and neophiles who like to come up with the most extreme scenario possible, and then back down when maybe it isn’t true. Russell got interested in AI safety by hanging out with sober researchers who like to be as boring and conservative as possible, and then accept new ideas once the evidence for them proves overwhelming. At some point one hopes we meet in the middle. We’re almost there.

But maybe we’re not quite there yet. My reaction to this book has been “what an amazing talent Russell must have to build all of this up from normality”. But maybe it’s not talent. Maybe Russell is just recounting his own intellectual journey. Maybe this is what a straightforward examination of AI risk looks like if you have fewer crazy people in your intellectual pedigree than I do.

I recommend this book both for the general public and for SSC readers. The general public will learn what AI safety is. SSC readers will learn what AI safety sounds like when it’s someone other than me talking about it. Both lessons are valuable.

Posted in Uncategorized | Tagged , | 310 Comments

Assortative Mating And Autism

Posted on January 28, 2020 by Scott Alexander

Introduction

Assortative mating is when similar people marry and have children. Some people worry about assortative mating in Silicon Valley: highly analytical tech workers marry other highly analytical tech workers. If highly analytical tech workers have more autism risk genes than the general population, assortative mating could put their children at very high risk of autism. How concerned should this make us?

Methods / Sample Characteristics

I used the 2020 Slate Star Codex survey to investigate this question. It had 8,043 respondents selected for being interested in a highly analytical blog about topics like science and economics. The blog is associated with – and draws many of its readers from – the rationalist and effective altruist movements, both highly analytical. More than half of respondents worked in programming, engineering, math, or physics. 79% described themselves as atheist or agnostic. 65% described themselves as more interested in STEM than the humanities; only 15% said the opposite.

According to Kogan et al (2018), about 2.5% of US children are currently diagnosed with autism spectrum disorders. The difference between “autism” and “autism spectrum disorder” is complicated, shifts frequently, and is not very well-known to the public; this piece will treat them interchangeably from here on. There are no surveys of what percent of adults are diagnosed with autism; it is probably lower since most diagnoses happen during childhood and the condition was less appreciated in past decades. These numbers may be affected by parents’ education level and social class; one study shows that children in wealthy neighborhoods were up to twice as likely to get diagnosed as poorer children.

Given that respondents are likely wealthier than average, we might expect a rate of 2.5% – 5%. Instead the rate is noticeably higher than that, consistent with the hypothesis that this sample will be more autistic than average. About 4% of the SSC survey sample had a formal diagnosis of autism, but this rose to 6% when the sample was limited to people below 30, and to 8% below 20. This sample is plausibly about 2-3x more autistic than the US population. Childhood social class was not found to have a significant effect on autism status in this sample.

Results

I tried to get information on how many children respondents have, but I forgot to ask important questions about age until a quarter of the way through the survey. I want to make sure I’m only catching children old enough that their autism would have been diagnosed, so the information below (except when otherwise noted) comes from the three-quarters of the sample where I have good age information. I also checked it against the whole sample and it didn’t make a difference.

Of this limited sample, 1,204 individual parents had a total of 2,459 children. 1,892 of those children were older than 3, and 1,604 were older than 5. I chose to analyze children older than 3, since autism generally becomes detectable around 2.

71 children in the 1,892 child sample had formal diagnoses of autism, for a total prevalence of 3.7%. When parents were asked to include children who were not formally diagnosed but who they thought had the condition, this increased to 99 children, or a 5.2% prevalence. Both numbers are much lower than the 8% prevalence in young people in the sample.

What about marriages where both partners were highly analytical? My proxy for this was the following survey question:

I’ll be referring to these answers as “yes”, “sort of”, and “no” from here on, and moving back to the full sample. 938 parents answered this question; 51 (5.4%) yes, 233 (24.8%) sort of, and 653 (69.4%) no. Keep in mind the effective sample is even smaller, since both partners in two-partners-read-SSC-families may have filled out the survey individually about the same set of children (though this should not have affected the “sort of” group). Here is the autism rate for each group, with 95% confidence interval in black:

There is little difference. If we combine the latter two groups, the confidence interval narrows slightly, to 2.7 – 6.5.

I asked respondents about the severity of their children’s autism.

People who hadn’t previously reported any children with autism gave answers other than N/A for this one, which was confusing. Instead of the 71 children we had before, now it’s up to 144 children. I’m not sure what’s going on here. Of these phantom children, 101 had mild cases, 31 moderate, and only 12 severe. Severe autism was only present in 0.6% of the children in the sample. There was no tendency for couples where both partners were highly analytical to have children with more severe autism.

Discussion

Autism rates in this survey were generally low. Although the general rate of 3.7% was higher than the commonly-estimated US base rate of 2.5%, this is consistent with the slight elevation of autism observed in higher social classes.

There was no sign of elevated risk when both partners were highly analytical. The sample size was too small to say for certain that no such elevation exists, but it can say with 95% confidence that the elevated risk is less than three percentage points.

This suggests that the answer to the original question – does assortative mating between highly analytical people significantly increase chance of autism in offspring – is at least a qualified “no”.

Why should this be? It could just be that regression to the mean is more important in this case than any negative effects from combining recessive genes or mixing too many risk genes together. Or maybe we should challenge the assumption that being a highly analytical programmer is necessarily on a continuum with autism. It seems like p(highly analytical|on autism spectrum) is pretty high, but p(on autism spectrum|is highly analytical) might be much lower.

Obvious limitations of this survey include the small sample size of both-partners-highly-analytical couples, the weak operationalization of highly analytical as “member of the SSC, rationalist, and effective altruist communities”, and the inability to separate non-autistic children from children who are not yet diagnosed. Due to these limitations, this should only be viewed as providing evidence against the strongest versions of the assortative mating hypothesis, where it might increase risk by double, triple, or more. Smaller elevations of risk remain plausible and would require larger studies to assess.

I welcome people trying to replicate or expand on these results. All of the data used in this post are freely available and can be downloaded here.

Posted in Uncategorized | Tagged , | 108 Comments

Open Thread 146

Posted on January 26, 2020 by Scott Alexander

This is the bi-weekly visible open thread (there are also hidden open threads twice a week you can reach through the Open Thread tab on the top of the page). Post about anything you want, but please try to avoid hot-button political and social topics. You can also talk at the SSC subreddit – and also check out the SSC Podcast. Also:

1. In 2016, I made a bet with bhauth that the US median income growth under Donald Trump wouldn’t significantly outperform the trendline for the past 25 years. It did, so I lost. As his prize, bhauth asks me to signal-boost his work on a new type of battery that outperforms lithium-ion.

2. Some people have already gotten a nice head start analyzing the SSC survey; see eg wulfrickson on autogynephilia and jsmp on various things.

3. The SSC podcast is still trying to recoup its costs, so it’s started offering ads. You can get your ad read on the podcast for $100/month; they get about 1500 downloads per episode, and there are 10-ish episodes per month. Email slatestarpodcast[at]gmail[dot]com for details.

4. I need to make my inbox more manageable, so I am going to ask you not to send me emails asking for comments on your manifestos or ideas or interesting links you found. I find myself feeling annoyed if I spend time on them and guilty if I don’t, and it’s unfair to you to have to listen to me saying I will answer you and then never doing so. If you have interesting things like this you want to bring to my attention, post them on the SSC subreddit, which I read pretty often. I continue to accept other types of emails. Sorry about this.

Posted in Uncategorized | Tagged | 708 Comments

Book Review Review: Little Soldiers

Posted on January 22, 2020 by Scott Alexander

Little Soldiers is a book by Lenora Chu about the Chinese education system. I haven’t read it. This is a review of Dormin111’s review of Little Soldiers.

Dormin describes the “plot”: The author is a second-generation Chinese-American woman, raised by demanding Asian parents. Her parents made her work herself to the bone to get perfect grades in school, practice piano, get into Ivy League schools, etc. She resisted and resented the hell she was forced to go through (though she got into Stanford, so she couldn’t have resisted too hard).

Skip a decade. She is grown up, married, and has a three year old child. Her husband (a white guy named Rob) gets a job in China, so they move to Shanghai. She wants their three-year-old son to be bilingual/bicultural, so she enrolls him in Soong Qing Ling, the Harvard of Chinese preschools. The book is about her experiences there and what it taught her about various aspects of Chinese education. Like the lunches:

During his first week at Soong Qing Ling, Rainey began complaining to his mom about eating eggs. This puzzled Lenora because as far as she knew, Rainey refused to eat eggs and never did so at home. But somehow he was eating them at school.

After much coaxing (three-year-olds aren’t especially articulate), Lenora discovered that Rainey was being force-fed eggs. By his telling, every day at school, Rainey’s teacher would pass hardboiled eggs to all students and order them to eat. When Rainey refused (as he always did), the teacher would grab the egg and shove it in his mouth. When Rainey spit the egg out (as he always did), the teacher would do the same thing. This cycle would repeat 3-5 times with louder yelling from the teacher each time until Rainey surrendered and ate the egg.

Outraged, Lenora stormed to the school the next day and approached the teacher in the morning as she dropped Rainey off. Lenora demanded to know if Rainey was telling the truth – was this teacher literally forcing food into her three-year-old son’s mouth and verbally berating him until he ate it. The teacher didn’t even bother looking at Lenora as she calmly explained that eggs are healthy and that it was important for children to eat them. When Lenora demanded she stop force-feeding her son, the teacher refused and walked away.

Or the seating:

As Lenora hears more crazy stories from her son and friends, she keeps coming back to one question: “what does Rainey actually do in school?” Lenora tries to ask Rainey, but he always replies, “we sit still.” He also occasionally mentions painting and eating, but that’s it.

So Lenora goes to Rainey’s teacher one day and asks to sit in on classes to observe. Lenora is told that this is not possible. So she asks if she can know a little more about what the school is teaching Rainey. The teacher tells her that she is already told everything she needs to know, and that this is the “Chinese way.”

Since Lenora couldn’t get a look into Soong Qing Ling, she went to another local school and bribed her way into a classroom-observation post with some well-placed handbags. She discovered that Rainey was basically right. Chinese preschool really does seem to consist of sitting still. Unless given different orders, all students were required to sit in their seats with their arms at their sides, and their feet flat on a line of tape on the ground. This is not an easy task for three-year-olds.

There were two teachers in the classroom with a classic good cop/bad cop dynamic. The good cop stood in the front of the room with the desks splayed out before her. She would give simple instructions like orders to get food, water, or sometimes paint, though usually she said nothing at all. The bad cop was another teacher who prowled the classroom. Any time she saw a student remove a foot from the line, move arms from his side, or otherwise deviate from the instructions, she would yell at the student to fall back in line. Lenora spent about a week watching tiny kids get screamed at for trying to get water, shifting in their chairs, or talking to classmates.

Or art class:

When Lenora sat in on a kindergarten class, she witnessed an art lesson where the students were taught how to draw rain. The nice teacher drew raindrops on a whiteboard, showing precisely where to start and end each stroke to form a tear-drop shape. When it was the students’ turns, they had to perfectly replicate her raindrop. Over and over again. Same start and end points. Same curves. For an hour. No student could draw anything else. Any student who did anything different would be yelled at and told to start over.

The point of this exercise was not to teach students how to draw raindrops. Drawing raindrops is not an important life skill, and drawing them in a particular way is especially not important. Even the three-year-old students in the class seemed to realize this as many immediately created their own custom raindrop shapes and drew landscapes, all to be crushed under the mean teacher’s admonishment. The real point of the exercise was to teach students to follow directions from an authority figure. But more than that, the point was to follow pointless and arbitrary directions. The more pointless and arbitrary the directions are, the more willpower is required to follow them.

Chinese people presumably put up with this because it makes sense within their culture; why did Chu put up with it? Dormin half-jokingly suggests maybe she really wanted to write the book she eventually wrote, and this was her research. But Chu herself says it eventually got results:

After spending 75% of the book relentlessly complaining about her son’s Chinese education, with the occasional anecdote about how horrible her own culturally Chinese upbringing was, Lenora decides Chinese schools aren’t so bad.

After a few years in China, Rainey changed. Though Lenora constantly worried if Rainey’s creativity and leadership potential was being snuffed out, she couldn’t help but be impressed by his emerging self-control. He could sit still for longer. He always greeted people politely. He finished eating his food. He asked permission a lot.

Lenora didn’t realize what Rainey had become until she took him back to the US for a few weeks to visit family. There, the contrast between Rainey and his same-aged American counterparts become stark. Lenora’s friends’ kids ate junk food all day while Rainey asked for vegetables. They couldn’t read or do basic addition while Rainey was close to being bilingual and had started double-digit addition and subtraction by first grade. They wandered obliviously in their own worlds while Rainey’s Chinese grandparents were thrilled to receive respectful greetings every time Rainey entered the room […]

What really sold Lenora on Chinese education was that it apparently worked. At the time of writing the book, Shanghai was scoring first place in the world on the PISA exams, beating heavy-hitters like Norway and Singapore. Supposedly, education scholars and professionals all over the world were looking at China for wisdom. They all saw the bad, but they saw a lot of good too.

(before going forward, I should interject that China’s great PISA scores are kind of fake. China struck a deal with the OECD (the group that administers PISA) to let it conduct testing only in its four richest and best-educated provinces. Rich and well-educated places always do well on PISA. That China’s four best provinces outperform the average score of other countries is unsurprising. This article points out that if the US were allowed to enter only its best-educated state (Massachussetts, obviously) we would be right up there with China. So this probably isn’t as impressive as Ms. Chu thinks.)

This is just a sample of the great stuff in Dormin’s review of Little Soldiers, and I strongly recommend you read the whole thing. You should also read the comments, which point out that this may be more about a few elite Chinese schools than about an entire country. But I want to use these excerpts as a jumping-off point to talk about the US education system, unschooling, and child development in general.

I predict most of my Bay Area friends would hate the Chinese education system as Chu describes it. I predict this because they already hate the US education system, which is only like 10% as bad. I’m especially thinking of @webdevmason and @michaelblume, who often write about the ways American education is frustrating, regressive, and authoritarian. Bright-eyed, curious kids come in. They spend thirteenish years getting told to show their work, being punished for reading ahead in the textbook, and otherwise having their innate love of learning drummed out of them in favor of endless mass-produced homework assignments (five pages, single-spaced, make sure you use the right number of topic sentences).

People with this position usually make two claims. One, US public school as it currently exists is awful, basically institutionalized child abuse. Two, this is bad for the economy. I’ve been through too much school myself to feel like challenging the first, so I want to focus on the second.

Salman Khan, John Gatto, and other education rebels trace the current school systems back to the Prussians, who invented compulsory education to prepare children for a career as infantrymen or factory workers. It’s a great story. Like most great stories, it’s kind of false. But like most kind-of-false things that catch on, it has an element of truth. Children who can sit still in a classroom and do what their teachers say are well-placed to become adults who can sit still in an open office and do what their bosses say. So (according to this logic), even if our schools are awful, they were well-suited to the Industrial Age economy. Some hypothetical mash-up of Otto von Bismarck and Voldemort, who wanted the country to produce as much as possible and didn’t care how many children’s souls were crushed in the process, might at least endorse the education system on widget-maximization grounds.

But (these same people argue), the Industrial Age is over. The most important skills now are entrepreneurship and creative problem solving. Reinventing yourself, selling yourself, carving out a new niche for yourself. Figuring out what’s going to be the next big thing and pursuing it without anyone else watching over you. We’re in XKCD’s world now, where 900 hours of classes and 400 hours of homework matter less to your career success than one weekend messing around with a programming language in 11th grade. The Prussian model of education stamps out the kind of independent agency that could help people navigate the weird, formless 21st century world.

How might the personified Chinese education system respond?

What if it said “I don’t know what you 老外 are doing in America, but I’m not crushing anybody. I’m just telling kids to sit here drawing 1,000 raindrops in a row without moving or protesting. If after that you decide you don’t want to found the next Uber, that’s on you. But if you do decide to found the next Uber, I will have taught you the most important skill: discpline. Learning how to sit still and obey others is the necessary prerequisite to learning how to sit still and obey yourself.”

If it was really mean, it might go further. “I notice most of you Americans suck at this skill. I notice you’re always whining about how you don’t have enough discipline to pursue your interests. Some of you are writers who spend years fantasizing about the novel you’re going to publish, but can never quite bring yourself to put pen to paper. Others want to learn another language, but reject real work in favor of phone apps that promise to ‘gamify’ staying at a 101 level for the rest of your life. You don’t need to feel bad about having no self-control; after all, nobody taught you any. If you’d gone to 宋庆龄幼儿园, you would have spent your formative years learning to sit still and focus, having your natural impulse to slack off squeezed out of you. Then you could have pushed through and written your novel, or learned 官話, or if you wanted to start Uber you could start Uber. At the very least you’d be doing something other than lying in bed browsing Reddit posts about how adulting is hard.”

My Bay Area friends treat people as naturally motivated, and assume that if someone acts unmotivated, it’s because they’ve spent so long being taught to suppress their own desires that they’ve lost touch with innate enthusiasm. Personified China treats people as naturally unmotivated, and assumes that if someone acts unmotivated, it’s because they haven’t been trained to pursue a goal determinedly without getting blown around by every passing whim.

What evidence is there in favor of one education system or the other?

I can’t find any good studies directly supporting or opposing either of these claims. The best I can do is The Development Of Executive Functioning And Theory Of Mind: A Comparison Of Chinese And US Preschoolers. They find that on various tests of executive function, “Chinese [preschool-age] children’s performance was consistently on par with that of US children who were on average 6 months older” (other sources say 1-2 years). But lots of interventions change things in childhood; this isn’t interesting unless it persists into adulthood, and I don’t see any work on this. This study on racial differences in personality traits found weak and inconsistent white-Asian differences on adult conscientiousness, but the Asian sample was Asian-American and differences in education were probably pretty minor.

What about circumstantial evidence?

First and most important, since extreme cultivation of discipline vs. laissez-faire childrearing is a property of parents as much as schools, any claimed effect would run afoul of all the twin studies showing that shared environment has few long-term effects on any trait. For example, this meta-analysis of factors affecting self-control that finds “no or very little influence of the shared environment on the variance in self-control”. But we can always invoke the usual loophole in shared environment findings: maybe the US doesn’t contain anything as extreme as the Chinese education system, so US-only studies can’t capture its effects.

Second, both Westerners and Chinese seem to include some very impressive and some less impressive people. It certainly doesn’t seem wrong to say that Chinese people seem more diligent and Westerners seem more independent, but there are so many potential biases at work that I would hate to take this too seriously as evidence for or against one form of education. Also, Chinese-Americans who are educated in US schools also seem more diligent than white Americans, so maybe the education system doesn’t contribute too much to this. Maybe Chinese culture promotes diligence better in general, this causes diligence-focused school systems, but the diligence-focused school systems don’t themselves cause the diligence.

Third, we could try to find more extreme versions on both sides and see what happens there. Pre-industrial populations with no education were famously bad at the discipline needed for factory work. From Pseudoerasmus:

The earliest factory workers were lacking in what Mokyr & Voth call “discipline capital” — non-cognitive ‘skills’ like punctuality, sobriety, reliability, docility, and pliability. Whether they had been peasants or artisans, early workers were new to industrial work habits and they had a strong preference for autonomous work arrangements. They were accustomed to setting their own pace of work in farming, domestic outwork, or artisanal workshops, and disliked the time rules and strict supervision of the factories.

All this is consistent with colourful descriptions of the early history of the textile industry in the Global South, including Japan. Mills were described as places of chaos and disorder. They were supposedly filled with workers ‘idling’, ‘loitering’, ‘socialising’, smoking, tea-drinking, or just disappeared for the day. In Japan, “twenty percent of the female operatives…absent themselves after they receive their monthly pay check” (Saxonhouse & Kiyokawa 1978). In Shanghai, it was said female mill workers could be found breast-feeding infants during work hours (Cochran 2000). Or at Mumbai mills, workers “bathed, washed clothes, ate his meals, and took naps” (Gupta 2011).

But this could be as much about expectations as about abilities.

Which historical culture had the most authoritarian-instillment-of-virtue-focused approach to child-rearing? Surely the New England Puritans were up there – remember that eg Puritan parents would traditionally send children away to be raised by other families, in the hopes that the lack of familiarity would make the child behave better”. They certainly ended out industrious. But they were also creative and self-motivated, sometimes almost hilariously so. On the other hand, I’m not sure that the Puritans who ended up incredibly creative were exactly the same Puritans who suffered extreme strict child-rearing – there seems about a century gulf between the evidence of authoritarian parenting in the 1600s and the crop of geniuses born in the late 1700s – so I’m not sure how seriously to take this.

Fourth, we could look at US trends over time. Both US parenting and US schooling seem to be getting less authoritarian over time; 31 states have banned corporal punishment since 1970, and the teachers I know confirm a shift away from most forms of discipline. Over the same time period, children have gotten weirdly better behaved – less crime, less teenage pregnancy, more willing to jump through various stupid hoops to get into a good college. This seems to contradict the Chinese theory – the children are no worse at controlling their impulses. But there are other findings that contradict the Bay Area theory – entrepreneurship is decreasing; more top students are choosing to go work for a boss at a big bank rather than go do something weird. I think the better behavior is probably just caused by lower lead; I have no idea why people are more risk-averse. Secular decline in testosterone, maybe?

Fifth, we could look at research on the effects of preschool more generally. Some studies find that US preschools do not make children smarter, but still improve life outcomes like graduation rates, crime rates, and employment. Although there are lots of theories about the “noncognitive skills” that accomplish this (including that they don’t exist and the improvement is an artifact of bad experimental technique), this is certainly consistent with preschool teaching children discipline at a critical window. If this hypothesis were true, the effect of preschool would be much larger in China, but I don’t know of any Chinese studies on the topic.

Sixth, we could look at the research on meditation for very young kids. The Chinese theory casts preschool as a sort of dark-side form of mindfulness. In traditional Buddhist settings, monks would sit perfectly still and concentrate on the most boring thing imaginable, and the head monk would slap them with a bamboo stick if they moved. The resemblance to the school system is uncanny. So maybe school’s effects on self-control could be modeled as a sort of less-intense but much-more-drawn-out meditation session. Unfortunately, the studies surrounding mindfulness in kids are crap, so this doesn’t help either.

Really none of this seems very helpful and we’re kind of left with our priors. And maybe one of our priors is “don’t abuse children”, so there’s that.

But what about the Polgars? They turned all three of their children into chess prodigies through a strategy that seemed based around exposing them to absurd amounts of chess at a very young age. If we generalize, it does look like very young children might have very plastic minds that you can shape through out-of-distribution experiences. But Lazslo Polgar insisted that his technique didn’t use force; the point was to interest his children in the material so avidly that they inflicted near-Chinese levels of intensity on themselves in order to study it more successfully.

One problem with the physical universe is that even after you study a question in depth and decide more evidence is needed, there are still real children you have to educate one way or the other. I have no general solution for this, but the Polgar strategy seems like a good deal if you can pull it off.

Posted in Uncategorized | Tagged , | 452 Comments

SSC Survey Results 2020

Posted on January 20, 2020 by Scott Alexander

EDIT: This post presents open-access data from a large survey that anyone is allowed to download and analyze. It’s gone viral because a Twitter user named Philippe Lemoine downloaded the data and used it to investigate politics and mental illness. That analysis is not described here, it’s not my analysis, and I have various caveats about it, which are described here. Sorry for any confusion.

Thanks to the 8,043 people who took the 2020 Slate Star Codex survey.

See the questions for the SSC survey

See the results from the SSC Survey (click “see previous responses” on that page)

Some people expressed concern about privacy on the survey. Originally, respondents could see aggregate responses, including the responses of people who marked their answers private. I figured this was okay because nobody’s responses could be connected – ie you could see that one person put their age as 83, and another person put their country as Canada, but because the table order wasn’t the same you couldn’t link these together to form a coherent picture of an 83 year old Canadian. Some people still expressed concern about a few of the long answers, since some people might have put personal information in there. There’s no way for me to eliminate only the private people’s responses from Google Forms and still display the information to you like this, so instead I’ve removed all long answer questions. If you’re interested in those, you can find them in the downloadable data files. Sorry for not doing this earlier, and I hope this compromise is okay to everyone. I’ll try to get a clearer picture of what people want before the next survey.

I’ll be publishing more complicated analyses over the course of the next year, hopefully starting later this week. If you want to scoop me, or investigate the data yourself, you can download the answers of the 7000 people who agreed to have their responses shared publicly. The public datasets will not exactly match the full version, some overly identifiable questions (eg age) will be binned, and a few sensitive subjects will not be included.

Download the public data (.xlsx, .csv)

Finally, the game results. I randomly selected Game 3, “Prisoner’s Dilemma Against Your Clone”, chose a random respondent as the prisoner, and found someone similar to be his clone. Of the two clones, one cooperated and one defected, so the defector gets the full prize. That defector’s public key is “gwern is my waifu and paperklipot maximizer”. Please email me within one week at scott[at]slatestarcodex.com with your private key and a Paypal account where I can send you money (or a charity you want me to donate to).

Posted in Uncategorized | Tagged | 174 Comments

Last Chance To Take The Survey

Posted on January 17, 2020 by Scott Alexander

This is your last chance to take the 2020 SSC survey. Remember, it’s open to anyone who has read posts on this blog, it should take about 30 minutes to finish, and don’t click on a link unless you’re sure it will open in a new tab and not throw out your responses so far.

Thanks to everyone who’s taken it already. I look forward to sharing some interesting results with you soon.

Posted in Uncategorized | 50 Comments

Contra Contra Contra Caplan On Psych

Posted on January 15, 2020 by Scott Alexander

I.

In 2006, Bryan Caplan wrote a critique of psychiatry. In 2015, I responded. Now it’s 2020, and Bryan has a counterargument. I’m going to break the cycle of delay and respond now, and maybe we’ll finish this argument before we’re both too old and demented to operate computers.

Bryan writes:

1. With a few exceptions, Scott fairly and accurately explains my original (and current) position.

2. Scott correctly identifies several gray areas in my position, but by my count I explicitly acknowledged all of them in my original article.

3. Scott then uses those gray areas to reject my whole position in favor of the conventional view.

4. The range of the gray areas isn’t actually that big, so he should have accepted most of my heterodoxies.

5. If the gray areas were as big as Scott says, he should reject the conventional view too and just be agnostic.

I think the gray areas are overwhelming and provide proof that Bryan’s strict dichotomies don’t match the real world.

I also think, as a general philosophical point, that we ought to be suspicious of arguments of the form “the gray areas are small”. Even if this is true, and your model only fails in a few places, controversial questions are likely to be controversial questions precisely because they’re located where your model fails. Nobody challenges a model on an exactly typical case where everything makes sense. So if a point is under debate, let’s say in a fifteen year back-and-forth argument between two bloggers that’s attracted hundreds of total comments, the a priori size of the gray areas doesn’t matter. Even if your model is good at most things, you have strong evidence this isn’t one of them.

In this case, the model we’re debating is Bryan’s idea of constraints vs. preferences. My previous summary of this (which Bryan endorses) goes like this:

Consumer theory distinguishes between two different reasons why someone might not buy a Ferrari – budget constraints (they can’t afford one) and preferences (they don’t want one, or they want other things more). Physical diseases seem much like budget constraints – the reason a paralyzed person can’t run a marathon is because it’s beyond her abilities, simply impossible. Psychiatric diseases seem more like preferences. There’s nothing obvious stopping an alcoholic from quitting booze and there’s nothing obvious preventing someone with ADHD from sitting still and paying attention. Therefore they are best modeled as people with unusual preferences – the one with a preference for booze over normal activities like holding down a job, the other with a high dispreference for sitting still and attending classes. But lots of people have weird preferences. Therefore, psychiatric diseases should be thought of as within the broad spectrum of normal variation, rather than as analogous to physical diseases.

I countered by pointing out that this was in fact very analogous to physical diseases:

Alice has always had problems concentrating in school. Now she’s older and she hops between a couple of different part-time jobs. She frequently calls in sick because she feels like she doesn’t have enough energy to go into work that day, and when she does work her mind isn’t really on her projects. When she gets home, she mostly just lies in bed and sleeps. She goes to a psychiatrist who diagnoses her with ADHD and depression.

Bob is a high-powered corporate executive who rose to become Vice-President of his big Fortune 500 company. When he gets home after working 14 hour days, he trains toward his dream of running the Boston Marathon. Alas, this week Bob has the flu. He finds that he’s really tired all the time, and he usually feels exhausted at work and goes home after lunch; when he stays, he finds that his mind just can’t concentrate on what he’s doing. Yesterday he stayed home from work entirely because he didn’t feel like he had the energy. And when he gets home, instead of doing his customary 16 mile run he just lies in bed all day. His doctor tells him that he has the flu and is expected to recover soon.

At least for this week Alice and Bob are pretty similar. They’d both like to be able to work long hours, concentrate hard, and stay active after work. Instead they’re both working short hours, calling in sick, failing to concentrate, and lying in bed all day.

But for some reason, Bryan calls Alice’s problem “different preferences” and Bob’s problem “budgetary constraints”, even though they’re presenting exactly the same way! It doesn’t look like he’s “diagnosing” which side of the consumer theory dichotomy they’re on by their symptoms, but rather by his assumptions about the causes.

But Bryan doesn’t budge:

I’m unimpressed, because I not only anticipated such objections in my original paper, but even proposed a test to help clarify the fuzziness…can we change a person’s behavior purely by changing his incentives? If we can, it follows that the person was able to act differently all along, but preferred not to; his condition is a matter of preference, not constraint. I will refer to this as the ‘Gun-to-the-Head Test’. If suddenly pointing a gun at alcoholics induces them to stop drinking, then evidently sober behavior was in their choice set all along. Conversely, if a gun-to-the-head fails to change a person’s behavior, it is highly likely (though not necessarily true) that you are literally asking the impossible.

I then presented multiple forms of evidence that a wide range of alleged mental illnesses are responsive to incentives. Scott barely mentions said evidence.

Still, does this mean that the flu isn’t “really” an illness either? No. Rather it means that physical illness often constrains behavioral and changes preferences. When sick, the maximum amount of weight I can bench press falls. (Yes, I’ve actually tried this). Yet in addition, I don’t feel like lifting weights at all when I’m sick. Anyone who has worked while ill should be able to appreciate these dual effects. If you literally get sick, your ability and desire to work both go down. When you metaphorically get “sick of your job,” in contrast, only your desire goes down.

I reject the heck out of this answer. I agree the “gun to the head” test is a good summary of Bryan’s position, but we already agreed what Bryan’s position is. The only thing he’s adding here is a claim that the flu still qualifies as a real disease because it sometimes constrains behavior (the amount of weight Bryan can lift). But nobody cares how much weight they can lift during a flu! When we talk about having the flu being bad, we’re talking 0% about how much weight we can lift, and 100% about the sorts of problems Bob has – feeling too ill to go to work, not wanting to do things, etc. If Bryan searches hard enough, he can find a way the flu results in slightly weaker muscle strength. But if I search hard enough, I can find a way depression results in slightly weaker muscle strength. Neither of these things are what the average person thinks about when they think of “flu symptoms” or “depression symptoms”, and I consider them both equally irrelevant.

But if a change in weight-lifting ability really disqualifies the flu for Bryan, we can talk about other diseases.

What about shingles? It’s a viral infection that causes a very itchy rash. But sometimes (herpes sine zoster) the rash isn’t visible, and you just get really itchy for a few days. Like, really itchy. I had this condition once and it was just embarrassing how much I was scratching myself. But if you had put a gun to my head and said “Don’t scratch yourself, or I’ll kill you”, I would have sat on my hands and suffered quietly. For Bryan, an itch is just a newfound preference for scratching yourself. Shingles, like depression or ADHD, is just a preference shift, and so doesn’t qualify as a real disease.

Or what about respiratory tract infections that cause coughing? My impression is that, put a gun to my head, and I could keep myself from coughing, even when I really really felt like it. Coughing is a preference, not a constraint, and Bryan, to be consistent, would have to think of respiratory infections as just a preference for coughing.

Or what about migraines? Sure, people with migraines say they feel pain, but that’s no better grounded than someone with depression saying they feel sad. If Bryan is allowed to bring in concepts like “pain”, I’m allowed to bring in concepts like “sadness”, “anxiety”, etc. And since an anxious person feels anxiety and cannot stop feeling it even if threatened with a gunshot, the anxiety counts as a constraint, and so mental disorders are constraining. For Bryan’s constraints-vs-preferences dichotomy to work at all, he has to endorse a sort of behaviorism, where we need not believe anything that doesn’t express itself as behavior. And the only behavior we see in a migraine is somebody lying in bed, turning off all the lights, and occasionally clutching their head and saying “auggggh”. But put a gun to their head and demand they be in a bright room with lots of loud music, and they’ll go to the bright room with lots of loud music. Threaten to shoot them unless they stop clutching their head and moaning, and they’ll stop clutching their head and moaning. In Bryan’s model, migraines are just a newfound preference for saying “auggggh” a lot. Why medicalize this? Some people like saying “auggggh” and that’s valid!

Bryan’s preference vs. constraint model doesn’t just invalidate mental illness. It invalidates many (maybe most) physical illnesses! Even the ones it doesn’t invalidate may only get saved by some triviality we don’t care about – like how maybe you can lift less weight when you have the flu – and not by the symptoms that actually bother us.

II.

We need a model that lets us describe shingles as something more than “this person has a preference for scratching themselves frantically, and that preference is valid, nothing to worry about here”. I don’t have a beautiful elegant version of a model like this yet, but I think Bryan himself has gone most of the way to an at-least-adequate one.

In his post The Depression Preference, Bryan admits that most depressed people don’t want to be depressed. But he terms this a meta-preference – a preference over preferences. They have depressive preferences – for example, a preference for sitting around crying rather than doing work. They would meta-prefer not to have those preferences. But they do have them.

I agree this is a fruitful way to look at things, but I think we have to be really careful here, and that using the same term for endorsed meta-preferences and unendorsed object-level preferences is preventing this level of care. Let’s call endorsed preferences which people meta-prefer to have “goals”, and unendorsed preferences which people would meta-prefer not to have “urges”. I think this closely matches our intuitive understanding of these terms.

Suppose I created a sinister machine that beamed mind control rays into Bryan’s head and gave him an urge to constantly slap himself in the face. This urge could theoretically be resisted, but it’s so strong that in practice he never managed to resist it. It didn’t make him enjoy slapping himself in the face, or think this was a reasonable thing to do. It just made him compulsively want to keep doing it. He loses his job, his friends, and his dignity, because nobody wants to be around someone who’s slapping himself in the face all the time. I hope we can common-sensically agree on the following:

1. This is bad
2. Bryan would want to find and destroy the sinister machine
3. That would be a pretty reasonable goal for Bryan to have, and society should support him in this

This seems a lot like the shingles case. A sinister outside imposition (the viral infection) gives its victim an urge to constantly scratch themselves. It doesn’t make them enjoy scratching themselves, or think this is a reasonable thing to do. These people want to cure their shingles infection, and everyone agrees this desire is reasonable.

But this also seems a lot like some cases of OCD. Did you know that a subset of childhood OCD is caused by a streptococcal infection? So again, you get a sinister outside imposition (an infection) that gives its victim an urge to, let’s say, wash their hands fifty times a day. It doesn’t make them enjoy washing their hands, or think this is a reasonable thing to do (some OCD patients do believe their rituals are necessary, others don’t). These people want to cure their OCD, and I at least agree this desire is reasonable.

If you would support the sinister machine victim and the shingles victim, it’s hard for me to see a case for putting the OCD victim in a different category. I agree I’m using as clear a case as possible (most mental disorders aren’t obviously due to infections), but both Bryan and I are trying to avoid bringing specific facts about biology into this mostly-philosophical debate. The distinction between goals and urges turns what looked like an acceptable situation (these people are following their preferences, which is good) into an unacceptable situation (these people’s goals are being thwarted by unwelcome urges which they can’t resist).

I expect most of Bryan’s skepticism to focus on those last two words – “can’t resist”. He will no doubt bring up his gun-to-the-head test again. If we put a gun to the head of a shingles patient, they could stop scratching. So although we can be sympathetic to the trouble their unwanted new preference causes them, how can we recommend anything other than “just suck it up and resist the preference”?

The best model of decision-making I know of comes from research on lampreys. Various areas of the lamprey brain come up with various plans – hunt for food, hide under a rock, wriggle around – and calculate the “strength” of the “case” for each one, which they convert into an amount of dopamine. They send this dopamine to a part of the brain called the pallium, and then the pallium executes whichever plan has the most dopamine associated with it.

Suppose I have shingles. I’m giving a speech to a group of distinguished people whom I desperately want to impress. Then I get a very strong itch. Part of my brain calculates the expected value of continuing to speak in a dignified way, and converts that into dopamine. Another part calculates the importance of scratching myself vigorously, and converts that into dopamine. The pallium compares these two amounts of dopamine, one is larger than the other, and the decision gets made. If the itch is bad enough, and if whatever lizard-brain nucleus makes me want to scratch itches has enough dopamine to spare, then I never had a chance.

“But,” Bryan objects, “if I put a gun to your head, and threatened to shoot you if you scratched the itch, you wouldn’t do it, would you?”

In that case, a part of my brain calculates the expected value of continuing to speak in a dignified way plus not getting shot. This is a very high expected value! It sends lots and lots of dopamine to my pallium. The part of my brain calculating the expected value of scratching the itch and getting shot calculates this as a very low-expected-value course, and sends some a very low (maybe negative?) signal. The pallium decisively selects the plan to keep speaking and not get shot.

To summarize: the brain compares the strength of various preferences and executes the strongest. Anything that strengthens your urges at the expense of your goals makes you more likely to do things you don’t endorse, and makes you worse off. In a counterfactual world where a threatened gunshot is also weighing down the scale, maybe the calculus would come out different. But in the non-counterfactual world where there is no gunshot, the calculus comes out the way it does.

(also, if Bryan uses his gunshot analogy one more time, I am going to tell him about all of the mentally ill people I know about who did, in fact, non-metaphorically, non-hypothetically, choose a gunshot to the head over continuing to do the things their illness made it hard for them to do. Are you sure this is the easily-falsified hill you want to die on?)

This model doesn’t use the word or the concept of “choice” anywhere. There are various algorithms mechanically evaluating the expected reward of different actions, and a more central algorithm comparing all of those evaluations. Those algorithms could have resolved differently in different situations, and you can be uncertain how they will resolve in the same situation, but there’s no point at which they actually could resolve differently in the same situation. If this makes you want to start debating free will – in either direction – I cannot recommend this Less Wrong post highly enough.

A few examples to hammer this in:

1. Most weekends, Alice stays in and reads a book (preference strength 20). But today is her firstborn child’s wedding, which she has been looking forward to for years (preference strength 100). Just before she leaves for the chapel, she gets a terrible migraine, and she feels like it would be unbearable to go out of her room (preference strength 200). Since 200 is greater than 100, Alice misses the wedding and feel miserable, since she would have meta-preferred to go to the wedding. If you had threatened to shoot her unless she went to the wedding, she would have gone to the wedding and been miserable the whole time, because she is terrified of death (preference strength 9999) and 9999 is greater than 200.

2. Bryan is a responsible member of society and wants to work hard and take care of his family (preference strength 100). He drinks some alcohol, but because he has no genetic or environmental risk factors for alcoholism, it doesn’t make him feel any urge to drink himself to death (preference strength 0), so he doesn’t. If we CRISPRed him to give him every single alcoholism risk gene plus crippling anxiety, then drinking the alcohol would make him feel a very strong urge to drink himself to death (preference strength 200), and he would drink himself to death instead of caring for his family.

3. CRISPRed alcoholic Bryan goes to an addiction doctor. The doctor advises him to take the anti-alcoholism drug naltrexone (-20 preference strength for alcoholism). Then the doctor advises him to go to Alcoholics Anonymous and get a whole new friend group in which his status depends entirely on his ability to remain sober (+20 for staying sober). Now his preferences are “stay sober and take care of my family” (strength 120) vs. “drink myself to death (strength 180), but the preference to drink is still stronger, so he does.

4. Bryan goes to a therapist who asks him to visualize the things he loves about his family and why he thinks it’s important to take care of them, which makes this more vivid in his mind (preference +10 for sobriety). Bryan’s boss threatens to fire him if he misses one more day of work because of drunkenness (preference +20 for sobriety). Now he’s at 150 for sobriety vs. 180 for drinking. He gives $20,000 to Beeminder, which they will only give him back if he stays sober for the next year (+20 for sobriety), and he reads George Ainslie’s Picoeconomics which describes ways to reconceptualize choices across time to better account for all of their implications (+20 for sobriety). Now he’s at 190 for sobriety vs. 180 for drinking, so he stays sober.

5. A few months later, Bryan’s friend dies in an accident. He feels angry, depressed, and anxious. This makes alcohol seems more attractive, since it would temporarily help him forget these feelings (+20 for drinking). At the same time, he stops going to AA because it’s annoying and far away (-20 for staying sober). Now he’s at 170 for sobriety vs. 200 for drinking, so he falls off the wagon.

I’m not claiming this lamprey model is exactly literally true for humans. And I’m not claiming there’s a perfect binary distinction between endorsed goals and unendorsed urges. This model is full of complications and gray areas. I’m just saying it’s a better model, with fewer gray areas, than trying to separate everything into just “preference” or “constraint”, and shooting yourself in the foot again and again like some kind of tipped-over Gatling gun.

And it goes a lot of the way to modeling mental illness: the mentally ill have conditions that give them strong unendorsed urges. For any given strength of goal, having strong urges will make people less able to pursue that goal, in favor of pursuing the urges instead, and that will make them worse off, for a definition of “well off” that involves being happy and achieving goals. These people very reasonably want to stop having these weird urges so they can pursue their goals in peace.

Bryan will correctly point out that there are awkward implications in identifying “unexpected generator of strong unendorsed urges” with “disease”. For example, gay people in a traditional religious community will have strong urges to have homosexual relationships, and they won’t endorse those urges – they would probably rather be straight instead.

Or: obese people feel an urge to eat which they don’t endorse. Should we call obesity a disease, and describe them as having a disease which produces urges contrary to their preferences? Some people say yes (and keep in mind that both genetics and viral infections can induce obesity). But suppose some normal-weight person would rather be supermodel-thin, and their perfectly normal urge to eat a normal amount prevents them from looking like a broomstick. Is their normal level of hunger a disease? A naive equation of “biological generator of unendorsed urges” and “disease” would say yes!

We want some criteria that let us call shingles a disease, but don’t let us call “being thin but wanting to be even thinner” a disease. Unfortunately, there is no perfect solution to this problem. People have wanted perfect solutions to definitional questions ever since Plato defined man as “a featherless biped”, and it’s never worked. Luckily, there are kludgy, good-enough solutions, which I describe in Dissolving Questions About Disease, the fourth most popular Less Wrong post of all time. If you still think this is confusing, please read it. If it’s still confusing even after that, try The Categories Were Made For Man, Not Man For The Categories.

I think Bryan should be happy with this solution. It’s very libertarian. It says that it’s up to every individual to decide how to satisfy their own preferences (including meta-preferences). If your problem is constraints (you want to go to Hawaii, but you don’t have enough money), you can work to resolve those constraints (eg go to work and earn more money). If your problem is urges (you want to go to Hawaii, but you’re too anxious to leave your room), you can work to resolve those urges (eg go to a psychiatrist and get medication). The job of a good liberal society is to support people in achieving their own goals as they understand them, and this includes supporting their decision to get the job they want and their decision to get the psychiatric treatment they want.

As I write this essay, I’m a little bit caffeinated. I looked at my preference set – which included an urge to get back in bed instead of writing blog posts – decided it didn’t achieve my goals, and took a psychotropic drug to shift my preference set to one I liked better. And if we’re willing to accept this in relatively trivial cases, the argument for accepting it is even stronger for people whose preference sets have been deranged by obvious bizarre causes – infections, hormone imbalances, brain injuries, addictive substances, genetic defects – and for people whose irresistible urges are ruining their lives in preventable ways.

Posted in Uncategorized | Tagged | 458 Comments

2019 Adversarial Collaboration Winners

Posted on January 13, 2020 by Scott Alexander

Thanks to everyone who participated and/or voted in the 2019 Adversarial Collaboration Contest. And the winner is…

…

…

Adrian Liberman and Calvin Reese, for Does Calorie Restriction Slow Aging?.

An extraordinarily close second place (26.9% vs. 26.2% of votes) goes to David G and Froolow, for Is Eating Meat A Net Harm?.

Both of these did great research and were written up well. I especially like them as winners because they have such different strengths.

The calorie restriction collaboration was carefully focused on a factual question. I think this is a promising model for adversarial collaborations, and that others failed the further they deviated from this. For example, the circumcision collaboration did a good job assessing the quantifiable benefits and harms of the practice, but it turned out that most people who disagreed about it weren’t disagreeing because they assessed quantifiable benefits and harms differently. The abortion collaboration ended up in a similar place. By focusing on a topic where there really was debate about what the research showed, and by hitting the lit review portion out of the park, Adrian and Calvin helped deconfuse a lot of previously confused people.

And the meat collaboration managed to succeed without being like this at all! It was unabashedly just a lengthy review of every single plausible argument for and against vegetarianism, and bulldozed over the immense difficulties with this approach by putting in more work than any reasonable person would have thought possible. And if it didn’t get quite as many votes as calorie restriction, it won on another metric – here are some of voters’ comments (plus some extra from the blog):

– After this, I expect to restrict more strictly to chicken and fish, and alternate more aggressively towards fish as a hedge against the possibility I’m undervaluing chicken sapience.

– Got me to significantly change my diet (at least over the past few weeks) towards more fish and much less chicken.

– I read it, and I changed what I believe and how I changed my life accordingly.

– My final pick is not necesarily based on the article‘s excellence but rather due to me going to change my eating behaviour, which I find an impressive thing Form an article to do.

– This has practical implications for my life. I’ve stopped eating pork because of it (baby steps, working towards less meat generally)

– I actually resolved to stop eating chicken (the only meat I can regularly eat, due to dietary restrictions) based on that piece, so I’d say it was pretty effective in informing me about things.

– It convinced me not to buy chicken that isn’t organic/free-range

– This article will result in all my family eating less meat. It’s actually going to change our lives, health, and the environment!

– As a result of the adversarial collaboration on the ethics of eating meat, have dramatically scaled back my meat consumption to probably 20% of my previous value, and the meat I do eat now is almost exclusively fish and invertebrates.

– Learning about the harm of factory farming from their dispassionate and empirical analysis has prompted me to greatly reduce my red meat and poultry consumption.

Some voters brought up a reasonable complaint: the end result ended up being pretty (though not completely) pro-vegetarian. How do we tell the difference between “a good faith effort by intelligent people naturally converges on vegetarianism” vs. “the anti-vegetarian collaborator slacked off”?

In this case, we tell because the anti-vegetarian collaborator posted a comment about his thought process and what convinced him. But there were other cases where people had the same question, and still other cases where one collaborator did a good job representing their own anti-X position, but other people were anti-X for different reasons that didn’t get represented.

If I had infinite resources, I would fund adversarial collaborations between well-known and universally-recognized intellectuals on different sides of a topic, who everybody trusted to stick to their guns. As it is, I can only say I’m delighted to have stumbled into the one part of the world where “people are too likely to change their mind when presented with new evidence on controversial issues” is a problem.

Some thoughts on the other collaborations:

Circumcision: I loved this one. I’d never seen a good assessment of exactly what health risks circumcision was supposed to prevent, and I didn’t know how weak the evidence was that the foreskin helps with sexual pleasure. But the conclusion ended up being “the quantifiable benefits of circumcision are nonzero but pretty low; the quantifiable harms are not obviously distinguishable from zero but who knows”, which leaves a lot of space for people’s ethical intuitions, which turned out to be REALLY STRONG. One reader said they were going to boycott my blog from now on for not having no-platformed this ACC, and a few others seemed only slightly less angry. On the other hand, it also did better than average among voters, so good job there. I take a small amount of blame for this one not being more popular – I retitled it to be about the ethics of circumcision, whereas the original title had been about benefits vs. harms. But I think it’s naturally hard to write something about benefits and harms without it sounding like you’re talking about ethics, and in this case the ethics were too complicated to fit in the model provided for them. Some positive comments from the survey: “This…actually changed my opinion from circumcision being mildly ethically wrong back to neutral”, “I gained a much more nuanced understanding of the benefits position to the point that my mind was changed to be in favor (maybe too strong), or at least not opposed to, it for developing countries”, “It tidily presented the pros and cons and presented a lot of useful information, with a clear conclusion. It shifted my thinking the most of all of them.”

Space Colonization: You guys presented a lot of evidence for one side, then at the end switched to the opposite side based on a one paragraph explanation of something you’d never brought up before. If that was your crux, I wish you had analyzed it in more depth. If the whole point is to make something that can’t be defunded, couldn’t the government (or whoever) give the money to a private foundation with really good trustees, no takebacks? Maybe there’s a problem with that idea, I don’t know, but if you’re going to make defundability the center of your conclusion, I wish you had examined it more closely. Some positive comments from the survey: “Excellent selection of question, manages to present both sides fairly and come to an insightful conclusion”, “I think this ACC did the best job of covering the entire scope of the question they assigned to themselves, while still presenting a shared conclusion”, “Interesting non-obvious conclusion, subject I care a lot about, pretty pictures”.

Gene Editing: This one seemed to spend a lot of time on very knowledgeable and very well-cited assessments of the current state of the technology and how and why it worked, but didn’t really get around to assessing the “should” question in the title. It also had a few factual missteps – maybe no more than the others, but more obvious since it was so fact-based. While it was an impressive work of scholarship I’m not sure it came together as an adversarial collaboration. Some positive comments from the survey: “Very nicely presented ACC. It was thought provoking and totally enrapturing!”, “well-reasoned collaboration on a difficult question”, “Great, nuanced answer to a complex question”, “This collaboration caused me to reconsider my enthusiasm for CRISPR based on the narrative provided in most press releases. The topic is much more involved than I’d initially realized.”

Abortion: An adversarial collaboration on a completely moral question – you guys didn’t make this easy for yourself, did you? I don’t think you made any particular missteps given the difficult task you set yourselves, but this is another one that I feel like didn’t quite come together. Some positive comments from the survey: “Most interesting (and politically relevant) topic, plus it seems icerun’s position actually shifted somewhat by virtue of having to marshall arguments for it, proving the whole endeavor to be more than just an exercise in futility”, “I thought it best captured the spirit of an Adversarial Collaboration”, “This was a nice, cautious walk-through of an extremely divisive subject. i never thought i would enjoy reading a “point/counterpoint” on abortion, but i enjoyed this one”, “Lots of adversity, focused on the actual disagreement, and balanced data and philosophy well.”

Automation: Seemed broadly correct and helpful. I didn’t find it too exciting because I felt like I had already covered most of the same beats in this article (which they cited), but I’m surprised other people didn’t vote for it more. Some positive comments from the survey: “Importance of the issue and the thoroughness with which it was explored”, “The most fitting, thought-out and the one that draws actual conclusions”, “Highest rationality-to-contentiousness ratio”.

Spiritual Experience: This was another one that was long, fascinating, and didn’t seem to be making much of an attempt to come to a conclusion. I especially liked the section on near death experiences, and I’ll be thinking about it a lot, but I didn’t feel like this collaboration gave me the tools I would need to generate or test hypotheses about what might be going on. Some positive comments from the survey: “The most polished and one which most likely caused me to reconsider things”, “Most informative. Best at following an ideal format”, “This is the one that 1) is most interesting to me, 2) seems like it had a strong difference of opinion as a starting point.”

I included the positive comments because I think comments on these kinds of things (mine and others) naturally tends to skew negative. Certainly the comments in the comments section were overwhelmingly negative even for the winning collaborations (seriously, what was up with this?) So I want to counter this by pointing out that every collaboration got at least 25 votes, and the comments on the voting survey were mostly positive. It’s easier to nitpick than to give praise where praise is due, but people put in a lot of work here and it was generally appreciated.

I promised that I would come up with some fair way of dividing the prize money, with at least 50% going to the first place winner. Because the top two entries were so close, and because I was so impressed with the second place winner, I choose to give $1,300 to Calvin and Adrian ($650 each), and $1,200 to David and Froolow ($600 each). Please send me an email at scott[at]slatestarcodex[dot]com telling me where to send your share of the money – I can PayPal it to you or donate it to a charity of your choice. Thanks to SSC Patreon supporters for making this possible.

As much as I enjoyed this, I don’t expect to do another contest next year. For one thing, I think requiring two people made it a lot harder – 22 out of 30 teams dropped out before the deadline, and I worry some of that involved a lot of wasted work. For another thing, it involved a surprising amount of work on my part converting whatever Word or Google Docs file people sent me into a format I could use on the blog. Finally, I feel like the past two years did a good job exploring this medium, and now it’s up to other people with real questions to see if they can adapt it to their needs.

Most likely I’ll be replacing this with a book review contest sometime towards the end of next year, so if you read any good books, keep them in mind.

But I continue to be interested in adversarial collaborations. If you happen to do one, please tell me – there’s a decent chance I’ll publish it.

Posted in Uncategorized | Tagged | 166 Comments

Open Thread 145

Posted on January 12, 2020 by Scott Alexander

This is the bi-weekly visible open thread (there are also hidden open threads twice a week you can reach through the Open Thread tab on the top of the page). Post about anything you want, but please try to avoid hot-button political and social topics. You can also talk at the SSC subreddit – and also check out the SSC Podcast. Also:

1. In last month’s links, I posted about concern that Alcor was getting too close to a weird cult. Alcor leader Max More commented with an explanation of why that wasn’t a fair claim, which I believe. I apologize for helping spread an exaggerated and poorly-contextualized version of the story.

2. Also in the spirit of “very long comments by very important people” – Brian Earp of Yale University’s Program In Ethics And Health Policy posted some thoughts on the adversarial collaboration about circumcision.

3. But if you prefer very short comments by very important people, here’s Gary Marcus on Twitter about GPT-2 chess. Gary Marcus calling your work trivial is how you know you’ve really made it in AI! Also, with all of the great Gwern stuff I mentioned on that post I should have mentioned that Gwern has a Patreon that helps fund his projects. Also, it looks like a commenter made a better GPT-2 chessbot, although you can’t play it.

4. And if you prefer medium-length comments by non-famous people – chaosmage tries to extend my Why Doctors Think They’re The Best post to programmers.

5. This is your absolute last chance to vote for the winner of the adversarial collaboration contest. I’ll be naming the victor and distributing prizes this coming week.

6. And this is your second-to-last chance to take the 2020 SSC survey if you didn’t already; I’ll probably post one more reminder this week, then close it on Friday or so.

Posted in Uncategorized | Tagged | 741 Comments

What Intellectual Progress Did I Make In The 2010s?

Posted on January 8, 2020 by Scott Alexander

One of the best parts of writing a blog is being able to answer questions like this. Whenever I felt like I understood new and important, I wrote a post about it. This makes it easy to track what I learned.

I think the single most important thing I discovered this decade (due to a random comment in the SSC subreddit!) was the predictive coding theory of the brain. I started groping towards it (without knowing what I was looking for) in Mysticism And Pattern-Matching, reported the exact moment when I found it in It’s Bayes All The Way Up, and finally got a decent understanding of it after reading Surfing Uncertainty. At the same time, thanks to some other helpful tips from other rationalists, I discovered Behavior: The Control Of Perception, and with some help from Vaniver and a few other people was able to realize how these two overarching theories were basically the same. Discovering this area of research may be the best thing that happened to me the second half of this decade (sorry, everyone I dated, you were pretty good too).

Psychedelics are clearly interesting, and everyone else had already covered all the interesting pro-psychedelic arguments, so I wrote about some of my misgivings in my 2016 Why Were Early Psychedelicists So Weird?. The next step was trying to fit in an understanding of HPPD, which started with near-total bafflement. Predictive processing proved helpful here too, and my biggest update of the decade on psychedelics came with Friston and Carhart-Harris’ Relaxed Beliefs Under Psychedelics And The Anarchic Brain, which I tried to process further here. This didn’t directly improve my understanding of HPPD specifically, but just by talking about it a lot I got a subtler picture where lots of people have odd visual artifacts and psychedelics can cause slightly more (very rarely, significantly more) visual artifacts. I started the decade thinking that “psychedelic insight” was probably fake, and ended it believing that it is probably real, but I still don’t feel like I have a good sense of the potential risks.

In mental health, the field I am supposed to be an expert on, I spent a long time throwing out all kinds of random ideas and seeing what stuck – Boorsboom et al’s idea of Mental Disorders As Networks, The Synapse Hypothesis of depression, etc. Although I still think we can learn something from models like those, right now my best model is the one in Symptom, Condition, Cause, which kind of sidesteps some of those problems. Again, learning about predictive processing helped here, and by the end of the decade I was able to say actually useful things that explained some features of psychiatric conditions, like in Treat The Prodrome. Friston On Computational Mood might also be in this category, I’m still waiting for more evidence one way or the other.

I also spent a lot of time thinking about SSRIs in particular, especially Irving Kirsch (and others’) claim that they barely outperform placebo. I wrote up some preliminary results in SSRIs: Much More Than You Wanted To Know, but got increasingly concerned that this didn’t really address the crux of the issue, especially after Cipriani et al (covertly) confirmed Kirsch’s results (see Cipriani On Antidepressants). My thoughts evolved a little further with SSRIs: An Update and some of my Survey Results On SSRIs. But my most recent update actually hasn’t got written up yet – see the PANDA trial results for a preview of what will basically be “SSRIs work very well on some form of mental distress which is kind of, but not exactly, depression and anxiety”.

One place I just completely failed was in understanding the psychometrics of autism, schizophrenia, transgender, and how they all related to each other and to the normal spectrum of variation. I kind of started this program with Why Are Transgender People Immune To Optical Illusions? (still a good question!), fumbled around by first-sort-of-condemning and then sort-of-accepting the diametrical model of autism and schizophrenia, and then admitting I just didn’t know what was going on in this area and not talking about it much more. I still sometimes have thoughts like “Is borderline the opposite of autism?” or “Are schizoid people unusually charismatic, unusually uncharismatic, or somehow both?”, and I still have no idea how to even begin answering them. Autism And Intelligence: Much More Than You Wanted To Know at least helped address a very tangentially related question and is probably the closest thing to a high point this decade gave me here.

The Nurture Assumption shaped my 2000s views of genetics and development. Ten years later, I’m still trying to process it, and in particular to square the many behavioral genetics studies showing nonshared environment doesn’t matter with the many other studies suggesting it does (see eg The Dark Side Of Divorce and Shared Environment Proves Too Much). I think I started to get more of handle on attachment theory and cPTSD as both being different aspects of the same basic predictive processing concept of “a global prior on the world being safe” – see Mental Mountains and Evolutionary Psychopathology for two different ways of approaching this concept. This made me conclude that I might have been wrong about preschool (though see also Preschool: Much More Than You Wanted To Know). Honestly I am still confused about this. The one really exciting major good update I made about genetics this decade was understanding and fully internalizing the omnigenic model.

One of the big motivating questions I keep coming back to again and again is – what the heck is “willpower”? I started the decade so confused about this that I voluntarily bought and read Baumeister and Tierney’s book Willpower and expected it to be helpful. I spent the first few years gradually internalizing the lesson (which I learned in the 2000s) that Humans Are Not Automatically Strategic (see also The Blue-Minimizing Robot as a memorial to the exact second I figured this out), and that hyperbolic discounting is a thing. Since then, progress has been disappointing – the only two insights I can be even a little happy about are understanding perceptual control theory and Stephen Guyenet’s detailed account of how motivation works in lampreys. If I ever become a lamprey I am finally going to be totally content with how well I understand my motivational structure, and it’s going to feel great.

Speaking of Guyenet, if nothing else this last decade has taught us that Gary Taubes did not solve all of nutrition in 2004, that Atkins/paleo/keto are good for some people and bad for others, and that diet is still hard. See the various Guyenet vs. Taubes and Taubes vs. Guyenet posts, and my 2015 The Physics Diet on where I was at that point. So what is going on with diet? Compressing an entire decade’s worth of research into two words, the key phrase seems to be “set point” (which, credit to Taubes, he was one of the first people to popularize). See eg Anorexia And Metabolic Set Point and Del Giudice On The Self-Starvation Cycle. But what is the set point and how does it get dysregulated? See my book review of The Hungry Brain for the best answer to that I have now (not so good). This whole mess helped me get a better understanding of Contrarians, Crackpots, and Consensus, and eventually ended up with me Learning To Love Scientific Consensus.

In terms of x-risk: I started out this decade concerned about The Great Filter. After thinking about it more, I advised readers Don’t Fear The Filter. I think that advice was later proven right in Sandler, Drexler, and Ord’s paper on the Fermi Paradox, to the point where now people protest to me that nobody ever really believed it was a problem. AI has been the opposite – I feel like the decade began with people pooh-poohing it, my AI Researchers On AI Risk was part of a large-scale effort to turn the tide, and now it’s more widely accepted as an important concern. At the same time, the triumphs of deep learning has made things look a little different – see How Does Recent AI Progress Affect The Bostromian Paradigm? and Reframing Superintelligence – and I’ll be reviewing Human Compatible soon. I also got some really great insights on what “human-level intelligence” means from the good people at AI Impacts, which I wrote up as first Where The Falling Einstein Meets The Rising Mouse and later Neurons And Intelligence: A Bird-Brained Perspective (see also Cortical Neuron Number Matches Intuitive Perceptions Of Moral Value Across Animals and all the retractions and meta-retractions thereof). Overall I think I’ve updated a little (though not completely) towards non-singleton scenarios and not-super-fast takeoffs, which combined with the increased amount of effort being put into this area is cause for a little more optimism than I had in 2010. I know some smart people disagree with me on this.

In the 2000s, people debated Kurzweil’s thesis that scientific progress was speeding up superexponentially. By the mid-2010s, the debate shifted to whether progress was actually slowing down. In Promising The Moon, I wrote about my skepticism that technological progress is declining. A group of people including Patrick Collison and Tyler Cowen have since worked to strengthen the case that it is; in 2018 I wrote Is Science Slowing Down?, and late last year I conceded the point. Paul Christiano helped me synthesize the Kurzweillian and anti-Kurzweillian perspectives into 1960: The Year The Singularity Was Cancelled.

In 2017, I synthesized some thoughts that had been bouncing around about rising prices into Considerations On Cost Disease, still one of this blog’s most popular posts. I felt like early responses were pretty weak, although they brought up a few interesting points on veterinary medicine, cosmetic medicine, and other outliers that I still need to transform into a blog post; Alon Levy’s work on infrastructure in particular has also been great. The first would-be-general-answer that made me sit up and take notice was Alex Tabarrok’s book (link goes to my review) The Prices Are Too Damn High – but I explain there why I don’t think it can be the full answer. The most recent thing I learned (tragically underhighlighted in my wage stagnation post) is that a lot of apparent wage stagnation is due to cost disease – consumer services ballooning in cost means the consumer inflation index rises faster than the business inflation index, productivity gets measured by business inflation, wages get measured by consumer inflation, and so it looks like productivity is outpacing wages. This is still only half of the apparent decoupling, but it’s still a big deal.

The highlight/lowlight of the decade in social science was surely the replication crisis. My first inkling that something like this might exist was in December 2009, from the Less Wrong post Parapsychology: The Control Group For Science. There were a couple of years where people were trying to figure out how bad the damage was; of these, my 90% Of All Claims About Problems With Medical Studies Are Wrong was more optimistic, and my slightly later The Control Group Is Out Of Control was more pessimistic (I still stand by both). As the decade continued, I think we got better about realizing that many to most older studies were wrong, in a way that didn’t make us feel like total Cartesian skeptics or like we were going to have to throw out evolution or aspirin or any of the things on really sound footing. After that it just became fun: my “acceptance” stage of grief produced some gems like 5-HTTLPR: A Pointed Review.

On SSC, I particularly examined some of the replication issues of growth mindset. I started in 2015 by pointing out that the studies seemed literally unbelievable, but so far nobody had tried attacking them. I claim to have been way ahead of the curve on this one – if you don’t believe me, just read the kind of pushback I got. But by 2017, that situation had changed – Buzzfeed posted an article that called the field into question, but still without clear negative evidence. Finally, over the past few years, the negative studies have come pouring in, accented by supposedly “positive” studies by Dweck & co showing effect sizes only a tiny fraction of what they had originally claimed. The latest research (can’t find it right now) is that praising students for effort rather than for ability has no effect on how hard-working or successful they are, debunking the original headline result that got most people interested in the field and nicely closing the circle.

In 2010 I worked with a medical school professor who studied the placebo effect and realized I didn’t understand it at all. Over the past few years I gradually became more convinced of the heterodox position of Gøtzsche and Hróbjartsson, who believe placebo effect doesn’t apply to anything except pain and a few other purely mental phenomena (The Placebo Singers, Powerless Placebos). I’ve since become less convinced that’s true (just today I treated a patient who I’m pretty sure has psychosomatic vomiting from what he falsely believes was a medication side effect, and if belief can cause vomiting, surely it can also treat it). As with so many other things, it was predictive processing to the rescue – see section IV part 7 of my Surfing Uncertainty review. I now think I have a pretty good understanding of how placebos can treat both purely mental conditions and conditions heavily regulated by the nervous system, while still mostly sticking to Gøtzsche and Hróbjartsson’s findings.

I started this decade confused about how to understand ethics given all the paradoxes of utilitarianism. I’m still 90% as confused now as I was then, but I still feel like I’ve made some progress. A lot of my early thinking involved folk decision theory and contractualism – how would you act if you expected everyone else to act the same way? I explored the edges of this idea in You Kant Dismiss Universalizability and Invisible Nation. I’m not how much it helped my search for metaethical grounding, but it helped me get a more robust understanding of liberalism and clarify my views on some practical questions, eg Be Nice, At Least Until You Can Coordinate Meanness and The Dark Rule Utilitarian Argument For Science Piracy. In general I think this has given me a more cautious theory of decision-making that’s occasionally (and terrifyingly) set me against other more anti-Outside-View rationalists. I think the most important shift in my understanding of ethics this decade was the one I wrote up in Axiology, Morality, Law (formerly titled “Contra Askell On Moral Offsets”), which isn’t related to grounding utilitarianism at all but sure helps make the problem less urgent

Despite my better judgment, I waded into politics a lot this decade. I Can Tolerate Anything Except The Outgroup produced this blog’s first “big break”, but it admitted it didn’t really understand the factors underlying “tribe”. Since then Albion’s Seed helped provide another piece of the puzzle, and a better understanding of class provided another. I went a little further discussing why tribes have ideologies associated with them in The Ideology Is Not The Movement, how that is like/unlike religion in Is Everything A Religion?, and hammered it home unsubtle-ly in Gay Rites Are Civil Rites.

I wrote the Non-Libertarian FAQ sometime around 2012 and last updated it in 2017. Sometime, possibly between those dates, I read David Friedman’s A Positive Account Of Property Rights, definitely among the most important essays I’ve ever read, and got gold-pilled (is that a term? It should be a term). I’ve since been trying to sort this out with things like A Left-Libertarian Manifesto, and trying to move them up a level as Archipelago. James Scott’s Seeing Like A State and David Friedman’s Legal Systems Very Different From Ours were also big influences here. Like all platitudes, “government is a hallucination in the mind of the governed” is easy to understand on a shallow level but fiendlishly complicated on a deep level, but I feel like all of these sources have given me a deep understanding of exactly how it’s true.

The rightists (especially Moldbug) get the other half of the credit for helping me understand Archipelago, and also deserve kudos for teaching me about cultural evolution. My first attempts to engage with this topic were nervous and halting – see eg The Argument From Cultural Evolution. I got a much better feel for this after reading The Secret Of Our Success, and was able to bring this train of thought back to its right-wing roots Addendum To Enormous Nutshell: Competing Selectors. I’m grateful to the many rightists who argued about some of these points with me until they finally stuck.

I had more trouble engaging with leftists. I started with Does Class Warfare Have A Free-Rider Problem, and it took me way too long to figure out that this was one of the major questions sociology was asking, and that “an answer” would look less like “your game theory analogy is missing this one variable” and more like a whole library full of books on what the heck society was. Later the same engagement produced Conflict Vs. Mistake, which I am informed is still unfair and partially inaccurate, but which (take my word for it) is a heck of a lot better than the stuff I was thinking before I wrote it. More recently I’ve been trying to figure out a sympathetic account of activism (as opposed to the unsympathetic account that it’s virtue signaling and/or people who are really bad at figuring out what things are vs. aren’t effective). You can sketch the outline at Respectability Cascades and Social Censorship: The First Offender Model, and I’ll sketch the whole thing out sometime when I have enough emotional energy to deal with the kind of people who will have opinions on it.

I also had to grapple with the sudden rise of social justice ideology. I’m proud of my work on gender differences – both what I learned, how I wrote it up, and the few bits of original research I did (eg Sexual Harassment Levels By Field). My knowledge and claims started off kind of weak (Gender Differences Are Mostly Not Due To Offensive Attitudes), but I eventually feel like I got a really great evidence-based basically-airtight theory of what is going on with gender imbalances in different fields, which I posted most of in Contra Grant On Exaggerated Differences (I’m still thankful for the commenter who solved that one remaining paradox about math majors). And despite all the mobs and vitriol I think sound science has basically triumphed here – I was delighted to recently see as mainstream a blog as Marginal Revolution recently publish, without any caveats or double-talk, a post called Sex Differences In Personality Are Large And Important and get basically no pushback. I was a lot more pessimistic around 2017 or so and described some thoughts on how to make a strategic retreat in Kolmogorov Complicity And The Parable Of Lightning, which I still think is relevant in some areas. But I actually start the new decade really optimistic – I haven’t written up an explanation of why, but careful readers of New Atheism: The Godlessness That Failed may be able to figure it out, especially if they apply some of the same metrics I used there to track how social justice terms have been doing recently.

Upstream of politics, I think I got a better understanding of…game theory? Complex system dynamics? The most important post here was Meditations On Moloch; the sequel/expansion, whose thesis I have yet to write up in clear prose, is The Goddess Of Everything Else. Reading Inadequate Equilibria was also helpful here.

My understanding of “enlightenment” went from total mystical confusion to feeling like I have a pretty good idea what claims are being made, and mostly believing them. This line of thinking started with the Mastering The Core Teachings Of The Buddha review, and then was genuinely helped by Vinay Gupta’s contributions summed up in Gupta On Enlightenment, despite the disaster in the comments of that post. From there I progressed to reading The Mind Illuminated, and Is Enlightenment Compatible With Sex Scandals led me to discover The PNSE Paper, which as much as anything else helped ground my thinking here (the comments there were pretty good too).

And thanks to all of you who took the survey, I went from skepticism of birth order effects to saying Fight Me, Psychologists: Birth Order Effects Exist And Are Very Strong. This was bolstered by Eli Tyre and Bucky’s posts on Less Wrong about birth order in mathematicians and physicists respectively. Last year I expanded on that with a post on how birth order responded to age gaps (somewhat updated and modified here, thanks Bucky). Once this year’s survey results are in I expect to have a lot more data on exactly what causes birth order effects and maybe how to deal with them. If you haven’t taken the SSC survey this year, consider this your reminder to do it here.

Not many of these were total 180 degree flips in my position (though birth order, preschool, psychedelic insight, and the rate of scientific progress are close). And not many of them completely resolved a big question that had been bothering me before (though the Fermi Paradox paper, omnigenic model, and animal neuron work did). A few of them confirmed things I had only suspected before (growth mindset, gender imbalances, diet). Many of them feel like what MIRI calls “deconfusion”, turning a space full of unknown unknowns to one where you feel like you have a decent map of where the major problems are and what it would feel like to solve them. The enlightenment research seems to fit here – I went from “I have no idea how to even think about this question or whether it’s all fake” to “I don’t know exactly what’s going on here, but I know what needs to be explained, and it looks like the explanation will have a shape that fits nicely into the rest of my ontology.”

There’s an argument that I should learn less each decade, since I’ll be picking higher and higher fruit. My own knowledge can advance either because civilization advances and I hear about it, or because I absorb/integrate older knowledge that I hadn’t noticed before. Civilization advances at a decade per decade (or maybe less; see the Cowen & Southwood paper above), but each year it becomes harder and harder to find relevant older knowledge that I haven’t integrated yet. I plausibly only have five more decades to live, and I don’t think I’d be happy only advancing five times this amount over the rest of my life, let alone less than that.

But I notice I only started SSC about halfway through the decade, and that my progress picked up a lot after that. I don’t think it’s just recall bias from being able to track myself better. I think being able to put ideas out there and have you guys comment on them and link me to important resources I might have missed has been great for me. I only started taking full advantage of that around 2015; this decade I have a head start. And maybe I’ll discover other useful tools that will speed things up further.

Thanks for sticking around with this blog, and have a happy third decade of the twenty-first century.

Posted in Uncategorized | 166 Comments