Thanks to everyone who participated and/or voted in the 2019 Adversarial Collaboration Contest. And the winner is…
Adrian Liberman and Calvin Reese, for Does Calorie Restriction Slow Aging?.
An extraordinarily close second place (26.9% vs. 26.2% of votes) goes to David G and Froolow, for Is Eating Meat A Net Harm?.
Both of these did great research and were written up well. I especially like them as winners because they have such different strengths.
The calorie restriction collaboration was carefully focused on a factual question. I think this is a promising model for adversarial collaborations, and that others failed the further they deviated from this. For example, the circumcision collaboration did a good job assessing the quantifiable benefits and harms of the practice, but it turned out that most people who disagreed about it weren’t disagreeing because they assessed quantifiable benefits and harms differently. The abortion collaboration ended up in a similar place. By focusing on a topic where there really was debate about what the research showed, and by hitting the lit review portion out of the park, Adrian and Calvin helped deconfuse a lot of previously confused people.
And the meat collaboration managed to succeed without being like this at all! It was unabashedly just a lengthy review of every single plausible argument for and against vegetarianism, and bulldozed over the immense difficulties with this approach by putting in more work than any reasonable person would have thought possible. And if it didn’t get quite as many votes as calorie restriction, it won on another metric – here are some of voters’ comments (plus some extra from the blog):
– After this, I expect to restrict more strictly to chicken and fish, and alternate more aggressively towards fish as a hedge against the possibility I’m undervaluing chicken sapience.
– Got me to significantly change my diet (at least over the past few weeks) towards more fish and much less chicken.
– I read it, and I changed what I believe and how I changed my life accordingly.
– My final pick is not necesarily based on the article‘s excellence but rather due to me going to change my eating behaviour, which I find an impressive thing Form an article to do.
– This has practical implications for my life. I’ve stopped eating pork because of it (baby steps, working towards less meat generally)
– I actually resolved to stop eating chicken (the only meat I can regularly eat, due to dietary restrictions) based on that piece, so I’d say it was pretty effective in informing me about things.
– It convinced me not to buy chicken that isn’t organic/free-range
– This article will result in all my family eating less meat. It’s actually going to change our lives, health, and the environment!
– As a result of the adversarial collaboration on the ethics of eating meat, have dramatically scaled back my meat consumption to probably 20% of my previous value, and the meat I do eat now is almost exclusively fish and invertebrates.
– Learning about the harm of factory farming from their dispassionate and empirical analysis has prompted me to greatly reduce my red meat and poultry consumption.
Some voters brought up a reasonable complaint: the end result ended up being pretty (though not completely) pro-vegetarian. How do we tell the difference between “a good faith effort by intelligent people naturally converges on vegetarianism” vs. “the anti-vegetarian collaborator slacked off”?
In this case, we tell because the anti-vegetarian collaborator posted a comment about his thought process and what convinced him. But there were other cases where people had the same question, and still other cases where one collaborator did a good job representing their own anti-X position, but other people were anti-X for different reasons that didn’t get represented.
If I had infinite resources, I would fund adversarial collaborations between well-known and universally-recognized intellectuals on different sides of a topic, who everybody trusted to stick to their guns. As it is, I can only say I’m delighted to have stumbled into the one part of the world where “people are too likely to change their mind when presented with new evidence on controversial issues” is a problem.
Some thoughts on the other collaborations:
Circumcision: I loved this one. I’d never seen a good assessment of exactly what health risks circumcision was supposed to prevent, and I didn’t know how weak the evidence was that the foreskin helps with sexual pleasure. But the conclusion ended up being “the quantifiable benefits of circumcision are nonzero but pretty low; the quantifiable harms are not obviously distinguishable from zero but who knows”, which leaves a lot of space for people’s ethical intuitions, which turned out to be REALLY STRONG. One reader said they were going to boycott my blog from now on for not having no-platformed this ACC, and a few others seemed only slightly less angry. On the other hand, it also did better than average among voters, so good job there. I take a small amount of blame for this one not being more popular – I retitled it to be about the ethics of circumcision, whereas the original title had been about benefits vs. harms. But I think it’s naturally hard to write something about benefits and harms without it sounding like you’re talking about ethics, and in this case the ethics were too complicated to fit in the model provided for them. Some positive comments from the survey: “This…actually changed my opinion from circumcision being mildly ethically wrong back to neutral”, “I gained a much more nuanced understanding of the benefits position to the point that my mind was changed to be in favor (maybe too strong), or at least not opposed to, it for developing countries”, “It tidily presented the pros and cons and presented a lot of useful information, with a clear conclusion. It shifted my thinking the most of all of them.”
Space Colonization: You guys presented a lot of evidence for one side, then at the end switched to the opposite side based on a one paragraph explanation of something you’d never brought up before. If that was your crux, I wish you had analyzed it in more depth. If the whole point is to make something that can’t be defunded, couldn’t the government (or whoever) give the money to a private foundation with really good trustees, no takebacks? Maybe there’s a problem with that idea, I don’t know, but if you’re going to make defundability the center of your conclusion, I wish you had examined it more closely. Some positive comments from the survey: “Excellent selection of question, manages to present both sides fairly and come to an insightful conclusion”, “I think this ACC did the best job of covering the entire scope of the question they assigned to themselves, while still presenting a shared conclusion”, “Interesting non-obvious conclusion, subject I care a lot about, pretty pictures”.
Gene Editing: This one seemed to spend a lot of time on very knowledgeable and very well-cited assessments of the current state of the technology and how and why it worked, but didn’t really get around to assessing the “should” question in the title. It also had a few factual missteps – maybe no more than the others, but more obvious since it was so fact-based. While it was an impressive work of scholarship I’m not sure it came together as an adversarial collaboration. Some positive comments from the survey: “Very nicely presented ACC. It was thought provoking and totally enrapturing!”, “well-reasoned collaboration on a difficult question”, “Great, nuanced answer to a complex question”, “This collaboration caused me to reconsider my enthusiasm for CRISPR based on the narrative provided in most press releases. The topic is much more involved than I’d initially realized.”
Abortion: An adversarial collaboration on a completely moral question – you guys didn’t make this easy for yourself, did you? I don’t think you made any particular missteps given the difficult task you set yourselves, but this is another one that I feel like didn’t quite come together. Some positive comments from the survey: “Most interesting (and politically relevant) topic, plus it seems icerun’s position actually shifted somewhat by virtue of having to marshall arguments for it, proving the whole endeavor to be more than just an exercise in futility”, “I thought it best captured the spirit of an Adversarial Collaboration”, “This was a nice, cautious walk-through of an extremely divisive subject. i never thought i would enjoy reading a “point/counterpoint” on abortion, but i enjoyed this one”, “Lots of adversity, focused on the actual disagreement, and balanced data and philosophy well.”
Automation: Seemed broadly correct and helpful. I didn’t find it too exciting because I felt like I had already covered most of the same beats in this article (which they cited), but I’m surprised other people didn’t vote for it more. Some positive comments from the survey: “Importance of the issue and the thoroughness with which it was explored”, “The most fitting, thought-out and the one that draws actual conclusions”, “Highest rationality-to-contentiousness ratio”.
Spiritual Experience: This was another one that was long, fascinating, and didn’t seem to be making much of an attempt to come to a conclusion. I especially liked the section on near death experiences, and I’ll be thinking about it a lot, but I didn’t feel like this collaboration gave me the tools I would need to generate or test hypotheses about what might be going on. Some positive comments from the survey: “The most polished and one which most likely caused me to reconsider things”, “Most informative. Best at following an ideal format”, “This is the one that 1) is most interesting to me, 2) seems like it had a strong difference of opinion as a starting point.”
I included the positive comments because I think comments on these kinds of things (mine and others) naturally tends to skew negative. Certainly the comments in the comments section were overwhelmingly negative even for the winning collaborations (seriously, what was up with this?) So I want to counter this by pointing out that every collaboration got at least 25 votes, and the comments on the voting survey were mostly positive. It’s easier to nitpick than to give praise where praise is due, but people put in a lot of work here and it was generally appreciated.
I promised that I would come up with some fair way of dividing the prize money, with at least 50% going to the first place winner. Because the top two entries were so close, and because I was so impressed with the second place winner, I choose to give $1,300 to Calvin and Adrian ($650 each), and $1,200 to David and Froolow ($600 each). Please send me an email at scott[at]slatestarcodex[dot]com telling me where to send your share of the money – I can PayPal it to you or donate it to a charity of your choice. Thanks to SSC Patreon supporters for making this possible.
As much as I enjoyed this, I don’t expect to do another contest next year. For one thing, I think requiring two people made it a lot harder – 22 out of 30 teams dropped out before the deadline, and I worry some of that involved a lot of wasted work. For another thing, it involved a surprising amount of work on my part converting whatever Word or Google Docs file people sent me into a format I could use on the blog. Finally, I feel like the past two years did a good job exploring this medium, and now it’s up to other people with real questions to see if they can adapt it to their needs.
Most likely I’ll be replacing this with a book review contest sometime towards the end of next year, so if you read any good books, keep them in mind.
But I continue to be interested in adversarial collaborations. If you happen to do one, please tell me – there’s a decent chance I’ll publish it.