Slate Star Codex

Thanks to everyone who participated or voted in the adversarial collaboration contest. The winners are:

Grand Prize ($1000): Does The Education System Adequately Serve Advanced Students?

Editor’s Choice ($500): Should Transgender Children Transition?

Honorable Mentions ($250): Should Childhood Vaccination Be Mandatory?, Are Islam And Liberal Democracy Compatible?

I’m sorry for jerking the number and value of the prizes around so many times, but I wanted to balance my preferences, the contestants’ preferences, and readers’ preferences – and this was the best way I could think of to do it. Nobody has gotten less money than they expected, although some prize categories have gotten more money than I originally said. In the end I could not in good conscience let any of these escape without getting a prize. Thanks to this blog’s Patreon supporters for making this possible. All winners should email me with their preferred form of payment (I can do Paypal, Bitcoin, or donations to a charity of their choice).

The overwhelming winner of the popular vote was the collaboration on education. I agree this one was excellent. It cited a lot of research, analyzed it very well, and mostly came to conclusions. Its only flaw from my perspective was a lack of focus; it discussed many different educational interventions, some of which were similar enough that it was hard for me to keep track of what was going on.

I chose the collaboration on transgender children. I thought it did an exceptional job of addressing a specific hot-button issue many people are concerned about, presenting all the evidence on both sides, and mostly coming to conclusions. My strongest complaint was that it ignored some of the potential side effects of puberty blockers which commenters pointed out, and sort of trivialized bone problems that are not trivial; given that the side effects of puberty blockers was a major crux of this question, I found that to be a major weakness. I was still very impressed with the piece’s ability to break down and navigate such a controversial question.

The first honorable mention, the collaboration on childhood vaccination, was also a great example of navigating a controversial topic. It seemed to come closest to outright saying one of the two sides was correct – though see the commentary below for more on that. I had to take off points because the conclusion – that lots of countries get away without having mandatory childhood vaccinations, so surely America can too – seemed overly simplistic. Those other countries might maintain high childhood vaccination rates because of cultural differences, or a different (eg socialized) health system, or lots of other things that might not generalize to the US. Again, this was the crux of the issue, so minor flaws here are very important. The section on the hygiene hypothesis, while fascinating, seemed kind of unfocused to me and probably could have been much shorter.

The second honorable mention, the collaboration on Islam and democracy, was maybe the most thoroughly researched, but also seemed the least structured. The writers chose to list democratic and undemocratic features of six Islamic countries. Since they found one Islamic country (Tunisia) that seemed pretty much like a liberal democracy, they concluded the two were compatible. But in real life, when people ask whether Islam and democracy are compatible, I imagine them wondering things like “Are Muslim countries less likely to be democratic than other countries?” or “Is the Muslim religion one reason why there is so little democracy in the Middle East?” or “In mixed countries, do Muslim populations resist democratic norms?” The collaboration’s focus on listing the particular features of particular Muslim countries’ institutions seemed like a really complete answer to a really boring question.

I asked the various collaborators to send me their reports on the process. Their answers are below, or scroll down to the end of the blockquotes for my summary.

TracingWoodgrains from the education collaboration:

My adversarial collaboration was similar to how Kahneman describes his experience (https://pastebin.com/d6cbN765): a “failure to disagree.” When Michael and I zoomed into an issue at the object level, we usually agreed. Then we’d zoom out and realize we used that object-level detail as part of two drastically different narratives. I have no education credentials besides having been a deeply frustrated student; he was a happy student and now enjoys teaching. That dichotomy between teacher and student, happy and unhappy, defined our perspectives.

Going in, I was confident that tailored curriculum involving ability grouping could help advanced students progress up to 4-5 grade levels above where they are now, that school is a deeply harmful place for many advanced students academically and socially, and that education should follow the principles of game design (using online tools to allow students to progress at their own paces while encouraging them to progress quickly).

Our project was extensive and far-reaching, mostly due to our disagreement and Michael’s tireless willingness to keep working with me. We approached it as a research project more than anything else, which mostly meant “toss all relevant-looking research and related writing towards each other, read it manically, and see where and how disagreements bubbled up as we were learning.” At the start, I was coming from a position of near-complete ignorance towards the state of specific research on ability grouping, though I had passing familiarity with acceleration research and what I felt was a comfortable understanding of the (poor) state of research around online education. So a lot of my “position-shifting” was learning:

I learned that tracking, in a traditional sense, had much weaker evidence than I was anticipating, but the more it looked like what my intuitive view of its ideal, the better it fared. So: few peer effects, strong effects based on curriculum change, strongest impact based on diagnostic testing and precisely tailored curriculum. My views on acceleration as a useful, underfocused stopgap measure did not change. My understanding of the complexities of the education system, and why it was so far from what I hoped to see, grew tenfold. Michael’s pressing reminders of the competing purposes of education and experience in the field helped me a lot there.

That’s most of the small stuff. I had two genuinely paradigm-shifting experiences: First, on the happiness of advanced students. Typical mind here–I’d always just assumed that most advanced students were bored, held back, and miserable in school. It seemed one of the most obvious things in the world to me. But it’s really, really hard to find any of that in the research. Most studies indicate them enjoying school more than other students, (tautologically) performing better, and having lower incidence of mental health issues. My view shifted to intelligence as a potential intensifier for other risk factors (ADHD, autism, etc.), but not as the core cause of student frustration in school. My belief that the “smart kids” who are bored and miserable in school tend to be underprioritized and mis-served remains largely unchanged, but I have a very different view of this group.

Second: Michael pointed me towards noticing the apparent mediocrity of a lot of schools implementing some of the digital/in-person hybrid instruction I dreamed of. Rocketship Schools, School of One, Khan Academy, and others grounded my expectations of any sort of revolution. At the same time, we discovered Art of Problem Solving, which is probably the best thing in the world, and seeing what they’ve done with online education and specifically math made me realize that people smarter than me have been doing a really good job on “my” project. So now I see less need of creating the right resources, more of expanding their availability and sifting them out from the sea of garbage.

Less paradigm-shifting, but significant, is that I grew to trust experts more as I noticed that when you dug deep enough, most of the serious researchers came to pretty similar conclusions but wrapped them in different narratives, and really bad ideas that filtered through were less because the serious researchers had really bad ideas, more because their messages got distorted or ignored to better fit political agendas and the vagaries of people’s opinions.

(Bonus areas where I didn’t change my view, just properly grew one because I didn’t know enough to have a view before: direct instruction as intriguing and mostly good, Joplin plan and nongraded schools as intriguing and oddly ignored, early childhood education as incredibly high potential in specific fields, a growing frustration with just how much politics has infected and damaged education)

One example best outlines our differences: Direct Instruction. Michael actually brought it up to highlight problems he saw with online schools–basically, pointing out that it works, but isn’t attractive at all, except in educational emergencies. I hadn’t heard of it before and got about as far as “it works” before getting really fascinated by its whole story. I looked at schools that use it, interviewed someone closely involved with it, read into it, and concluded that it had a lot of potential that was being neglected since people felt like it wasn’t how they “should” learn. That whole arc was never supposed to happen. It was an offhand example that he raised to dismiss. And my response was basically “this is amazing; how have I never heard of this before?”

That happened again and again–where I would see an unusual, academically intensive approach that got good results, we’d talk about it, and he’d ask, “right, but what’s the point?” Sorting by aptitude over age level, high-intensity accelerated math programs, early (pre-K) academics, so forth. It’s worth repeating–we almost never disagreed about what studies showed, just on the importance of particular studies and particular points. And that was enough to fuel a hundred pages or so of disagreement.

So, advice for future adversarial collaborations: First off, it can only work if both are very, very willing to talk about the topic–the whole topic, not only their pet issues within it. If you find yourself uninterested in a part of the topic the other person is passionate about, pay close attention, since that’s often the most important part of disagreement. Second, it takes a lot of time. We took more time than strictly needed because we both enjoyed the research and conversations, but I’d guess 50-100 hours is a reasonable amount of time to set aside. Third–expect to find a lot of disagreements that boil down to differing priorities and interpretations of object-level facts you both agree on. If you’re both reasonable and willing to work together, you may find little factual ground you disagree on, even while telling two very different stories of the big picture.

One last thing: Adversarial collaborations are a phenomenal thinking tool. This was the most satisfying project I’ve worked on, bar none, and the best learning experience of my life. Talking in depth with someone who disagrees with you is incredible. If both agree, it’s hard to sustain a conversation, but the second someone says something wrong-seeming, it spurs a sort of need to respond, a creative burst. I read more research and focused more on that topic than I have at any other point, even though education has been one of my core interests for a while. Michael was the best partner I could have asked for on the project. I expect that we’ll still talk about wildly different things moving forward: I still love the potential in online learning, non-graded schools, and accelerated/intensive tools that depart dramatically from the current school model. Michael still loves teaching and working within the system mostly as it stands, and doesn’t trust most radical changes.

Thank you very much for organizing this contest. It was an incredible opportunity and a vital experience.

Michael from the same education collaboration:

I really want to give you something really detailed and good about our collaboration, but that would take a lot of work. Instead I’ll just give you the quick, stream of consciousness version.

When I first saw TW’s comment on your post, I thought his views on education were absolutely naive. I heard him expressing a love for game-based education, which I thought to be faddish and flashy in education but without much value. He also expressed that the strongest students were especially poorly-served by education. I thought that was dead wrong; smart kids would do fine, in general, out of school. Without checking, I can’t remember if he said anything about ability grouping. That wasn’t the point, anyway. What I heard was (sigh) another technophile education utopian who thinks that their own particular experience should be the basis of a system tasked with educating millions of children.

So we started chatting, and pretty soon we’d created an 150 page google doc. The doc contained our debates, annotated bibliographies, brief memoirs documenting our own experiences in education, etc.

I’m coming from all of this as a guy who was pretty conventionally successful in my academic life, and ended up in teaching almost by accident. In my professional life I’ve given a lot of attention to students who aren’t succeeding in my classes. Those are the situations that have made me the most passionate to put in extra efforts to help them. So things like volunteering to tutor a kid outside of class, to go over homework during my planning period, rethinking my pedagogical approaches.

At the same time, I had recently experienced a few cases in my honors elementary age classes that were making me think. And I was going back to work at a camp for talented math students.

So I’d say the real thing that changed me, through this experience, is I spent a lot of time thinking about students who find a class too easy, and what that can feel like. That was the most significant way that my views changed through working with TW. He consistently drew my attention towards students who are talented but unhappy.

I feel as if with what we wrote about technology, my views didn’t significantly develop through our collaboration. I think TW quickly learned that his tech optimism is future-optimism, not present-optimism, and that’s a change for him. But otherwise what we did there was more along the lines of finding a common narrative that we both thought important for people to understand the situation with technology and learning. That involved a lot of back and forth between us, as it’s hard to put everything in a way that satisfies both a pessimist (me) and an optimist (him). That’s how we conceived of this work, though.

A lot of the other things we did were more about fleshing out new conceptual territory than changing my views. I didn’t really have strong opinions about tracking or ability grouping. I was eager to learn something new, and TW was a great partner for working through the literature.

There was a lot of debate during this project, and we still disagree deeply. It’s hard to say who “won” our collaboration or who pushed who further etc. I’ve been reading about a lot of this stuff over the past 8 years. I don’t know how old TW is but he’s younger than me. I certainly urged him to read certain people who I thought were foundational, like Larry Cuban or David Labaree. A lot of this was very new to me though, and I was trying to integrate my experience in the classroom with my reading. I feel as if my strong views on gamification were quickly confirmed, and I also think that my view that smart kids mostly turn out alright were vindicated too. I’m sure TW has complementary ways in which he feels he shifted my views, though I’m not exactly sure what he’d say.

One thing that I think people haven’t given enough thought to is the differences between adversarial collaborations in different fields. Certainly there is a difference between medical research, economics research, comparative politics and education research. Each field has a different epistemology, and education research is a big tent containing many individual subfields. That means that TW and I had to do something that I love doing, which is figuring out how cog psych and big metanalysis and economic studies and case studies and history of education all fit together in a coherent whole.

So while I can imagine a briefer more focused write up for our collaboration, I think it’s possible that making sense of classroom questions is going to involve a lot more fuzziness and wandering than for certain other questions. We can’t just easily answer the question of whether top students are harmed by schooling. Compared to what? What’s the vision? What’s the purpose of schooling? That’s a question you don’t really need to ask about e.g. vaccines. The goal is to keep people alive without harming anybody else. I know there is some fuzziness there, but it’s just not the same magnitude.

One more thing, before I call it a night: by design, TW were non-adversarial in our collaboration. Early on, I sent TW an email where I said, you know what? We’ve figured out that we deeply disagree. I’m not interested in tallying points on whether my beliefs have been weakened or changed or your’s have. Instead, let’s just try to figure things out together. And that’s what we did.

I think a major question going forward is whether adversarial collaborations work best when you have an opponent or when things are framed so as to reduce opposition.

John and Christian from the Islam collaboration:

WHAT WERE YOUR INITIAL POSITIONS?
JohnBuridan’s initial position was that it was highly unlikely that Islam consistently opposed liberal democracy. ChristianFlanery’s position was that intrinsic components of Islam operate contrary to liberal democracy.

HOW MUCH DID YOUR POSITION SHIFT, AND HOW?

JohnBuridan’s position did not shift towards the other side of the aisle; he still think the opposing position holds too large a generalization. He did shift his approach to the question from a theoretical approach to a more descriptive one. ChristianFlanery convinced JB that the logic of surveying the variability in Islamic political thought would require immense time and resources. It would end up requiring us to determine what different groups of Muslim thinkers and activists think of each other. CF convinced JB that a descriptive and historical framework would come closer to providing insights into how the diversity of Islamic political thought caches out in real modern day politics.

CF became more conscious about how new Islam’s interactions with liberal democracy are. This made him less confident in his initial position. Furthermore, CF now has high confidence that liberal democratic polities can be achieved in the right conditions. CF continues to believe that a fundamentalist component in Islam will always exist, but he is now agnostic about whether Muslim populations will be experimenting with liberal democracy well into the future.

HOW MUCH DEBATE AND ARGUMENT WAS THERE?

Constant unceasing debate, but in a context of high levels of trust. We did not fall victim to one-upsmanship or point scoring debates. Preexisting intellectual rapport allowed us to trust each other not to load the dice too much. Although each of us probably gave too much ground to other person in certain places throughout the paper – many of which the commentariat picked up on – we think that such tradeoffs made the work possible.

The key to resolving debate for us was an established process for the paper. Each had three countries to research and do the write up on, and each of us edited each other’s rough drafts. We wrangled over essentially every line of the introduction but had the patience for very long phone calls to resolve them. Oftentimes we would disagree and circle back to the disagreement a week later and come to a solution. We both agreed beforehand to try to only pick battles we thought worth fighting. Tense moments surrounded our discussion of the introduction, the conclusion, constitution, Lebanon, and Iran. Our longest back-and-forths concerned the introduction and conclusion.

IS THE CONCLUSION CLOSER TO ONE PERSON’S ORIGINAL POSITION?

Our conclusion is closer to the side of commensurability. Nonetheless it is fairly neutral, and we try to leave the door open for the reader’s own knowledge and expertise to affect the conclusion. Our goal was to provide enough background knowledge of some slice of the Islamic world from which a general reader could inform their own conclusions.

– Lots of readers were deeply concerned about how we chose the countries we did. We chose the countries before doing deep research, and we were looking for a few key traits:
1) majority-supermajority Muslim
2) culturally diverse
3) geographically distant from one another.
4) tractable to discuss (we did not feel that we could do Saudi Arabia or Pakistan justice, and we considered the possibility of many other countries.)

WHAT ADVICE WOULD YOU GIVE FUTURE ADVERSARIAL COLLABORATORS?

Our main points were:
1) Establish high levels of trust beforehand. Discuss all sorts of topics and allow for a lot of freewheeling conversation about fundamentals before you buckle down to work.
2) Design a method and framework and develop a clear division of labor.
3) Split the difference wherever possible. We veered towards choosing the least controversial phrasing, but we also believe that a noncontroversial statement said well does a lot of good. Consistently work then discuss.
4) The fact that our topic was broad gave us plenty of thought-space to work with each.
5) Have a partner that is incentivized. We were both were enthusiastic about the Ad Collab format, confident we could make a contribution to the field of our topic, and the possibility of money was meaningful.
6) We both had time in our schedules. It was summer, and we both worked in academic and educational institutions at the time.

From Mark W, who wrote the pro-vaccine side of the vaccination collaboration:

WHAT WERE YOUR INITIAL POSITIONS?

My initial position was that herd immunity is a compelling government interest, and that it is well within the power of a reasonable government to implement a mandatory policy in order to achieve this end.

Interestingly, the first part of this statement (that herd immunity is a compelling government interest) was not challenged by my collaborator, so I think we spent less time making the case in favor of herd immunity than we probably should have. This was likely a mistake on my part, as I should have been spending more time shoring up my side – even those parts we agreed on – as this was one of the areas some commentators found lacking.

(Of course, if you are adversarially collaborating in bad faith, you could choose to agree quickly and dismiss significant parts of your opponent’s argument so as to focus less energy defending their strongest points. I’m not saying my collaborator did that; I think he was arguing in good faith. I’m just saying that you could…)

On the second point, Mark Davis began by saying that he believes vaccines are a major causal factor in autoimmune and autoinflammatory disease. I responded that we looked at that angle back in grad school (about 10 years ago for me) and my recollection of the evidence was that vaccines are not a good explanation for this phenomenon.

HOW MUCH DID YOUR POSITIONS SHIFT, AND IN WHAT WAYS?

My position that mandatory vaccinations is good policy shifted to the extent that we showed it’s not required to achieve herd immunity. I would say that as a matter of public policy influence, the burden falls on the side opposed to mandates to persuade the public that there’s a better way. On the side of vaccines causing autoimmunity, I was not swayed much. I think it’s possible that mycobacterial vaccines contribute a little, but the problem is that there are too many types of mycobacteria, and families of protective/sensitizing commensals are so broad that a highly-specific intervention such as vaccination is unlikely to be able to wipe out a whole family like that and leave you sensitized to developing autoimmunity. Mark Davis wanted to put forward the hypothesis that not only are vaccines bad (because, he believes, they cause autoimmunity) but that the bad outweighs the good they otherwise cause. I felt that one point we made in the essay was that even if it turns out that vaccines contribute significantly to autoimmunity (a claim I don’t find much evidence for) he is presenting a false choice. The public health implications are not either disease control and unchecked autoimmunity, or unchecked disease and autoimmunity control. There are populations that don’t have disease exposure and that don’t have autoimmunity, and there’s no good reason to jettison vaccines and the positive good that they bring (which we both agreed on) out of a belief that otherwise we have to live with autoimmunity.

There’s one point that didn’t make it into the final essay – due to time constraints and narrative flow – I wanted to add. If you believe that autoimmunity is prevented by exposure to pathogens that are vaccinated against, this has dire ethical implications. It’s one thing to believe that the measles vaccine caused your child to get asthma. Even if you’re wrong and unpersuadable, your proposed solution will be to not vaccinate your child. If you believe that a lack of disease exposure caused your child to get asthma, you’ll want to actively work to ensure that all your children contract each disease you refuse to vaccinate against. Otherwise, it makes no sense to not vaccinate against something, if you think the causal mechanism requires exposure, too. Thus, there is an ethical problem here, not with allowing parents to opt-out of vaccination, but with allowing parents to actively seek to spread disease through their communities. If a parent drives around with an 18-month-old in the back seat with no seatbelt, then intentionally gets into a car wreck, they would be criminally liable – even if they were only caught attempting such a feat. There’s a good case to be made that intentionally infecting your child with a disease known to cause morbidity and/or mortality should carry criminal liability. Davis didn’t make this exact case, but it’s really the only natural extension of his claim. If you oppose vaccination on the grounds that you believe it’s necessary to experience disease exposure in order to protect against autoimmunity, you have to be willing to expose your children to disease.

HOW MUCH DEBATE AND ARGUMENT WAS THERE?

Most of the debate was about whether vaccines can lead to autoimmunity. This was the driving factor behind our time delays. We went through multiple drafts surrounding this point as a main focus of conflict. Davis sent a couple of articles suggesting some causal link between vaccines and autoimmunity. I sent back about a dozen more that argued causal link, no effect, and protection. I also pointed out that the review he sent me actively argued against his point, as did every other review on the subject I could find and multiple meta-analyses.

HOW DID YOU RESOLVE IT?

Davis maintained that this body of evidence was not sufficiently strong to persuade him. At the outset, we had agreed that the burden of proof lay with his side in defending an active harm perpetrated by vaccination. Given that the evidence trended in the opposite direction, he conceded to allow language that public policy should not discourage vaccination – on the condition that private individuals be enabled to disagree with public consensus.

IS THE CONCLUSION CLOSER TO ONE PERSON’S ORIGINAL POSITION?

I’m not sure. I’d say the first part (about mandatory vaccination) preserves my stipulation that elimination of mandates is only okay insofar as the objective of herd immunity is maintained. Thus, even though I capitulated, I did so within the realm of broader public policy objectives I believe in. For the second part, we probably stuck closer to my side, in that there was simply no way to advise public policy changes based on weak evidence of hypothesized harms. That said, there were lots of caveats and hedging.

WHAT ADVICE WOULD YOU GIVE FUTURE ADVERSARIAL COLLABORATORS?

Some thoughts on my experience:

– Time was definitely a factor. Repeat back-and-forth drafts took the most time, because one person would usually have it for a week or more before turning it back to the other person.
That means at least two weeks per draft. If something like this were to become a regular feature, I would advise future collaborators to allow extra time, and maybe wait to submit in a future round. – Every time I tried to add a strong statement, such as “we have no evidence to support [X]” “there is no reason to believe [X] and ample reason to believe [Y] instead” he would object. I suspect there’s a strong tendency as a collaborator to not capitulate your side.
– That said; I did end up capitulating on part of my side (the mandate part) because we stated at the outset under what conditions we would do so. My conditions were met (other countries without mandates achieve herd immunity, so clearly it’s not an absolute requirement, and some countries with mandates do not achieve herd immunity, so clearly it’s not sufficient), so I conceded that point.
– My collaborator stopped there, however, so the most we could recommend was, “sure they do things differently elsewhere, but we don’t really know what that entails; go look it up yourself.”
– We spent most of our time on the particularly contentious second part of the collaboration. This may have been why the first part was weaker than it could have been.
– The assigned titles tended to focus public discussion on what they thought was the main debate, but often that was either not the whole debate, or the debate had shifted as a result of the collaboration.

If you choose to do this again in the future, I would recommend a few changes:
– Have collaborators choose titles.
– Require collaborators to stake out one or two specific claims they wish to refute/defend a priori, and have that be outlined in the first part of the final submission. I know we mostly did this with the bolded statements, but not all of the positions made it into the final piece, so it was hard to tell where people were coming from.
– Have the collaborators include conditions that would be necessary for them to concede their positions. I feel like it would be within the role of a moderator to remand any concessions that are unrealistic, or not well-matched to the claim. (i.e. “I will be convinced once 1,000 years’ of RCTs have been completed.”)
– Require a concluding section that states whether the conditions were met, rejected, or uncertain – as well as what should happen moving forward.
– Where possible, include a third, impartial, moderating collaborator. This is theoretically the person with no horse in the race. If there’s disagreement, they can say to one side, “sure, but you’re not arguing based on the evidence anymore, you’ve been bested on that front; we need to draft stronger language that reflects the actual state of the evidence.”
– The SSC community is a great resource. In retrospect, it would have been nice to have used part of an open thread to discuss elements of the ACC. For example, asking an
international audience about what their on-the-ground vaccination experience is like in the UK/Ireland/France/etc. I should have done this unofficially, but it would also have been helpful to have an official, “first comment is for ACC contributors to ask the community for input”. That would also allow the rest of the community to contribute to the contest.

From a_reader, who wrote the anti-transition side of the transgender collaboration:

WHAT WERE YOUR INITIAL POSITIONS?

My position was that “transgender children” (gender dysphoric children) shouldn’t transition in childhood – neither socially nor by suppressing puberty at 12 (or even earlier!)

Because a lot of them – percent differs among sources, but something between 30-90% – desist, cease to feel and act like the opposite sex. And that’s a better outcome: not needing surgery and hormones for the entire life, having less problems in society acceptance (even as gays/lesbians, as most desisters become) and in finding partners.

If dysphoria persists from 3 to 18, that’s it, transition may be necessary, but it’s better if it’s not necessary, if the problem ceases spontaneously.

Social transition in childhood (before adolescence) – that means name, pronouns, hairstyle, clothes of the desired gender – makes desisting more difficult. But puberty blockers at 12, I thought, can do worse: prevent the desistance from happening, stopping exactly the natural process that may have cured gender dysphoria, by the action of puberty hormones on the brain.

HOW MUCH DID YOUR POSITIONS SHIFT?

My position didn’t shift that much, but now I understand better why so many parents choose to let their children transition and take puberty blockers, how distressful gender dysphoria is for those children – anxiety, depressing symptoms, sometimes (at puberty) even suicide attempts – and social transition and even puberty blockers seem to alleviate it substantially.

I still consider (spontaneous) desistance a better outcome. And I still tend to think that maybe puberty blockers may stop that process. But I admit I didn’t found conclusive evidence (only circumstantial: all children on puberty blockers persist).

HOW MUCH DEBATE AND ARGUMENT WAS THERE?

Not much. In the beginning, flame7926 created a document shared by Google Docs, for documentation, where both of us wrote about the studies (and media articles and blog posts describing cases) we found. One found a study and wrote a few lines about it, then the other added their own observations – using different colors each of us at different times, so that in the end that document looked like a rainbow, matching the subject.

HOW DID YOU RESOLVE IT?

After the documentation stage, flame7926 asked me if I changed my position, I said not really, so after a short discussion, we agreed to not recommend for or against childhood transition, to just present all the evidence, the risks and benefits.

IS THE CONCLUSION CLOSER TO ONE PERSON’S POSITIONS?

The conclusion seems to be that there isn’t yet enough evidence for a firm, definitive conclusion. So it tries to keep a balance between our positions, resuming the evidence and admitting uncertainties.

WHAT ADVICE WOULD YOU GIVE FUTURE ADVERSARIAL COLLABORATORS?

I think besides conscientiousness, you need a degree of agreeableness and being a mistake theorist, not a conflict theorist (at least not about the subject); it is also important to have enough time (that is, quite a lot) and to find the subject interesting (but not essential, not among your main values – otherwise you risk to become a conflict theorist).

I was really lucky to have such a good “adversary” like flame7926, who in the end worked more than me: he wrote most of the final entry (as you probably have already noticed, I’m not a native English speaker). Seeing how many people were abandoned by their partners, I realize how lucky I was.

And from flame7926, who wrote the pro-transition side of the same collaboration:

I enjoyed the adversarial collaboration process and like the idea a lot. I think that it has the potential to produce very productive discussions and papers. I think it definitely can moderate individuals’ opinions – get them to recognize that many things aren’t so black and white.

To begin, I thought that transgender children should be allowed whatever options they want in terms of socially transitioning, while I had less strong of an opinion on puberty blockers (mostly because I didn’t know much about them). Honestly, I didn’t know that much about gender dysphoria or transgender expression in children and youth before commencing with this project, so my opinion wasn’t very formed. I did feel strongly though that we not present “desistence” as a preferred option because I think it is at least reminiscent of transphobic and homophobic attempts to convert individuals away from their identity – an often very traumatic and distressing process. And I thought that once you remove the idea that desistence is preferred (because I think it is very difficult to have that macro-level preference while remaining supportive of individual children on a micro-level) then social transitioning becomes preferred. I also thought that these cases of “desistence” might just be people expressing cross-gender behavior and not actual transgender identities.

I think my position moderated some towards a more even approach, and I can understand why parents and others would want children to desist from transgender identities (as long as this happens without trauma). I also stand stronger behind current research on desistence, in the sense that I do think there is strong evidence that a sizable fraction of children who express desire to either be the other gender or that they are the other gender will not express those same desires later in life. Yet I still think (and believe the evidence shows) that social support of children is extremely important (in terms of mental health), and I think denying their desire to socially transition can run counter to that. I also think the example of Samoa and other societies shows that gender dysphoric feelings are not constrained to our culture, but that the mental health associations are culturally contingent.

I would say there was some debate and argument. As a_reader describes in their response, we had a collaborative Google Doc broken down by topic, like social transitioning, puberty blockers, mental health etc. One person would pull some evidence from an article, the other person might pull some contrary evidence. Debate proceeded in that fashion, with more and more sources being brought in. There wasn’t that much overarching debate, but I think our topic was broad enough that it needed to be broken down in components in which we actually examined the evidence.

When it came time to write our paper, it seemed that neither of our positions had shifted that much. We presented the evidence, but I don’t believe there is currently enough evidence on puberty blockers to make a recommendation either way. Given that we did not end up agreeing on social transitioning, we could not recommend either way. I did end up writing most of the paper (at the request of my co-author) and am glad that we were able to produce something that moderated and expressed both viewpoints. I’m also proud of how we integrated quantitative evidence with more qualitative sources and anecdotal accounts, which I also feel are important when dealing with issues that are, necessarily, subjective.

In terms of advice, I think just being able to put in the time is important. Seeing the small number of projects that were completed, I wonder why so many dropped out. I’m glad I had such a motivated partner (I would say that they did more work than I did). I think that it is also important to be able to moderate your viewpoints and understand where your disagreements come from, even if you don’t end up reaching the same conclusion. I think that having someone else working can keep you on track, if you see them making process. I am curious how other groups divided up the writing on their projects (especially since I believe that most of them ended up longer than ours?).

My executive summary: almost no one changed their mind on the overall issue (a few people changed on smaller subpoints), but almost everyone moderated their opinion at least a little.

Some of the collaborations raised concerns I hadn’t considered. Although John and Christian were happy with their result and many other people liked their collaboration, I still can’t shake the feeling that they solved their intractable disagreement by avoiding their actual cruxes and searching under a streetlight they didn’t lose their keys near. If you force two people to write a mutually agreeable essay on a topic they can’t agree on, it’s probably hard to prevent them from inadvertently shifting to a slightly different topic where everything is cut and dried and factual and agreement is easy.

I was originally very excited to see the vaccine collaboration come out strongly in favor of “vaccines do not raise disease risk”, since it looked like someone had really changed their mind on a controversial topic – then disappointed again once I learned that vaccine opponent Mark Davis had not been convinced, but just agreed that the burden of proof was on him and he hadn’t met it by convincing his adversary. If I were a vaccine opponent, I would be pretty upset that the adversarial collaboration comes out strongly in favor of vaccines even though the two collaborators weren’t both on board.

And as a vaccine proponent, I worry that Davis’ magnamity in allowing the final result to be mostly pro-vaccine convinced Webb to “reciprocate” by being less forceful than he could have been in other areas, and not fight back too hard against the technically-true-but-fraught claim that other countries had higher vaccination rates than the US without mandatory vaccination programs. I don’t know if this really happened in this collaboration (from the write-up it looks more like they agreed on the preconditions under which they would give in, and maybe those weren’t thought out too well) but I think the two adversaries agreeing to “compromise” by each giving up on one of their key points, rather than really come to agreement or present their disagreement exhaustively, is a failure mode to be watched for.

This particularly bothered me in the vaccine collaboration because of another factor I hadn’t considered. I previously hoped adversarial collaborations could be a good match for “pseudoscience”, ie questions where the mainstream is overwhelmingly on one side but there are still some holdouts with an alternate narrative. Certainly these topics cry out for explanation, but the current paradigm – RationalWiki-type “skeptics” who make fun of pseudoscience with cheap shots – are really bad at convincing believers and usually mangle the theories they try to criticize. An adversarial collaboration format could make sure that debunkers have to address pseudoscientists’ strongest arguments – and that pseudoscientists’ supposed counterarguments to the debunking get addressed also.

I now worry that such collaborations would systematically end up legitimizing pseudoscience. If nearly everyone believes that some theory (UFOs, Bigfoot, etc) is false, and some prominent proponent and opponent of the theory do a collaboration, then even if everything goes great and the end result is mostly skewed to the opponent’s side, then there will still be a few compromises and places where the opponent gives into a little social pressure to frame things in a less-than-maximally-hostile way. So if the end result comes out 90% anti-Bigfoot – well, believing in Bigfoot 10% is more than most people currently believe in Bigfoot, so most credulous readers would update in a pro-Bigfoot direction. True, any debate between pro- and anti- Bigfoot factions will involve some pro-Bigfoot arguments that you wouldn’t have heard otherwise, but the adversarial collaboration format risks importing them into the final product in a way that makes it seem like they have a stamp of approval. I’m not sure this format is a good match for pseudoscience unless somebody finds a way around this.

Thanks to everyone who participated. I would like to do more with this format and will probably devote at least a similar amount of time and money to it next year; if anyone has ideas to improve the contest, or better ideas for promoting adversarial collaborations than another contest like this one, please let me know.