codex Slate Star Codex


Highlights From The Comments On Survey Harassment Rates

[Content warning: harassment. This discusses the comments to SSC Survey Results: Sexual Harassment Levels By Field]

brmic writes:

Thank you for posting this and the data file. FWIW, I tried to reproduce the results and couldn’t reproduce the correlations between female victimization, male victimization and male perpetration. fem vic vs. male vic is 0.65, same as yours. fem vic vs. male perp is 0.01 for me, and male vic vs. male perp is 0.21 for me. Everything else more or less checks out.

As a reviewer, I’d say the combination score is not convincing, especially since it ignores all considerations of different male to female ratios in the various industries.
Also, if you have two measures with r = 0.8, Fig 6 is not a good idea IMHO. It’s probably just noise. (Also, it should be a dotplot centered around 1, because the relevant info is distance from 1:1 ratio.)
Instead, I’d focus on the correlation between female victimization at work and female victimization outside work of 0.65 (for me) and the same for males at 0.59, which also leads to the conclusion that there’s a strong ‘people in fields’ effect, without having to go through the combination score. If you’re so inclined, you might then do the at-work by outside-work ratios and end up a kind of cross-validation set, where you can see whether the bad fields for women are bad for men as well. Of course, once you then consider sex ratios per field. it’s story time all over again. Still, e.g. men report similar levels of out of work victimization in computers (20%) and Health Care (24%), but at work victimization of 4% and 12% respectively, which strongly suggests that Health Care is worse.

Their code is available here. Thanks for doing the work to try to replicate my results. I’ve removed the non-confirmed correlations from my post until I can figure out what’s going on with them. I agree that Figure 6 was barely worth it, which is why I tried to make Figure 4 (the unadjusted version) the center of my thesis.

Chris quotes a TIME article that argues that predominantly-male communities generally have lower harassment rates than predominantly-female communities:

Given the epidemic of campus rape, teenage girls and their parents are justifiably concerned about safety, just as teenage boys and their parents are worried about false accusations. What does any of that have to do with gender ratios? Well, there have been multiple studies showing a correlation between gender ratios and rates of sexual assault. As counterintuitive as it may sound, elevated rates of sexual assault are a predictable feature of communities with oversupplies of women, according to studies by sociologists Nigel Barber and Robert O’Brien.

The opposite is true of communities with oversupplies of men. Columbia University economics professor Lena Edlund investigated the impact of lopsided sex ratios in China, where young men now outnumber women by 20% due to sex selection, abortion, female infanticide, and other outgrowths of China’s old “One Child” policy. Edlund and her co-authors discovered that although overall crime rates went up in China as the gender ratio skewed more male—not surprising given that men are more prone to criminality—there was a precipitous decline in rape. It seems that men treat women better, and protect them more, when women are in shorter supply.

Can I prove beyond all doubt that Edlund’s and Barber’s findings also apply to college campuses—i.e. that rape is less common at schools that are at least half male? No, because the available data on campus rape tends to reveal as much about how forthright colleges are in handling sexual assaults—and how comfortable women feel reporting them—as it does about the actual frequency of assaults on a particular campus.

That said, I was intrigued by a recent Washington Post story on the topic. The article ranked 27 top colleges by their sexual assault rates, and I couldn’t help but notice which college had the lowest rate.

It was CalTech, a school that is 59% male.

But some commenters in the subreddit bring up contrary evidence. DinoInNameOnly links to a Pew poll finding that women in predominantly-male workplaces report more harassment. And Hepatitis Andronicus brings up Alaska, a heavily male-skewed state considered to be in “a sexual assault state of disaster”.

Rolaran argues:

I feel like a person who has had bad experiences in the tech industry is less likely to have continued reading a blog that argues (among other things) that problems in that field are overemphasized, and a person who has had nothing but good experiences in tech is more likely to have begun hanging around a blog where they felt like the author was sticking up for them. This is useful data for determining the experiences, preferences and demographics of SSC readers, but I would be considerably surprised if that could be generalized without some serious legwork.

I did consider this, but first of all, only the tiniest percent of my posts are on tech, and I think I might have only had one previous post that could by any stretch of the imagination have been considered about harassment in tech. Second, it’s not clear which direction this should skew things – some would argue that guilty people are more likely to push the “we have nothing to apologize for” line, and innocent people are more willing to critically self-examine their (or their tribe’s) problems. Third, I’m not sure this would really skew self-reported female victimization, self-reported male victimization, and self-reported male perpetration all in the same way.

Rachael writes:

I agree with most of your conclusions, but there’s still a major flaw in the survey methodology. It asked what industry people currently work in, and whether they’ve *ever* been harassed at work. Plenty of people have switched industries over the course of their career, and nearly everyone worked in retail or food service or similar as a teenager or young adult. So we have no idea how much of the reported harassment took place in industries other than the ones people currently work in, and probably skewed towards the industries that hire teenagers.

This is not only a good point, it’s a good point that would have been easy for me to fix if I had thought about it. Sorry.

Leah Velleman writes:

I wonder whether something gets lost here when we conflate the startup world (stereotypically an unsupervised bunch of pushy, risk-taking young single people) with — and I use this term lovingly — dinosaur tech companies (stereotypically a bunch of mild-mannered old married people, with a strong HR department looking over things).

The cultural gap between startups and dinosaur tech seems much bigger than, say, the gap between independent and chain restaurants, or small and large universities. If that’s true, then that’s an argument for treating startups and dinosaur tech as separate sectors. (And while some companies might be hard to sort into one category or the other, that’s true for any set of categories. “Is this job in econ or finance?” and “Is this job in business or law?” will turn up ambiguous cases too, but that’s not a reason to ditch those distinctions.)

I’m pushing this question because I have a hypothesis, which is that startups really do have a harassment problem — but that the people speaking up about it, and the news outlets quoting them, have misrepresented this as a problem across all of tech, ignoring the fact that most tech workers are at boring old companies with low harassment rates. I suspect this misrepresentation has happened partly because startup folks like to represent themselves as The Real Tech Industry, The One That Counts, and partly because companies tend to transition from “startup” to “dinosaur” over time and news outlets can’t be bothered to keep track.

Riceowlguy on the subreddit writes

I think it would be worthwhile to try and distinguish between “people whose job is actually writing code or keeping IT systems running smoothly” versus “people who have jobs at tech companies but whose work is not technical”. My recollection is that a lot of the anecdotal horror stories about harassment in Silicon Valley involve VCs, sales/marketing people, upper management, etc.

One question is whether people in that latter category (if any of them read SSC at all!) answered “computers” or “business” for their profession.

Doug writes:

Maybe men in these groups are just less sexually aggressive and more introverted. It’d be interesting to aggregate # of lifetime sexual partners by industry and correlate with levels of sexual harassment. Not that I’ve ever been involved in sexual harassment on either side of the table… But my guess is that most cases don’t look like Alice walking over to Bob’s desk and grabbing his crotch out of the blue. Most situations probably escalate from some fun, flirty, or social interaction that one party takes to far. Programmers don’t know how to flirt and avoid socializing, so they never get into these situations. Existing literature on sexual assault shows that it disproportionately occurs in environments of “revelry” and “carousing”. I don’t see any reason workplace harassment would deviate from this pattern. Pharma reps get drunk with their coworkers a hell of a lot more than mathematicians.

I’d been thinking of “introversion” as the default hypothesis, not requiring explanation in the same way as the others, but I realize I might have forgotten to mention it at all. Sorry.

But Terran has analyzed (original comment, small correction) whether these results are explained by the Big 5 personality traits of people in each field. He finds they generally are not – profession remains important even after looking at traits (including introversion). Stezinec on the subreddit tries the same thing – their analysis finds that, after adjusting for traits, the only significant result is that computer-related occupations have lower harassment – but guessing this is just a sample size issue and the computer industry was the only one with a good sample size.

In case you are interested, here are some other factors that affected sexual harassment in the survey. I’m not putting error/significance bars on because that would be too much work – but if I have to have a nice-sounding explanation, it’s because I want to discourage people from using these to say “Your ingroup is more harassy than my ingroup, proving you’re bad”. The best one can do without error bars is disconfirm statements like that – which I think is generally a more prosocial activity. This is also part of why I am making this section less prominent than it might otherwise be. I think this especially important to remember given that groups will be penalized for honesty – ie people who are more likely to confess to sexual harassment, or people who are more willing to admit that certain behavior crosses the line.

All analyses were limited to cisgender subjects except where stated otherwise. All analyses of women have a sample size around 600, all of men around 7000. Each subgroup has a sample size of at least 100 except where stated otherwise. Note that each graphic combines information on victimization and offending by one gender; offending results are often very low and slight variations should be taken with a grain of salt.

Rates by relationship status (male, female)
Rates by sexual orientation (male, female)
Rates by transgender status (male, female)
Rates by asexual vs. sexual (male, female)
Rates by political affilitation (male) [not enough data for full female analysis, but binary left/right graph here]
Rates by feminist self-identification (male, female)
Rates by social class of family of origin (male, female)

If you want to analyze this further or double-check any of my results, you can download the raw data here.

SSC Survey Results: Sexual Harassment Levels By Field

[content note: sexual harassment]


Recent discussion of sexual harassment at work has focused on a few high-profile industries. But there has been relatively little credible research as to how rates really differ by occupation type.

There are many surveys of harassment rates in specific industries, but they can’t be credibly compared with one another. The percent of people who report sexual harassment varies wildly from survey to survey – thus studies finding that anywhere from 12 percent to 48 percent to 60 percent to 85 percent of women have been harassed at work. If a survey shows that 60% of female nurses get sexually harassed at work, does that mean nurses are victimized particularly often (because more than 12%) or are unusually safe (because less than 85%)? It doesn’t matter, because another study says only 19% of nurses get harassed.

Why do all these numbers differ so dramatically? The most important issue seems to be how you ask the question. “Have you ever been harassed?” gets numbers more like 12%; giving a long list of specific behaviors and asking “Have you ever experienced any of these?” gets numbers closer to 85%, depending on what the behaviors are. Surveys also differ on whether they ask all employees or just women, whether they include a time frame (eg “…in the past two years”), whether they specify that it had to be at work vs. work-related events, and whether they include witnessing someone else’s harassment. Taking these surveys entirely seriously would lead to the conclusion that Uber has the lowest sexual harassment rate of any company or industry in the world; I choose not to take them seriously.

This means we need investigations that use the same methodology across multiple fields. Whenever the media talks about this – see eg the Washington Post’s The Industries With The Worst Sexual Harassment Problem – they’re working off of the Equal Employment Opportunity Commission’s records. But these are totally unsuitable for the task – they just report raw number of claims per industry. The industries that rank lowest in EEOC’s data tend to be small industries with very few women – for example, taken seriously the WaPo’s graph shows that mining has the least problem with sexual harassment of any industry in the world. Is this thanks to their uniquely progressive culture – or because there are practically no female miners? I’m going to say the second one. The takeaway that most real researchers take from the EEOC claims is that the lowest-paying and most mundane occupations – retail, restaurant work, hotel work, etc – have much higher sexual harassment rates than the prestigious occupations people generally talk about. Eyeballing the data, this looks basically true. But trying to get anything more fine-grained than that out of EEOC is basically hopeless.

I only know of two surveys that have even attempted to compare different fields in a principled way, and neither really inspires confidence.

First is a survey by Cosmopolitan magazine, that asked 2235 women in different fields about sexual harassment in their industry. About 33% reported being harassed at work. Women in the retail and restaurant industries reported the highest rates of sexual harassment; despite media coverage’s focus on STEM and entertainment, both reported average to just-below-average harassment rates.

Second is the Project XX Survey, where the automotive industry decided to survey their workers using methodology previously used in Silicon Valley, which made their results at least somewhat comparable. The advertising and market research industries seem to have joined in later. They find the automotive industry has the most harassment, followed by Silicon Valley, advertising, and market research – but there aren’t really big differences and industry rankings frequently reverse across questions.

I interpret existing data in this area as being basically useless, but at least suggestive that the media focus on a few prestigious industries is mistargeted.


The Slate Star Codex survey is an online poll of readers of this blog about various aspects of their life and personality. This year 8,077 people participated. The survey asked a set of questions on sexual harassment, providing a unique dataset with which to investigate the question.

Due to the blog’s tech focus, readers skew well-off, male, and STEM-focused. This made it impossible to investigate the restaurant and retail industries that previous research suggests have the most problems. But there were still enough responses to get a wide cross-section of white-collar occupations and academic fields.

Respondents were asked four questions:

1. Have you ever been sexually harassed or assaulted at work?

2. Have you ever been sexually harassed or assaulted outside of work?

3. Have you ever made unwanted sexual advances to someone at work that you think they interpreted as sexual harassment or assault?

4. Have you ever made unwanted sexual advances to someone outside of work that you think they interpreted as sexual harassment or assault?

Respondents were given a “prefer not to answer” option in case the questions made them upset; in order to make people entirely comfortable using the option, they were asked to select it automatically if the last digit of the time was a “1”.

Data were analyzed about female victims, male victims, and male perpetrators. Although some women admitted to perpetration, the sample size across fields was too small to be useful. Industries were included if they had a sample size greater than thirty; because there were more male respondents and so all industries had a greater male sample size, some industries are included on the male graphs but not the female graphs.


Overall, 21% of women and 6% of men reported being sexually harassed at work. 1% of women and 2.2% of men reported committing sexual harassment at work.

Fields listed do not exactly match fields on the survey as some were combined in order to increase sample size, eg “physics” and “chemistry” into “hard sciences”. There’s a discussion involving pre-registered categories below.

Here’s all three measures combined into my best guess for the relative rates in each field:

Made by taking each of the three previous graphs, standardizing each field to percent of the worst field in that category, and then adding them up and dividing by three to get an average for each field

But this combines three different things. First, the actual rate of harassment. Second, a measure of how willing people are to report harassment. And third, a measure of how willing people are to consider some specific act to be harassing. This is a major problem. People from traditionalist cultures and subcultures may have a higher threshold for calling something harassing; if (for example), more traditional people go into Business, and more socially liberal people go into Art, that could skew the numbers.

In order to control for this and other factors, I asked subjects how often they are harassed outside of work. Of note, men, women, victims, perpetrators, everybody in every industry, reported much more sexual harassment outside of work than in it. Although there were enough female perpetrators to do statistics on this time, I left them out to keep it as similar to the in-work data as possible.

Here’s an overall summary, made through the same combination process as Figure 4.

Levels of at-work harassment and out-of-work harassment correlate at 0.81 across fields. This suggests that most of the differences observed here are differences related to the people in a field, rather than to any sort of field culture. Given how strong this effect is, it’s unclear whether it’s possible to adjust for it and get a measure of field culture. If we try (by dividing the at-work harassment levels by at-home harassment levels), we get this:

The most striking finding I find on all these graphs is that “nerdy” / STEM / traditionally-male jobs have the least harassment. The less nerdy / more verbal-personal-skills / traditionally-gender-balanced jobs have the most harassment.

Here’s Figure 4 again, but this time with nerdy-STEM-male occupations in red, verbal-personal-balanced occupations in blue, and edge cases in grey.

If categories like nerdy/STEM/traditionally-male seem unnatural, that’s fine – these are always a judgment call. But in December, I preregistered which clusters I would use, to prevent me from making them up to get the result I wanted. It’s not exactly the same as the clusters above – I ended out having to combine a few to get a big enough sample size to be meaningful. But using the original pre-registered clusters and the original pre-registered endpoint: in STEM/male-type fields, 12% of women (n = 212) reported being harassed at work. In non-STEM/balanced-type fields, 25% of women (n = 161) reported being harassed at work. p = 0.002. The preregistered analysis confirms the finding above.

I preregistered the hypotheses that STEM would have more female victimization (because there’s a higher ratio of heterosexual men to heterosexual women), but the same amount of male perpetration (because there is not an inherent difference in the sort of people who go into the field). Both preregistered hypotheses were wrong; female victimization and male perpetration were less.


Overall victimization rates were comparable to past surveys. These were on the low end of reported results, likely for several reasons. First, we strongly encouraged participants who were at all uncomfortable to say “prefer not to answer”, including asking 10% to say this outright regardless of experiences; this clearly depressed results and makes absolute numbers inaccurate. Second, participants were generally higher-income, and this group usually reports less harassment. Finally, we asked participants only about “sexual harassment” in general, and not about specific misbehaviors. Given these issues, the modest differences between these and past results (eg our 21% average female harassment rate compared to Cosmo’s 33%) are not surprising.

More surprising were the lower harassment rates in STEM and male-dominated fields, for two reasons. First, if most harassers are heterosexual, then a high percent male workforce in an industry should create a situation where a large number of harassers are focused on a small number of potential victims, raising the average harassment rate per victim. But this was not observed in our survey, nor in the Cosmo survey, nor (as far as can be eyeballed from the not-really-inter-comparable data) in the EEOC survey.

I don’t know of a simple explanation for the discrepancy. A friend suggested that it might be a customer effect. That is, lawyers / health-care-workers / etc often have to deal with sexist customers, whereas female programmers might not encounter anyone on the job except their coworkers. But this seems unlikely to make much difference; this survey finds that customers represent only 9% of sexual harassment incidents. Also, remember that the rates at which men admit to perpetrating sexual harassment match the rates at which women admit to experiencing it, and there’s no reason why the male-perpetration rates should have anything to do with customers. Also, social scientists don’t have customers any more than hard scientists do, but the social science harassment rate is higher. Perhaps the sort of people who go into STEM are socially oblivious in a way that prevents them from both noticing if they are sexually harassing someone else, and from noticing if they are sexually harassed. This would result in somewhat lower self-reported perpetration rates, and self-reported victimization rates that are even lower than what the low perpetration rates would predict. It would also match the high correlation between in-work and out-of-work harassment, which suggests that demographic factors are making most of the difference.

A second reason the low harassment rates in STEM were surprising is conventional wisdom claiming the opposite. In media coverage, STEM fields are portrayed as full of “techbros”, and uniquely unwelcoming to women. How do we square this with the data?

Some people who reviewed these data suggested that the media focuses more on a narrative where STEM workers are dismissive of women, rather than one where they harass them. I don’t find this accurate – a quick glance at any media source will show they focus on both narratives. But it seems possible that the dismissiveness narrative is correct, and the harassment narrative is a side effect of this. But this is somewhat contradicted by the auto industry survey mentioned above, which found tech was unexceptional either in harassment or in general dismissiveness. However, the other industries participating in the survey may themselves be unusually bad.

Another possibility is that the respondents to the survey may be skewed for some reason. For example, they may have been selected by this blog’s previous discussion of these issues, which have also argued that tech is not a uniquely bad industry.

But a final possibility is that the media coverage is inaccurate. The minimal amount of previous research in this area has all shown that the highest rates of sexual harassment are in the retail and restaurant industries. The media has ignored this. Before last year, the media almost never mentioned sexual harassment in Hollywood; now they hardly mention anything else. Given that already know media coverage here is generally inaccurate, maybe the data don’t need to be squared with it. Maybe the media is just wrong.

Every industry has enough sexual harassment to produce horror stories. If the media is disproportionately interested in the horror stories of relatively well-off people, and in stories that confirm their existing prejudices, we might hear many genuinely horrible stories about harassment in technology, but fewer about harassment in eg health care, philosophy, or law – even if objectively those fields are much worse. That might further encourage people in technology to come forward (consider what happened in Hollywood after the Weinstein revelations) and people in other fields to stay silent, contributing to a vicious cycle.

Compare all the anecdotes and popular lore about how immigrants are criminals. It’s totally false – immigrants have crime rates well below native-born citizens. But we only know that because there have been really good studies. If the studies hadn’t been done, and all we had to go on was the daily lurid stories about a Mexican guy knifing someone, who would believe it? In the absence of real studies, the media’s ability to spin a compelling narrative casting some people as monsters feeds on itself forever. We know that happens relatively often. Are we sure this time is different?

There’s only been one strong previous survey on this subject – the one in Cosmo I referenced earlier. They found the tech industry had an around-average to slightly-less-than-average rate of harassment. Anything else you’ve heard – any news story, any anecdote – is at the same level of evidence as the stories of immigrant crime.

It seems possible that this narrative grew up to explain why there are so few women in technology – maybe they’re all being harassed out by lecherous professors and bosses. But this explanation doesn’t hold water – the rate at which men vs. women choose tech courses in high school exactly parallels the rate at which they go into tech occupations. And gender balance in fields isn’t predicted by harassment levels in those fields anyway – otherwise you’d never see women working in restaurants, retail, or Hollywood.

I have been expecting results like these even before I did this survey, and this calls my data into question. Whenever someone with an unusual opinion produces an experiment that confirms their opinion, you should always be skeptical until it is verified by other sources.

So aside from appealing to the unimpeachable authority of Cosmopolitan as partially confirming my results, I urge anyone with the relevant skills to download the raw survey data themselves and see if they can replicate my conclusions. You can get them as .xlsx or .csv. Some people have complained of weird problems in the csv format and I recommend the xlsx if at all possible. I have removed the data of a few people who did not want their answers to be public, so you may not get exactly the same numbers I did, but they should be pretty close.

And more important, I appeal to anyone with an interest in this topic to do larger and more formal surveys comparing different fields. This may be the single topic where the extent of public interest is most disproportionate to the minimal amount of good research done. If people more prestigious than me or Cosmo were working on this, we would be in a much better position to know what to believe.

Recommendations vs. Guidelines

Medicine loves guidelines. But everywhere else, guidelines are still underappreciated.

Consider a recommendation, like “Try Lexapro!” Even if Lexapro is a good medication, it might not be a good medication for your situation. And even if it’s a good medication for your situation, it might fail for unpredictable reasons involving genetics and individual variability.

So medicine uses guidelines – algorithms that eventually result in a recommendation. A typical guideline for treating depression might look like this (this is a very over-simplified version for an example only, NOT MEDICAL ADVICE):

1. Ask the patient if they have symptoms of bipolar disorder. If so, ignore everything else on here and move to the bipolar guideline.

2. If the depression seems more anxious, try Lexapro. Or if the depression seems more anergic, try Wellbutrin.

3. Wait one month. If it works perfectly, declare victory. If it works a little but not enough, increase the dose. If it doesn’t work at all, stop it and move on to the next step.

4. Try Zoloft, Remeron, or Effexor. Repeat Step 3.

5. Cycle through steps 3 and 4 until you either find something that works, or you and your patient agree that you don’t have enough time and patience to continue cycling through this tier of options and you want to try another tier with more risks in exchange for more potential benefits.

6. If the depression seems more melancholic, try Anafranil. Or if the depression seems more atypical, try Nardil. Or if your patient is on an earlier-tier medication that almost but not quite works, try augmenting with Abilify. Repeat Step 3.

7. Try electroconvulsive therapy.

The end result might be the recommendation “try Lexapro!”, but you know where to go if that doesn’t work. A psychiatrist armed with this guideline can do much better work than one who just happens to know that Lexapro is the best antidepressant, even if Lexapro really is the best antidepressant. Whenever I’m hopelessly confused about what to do with a difficult patient, I find it really reassuring that I can go back to a guideline like this, put together by top psychiatrists working off the best evidence available.

This makes it even more infuriating that there’s nothing like this for other areas I care about.

Take dieting. Everybody has recommendations for what the best diet is. But no matter what diet you’re recommending, there are going to be thousands of people who tried it and failed. How come I’ve never seen a diet guideline? Why hasn’t someone written something like:

1. Try cutting carbs by X amount. If you lose Y pounds per week, the diet is working. If not, you’re probably resistant to cutting carbs because [two hours of mumbling about insulin] and you should move on to the next tier.

2. Try cutting fat by X amount. If you lose Y pounds per week, the diet is working. If not, you’re probably resistant to cutting fat because [two hours of mumbling about leptin], and you should move on to the next tier.

And so on until Step 7 is “get a gastric bypass”.

I agree nobody can ever make a perfect algorithm that works for all eventualities. But still. Surely we can do better than “Try the Paleo diet! I hear it’s great!”

What information do guidelines carry beyond a recommendation?

First, they have more than one recommendation. It may be that the Paleo diet is the best, but the guidelines will also include which is the second-best, third-best, et cetera.

Second, because they have more than one recommendation, they can tailor their recommendation to your specific circumstances. The person with depression and comorbid anxiety may want to start with Lexapro; the person whose main symptom is tiredness may want to start with Wellbutrin. Since I love bread, does that mean I should avoid carb-cutting diets? Does that mean it’s extra-important that I cut carbs? Does it not matter, and really it depends on whether I have a family history of diabetes or not?

Third, they acknowledge that some people might need more than one recommendation. If you hear “try the Paleo diet”, and then you try it, and it doesn’t work, you might believe you’re just a bad dieter, or that all diets are scams, or something like that. Guidelines implicitly admit that everyone is different in confusing ways, that something that’s expected to work for many people might not work for you, and that you should expect to have to try many things before you find the right one.

Fourth, because they admit you may need to try more than one thing, they contain (or at least nod at) explicit criteria for success or failure. How long should you try the Paleo diet before you decide it doesn’t work? How much weight do you need to lose before it qualifies as “working”? If it’s been three months and I’ve lost four pounds, should you stick with it or not?

Fifth, they potentially contain information about which things are correlated or anticorrelated. The depression guidelines make it clear that if you’ve already tried Lexapro and Zoloft and they’ve both failed, you should stop trying SSRIs and move on to something with a different mechanism of action. If I’ve tried five carb-cutting diets, should I try a fat-cutting diet next? If I hate both Mexican food and Chinese food, is there some other category of food which is suitably distant from both of those that I might like it? Guidelines have to worry about these kinds of questions.

My impression is that once you understand a field really well, you have something like a Guideline in your mind. I think if nobody had ever written a guideline for treating depression, I could invent a decent one myself out of everything I’ve pieced together from word-of-mouth and common-sense and personal experience. In fact, I think I do have some personal guidelines, similar to but not exactly the same as the official ones, that I’m working off of without ever really being explicit about it. Part of the confusion of questions like “What diet should I do?” is sorting through the field of nutrition until you can sort of imagine what a guideline would look like.

So why don’t people who have more knowledge of nutrition make these kinds of guidelines? Maybe some do. I can’t be sure I haven’t read dieting guidelines, and if I did I probably ignored them because lots of people say lots of stuff.

But I think that’s a big part of it – making guidelines seems like a really strong claim to knowledge and authority, in a way that a recommendation isn’t. Some idiot is going to follow the guidelines exactly, screw up, and sue you. I just realized that my simplified-made-up depression guidelines above didn’t have “if the patient experiences terrible side effects on the antidepressant, stop it”. Maybe someone will follow those guidelines exactly (contra my plea not to), have something horrible happen to them, and sue me. Unless you’re the American Psychiatric Association Task Force or someone else suitably impressive, your “guidelines” are always going to be pretty vague stuff that you came up with from having an intuitive feel for a certain area. I don’t know if people really want to take that risk.

Still, there are a lot of fields where I find it really annoying how few guidelines there are.

What about nootropics? I keep seeing people come into the nootropics community and ask “Hey, I feel bad, what nootropic should I use?” And sure, eventually after doing lots of research and trying to separate the fact from the lies, they might come up with enough of a vague map of the area to have some ideas. But this is an area where “Well, the first three things you should try for anxiety are…” could be really helpful. And I don’t know of anything like that – let alone something that tells you how long to try before giving up, what to look for, etc.

Or let’s get even broader – what about self-help in general? I don’t really believe in it much, but I would love to be proven wrong. If there were a book called “You Are Willing To Devote 100 Hours Of Your Life To Seeing If Self-Help Really Works, Here’s The Best Way For You To Do It”, which contained a smart person’s guidelines on what self-help things to try and how to go about them, I would absolutely buy it.

Posted in Uncategorized | Tagged | 274 Comments

Highlights From The Comments On DC Graduation Rates

Bizzolt writes:

DC Public Schools HS teacher here (although I’m not returning next year, as is the case with many of my colleagues). As noted, one of the biggest factors in the graduation rates is the unexcused absences–if you look at the results of our external audit and investigation here, you see that for many schools, a significant number of our seniors “Passed Despite Excessive Absences in Regular Instruction Courses Required for Graduation”–over 40% of 2017 graduates at my high school, for example.

So the attendance policy is being strictly enforced now, and you can see how from that alone, a ~30% drop in expected graduates is possible. Some more details about strictly enforcing the attendance policy though:

1: DCPS has what’s called the ’80 20′ rule: A student that is absent for at least 20% of their classes is considered absent for the whole day.
2: Most schools have 5 periods, so an absence in one class would be considered an absence for the whole day.
3: If you have 10 or more unexcused absences in a class, you automatically get an F for the term.
4: If you are over 15 minutes late for a class, that is considered an unexcused absence.
5: A majority of these absences are in first period.
6: A majority of students in my school and many others live in single parent households.
7: These students are typically responsible for making sure their younger siblings get to school, if they have any.
8: Elementary and middle schools in my neighborhood start at the exact same time as high school.
9: Their doors do not open until 5 to 10 minutes before the starting bell, presumably for safety reasons.
10: Refer to point 4.

There’s many other problems at DCPS to be sure, but this set of circumstances alone is causing the largest increase in failing grades and graduation ineligibility at my high school, and basically every other 90+% black school in the district. You could see how this accounts for quite a bit of the difference between white and black graduation rates as well. There’s a reason why across the board, DCPS schools were not strictly enforcing this policy in previous years.

It looks like most other school districts don’t have this policy; it seems plausible that this is the main difference between DC and other poor school districts that nevertheless manage to pass most of their kids.

Userfriendlyyy also focuses on the absences:

Looks to me like the policy they changed was losing credit for bad attendance. This might be from a few things. Kids might need to help out with the family finances. The only part of the job market that is doing well right now is low end unskilled workers who are willing to get paid crap (no matter how much the financial press wants to pretend otherwise, I listened to an hour of local NPR and the Topic was ‘call in and tell us how the booming job market is helping you out’, 20 callers not one had anything good to say and my state has one of the lowest unemployment rates in the country). If you know you don’t have the grades for a scholarship, your family is broke and since we have effectively made going to college impossible for anyone but the offspring of the oligarchy, and you can find a minimum wage job easily; what exactly is the utility of that little piece of paper compared to the ability to put food on the table tonight?

Static focuses on absences too:

The amazing thing to me is that they are largely failing due to unexcused absences. [See Washington Post:] I Feel Really Bad For The Class Of 2018: Graduation May Be Imperiled

I think that does point a little to the signalling difference between the GED and the diploma- can you show up every day?

That said, DC is planning to add an exam requirement for graduation. It would be better if they made that an alternative to attendance.

The absence requirement seems infuriating insofar as it probably fails mostly poor children, and if those children are failing due to the absence requirement rather than because they actually flunk their exams, then it seems mean-spirited to punish inputs rather than actual success.

Maybe they have it because they don’t actually grade students on exam performance (or have lots of different ways to make sure students who flunk exams graduate anyway), but they only want to extend this benefit-of-the-doubt to students who really try. I wonder if it would work to say you can graduate if you manage to make it to school enough times or pass your examinations. That would at least be fairer to poor children who can’t always attend but are able to figure out ways to succeed anyway.

Proyas writes:

I was friends with a guy who briefly worked as a teacher at a public high school in central DC (I’m 80% sure it was Cardozo High). He had an education background thanks to spending several years working as a youth camp counselor and as an after-school program counselor, and that was sufficient to qualify him for DCPS’ abbreviated teacher training program (such a thing existed in 2009 when he did it; I’m unsure if it is still around). During the training program, I remember him speaking about his enthusiasm for the teaching skills he was learning and about his eagerness to put them to use (in retrospect, I think some of this was a nervous attempt to convince himself the job wouldn’t be bad). After a break of several months, we spoke again, and he was almost totally disillusioned with the job and was already thinking of quitting. This is what I remember him saying:

1) On the first day of classes, there was no orientation for new teachers, no brief meeting where the Principal shook his hand and said “Welcome Aboard,” nothing. He had to go to the front office and ask a secretary what classroom was his and walk there by himself.

2) Unexcused absences were chronic and undermined his ability to teach anything. At the start of each of his classes, he had a written roster of students, and he had to check off which students were there. For any class, typically 20-30% of students would be missing, without explanation (This is a very important point to remember whenever anyone tries to blame DCPS’ poor outcomes on large class sizes–on paper, each class might have 35 students, but typically, only 23 are actually showing up). Additionally, the 20-30% of students who were absent each class varied from day-to-day, meaning one student didn’t know what was taught on Monday, the one next to him was there Monday but not Tuesday, the third was there the first two days but not Wednesday, etc.

3) Student misbehavior was atrocious. For example, out of the students who showed up to class, it was common for some to walk into the classroom late, again without any explanation and often behaving disruptively. As a rule, whenever a student did that, he was obligated to sign his name on a clipboard for the teacher’s attendance records (there was no punishment for tardiness–late students merely had to write their names down). Some late students would chronically resist doing this, either ignoring him and just going to their desks or yelling curses at him. My friend described an incident where one student–who was physically bigger than he was–yelled out he was a “FAGGOT” when asked to sign the clipboard, provoking laughs from all the other students, before sitting down without signing it. After seeing he could get away with that, the student started calling my friend “FAGGOT” all the time. Other examples of misbehavior included near-constant talking among the students during lessons and fooling around with cell phones.

4) Teachers received almost no support from the school administration. Had sane rules been followed at this high school, students would have been immediately sent to the office for formal punishment for these sorts of offenses I’ve described. However, under such a policy, the office would have been overwhelmed with misbehaving students and probably some of their enraged parents, so the administration solved the problem by forbidding teachers from sending students to the office for anything other than physical violence in the classroom. My friend had no ability to formally punish the student who liked to call him “FAGGOT” other than to use stern verbal warnings.

5) Most of the students were unwilling and in some cases unable to learn. During class sessions, the students were clearly disengaged from what he was teaching. Homework completion rates were abysmal. As the end of the academic semester neared, he saw that a huge fraction of them were on track to fail, so he resorted to pitiful cajoling, pizza parties, reward schemes, and deals involving large curves to everyone’s grades if they could only, for once do a little work, and it didn’t work. Some of his students were Latino and understood little or even no English, meaning they learned (almost) nothing, even when they tried. He resorted to seating the students who knew no English next to bilingual Latinos who could translate for them. That was the best he could do. In fairness, he spoke glowingly of some of his students, who actually put in some effort and were surprisingly smart.

6) At the time my friend was teaching, DCPS was in the grips of some harebrained, faddish teaching philosophy that said students of different academic abilities shouldn’t be put in different course tracks, but rather, should be deliberately put in the same class. This of course caused immediate problems since the curriculum was too hard for the weakest students and too easy for the strongest ones. I think my friend said his training program basically told teachers to “try harder” if any problems arose from the setup.

I’ll never forget how crestfallen and stressed out he was when he described these things to me. Having never taught in American public schools, I didn’t realize just how bad it was, and the detailed nature of his anecdotes really had an impact on me. I advised him to finish his year at the high school and then to transfer to ANY non-urban school in the area, even if it meant lower pay or a longer commute. We lost touch after that, but I can’t imagine he still works in DCPS.

Okay, maybe this goes deeper than just the absence thing.

MrApophenia writes:

One thing to keep in mind is that DC really is uniquely bad as a school district. A few years back, the Washington Post did a really in depth analysis of why it was so broken. (Sadly, with Google being totally swamped by the current scandal, I was unable to find a link.)

The conclusion they found was pretty interesting –

If you go back to the 60s, DC had some of the best urban schools in the country by every metric they had to track things back then. What changed? Well, see, back then DC still didn’t have home rule. They were almost entirely run by the federal government. So the highest elected office the city had was the DC Board of Education.

In 1971, an ambitious young politician by the name of Marion Barry got elected to the board, and almost immediately began farming out school administration positions as political rewards for his cronies. This practice caught on, and within a matter of years, the whole enterprise basically descended into naked corruption.

I recall they showed a figure in that article that the DC public school system spends the third most money per student of any district in the country (after NY and Boston); however, the Post also found that in terms of the quantity of money that is actually spent on students, DC was roughly at the level of the most poverty-stricken districts in the poorest Southern, rural school districts.

Michelle Rhee did not change any of this, Waiting for Superman or no.

This isn’t meant to argue with the premise of the rest of the article – the idea that everyone else is committing fraud seems quite plausible. But I wouldn’t necessarily reject the idea that DC’s school system really is a special, unique snowflake of terrible practices, either.

There’s also this comment on home rule and Puerto Rico which seems to reinforce this idea that districts carefully monitored by competent national authorities do well, and districts that have control of their own standards devolve into corruption and failure really quickly. How does this mesh with the standard federalist/localist/Seeing-Like-A-State style arguments that individual communities know what’s best for them and central planners usually make things worse?

I won’t quote it directly because it’s not from an SSC comment, but some people on the subreddit link to this Reddit post by a DC public school teacher.

See also this thread in the subreddit on how sub-Saharan African schools – despite being much poorer than anywhere in DC – are really well-run, quite safe (except that apparently “baboons are a huge problem”), and a delight to teach at.

Why DC’s Low Graduation Rates?

[Some changes to the conclusions in this post; see edit at the end and entry 21 on Mistakes page]

US News: DC Schools Brace For Catastrophic Drop In Graduation Rates. “Catastrophic” isn’t hyperbole; the numbers are expected to drop from 73% (close to the national average of 83%) all the way down to 42%.

There’s no debate about why this is happening – it’s because the previous graduation rate was basically fraudulent, inflated by pressure to show that recent “reforms” were working. Last year there was a big investigation, all the investigators agreed it was fraudulent, DC agreed to do a little less fraud this year, and this is the result. It’s pretty damning, given how everybody was praising the reforms and holding them up as a national model and saying this proved that Tough But Fair Education Policy could make a difference:

As far as scandals in the education policy world go, D.C. schools so profoundly miscalculating graduation rates at a time when the high-profile school district had been so self-laudatory about its achievements may be difficult to top […] Indeed, when Michelle Rhee took the reins of the flailing school system a decade ago, it galvanized the education reform movement, which had just begun blossoming around the country, and ushered in a host of controversial changes that included the shuttering of multiple schools, firing of hundreds of teachers and the institution of new teacher evaluation and compensation models.

The changes not only dramatically altered the local political landscape in Washington but also shined a national spotlight on D.C. schools that prompted other urban school districts and education policy researchers to consider the nation’s capital a bellwether for the entire education reform movement.

Well, darn.

But the interesting bit isn’t just that DC schools are doing worse than we thought. It’s that DC schools are doing amazingly, uniquely, abysmally bad, below what should even be possible. We make fun of states like Mississippi and Alabama, but both have graduation rates around 80%. The lowest graduation rate in any of the fifty states is in Oregon, which still has 69%. And we are being told DC is 42%!

When we discussed this in the last links thread, people had a couple of explanations:

1. Washington DC has a terrible school system, with uniquely incompetent administrators.

2. Washington DC is poorer, blacker, and more segregated than any other state, and that leads to unique challenges other school systems don’t face. Even though everyone is doing their best, they face insurmountable structural difficulties.

3. Maybe the fraud was so bad that DC over-corrected, and now has stricter standards than anywhere else.

Which of these is most important?

Let’s start by looking at test scores. Here’s a sample of DC’s NAEP scores compared to some other states (full list is here).

In both reading and math, in all grades, DC does abysmally. But they don’t do as abysmally as a 42% graduation rate might predict. They are sometimes last, sometimes second- or third- to last, and in any case they’re rarely that different from other low-performers like Alabama and Mississippi. They might be best described as a member in good standing of the lowest-performing tier of US states.

But there are massive racial inequalities in education, and DC is by far the blackest “state” in the nation. How does it do when we adjust for this?

The full table is here. DC has by far the highest white test scores in the country – probably because a lot of its white students are the kids of well-off bureaucrats and think-tank types. And its black test scores range from lowest-tier-member to mediocre.

Nor can this just be a Simpson’s Paradox, where both white and black students do fine but the difference is driven by a greater number of black students. The US average black graduation rate is 68%. No state has a black graduation rate lower than 56%. DC – again – is supposed to have a 42% total graduation rate.

This seems to refute hypotheses 1 and 2 – that DC just has a terrible school system, or just has an unusually disadvantaged population. Its white students do very well. Its black students do poorly but not too much worse than they would in other states. There is no gap – among either race or among both combined – that corresponds to the gap between Mississippi’s graduation rate (total 80%, black-only 77%), and DC’s graduation rate of 42%.

This leaves us with hypothesis 3 – that DC got burned so badly in the fraud investigations that its standards are now much higher than anywhere else’s. Maybe a graph will help:

The horizontal axis is each state’s test scores – specifically the average of its 8th grade NAEP reading and math. The vertical axis is its graduation rate. The previous, supposedly fraudulent DC rate clusters together more or less with everybody else. The new, supposedly non-fraudulent rate is an extreme outlier, showing a graduation rate about 20 percentage points lower than test scores would predict.

So DC’s old graduation rate was normal relative to their test scores, and their new graduation rate is an outlier. But their old graduation rate is widely considered to have been maintained by fraud and really low standards, and their new graduation rate is widely considered correct. Does that mean that everywhere else with the same levels of poverty and segregation as DC also uses fraud and really low standards to keep their graduation rates up?

Maybe. Detroit is often used as a symbol of inner-city educational dysfunction, but even the district with the worst Detroit schools has a 61.5% graduation rate. How do they do it? Given that fewer than 5% of their students pass exams, I assume they do it through fraud and really low standards. Los Angeles? Fraud and really low standards. Chicago? Fraud and really low standards. Baltimore? Given stories like the one where one of the city’s highest-graduation-rate schools has zero percent of students score at “meets expectations” or even “approaches expectations” on statewide exams, it looks like fraud and really low standards.

I understand this is a really strong claim. But others seem to agree, and it’s the only way I can make sense of DC’s abysmally low projected graduation rates, in the context of their merely-awful exam scores.

Or maybe the key word is “projected”. From the US News article:

The report estimated that just 42 percent of seniors are on track to graduate at the end of the current school year, down from 73 percent who graduated in the 2016-17 school year. It noted that 19 percent of students were “moderately off-track” and could still earn enough credits to graduate.

Plausibly, kindly school officials will find loopholes that allow all of those 19% of moderately-off-track children to graduate, DC’s actual graduation rate will be 62% (right on the trend line for the graph above), and this whole episode will be remembered as “that one time a scary report underestimating the graduation rate came out”. Plausibly, this is the main way that Detroit and Los Angeles and all those other cities keep their rates up, and the more dramatic scare stories are just that.

I hope this happens. Think how unfair it will be for DC students if it doesn’t. Somebody who would graduate comfortably from any other high school in the country will be held back because they happen to have been in DC the year it decided to enforce standards nobody else enforced. If the true value of education is signaling, then the most important thing a school district can do is make sure it’s speaking the same signaling-language as everyone else. Probably somebody should fix the system in general, but that needs to happen on a national level if it’s not going to leave thousands of unfairly-failed children as collateral damage.

[EDIT: More discussion at Highlights From The Comments On DC Graduation Rates. Main update is that I underestimated the importance of absences, which are what’s causing a lot of the non-graduations, which there might be more of in DC, and which DC might be stricter about than other areas.]

OT99: Alpine Thread

This is the bi-weekly visible open thread (there are also hidden open threads twice a week you can reach through the Open Thread tab on the top of the page). Post about anything you want, ask random questions, whatever. You can also talk at the SSC subreddit or the SSC Discord server. Also:

1. The rationalist community now has a community center – the former La Renaissance Cafe on 3045 Shattuck in Berkeley. There are scheduled meetings there throughout the week, or you can just drop by (it’s usually open from about 11 AM – 5 PM, and once you know the people there they can give you a key if you want to do something during other times). Sarah is currently managing this and paying for the space herself, but she can’t keep doing that forever and is looking for financial assistance. Please see their Patreon and donate if you feel so inspired. The Patreon also has a list of when the different meetups are and what kind of things go on there. If they’re able to stay afloat, I may move SSC meetups there and you won’t have to stand awkwardly in the university quad.

2. Comment of the week is nostalgebraist on the neurogenesis post, best read in combination with this new study arguing that the study I cited was wrong and there is adult human neurogenesis after all. In retrospect, I probably framed the original post incorrectly. I originally wrote of it as “here’s all of this research that claims to have nailed down subtle and specific details of adult human neurogenesis, when adult human neurogenesis doesn’t even exist”. It might have been equally interesting, and more correct, to frame it as “here’s all this research that claims to have nailed down subtle and specific details of adult human neurogenesis, when it’s still a hotly debated topic whether adult human neurogenesis even exists.” Not only would this have been more accurate, but I think it generalizes better too. The experience of reading science in these kinds of fields is rarely one where we have proof that anything is wrong, and more often one where we always have to worry that things are built on flimsy foundations that might or might not survive later research.

Posted in Uncategorized | Tagged | 716 Comments

Links 4/18: Siter Plate

Sumerian mythology includes a bunch of weird legendary debates, like The Debate Between Sheep And Grain, the Debate Between Winter And Summer, and the Debate Between Bird And Fish. In case you’re wondering, the winners were (spoiler alert) grain, winter, and bird respectively.

Stevenson and Wolfers find that liberalization of divorce laws significantly decreased rates of domestic violence and female suicide.

History’s first conspiracy theory? The Nero Redivivus legend said that Emperor Nero survived his apparent death in 68 AD and was going to reclaim the Roman Imperial crown; it inspired three rebellions by people pretending to be Nero.

And in the world of modern crazy conspiracy theories: some Pakistanis believe Malala Yousafzai was never shot by the Taliban at all; her shooting was staged by the CIA (or, in one version, Robert De Niro). Also: “In November 2014, just a month after she was awarded the Nobel Peace Prize, the All Pakistan Private Schools Federation – which claimed to represent 150,000 schools – announced an ‘I Am Not Malala’ day.”

Interview with Open Philanthropy Project director Holden Karnofsky on what academic science can and can’t tell us about how to do altruism. Related: Karnofsky’s AMA about job openings at OpenPhil.

New England Journal of Medicine claims firearm injuries go down during NRA conventions (because the people who are busy attending the convention are the same people who would otherwise be shooting themselves). But Andrew Gelman thinks we shouldn’t believe it. Related: firearms researcher Carlos Goes changes his mind, now says there is good evidence for guns causing more crime.

After Washington DC cracked down on fraudulently graduating students who didn’t meet requirements, only 42% of high school seniors are on track to graduate this year. Some interesting discussion here, including the observation that even with the widespread fraud, DC’s graduation rates were still well below the national average. Before we start blaming the DC education system, I hope someone checks that this isn’t exactly what we would predict based on DC’s racial composition and known racial disparities in education.

David Graeber on the new understanding of prehistory. Claims that recent research has converged around a model where prehistoric humans formed large communities reminiscent of “civilizations” in the off-season from hunting, and that agriculture was less of a sudden shock and more a transition to having civilization year-round. Not sure if his views are as consensus as he claims, interested in learning more from prehistorically knowledgeable readers.

Italian election ends with center-right and populists in power, likely a victory for anti-immigrant forces and Euroskeptics. Still unclear who will get to lead the government, prediction markets slightly favor Di Maio and Five Stars. No market on whether Italy will leave the Euro, but most people I’ve read seem doubtful.

Reddit discussion on why the South African decision to seize white land probably won’t come to anything – strongest evidence is they’ve tried this a bunch of times before and it’s never come to anything.

Given the magnitude of the decline in global insect populations, why aren’t we all dead yet?

Bad signs: when your government becomes so censorious that it bans the word “disagree”.

WeForum has numbers on the bullshit-jobs phenomenon: “In a 2013 survey of 12,000 professionals by the Harvard Business Review, half said they felt their job had no “meaning and significance,” and an equal number were unable to relate to their company’s mission, while another poll among 230,000 employees in 142 countries showed that only 13% of workers actually like their job. A recent poll among Brits revealed that as many as 37% think they have a job that is utterly useless.”

Nominative non-determinism: the town of Equality, Illinois was a historical center of the slave trade, and center of a perverse scheme for kidnapping Northern blacks and selling them into slavery called the Reverse Underground Railroad.

Two new major papers on growth mindset. A large pre-registered experiment (related Twitter discussion here) concluded that a growth mindset intervention had very modest (but statistically significant) benefits, and given that it was so cheap it might still be cost-effective to spam the school system with it in the hopes that a couple of students benefit a little. A large meta-analysis agreed, with the caveat that spamming the school system with basically anything else would be more cost-effective (“From a practical perspective, resources might be better allocated elsewhere than mind-set interventions. Across a range of treatment types, Hattie, Biggs, and Purdie (1996) found that the meta-analytic average effect size for a typical educational intervention on academic performance is 0.57. All meta-analytic effects of mind-set interventions on academic performance were less than 0.35, and most were null. The evidence suggests that the “mindset revolution” might not be the best avenue to reshape our education system.”) People who previously supported growth mindset are taking this as proof that at least it works. I admit I am pretty biased against this idea, but I have a different perspective. Imagine I claimed our next-door neighbor was a billionaire oil sheik who kept thousands of boxes of gold and diamonds hidden in his basement. Later we meet the neighbor, and he is the manager of a small bookstore and has a salary 10% above the US average (though below the average for our neighborhood). Should we describe this as “we have confirmed the Wealthy Neighbor Hypothesis, though the effect size was smaller than expected”? Or as “I made up a completely crazy story, and in unrelated news there was an irrelevant deviation from literally-zero in the same space”?

The Bible says God doomed Cain to wander forever, so where is he these days? Various theories advanced through history have included “it’s metaphorical”, “on the Moon”, and “in Tennessee”.

Business Insider: companies are publicly liberal on social issues mostly because liberals are a more valuable consumer demographic.

New paper The Moral Hazard Of Lifesaving Innovations concludes that when states promote freer distribution of the opiate-overdose-antidote naloxone, people are more likely to abuse opiates because it’s perceived as safer, and in the end there’s higher crime and no reduction in mortality. Interested in hearing what the thus-far-very-successful pro-naloxone movement thinks about this.

Nathan Cofnas debunks Kevin MacDonald’s anti-Semitic conspiracy theories. Interesting not so much because I expect many people to believe Kevin MacDonald’s anti-Semitic conspiracy theories, but because an attempt to disprove anti-Semitism on the merits got published in a journal and was generally-well-received, and because it shows the way that careful and intelligent study of group differences can be used to fight racism (the argument is basically “Jews’ success in various fields is as we would predict from their IQs, so there’s no need to posit any conspiracy theory”). I don’t think there’s a good way to debunk these kinds of conspiracy theories without citing this research, which is one reason I become so concerned when people try to suppress it.

Related: Wikipedia’s article on Jewtown, Pennsylvania has a certain kind of minimalist beauty to it.

The Darian calendar, created for future colonists to keep time on Mars. In case you have the same question as I do – no, the months aren’t named after Homestuck trolls, they’re named after the Sanskrit names of zodiac signs (which Homestuck trolls are also named after).

Businessman Andrew Yang will run in the 2020 presidential election on a platform of universal basic income. “I’m a capitalist, and I believe that universal basic income is necessary for capitalism to continue”. Also supports banning federal regulators from moving to jobs in the fields they regulate, and “turning April 15 into a national holiday”. Who knows, he might even beat Vermin Supreme to win the coveted First Place Among People Who Will Never Win award.

Let’s be fair to Bernie Sanders: he never actually said anything positive about Venezuela. In fact, let’s celebrate this: given how many socialists did praise Venezuela when it looked like it was doing well, this demonstrates admirable judgment and restraint.

The Long-Run Effect Of Teacher Strikes: Evidence From Argentina tries to measure the effect of teachers by seeing if students who are exposed to long teacher strikes do worse in life. It claims “robust evidence” that a standard (for Argentina) of three months’ of teacher strikes over one’s educational career lowers adult earnings by 3%. This goes against all my priors but potentially matches some related results by Chetty. Interested in seeing further discussion of this.

The Fatebenefratrelli Hospital in Italy is famous for the mysterious “Syndrome K” – a fake, supposedly contagious diagnosis they would give Jews in order to keep them out of the Nazi concentration camps. The Nazis never investigated the hospital “out of fear of contracting the disease”.

If your favorite websites have become more censorious lately or cracked down on mostly harmless activity, it’s not their fault – it’s a result of FOSTA, a new anti-sex-trafficking law that in practice enables a wide variety of legal crackdowns and censorship against the Internet. RIP most of Reddit’s darknet- and drug- related communities, and Craigslist personals.

Here’s a graph showing favorability of various groups/people among Democrats vs. Republicans. Republicans view women more favorably than they view the NRA; Democrats view Christians more favorably than they view Nancy Pelosi.

New paper claims that cutting back on stop-and-frisk in Chicago caused a spike in homicides. Some discussion on Reason (1, 2) and on an SSC open thread.

Judge finds Starbucks guilty of refusing to put “This product contains chemicals known to the State of California to cause cancer” warning on their coffee, exposing them and other coffee chains to potentially millions of dollars in fines. Relevant law seriously seems to say that if they can’t prove their coffee doesn’t cause cancer, they need to include the warning.

Does anyone else think the UK #knifefree campaign comes off as a little creepy and Orwellian?

Paul Christiano’s AI safety research is up online at, including a summary by Ajeya Cotra intended to by comprehensible by us mere mortals. There’s also been a lot of good discussion of his research program at the new Less Wrong, including a response by Wei Dai.

Also: not one, but two good comic-book-style illustrated guides to AI safety topics. Abram Demski on some of MIRI’s research, and Chris Noessel illustrating Stuart Armstrong’s Smarter Than Us: The Rise Of Machine Intelligence.

Results of all 833 of Aella’s twitter polls. Content warning for frequent graphic sexual content.

Posted in Uncategorized | Tagged | 637 Comments

Adult Neurogenesis – A Pointed Review

[I am not a neuroscientist and apologize in advance for any errors in this article. A recent study came out contradicting some of the claims mentioned here. See here for study, this comment for some discussion, and entry #20 on my Mistakes page.]

Hey, let’s review the literature on adult neurogenesis! This’ll be really fun, promise.

Gage’s Neurogenesis In The Adult Brain, published in the Journal Of Neuroscience and cited 834 times, begins:

A milestone is marked in our understanding of the brain with the recent acceptance, contrary to early dogma, that the adult nervous system can generate new neurons. One could wonder how this dogma originally came about, particularly because all organisms have some cells that continue to divide, adding to the size of the organism and repairing damage. All mammals have replicating cells in many organs and in some cases, notably the blood, skin, and gut, stem cells have been shown to exist throughout life, contributing to rapid cell replacement. Furthermore, insects, fish, and amphibia can replicate neural cells throughout life. An exception to this rule of self-repair and continued growth was thought to be the mammalian brain and spinal cord. In fact, because we knew that microglia, astrocytes, and oligodendrocytes all normally divide in the adult and respond to injury by dividing, it was only neurons that were considered to be refractory to replication. Now we know that this long accepted limitation is not completely true

Subsequent investigation has found adult neurogenesis in all sorts of brain regions. Wikipedia notes that “In humans, new neurons are continually born throughout adulthood in two regions of the brain: the subgranular zone and the striatum”, but adds that “some authors (particularly Elizabeth Gould) have suggested that adult neurogenesis may also occur in regions within the brain not generally associated with neurogenesis including the neocortex”, and there’s also some research pointing to the cerebellum.

Some research has looked at the exact mechanism by which neurogenesis takes place; for example, in a paper in Nature cited 1581 times, Song et al determine that astroglia have an important role in promoting neurogenesis from FGF-2-dependent stem cells. Other research has tried to determine the rate; for example, Cameron et al (1609 citations) find that there is “a substantial pool of immature granule neurons” that may generate as many as 250,000 new cells per month. Still other research looks at the chemical regulators – a study by Lie et al, cited 1312 times, finds that Wnt3 signaling is involved.

(which is making you more nervous – the fact that I keep emphasizing how many citations these studies have, or the fact that one of the principal investigators is named “Lie”?)

But the most exciting research has been the work identifying the many important roles that neurogenesis plays in the adult brain – roles vital in understanding learning, memory, and disease.

Snyder et al (775 citations) finds “a new role for adult neurogenesis in the formation and/or consolidation of long-term, hippocampus-dependent, spatial memories.” Dupret et al go further and find that “spatial relational memory requires hippcampal adult neurogenesis”. Aimone et al (633 citations) find “a possible role” for adult neurogenesis in explaining the “temporal clusters of long-term episodic memories seen in some human psychological studies”. And Jessberger et al (506 citations) finds a role in object recognition memory as well.

In terms of learning, one of the major studies was Gould et al in Nature Neuroscience (2207 citations) finding that Learning Enhances Adult Neurogenesis In The Hippocampal Formation. Lledo et al (1288 citations) find that neurogenesis plays a part in explaining the brain’s amazing plasticity, and is “highly modulated, revealing a plastic mechanism by which the brain’s performance can be optimized for a given environment”. Clemenson et al (17 citations) find that “from mice to humans”, enviromental enrichment improves neurogenesis, and this “may one day lead us to a way to enrich our own lives and enhance performance on hippocampal behaviors”.

But I’ve always been most interested in the link with depression. In 2000, Jacobs et al published Adult Brain Neurogenesis And Psychiatry: A Novel Theory Of Depression (961 citations). It’s important enough that I want to quote the whole abstract:

Neurogenesis (the birth of new neurons) continues postnatally and into adulthood in the brains of many animal species, including humans. This is particularly prominent in the dentate gyrus of the hippocampal formation. One of the factors that potently suppresses adult neurogenesis is stress, probably due to increased glucocorticoid release. Complementing this, we have recently found that increasing brain levels of serotonin enhance the basal rate of dentate gyrus neurogenesis. These and other data have led us to propose the following theory regarding clinical depression. Stress-induced decreases in dentate gyrus neurogenesis are an important causal factor in precipitating episodes of depression. Reciprocally, therapeutic interventions for depression that increase serotonergic neurotransmission act at least in part by augmenting dentate gyrus neurogenesis and thereby promoting recovery from depression. Thus, we hypothesize that the waning and waxing of neurogenesis in the hippocampal formation are important causal factors, respectively, in the precipitation of, and recovery from, episodes of clinical depression.

This theory got a boost from studies like Duman et al (522 citations), which found that antidepressant drugs like SSRIs upregulated neurogenesis – could this be their mechanism of action? And Ernst et al (327 citations) find that “there is evidence to support the hypothesis that exercise alleviates MDD and that several mechanisms exist that could mediate this effect through adult neurogenesis” – ie the antidepressant effects of exercise seem to work this way too. Electroconvulsive therapy, the most effective known treatment for depression? Works by promoting adult neurogenesis, at least according to Schloesser et al.

Is there anything that doesn’t have important neurogenesis-related effects? It would seem there is not. Sex, for example, “promotes adult neurogenesis in the hippocampus, despite an initial elevation in stress hormones” according to Leuner et al (124 citations). Drug addiction is modulated by neurogenesis. We need rock n’ roll to complete the triad, so here’s Music Faciliates The Neurogenesis, Regeneration, and Repair of Neurons.

A study in Nature Neuroscience that garnered over 3000 citations found that running increased neurogenesis. The popular science press was quick to notice. A slew of exercise-neurogenesis studies spawned articles like Psychology Today’s More Proof That Aerobic Exercise Can Make Your Brain Bigger. Dr. Perlmutter (“Empowering Neurologist!”) has a video about how you can Grow New Brain Cells Through Exercise. After this the pop sci world might have gotten a little carried away, until neurogenesis controls everything and is controlled by everything in turn. Slimland (of course there’s a site called Slimland) has a How To Grow New Brain Cells And Stimulate Neurogenesis page, suggesting you can “set yourself free and start flying” by removing toxins, eating a ketogenic diet, and meditating. boasts 11 Proven Ways To Generate More Brain Cells, Improve Memory, And Boost Mood, which advises…really? Do you really want to know what it advises? Come on.

Also, growth mindset. Of course growth mindset. Carol Dweck’s Mindsetworks helpfully provides an infographic for teachers, urging them to tell their students that each time they set a goal or become motivated to learn a new skill, “a new neuron is formed through a process called neurogeneis [sic]”.

So it’s no surprise that researchers in the area are calling adult neurogenesis “one of the most exciting and rapidly evolving areas of research in the field of neuroscience”.


Fun fact: there’s no such thing as adult neurogenesis in humans.

At least, this is the conclusion of Sorrells et al, who have a new and impressive study in Nature. They look at “59 post-mortem and post-operative slices of the human hippocampus” and find “that recruitment of young neurons to the primate hippocampus decreases rapidly during the first years of life, and that neurogenesis in the dentate gyrus does not continue, or is extremely rare, in adult humans.” Also, the subgranular zone, the supposed part of the brain where neurogenesis begins, isn’t even a real structure.

I am not a neuroscientist and am unqualified to evaluate it. But the Neuroskeptic blog, which I tend to trust in issues like this, thinks it’s legit and has been saying this for years. Ed Yong from The Atlantic has a really excellent review of the finding that interviews a lot of the major players on both sides and which I highly recommend. Both of these reinforce my feeling that the current study makes a really strong case.

I was kind of floored when I saw this, in a way that I hope I was able to replicate in you by preceding this with the literature review above. How do you get so many highly-cited papers speaking so confidently about every little sub-sub-detail of a phenomenon, if the phenomenon never existed in the first place?

As far as I can tell, this was entirely innocent, well-intentioned, and understandable. It happened like this:

Adult neurogenesis was discovered in rats. This was so surprising, and such a violation of established doctrine, that it quickly became one of the most-investigated areas in neuroscience. Hundreds of studies were done on rats to nail down every little detail of the process.

The work was extended to many other mammals, to the point where it seemed inevitable that it must be true of humans as well. This was difficult to test because the relevant studies involve dissecting brains, and there aren’t that many human brain specimens available with the necessary level of preservation. After a lot of work, a few people got a couple of brains, did some very complicated and contamination-prone tests, and found evidence of adult neurogenesis. This encouraged everyone to assume that the things they had discovered about rat neurogenesis were probably true in humans as well, even though they could never prove them directly because of the difficulty of human experimentation. Later some other researchers tried to replicate the complicated and contamination-prone tests and couldn’t find adult neurogenesis in humans, but everyone assumed they had just messed up some aspect of the complicated testing process.

And to complicate matters, everyone in the new study has been very careful to say they can’t prove with certainty that zero adult neurogenesis occurs – just that the levels are so low and hard to detect that they can’t possibly matter. Looking back on some past studies, it seems that “so low and hard to detect that they can’t possibly matter” was actually within their confidence intervals. So it may be that some team found some extremely tiny and irrelevant population of immature neurons in the brain, gave a confidence level that included that number, and then everyone just assumed we were talking about levels similar to the ones we saw in rats.

With real scientists taking not-entirely-sufficient care to distinguish rat from human results, the popular press felt licensed to totally ignore the distinction (did you even notice which of the studies in Part I were done on which species? Don’t worry, nobody else did either).

Meanwhile, synaptogenesis – the growth of new synapses from existing nerve cells – was getting linked to depression and all kinds of other things in a lot of interesting studies. When people started talking about neurogenesis’ role in depression, psychiatrists like me who have trouble keeping words ending with -genesis separate just sort of nodded and said “Oh, yeah, I heard about that” and didn’t give it the sort of scrutiny it deserved.

(I wonder if this is young-earth creationists’ problem too)

So it’s not like any one person made a spectacular mistake anywhere along the lines. Most of the studies done were in rats, and 100% correct. A few studies were done in humans, and may have gotten the wrong answer in a very difficult domain, while also hedging their bets and admitting they were trying something hard. It was only on a structural, field-wide level that all of this came together into people just assuming that adult human neurogenesis had to happen and be important.

…or at least, that’s the optimistic take on it. But I can’t help thinking – antidepressants work in humans, which suggests that the people who found neurogenesis was necessary for antidepressant effects must have just been plain wrong. And if exercise has antidepressant effects in humans, then the claim that those effects are neurogenesis-mediated must be wrong too. And, uh, humans form spatial and temporal memories, so unless we do this by a totally different mechanism than the ones rats use, people must have been wrong when they said neurogenesis was involved in that. ECT? Works in humans. Brain plasticity? Happens in humans. So maybe it would be better to say that the original claim that adult neurogenesis happens in humans seems innocent and understandable – but if the new study is true, that suggests that a lot of the followup claims must have been imaginary. Anything that focuses on a process that happens in humans and says “neurogenesis causes this” must not only be wrong to extend the results to humans, but must be under strong suspicion of being wrong even about rats, unless rat brains and human brains accomplish the same basic tasks through totally different mechanisms (eg antidepressants work on rats but for different reasons than in humans).

We know many scientific studies are false. But we usually find this out one-at-a-time. This – again, assuming the new study is true, which it might not be – is a massacre. It offers an unusually good chance for reflection.

And looking over the brutal aftermath, I’m struck by how prosocial a lot of the felled studies are. Neurogenesis shows you should exercise more! Neurogenesis shows antidepressants work! Neurogenesis shows we need more enriched environments! Neurogenesis proves growth mindset! I’m in favor of exercise and antidepressants and enriched environments, but this emphasizes how if we want to believe something, it will accrete a protective layer of positive studies whether it’s true or not.

I’m also struck by how many of the offending studies begin by repeating how dogmatic past neuroscientists were for not recognizing the existence of adult neurogenesis sooner. Remember Gage’s review above:

A milestone is marked in our understanding of the brain with the recent acceptance, contrary to early dogma, that the adult nervous system can generate new neurons. One could wonder how this dogma originally came about…

Or from Neurogenesis In Adult CNS: From Denial To Opportunities And Challenges For Therapy:

The discovery of neurogenesis and neural stem cells (NSC) in the adult CNS has overturned a long‐standing and deep‐routed “dogma” in neuroscience, established at the beginning of the 20th century. This dogma lasted for almost 90 years and died hard when NSC were finally isolated from the adult mouse brain. The scepticism in accepting adult neurogenesis has now turned into a rush to find applications to alleviate or cure the devastating diseases that affect the CNS.

From Adult Human Neurogenesis: From Microscopy To Magnetic Resonance Imaging:

The discovery of adult neurogenesis crushed the century-old dogma that no new neurons are formed in the mammalian brain after birth. However, this finding and its acceptance by the scientific community did not happen without hurdles. At the beginning of the last century, based on detailed observations of the brain anatomy reported by Santiago Ramon y Cajal and others, it was established that the human nervous system develops in utero (Colucci-D’Amato et al., 2006). In adult brains, it was thought, no more neurons could be generated, as the brain is grossly incapable of regenerating after damage (for a more detailed historical report see Watts et al., 2005; Whitman and Greer, 2009). This dogma was deeply entrenched in the Neuroscience community, and Altman’s (1962) discovery of newborn cells in well-defined areas of the adult rodent brain was largely ignored.

I’m bolding the word “dogma” because for some reason every article in this field uses it like a verbal tic. University of Washington’s “Neuroscience For Kids” page feels compelled to use the word even though they don’t expect their readers to know what it means:

The dogma (a set of beliefs or ideas that is commonly accepted to be true) that nerve cells in the adult brain, once damaged or dead, do not replace themselves is being challenged. Research indicates that at least one part of the brain in adults maintains its ability to make nerve cells.

I think Patient Zero in this use-of-the-word-dogma epidemic might be Neurogenesis In The Adult Brain: Death Of A Dogma, (880 citations) whose abstract says:

For over 100 years a central assumption in the field of neuroscience has been that new neurons are not added to the adult mammalian brain. This perspective examines the origins of this dogma, its perseverance in the face of contradictory evidence, and its final collapse. The acceptance of adult neurogenesis may be part of a contemporary paradigm shift in our view of the plasticity and stability of the adult brain.

The dogma-concern isn’t totally wrong. Previous neuroscientists thought there wasn’t neurogenesis in rats, and there is. That was a legitimate mistake and one worth examining. But is it possible that the reaction to that mistake – a field-wide obsession with talking about how dogmatic you had to be to miss the obvious evidence of mammalian neurogenesis, and a desire never to repeat that mistake – contributed to the less-than-stellar effort to make sure neurogenesis was happening in humans? Heuristics work until they don’t. Those who fail to learn from history are doomed to repeat it, but those who learn too much from history are doomed to make the exact opposite mistake and accuse anyone who urges restraint of “failing to learn from history” and “dogmatism”. From the Virtues of Rationality:

The Way is a precise Art. Do not walk to the truth, but dance. On each and every step of that dance your foot comes down in exactly the right spot. Each piece of evidence shifts your beliefs by exactly the right amount, neither more nor less.

Or maybe I’m just grasping for straws. But I feel like I have to grasp for something. I have nowhere near as much expertise as the actual neuroscientists writing about this result (and there are many). I’m sure I’ve made some inexcusable mistakes somewhere in this process. Perhaps I am missing some colossal flaw in the new study, and wrongly slandering dozens of neuroscientists doing great work.

But the reason I feel compelled to dabble in this subject anyway is that I don’t feel like anyone else is conveying the level of absolute terror we should be feeling right now. As far as I can tell, this is the most troubling outbreak of the replication crisis so far. And it didn’t happen in a field like social psychology which everyone already knows is kind of iffy. It happened in neuroscience, with dramatic knock-on effects on medicine, psychology, and psychiatry.

I feel like every couple of months we get a result that could best be summed up as “no matter how bad you thought things were, they’re actually worse”. And then things turn out to be even worse than that. We can’t just become 100% certain things are arbitrarily bad – that would be making the same mistake as the neuroscientists who were overly eager to reject the no-neurogenesis dogma. But that means we always have to be ready for disappointment.

From the Neuroskeptic article:

So what does this all mean? Sorrells et al. conclude by speculating, provocatively, that our lack of adult hippocampal neurogenesis may actually be part of what makes us human:

“Interestingly, a lack of neurogenesis in the hippocampus has been suggested for aquatic mammals (dolphins, porpoises and whales), species known for their large brains, longevity and complex behaviour.”

This hypothesis seems pretty wild to me. But it’s no wilder than some of the other theories that have long surrounded adult neurogenesis

Our total inability to ever change or get better in any way is what separates us from the animals. Inspiring!

Posted in Uncategorized | Tagged , , | 99 Comments

Highlights From The Comments On Twelve Rules

[Taken from the responses to this post.]

From sclmlw:

While I don’t agree with lots of Jordan Peterson, I think Scott fundamentally missed the boat in some of his criticisms because he systematically views things from a different perspective than Peterson, which was missed.

From what I can tell, Peterson is intensely interested in the idea, “Everyone has the capacity to become a Nazi war criminal. What causes that phenomenon?” His answer, and the central driving idea of his philosophy, seems to be, “Anarchy/chaos is worse for society/humanity than horrific, unimaginable cruelty. So evolution pushed society to develop in a way that will always choose cruelty over chaos. Thus, if you were in Stalin’s Russia, you’d run the gulags to stave off anarchy, and you’d kill hundreds of people if you had to. You may hate it, but it was required for humanity to soldier on, so it’s what evolutionary forces produced.” Peterson cares because he wants to understand how to steer societies away from the gulags and the killing fields.

This appears to be the foundation of his philosophy, and you can understand a lot of what he talks about as an outgrowth of this idea. For example his answers to, Why do good things happen to bad people? He accepts that good things happen to bad people as a given. However, when good things happen to bad people, those people have a choice to make: should I give in to chaos and ascribe meaninglessness, or should I accept some order of some kind? And Peterson would say that societies that would have chosen the chaos angle didn’t survive. Evolution now has the choice as a built-in function, where you will always accept order of some kind. And that order could manifest in a number of different ways, such as the killing fields of Cambodia, but what matters is that if your choice is killing fields or chaos you’ll choose killing fields every time. So Peterson wants to ensure we don’t get to the point where that’s the only choice left; he advises his acolytes not to destroy society (which leads to the chaos/authoritarian dilemma), but to recognize that they’re going to choose to follow some order of some kind, and that they should therefore intentionally follow even a flawed societal order, because it’s better than gas chambers and ethnic cleansing. The reality that “bad things happen to good people” shouldn’t persuade them to tear down society and try starting all over again, because that leads to the chaos/tyrant choice, and we can’t go there. So when bad things happen, you have to do your part to keep flawed society going, or else we get concentration camps. Go back and read the quotes above in that context and they all make sense. He’s not trying to answer “why do bad things happen”, he’s trying to direct what he sees as an appropriate response to when they do.

This also directs his motivations when talking to people about his theory. Fundamentally, he has hypothesized a reason so many people in the 20th century became horrible, and he sees the current non-awful state of civilization as unstable. He sees trends he believes could tear down society, and cause people to spiral back to the point where they will be willing to do anything to stave off the chaos. In some of his videos he gets passionate, and in most cases he reserves his passion for this basic idea in some form or another: everyone has the capacity to become a Hutu killing Tutsis; you would do it, even if you think you’re better than that; if you don’t follow certain ideas, you (and society in general) will devolve into that awful state.

I don’t know if any of that is true, or if it’s a different kind of psychobabble, but the fundamental observation that Scott is missing is that Peterson is thinking on a society-wide and philosophically-projected evolutionary development axis. Peterson’s pronouncements flow from this angle. He’s not thinking as a utilitarian or deontologist or consequentialist. He’s thinking, “What do populations do in these situations, and how can we nudge populations away from mass torture/murder?” That’s not utilitarian maximization, or negative utilitarianism. It’s sort of like Nassim Taleb’s concern about fat-tail events breaking fragile systems, and how to avoid that.

From theredsheep:

Have not read Peterson, have seen videos of his lectures where he seems unremarkable and uninteresting. Maybe it’s just that I’m not in the market for a belief system, I don’t know.

I agree; I’ve also watched his videos and find them unwatchable and boring. I don’t know why the book seems so much better – maybe I just respond more to the written word, or maybe he had a really good editor. I would wonder if this is what’s behind the high variance in how people respond to him, except I think a lot of the people who absolutely love him are working off the videos. Weird. It takes all kinds, I guess.

From Dry Raven:

As someone who is deep into the Peterson hole, I have a warning. Peterson is like an alien whose words travel through several layers of perceptual distortion before they come out to a regular human being. You think you understand what he means when he says something, but he means something entirely different. His words come out like static to people, and they make the mistake of thinking they understand his intent because the sentence still parses in English. But he’s changed the meaning of all the words. Reading 12 rules for life is like watching him attempt to simulate a normal human being with hilariously punchy sentences, but maps of meaning and his absurd recursive diagrams is where you should go if you want to get an idea of what’s really floating around in Peterson’s head.

Also an interesting data point in the “why do people have such different responses to his different works?” question.

From Macruise on the subreddit:

I think one could stress more the distinction between morality and meaning, because it seems central to the specific point Peterson is making. Alexander writes, for instance:

“I think he’s saying – suffering is bad. This is so obvious as to require no justification. If you want to be the sort of person who doesn’t cause suffering, you need to be strong. If you want to be the sort of person who can fight back against it, you need to be even stronger. To strengthen yourself, you’ll need to deploy useful concepts like “God”, “faith”, and “Heaven”. Then you can dive into the whole Western tradition of self-cultivation which will help you take it from there.”

No, he’s not merely saying ‘suffering is bad’. He is saying that it matters. Think about hell. One thinks, from one’s armchair and slippers, that there’s something conceptually fishy about hell, about the very idea of infinite suffering. One wants to say ‘Well, that would just be the new normal, and you’d adapt. It too will become devoid of meaning’. Habituation is such a common experience that you tend to think ‘That too shall pass’ about everything. Even sex, which you’d think would always mean something given its centrality to our animal existence, gets quotidian, just another 17 bus. But pain never gets old. It’s always meaningful. You cannot write The Myth of Sisyphus whilst getting waterboarded, and it’s not merely because the keyboard would get wet.

So it’s not ‘negative utility’ that Peterson is shooting for; it’s ‘negative meaning’. ‘Positive meaning’ is where someone comes along and says ‘Here’s what you should try to achieve in your life because it is inherently meaningful’. ‘Negative meaning’, however, is about the avoidance of suffering, from which certain hypothetical imperatives (subject to our best psychological and medical knowledge) spring.

The good news here is that this notion of ‘negative meaning’ can be divorced from the specifics of Peterson’s injunctions insofar as they’re based in Jungian woo (which they pretty much always are, as far as I can see). The general idea – that pain matters, and you want to live your life to avoid it as much as you can – seems fairly solid as an answer to existential angst.

The bad news is that I think Peterson is wrong about life being suffering. It’s certainly true that experiencing suffering will solve the meaning crisis, and that looking to avoid suffering is a solid plan. But the problem with it as a solution to the problem of meaning is its contingency, made worse by the fact that we live in a world where suffering has no real urgency. The reality is that it is entirely possible now to live your life in a way where true suffering is very delayed. You can make it all the way to 60 without experiencing a serious loss (parent, partner sibling, child), without experiencing painful illness, and without having any existential crisis. These people actually exist, as much as you want to tell yourself otherwise in order to avoid resentment.

If you’re the sort of person who has bipolar disorder and is in a manic phase, it might seem reasonable to say something like ‘life is suffering’. Depressed people too understand that all-powerful-but-transitory truth in their bones. But even those who have had a period of suffering tend to forget how bad it was, and you tend to look back on those periods as times where you were strapped in to the rollercoaster ride, and that you knew all along that it too would pass (even though at the time you didn’t, and that’s what was really awful about it – the sheer, unadulterated despair of thinking ‘this is it’).

Thus, I don’t think negative meaning works. Life isn’t suffering. That’s only a contingent fact that’s true of certain people at certain points of their lives. In general, we have it so good that we invent problems for ourselves and tell ourselves it’s real suffering, and indulge in phoney culture-wars where we can pretend that the damn postmodern, neo-marxist superjews are taking over the world and need to be stopped and that’s that. We all know in our hearts that it’s a load of shit, and the world will carry bumbling on, and that we’ll all die of something insultingly banal, doped to the eyeballs on morphine. We won’t, as our emo selves tell us, be lonely tonight, and lonely tomorrow; we’ll be made comfortable.

Thus, I find myself stuck in the absurd. I know that I’m the sort of being that has to have a purpose, and I don’t believe that one can invent it as Sartre enjoined us to do, but the universe doesn’t owe me an answer, and doesn’t seem disposed to giving me one. I accept that this is, in some sense, bad for me, but one cannot tell oneself that the lines are equal in length and then be able to see it that way.

I appreciate the clarification, but I don’t know if I agree.

Is this the same as just saying that you don’t need a logically coherent life goal, because your biological drives will goad you forward whether you have the goal or not? I’d agree with that one – except maybe in depression, when biological drives suddenly become a lot less convincing. But one of the questions of meaning is “why should I do anything other than follow my immediate biological drives?”

I can imagine Peterson saying (though I don’t know if he would really endorse this) that living a good life is just satisfying your biological drives in an intelligent way. You want to become a good person because then you’re less likely to end up low-status in a way that frustrates your drive for belonging and sex and so on. The argument that moral behavior is really the selfishly best thing to do over the long term is popular, venerable – and way too convenient for me to ever believe.

I find the strongest argument against this to be the question of suicide. Peterson is against it, for vague cliched reasons. But it cuts through all of this “well I guess I have to follow my biological drives” stuff with great finality. You really don’t. If your reason for living doesn’t present the argument against suicide, it’s not much of a reason for living.

From Roe_ on the subreddit:

“Jordan Peterson’s superpower is saying cliches and having them sound meaningful.”

Yes – this is absolutely his super-power. How did he get it?

Because he lives the cliches. The cliches are easy to say, but they’re difficult as fuck to actually implement in your life. And, unless you’ve done so, you’re just talking shit.

As far as I can tell, Peterson has done so. Which is why he “speaks with authority”.

“Rescue your father from the underworld” means precisely this – try to live the old wisdom (ie. the cliches) – which wasn’t easy for our fathers, and sure isn’t any easier now.

This is a good explanation of the “rescue your father from the underworld” thing, and reminds me of my despair about every generation having to learn everything all over again. It’s also why I get fed up with accusations of “that’s not original” or “you’re just reinventing the wheel”.

The part about living the cliches making you able to say them more confidently sounds nice, but is it really true? What would the mechanism for this be? Also, lots of the most charismatic preachers and cult leaders are moral catastrophes. And if this is sort of like saying “he’s so authentic”, I don’t know how that idea interacts with the “actually an alien” point above.

I think of Part I of Liber ABA as being about the process by which prophets become able to speak with confidence, and I sort of wonder whether a lifetime of very successful Jungian analysis might be able to accomplish something similar.

Rocket discusses the inevitable AI angle:

A bit of an aside from Peterson, but relevant to the idea of conflicting Order-Chaos drives and uncertainty-minimization: There was a paper about a month ago about getting AI agents to learn new and interesting behaviours, which touched surprisingly close to that.

Background: Lots of people have tried “reinforcement learning”, where you plop an AI agent in some environment in which it can take actions, and doing certain things nets it a ‘reward’, then you let it do its own thing to try to learn how to maximize that reward. The problem with this is that if reward is rare, as it can be in many scenarios, the AI gets very few opportunities to learn from it. Imagine a game where such an AI only got a reward for finishing a complex level, but no reward for incremental progress: The odds of it coincidentally stumbling into that reward are extremely low, and it will take it a vast, vast amount of time and random button-mashing to join up the dots and figure out which chains of actions got it to that state, and so learn how to get that reward more efficiently in future.

The paper took a different approach: Rather than purely seeking out reward, the AI agent should give itself its own goal, namely to understand how to manipulate the environment it’s in. They achieved that goal (and here’s where the Order-Chaos part comes in) by giving it two components. The first component looks at the input it’s getting (i.e. what the agent can see in the virtual world it’s in) and the action (i.e. what “buttons” the agent is pushing). It then tries to predict what the agent will see next – in other words, to predict the consequences of that action.

The fun part comes from the second component, though, which has an adversarial relationship with the first. It’s the bit that gets to choose actions, and it chooses actions so as to maximally challenge the first component. It’s the Chaos, in this analogy, but it’s also the driver of learning, because if the first component was in control, and the only goal was to minimize uncertainty, it would just sit still and stare at a blank wall. With the two components, though, the AI effectively “learns through play”, starting by moving itself around until the predictor component has mastered that before focusing on objects in the scene, and specifically how to toss them around in maximally confusing ways. (Insert your own analogy to toddlers here)

As well as being an interesting piece of computer-psychology (this is sort of becoming a field lately), the longer-term hope is that such agents will build a model of their environment even in the absence of external reward signals, and so when reward arrives, they’ll have some higher-level abstractions ready to go (“Maybe I need to move the blocks onto the button”) rather than having to painstakingly bootstrap their world-model from raw pixels and rare rewards. In other words, such agents may be far more effective at getting reward than ones that are solely motivated by getting reward.

From Jacek Lach:

“But that’s exactly the problem. I worry Peterson wakes up in the morning and thinks “How can I help add meaning to people’s lives?” and then he says really meaningful-sounding stuff, and then people think their lives are meaningful. But at some point, things actually have to mean a specific other thing. They can’t just mean meaning. “Mean” is a transitive verb. It needs some direct object.”

I would contest that! Yes, for a perfectly rational agent that is the case; you can just look at the effects of your life, see that it is on net positive, and call that ‘meaning’.

But what we really care about when talking about the people, is the *impression of meaning*. It is not important for your wellbeing whether your life is actually rational-meaningful. It is important if your system 1 is satisfied with your life in the certain ways, that it signals this satisfaction in certain ways that we call ‘meaningful’.

As such, it’s not strictly necessary to tie your meaning to some objective measures. If your life feels meaningful, and you have reasonable expectation that this feeling will persist (so presumably – if you accept the assumption that this feeling of meaning evolved, culturally or genetically, for ‘a reason’, i.e. meaning is adaptive – just cheating system1 with drugs is not the right long term answer), and not causing too many negative externalities… Then you’re good?

Obviously meaningfullness is not the only thing one should optimise their life for. But it’s definitely one of the things you should be looking at.

This helps bring into focus one of the things I wasn’t able to discuss in the original review.

When we say we want our lives to be meaningful, are we saying we want our lives to actually have meaning? Or that we want to feel and act as if our lives have meaning?

Before virtuous people like ourselves slam our fists on the table and insist on actual meaning, keep in mind – don’t we have actual meaning already? Most of us would endorse something like helping others as being inherently meaningful – we might add things like creating great art, discovering new scientific truths, and the like. If we could reprogram ourselves like robots, a lot of us would just program ourselves to do whatever helps others or achieves some conception of human values most efficiently, then say “problem solved”. If there’s something left after realizing we can do that, it’s not wondering what the meaning of life is, it’s having some kind of vague emotional will to go on.

This is one reason I respect Jordan Peterson’s pragmatism on a pragmatic level, even as I think it’s a crappy theory of truth. I can imagine a version of him saying (I don’t know if the real one does) “Look, I’m giving you all of these inspirational slogans. You can pick my science and philosophy and mythography apart if you really want, but are you sure you want to do that? You’ll just ruin my attempt to inspire you, and go back to lying on the couch all day wishing you had a reason to get up in the morning.”

Maybe aliens would view it as a tragic quirk of the human psyche that we have to conflate inspiration and truth. Maybe to them, inspiration is just another genre, closer to art or poetry than to an attempt to describe the world as it is. Maybe to them, if there’s an intuitively satisfying explanation of the meaning of life, asking “Is that really the meaning?” or “Is that really true?” would be just as stupid and annoying as nitpicking the lyrics of Ode To Joy. “Ode to Joy says ‘all creatures drink of joy’, but some creatures are unhappy, and joy is not a liquid! Politifact rates your symphony FALSE.”

It’s disappointing that nobody frames it this way: “Inspiring things should be taken as a work of art and not judged on their truth value”. Instead, it’s always some formulation like “Inspiring things are true in a way different from the way factual claims are true”, at which point I have to interject that truth is a useful word and insist on defending its “factually correct” meaning.

Karl Smith on Twitter:

Scott quibbles with Peterson’s tendency to waffle between pragmatism and platitudes. But, this waffling is actually the correct answer.

Maybe I’m misunderstanding this, but I interpret it as sort of related to the above. If we can waffle between factual truth and inspirational things in a way that lets us blur the distinction, so that we feel suitably inspired without necessarily believing false stuff in a way that could ever be pinned down, is this good or bad? Should we interpret this as an intellectual failure, or a cool skill that lets you do more than rationality alone? What about the claim that “rationalists should win”?

My own position is to question whether anyone is really good enough at this not to let their inspirational beliefs bleed over into the factual world. I discuss this a little bit here.

And my position on the larger problem of meaning is to notice that my life always seems really meaningful and great when I have coffee. If I’m going to try to figure out what the actual meaning of life is, in some sort of deep principled way, I’m going to do it with as much attention to Truth as possible. And if I’m going to give myself some emotional hack that lets myself go on and continue finding life worth living, I think caffeine probably has fewer side effects than falsehood, and is just as effective.

And if you don’t respond to caffeine as well as I do, then I think the overall lesson is that the emotional problem of meaning is a basically biological one, that doesn’t connect with the philosophical problem of meaning nearly as much as you think. Get a good psychiatrist and you’ll solve the emotional problem. The philosophical problem might not be solvable, but “helping others” or “creating a positive singularity” or “[your ingroup’s political goals here]” are, though not Perfectly Objectively Grounded, grounded enough that most people don’t really want to question them once the emotional problem is solved.

I find some of Peterson’s non-truth-value-having writing effective in the same sense as caffeine; it makes me more emotionally willing to follow the truths I know I should be following. Since, jokes aside, I can’t literally be drugged 100% of the time, I appreciate that. And maybe the drug would be stronger if I were to swallow his truth-value-having claims too. But that’s not a risk-benefit profile I’m okay with right now.

Userfriedlyyy writes:

My problem with Jordan Peterson’s world view is that he completely ignores the structural problems in the way of self improvement. It’s all well and good to do everything in your power to make yourself stronger and improve your lot in life but it risks turning all of his devotees into fervent believers in the Horatio Alger myth. There will always be a segment of society that can pull themselves up by their own bootstraps; but it is an absolute certainty that it will not work for everyone. Then the problems arise when all those Horatio Alger’s go ‘I did it my self, your moral failings must be why you didn’t so you deserve what you get.’

It just feeds into the individualistic narrative that everything about your life is 100% the result of things you have control over. Just ignore why your wages aren’t going up, just work harder to get paid more. Just ignore that you are making less than your parents when they were your age, just keep working harder to get paid more.

It completely ignores workers banding together to demand better treatment. I’m not even against some of the self improvement stuff but I cringe when it is offered as the sole solution.

Peterson turns Marx on his head and claims that political activism is the opiate of the masses. That is, it’s something people use to make themselves feel sort of vaguely good and self-satisfied, but which prevents them from engaging in the actually important work of spiritual struggle.

My interpretation of him (can’t be sure it’s right) says that he is worried that there are problems with society, and all else being equal he would like people to solve them. But he has the psychoanalyst’s usual worry that anything which is not the Work will be a defense mechanism that people use to avoid the Work. Here again I find a comparison with Lewis helpful (this is from his demon character Screwtape’s advice on how to tempt humans):

“I had not forgotten my promise to consider whether we should make the patient an extreme patriot or an extreme pacifist. All extremes, except extreme devotion to the Enemy [God], are to be encouraged. Not always, of course, but at this period. Some ages are lukewarm and complacent, and then it is our business to soothe them yet faster asleep. Other ages, of which the present is one, are unbalanced and prone to faction, and it is our business to inflame them. Any small coterie, bound together by some interest which other men dislike or ignore, tends to develop inside itself a hothouse mutual admiration, and towards the outer world, a great deal of pride and hatred which is entertained without shame because the “Cause” is its sponsor and it is thought to be impersonal.”

I think Peterson assumes that a psychologically undeveloped person starting to dabble in politics will be eaten alive by various virulent memes, chewed up, and spit out as a Hofferian True Believer in about five minutes. At best they will end up as an never-shutting-up slacktivist who calls people out for not changing their profile picture on Facebook quickly enough, and at worst as some kind of totalitarian. I think he would argue there’s a vicious cycle here – the less psychologically developed you are, the more political activism will destroy you, and the more political activism destroys you, the less likely you are to ever psychologically develop further.

One of his twelve rules, “Set Your House In Perfect Order Before Criticizing The World”, is about this, and doesn’t preclude the possibility of getting involved in politics after you’ve sorted out your own life. I don’t know what this is supposed to mean, since presumably this is an eternal project that is never completed. Clearly Peterson himself thinks he’s at the point where he can participate in politics, so I don’t know.

Do I agree with him here? From a consequentialist point of view, what would it mean to get the least psychologically developed 50% of people out of political activism? If you’re a mistake theorist, it might be great – it takes an equal number of people away from both sides, but raises the quality of discourse. If you’re a conflict theorist, it might be awful – it decreases the number of troops available to the People in their struggle to overcome inertia and fight the Elites.

Rather than try to resolve that, I would just note that “Jordan Peterson saying psychologically underdeveloped people shouldn’t get involved in politics” does not remove the least psychologically developed people from politics. It removes from politics some group of people weighted towards reading Jordan Peterson, being psychologically underdeveloped, and having enough humility to realize that they might be psychologically underdeveloped (which is itself possibly a sign of not being underdeveloped). Whether or not you think this is worth it depends on your opinion of the average Peterson reader.

Philipp questions my use of Lewis:

Scott, I’m surprised no one on this thread (I’ve not looked at the sub-reddit) has taken you up on the C.S. Lewis angle, so I guess, as something of a Lewis fan, that I’ll bite. I’ll state at the outset that I’ve not read Jordan Peterson, though I have watched a few clips of him and read quite a bit about him.

I realize that your talk of C.S. Lewis ‘hating’ Jordan Peterson is hyperbole, but it’s well worth realizing that he was actually a man of broad friendships and, much though he later rejected his youthful flirtations with non- or anti-Christian ideas, a broad personal experience. I mean, the man had variously been a dabbler in the occult (something he rejected with particular ferocity, after one of his spiritualist friends went insane), an idealist, and an atheist materialist, before he became a Christian, in part through the long intellectual influence of George MacDonald and G.K. Chesterton (two very different figures) and in part because of a late-night conversation with Hugo Dyson and the very Catholic J.R.R. Tolkien. Of his friends throughout his life, one of the closest was the anthroposophist Owen Barfield, and another, Charles Williams, was, well, it’s rather hard to figure out, but at least eccentric.

In fact, Peterson sounds, whenever I hear him described, as a kind of prophet of what Lewis called ‘the Tao’ in The Abolition of Man. Lewis is not, so far as I can see it, using the word in the technical sense–this is not Tao as ancient Chinese doctrine–but instead borrowing a word to refer to the basic sense of right and wrong, truth and goodness, and, at least to some degree, beauty and order as reflections thereof. It is, in other words, the natural law, understood not as an arcane system for judging morality in the abstract (the unfortunate impression some less-talented writers leave), but as it is really is: the basic moral order imprinted in the universe itself and in the hearts of mankind.

The central problem of The Abolition of Man (and a central problem also of the novel That Hideous Strength, which was likewise published in 1943 and is very close to it in thought) is the failure of modern education to teach that sense of right and wrong: its tendency to produce ‘men without chests’, or people whose actions are not grounded in the law of nature, in what is true and good, but ultimately only in their own preferences. From this perspective (or so I infer–I don’t have the book in front of me and I don’t remember whether he treats any kind of sophisticated utilitarian thought), utilitarianism, though it purports to judge what is moral, must fail, as it is not ultimately beholden to what is actually right in and of itself. At best, the utilitarian, despite his doctrine, still acts on the basis of his ingrained sense of rightness, and that sense of rightness needs to be shaped to the Tao, or it is nothing.

So, where does this leave Jordan Peterson? Not, I think, straightforwardly on the outside. I doubt Lewis would see in him a proto-Christian, since Peterson knows scripture and rejects Christian orthodoxy, but he is at least–or so it seems–a man with a strong apprehension of right and wrong, who is trying to grow the chests of people, especially young men, whose morality has been stunted. He is, as it were, on the side of the Tao.

To any Christian, and Lewis was one, there is a danger here, of course, and that is that Peterson is nevertheless still not on the side of God. Morality simply is not, because it cannot be, a matter of self-help and self-improvement. There, I think, your own characterization of Lewis goes wrong. He is not part of a “vast humanistic self-cultivation tradition”; though he does draw on ancient philosophy, he is a Christian lay theologian, and therefore believes in human frailty and the need for divine grace. ‘We have not got to try to climb up into spiritual life by our own efforts’, he wrote in Mere Christianity. ‘It has already come down into the human race. If we will only lay ourselves open to the one Man in whom it was fully present, and who, in spite of being God, is also a real man, He will do it in us and for us.’

One cannot speak for a dead man, and others know Lewis’s works better than I do. Nonetheless, I suspect that he would see in Jordan Peterson a man who sees clearly some of the worst errors of his own age, but who can only be a kind of stop on the way to the full truth, who is (of course) Christ himself. Like the world’s moral traditions which, Lewis believed, reflected the truth of the Tao, he might prepare for the Gospel; but making of his teaching a Gospel–treating it, that is, as if it really could make people good enough–would simply be to pave yet another path that leads away from God and real health of the soul.

I acknowledge his greater expertise on Lewis scholarship. I base my concerns on two points. First, although Lewis seems broadly friendly to non-Christians who approximate Christian morality, he is broadly hostile to people who do that and say “And this, not that boring literal Jesus stuff, is the true essence of Christianity”.

Again quoting Screwtape Letters:

“When the humans disbelieve in our [demons’] existence we lose all the pleasing results of direct terrorism and we make no magicians. On the other hand, when they believe in us, we cannot make them materialists and sceptics. at least, not yet. I have great hopes that we shall learn in due time how to emotionalise and mythologise their science to such an extent that what is, in effect, a belief in us, (though not under that name) will creep in while the human mind remains closed to belief in [God]. The “life force”, the worship of sex, and some aspects of Psychoanalysis, may here prove useful. If once we can produce our perfect work — the Materialist Magician, the man, not using, but veritably worshipping, what he vaguely calls “forces” while denying the existence of “spirits” — then the end of the war will be in sight.”

I don’t think Peterson exactly fits this description, but the idea of the pseudo-religious person who uses psychoanalysis but doesn’t quite believe in God is suggestive enough to make me think Lewis would at least be a little uncomfortable.

From St. Rev on Twitter:

I sort of see Peterson as the Meaningness of the normies.

One could do much worse.

Posted in Uncategorized | Tagged | 148 Comments

Are The Amish Unhappy? Super Happy? Just Meh?


Recently on Marginal Revolution: Are the Amish unhappy?

The average levels of life satisfaction [among the Amish] was 4.4; just above the neutral point…the Amish fall lower than members of many other groups. In a study of more than 13 thousand college students from 31 nations, for example, only students from Kenya (whose average life satisfaction was 4.0) scored lower than the Amish (Diener & Diener, 1995).

Sounds like Amish people are quite unhappy. This came as a surprise to me, since I’d heard from Jonah Lehrer and Business Insider that the average Amish person is as happy as the average non-Amish billionaire, proving once and for all that community and old-fashioned values are more important than money:

As an illustration of the striking disconnect between money and happiness, the average life satisfaction of Forbes magazine’s 400 richest Americans was 5.8 on a 7-point scale. Yet the average life satisfaction of the Pennsylvania Amish is also 5.8, despite the fact that their average annual salary is several billion dollars lower.

I actually care about getting this one right. There’s a lot of discussion over whether modern society produces ennui, meaninglessness, atomization, etc – and whether our material wealth has really brought us happiness. The data tend to support a story where more modern and developed countries are happier, but not without some ambiguity and contradiction (a few Latin American countries seem to do better than much richer European ones). But there’s always a concern that the least-developed areas today – like sub-Saharan Africa – are places that have absorbed the worst parts of modernity – like totalitarianism, pollution, and slums – but not the good parts like not-dying-of-cholera. The Amish are about as close as we can get to surveying 1700s-Europe. If we can figure out how happy they are, maybe it would tell us something new about the good life.

Unfortunately, we can’t. This field is full of conflicting data, shifting methods, unreplicated surveys, and – and I didn’t even realize this was a problem it was possible for a field to have – is super-confusing because everyone involved is named Diener. I tried to get the above-cited Diener & Diener 1995, but carelessly bought Diener, Diener & Diener 1995 instead. Cowen’s source is a book by Robert Biswas-Diener, who is comparing a study by Diener & Diener to a study by Biswas-Diener, Vittersø, & Diener, the last of which was do-able only because:

It was both coincidental, and helpful, that my surname — Diener — is also a relatively common Amish surname. This curious point of contact allowed me to introduce myself and my project.

Sure. Whatever.

The Marginal Revolution excerpt comes from Biswas-Diener, Vittersø, & Diener (after this: BDVD). This study was part of the authors’ project to prove that most people are happy. In a previous paper, they had determined that most people in modern societies are happy; in this one, they look at three different traditional societies (Amish, Inughuit Eskimos, and Maasai) to see if they are mostly happy was well. On a 1 – 7 scale, they find:

They don’t formally compare these to the modern societies numbers in the paper. The Biswas-Diener book linked by MR does compare them, finding the Amish are lower than every modern society except Kenya, but there are two important caveats.

First, the Biswas-Diener book compares the survey of Amish (mean age 44) to the Diener & Diener survey of college students in modern society. If college students are happier than 44-year-olds (they are), that’s a potential confounder.

Second, the book notes that the Amish’s self-reported total happiness is lower than their self-reported happiness with any individual facet of their lives. Just look at the table above – their romantic life is a 6.1, their health is a 5.7, their attractiveness is a 5.1 (those beards, right?) – but the two totals, self-satisfaction and life-satisfaction – are 4.2 and 4.4 respectively. It seems like they’re averaging a bunch of numbers and getting an average lower than any of the individual inputs. This is especially bad since modern people tend to do the opposite; report an average happiness higher than their happiness with any individual part of their lives. Biswas-Diener guesses that modern people like to present themselves well (the “have a Facebook feed full of spectacular parties and meticulously-prepared plates of food even when your life is falling apart” effect), and traditional societies are more likely to value humility and treat pride as a sin. As far as I know, there are no studies that have ever measured a non-Amish society using this exact breakdown.

Third, as far as I can tell, the Diener & Diener paper doesn’t actually show Kenyans having the lowest life satisfaction, or Kenyans having a life satisfaction of 4.0:

This is the male table. There’s another one for women, but it’s very similar, and neither the male table, the female table, nor the average of the two tables matches the claim that Kenya is 4.0 or that nobody else is less than 4.4. My guess is all the Dieners share preliminary data with each other, and Biswas-Diener is going off some older or unpublished version of this, but I’m not sure and I might just be missing something.

The public version of Diener & Diener does generally backs up the claim that 4.4 is kind of on the lowish side. But on the other hand, the Maasai number of 5.4 would be the highest one on the whole chart (tied with Finland), which would suggest there’s no clear traditional vs. modern society dichotomy.

So the result of this comparison seems to be “The Amish are above neutral happiness, but less happy than almost any modern society. On the other hand, the Maasai are more happy than almost any modern society. However, a lot of this could be about self-presentation, and the data for modern societies are kind of unclear.”


What about the “Amish are as happy as billionaires” claim?

The billionaire numbers come from Diener’s Happiness Of The Very Wealthy and seem to check out. It finds billionaires are happier, though not vastly happier, than everyone else. Billionaires have an average happiness of 5.8 – remember, the highest national sample above was Finland at 5.4.

This time, the Amish numbers come from Diener and Seligman’s Beyond Money: Toward An Economy Of Well-Being, which presents this graph:

It doesn’t explain where the table comes from, but the use of the same three traditional societies (Maasai, Inughuit, and Amish) suggest the BDVD paper above (which shares one of its Dieners with the Beyond Money paper). But its numbers for all three groups are very different, and its headline result – the one about Pennsylvania Amish as happy as billionaires – isn’t in the BDVD paper at all and has no citation.

The only clue to this discrepancy is that the Beyond Money paper cites the BDVD paper as “manuscript submitted for publication”. Perhaps the peer reviewers made comments which caused BDVD to drop their Pennsylvania Amish result and analyze some of the other results differently? In any case, since the Amish = billionaires data seems to have been quietly dropped by the authors, we probably shouldn’t put much stock in it.

Does this mean we should default to the “Amish less happy than anyone else except Kenyans” data? I say no. If the study authors got data consistent with “Amish are as happy as billionaires”, and later on it got changed to “Amish less happy than anyone”, plus they changed the Kenyan stuff around too, then I really don’t care about the “correction”, this research isn’t rigorous enough, or fixed enough, to convince me of anything.

All of this is from long before the replication crisis started improving methodologies, and I don’t trust it enough to consider it worth trying to smush everything together into a coherent whole. Sweep the billionaires/Kenyans issue under the rug, and there are still too many questions. Is there really a gulf between Pennsylvania Amish and Illinois Amish as vast as that between Swedes and Calcutta slum-dwellers? Are the Maasai really so much happier than modern societies, even as the Amish are so much less happy? Is there really an entire scientific field where everyone is named “Diener”? We just don’t know.

Posted in Uncategorized | Tagged | 148 Comments