SELF-RECOMMENDING!

A Failure, But Not Of Prediction

I.

Vox asks What Went Wrong With The Media’s Coronavirus Coverage? They conclude that the media needs to be better at “not just saying what we do know, but what we don’t know”. This raises some important questions. Like: how much ink and paper is there in the world? Are we sure it’s enough? But also: how do you become better at saying what you don’t know?

In case you’ve been hiding under a rock recently (honestly, valid) the media not only failed to adequately warn its readers about the epidemic, but actively mocked and condescended to anyone who did sound a warning. Real Clear Politics has a list of highlights. The Vox tweet saying “Is this going to be a deadly pandemic? No.” Washington Post telling us in February “Why we should be wary of an aggressive government reponse to coronavirus (it might “scapegoat marginalized populations”). The Daily Beast complaining that “coronavirus, with zero American fatalities, is dominating headlines, while the flu is the real threat”. The New York Times, weighing in with articles like “The pandemic panic” and “Who says it’s not safe to travel to China”. The constant attempts to attribute “alarmism” over the virus to anti-Chinese racism. Etc, etc, etc.

One way people have summed this up is that the media (and the experts they relied on) did a terrible job predicting what would happen. I think this lets them off too easy.

Prediction is very hard. Nate Silver is maybe the best political predicter alive, and he estimated a 29% chance of Trump winning just before Trump won. UPenn professor Philip Tetlock has spent decades identifying “superforecasters” and coming up with complicated algorithms for aggregating their predictions, developing a prediction infrastructure that beats top CIA analysts, but they estimated a 23% chance Britain would choose Brexit just before it happened. This isn’t intended to criticize Silver or Tetlock. I believe they’re operating at close to optimum – the best anyone could possibly do with the information that they had. But the world is full of noise, and tiny chance events can have outsized effects, and there are only so many polls you can scrutinize, and even geniuses can only do so well.

Predicting the coronavirus was equally hard, and the best institutions we had missed it. On February 20th, Tetlock’s superforecasters predicted only a 3% chance that there would be 200,000+ coronavirus cases a month later (there were). The stock market is a giant coordinated attempt to predict the economy, and it reached an all-time high on February 12, suggesting that analysts expected the economy to do great over the following few months. On February 20th it fell in a way that suggested a mild inconvenience to the economy, but it didn’t really start plummeting until mid-March – the same time the media finally got a clue. These aren’t empty suits on cable TV with no skin in the game. These are the best predictive institutions we have, and they got it wrong. I conclude that predicting the scale of coronavirus in mid-February – the time when we could have done something about it – was really hard.

I don’t like this conclusion. But I have to ask myself – if it was so easy, why didn’t I do it? It’s easy to look back and say “yeah, I always secretly knew it would be pretty bad”. I did a few things right – I started prepping half-heartedly in mid-February, I recommended my readers prep in early March, I never criticized others for being alarmist. Overall I give myself a solid B-. But if it was so easy, why didn’t I post “Hey everyone, I officially predict the coronavirus will be a nightmarish worldwide pandemic” two months ago? It wouldn’t have helped anything, but I would have had bragging rights forever. For that matter, why didn’t you post this – on Facebook, on Twitter, on the comments here? You could have gone down in legend, alongside Travis W. Fisher, for making a single tweet. Since you didn’t do that (aside from the handful of you who did – we love you, Balaji) I conclude that predicting it was hard, even for smart and well-intentioned people like yourselves.

Does that mean we can’t put everyone’s heads on spikes outside the Capitol Building as a warning for future generations? I would be very disappointed if it meant that. I think we can still put heads on spikes. We just have to do it for more subtle, better-thought-out reasons.

II.

I used to run user surveys for a forum on probabilistic reasoning

(I promise this will become relevant soon)

A surprising number of these people had signed up for cryonics – the thing where they freeze your brain after you die, in case the future invents a way to resurrect frozen brains. Lots of people mocked us for this – “if you’re so good at probabilistic reasoning, how can you believe something so implausible?” I was curious about this myself, so I put some questions on one of the surveys.

The results were pretty strange. Frequent users of the forum (many of whom had pre-paid for brain freezing) said they estimated there was a 12% chance the process would work and they’d get resurrected. A control group with no interest in cryonics estimated a 15% chance. The people who were doing it were no more optimistic than the people who weren’t. What gives?

I think they were actually good at probabilistic reasoning. The control group said “15%? That’s less than 50%, which means cryonics probably won’t work, which means I shouldn’t sign up for it.” The frequent user group said “A 12% chance of eternal life for the cost of a freezer? Sounds like a good deal!”

There are a lot of potential objections and complications – for one thing, maybe both those numbers are much too high. You can read more here and here. But overall I learned something really important from this.

Making decisions is about more than just having certain beliefs. It’s also about how you act on them.

III.

A few weeks ago, I wrote a blog post on face masks. It reviewed the evidence and found that they probably helped prevent the spread of disease. Then it asked: how did the WHO, CDC, etc get this so wrong?

I went into it thinking they’d lied to us, hoping to prevent hoarders from buying up so many masks that there weren’t enough for health workers. Turns out that’s not true. The CDC has been singing the same tune for the past ten years. Swine flu, don’t wear masks. SARS, don’t wear masks. They’ve been really consistent on this point. But why?

If you really want to understand what happened, don’t read any studies about face masks or pandemics. Read Smith & Pell (2003), Parachute Use To Prevent Death And Major Trauma Related To Gravitational Challenge: Systematic Review Of Randomized Controlled Trials. It’s an article in the British Journal Of Medicine pointing out that there have never been any good studies proving that parachutes are helpful when jumping out of a plane, so they fail to meet the normal standards of evidence-based medicine. From the Discussion section:

It is a truth universally acknowledged that a medical intervention justified by observational data must be in want of verification through a randomised controlled trial. Observational studies have been tainted by accusations of data dredging, confounding, and bias. For example, observational studies showed lower rates of ischaemic heart disease among women using hormone replacement therapy, and these data were interpreted as advocating hormone replacement for healthy women, women with established ischaemic heart disease, and women with risk factors for ischaemic heart disease. However, randomised controlled trials showed that hormone replacement therapy actually increased the risk of ischaemic heart disease, indicating that the apparent protective effects seen in observational studies were due to bias. Cases such as this one show that medical interventions based solely on observational data should be carefully scrutinised, and the parachute is no exception.

Of course this is a joke. It’s in the all-joke holiday edition of BMJ, and everyone involved knew exactly what they were doing. But the joke is funny because it points at something true. It’s biting social commentary. Doctors will not admit any treatment could possibly be good until it has a lot of randomized controlled trials behind it, common sense be damned. This didn’t come out of nowhere. They’ve been burned lots of times before by thinking they were applying common sense and getting things really wrong. And after your mistakes kill a few thousand people you start getting really paranoid and careful. And there are so many quacks who can spout off some “common sense” explanation for why their vitamin-infused bleach or colloidal silver should work that doctors have just become immune to that kind of bullshit. Multiple good RCTs or it didn’t happen. Given the history I think this is a defensible choice, and if you are tempted to condemn it you may find this story about bone marrow transplants enlightening.

But you can take this too far. After highlighting the lack of parachute RCTs, the paper continues:

Only two options exist. The first is that we accept that, under exceptional circumstances, common sense might be applied when considering the potential risks and benefits of interventions. The second is that we continue our quest for the holy grail of exclusively evidence based interventions and preclude parachute use outside the context of a properly conducted trial. The dependency we have created in our population may make recruitment of the unenlightened masses to such a trial difficult. If so, we feel assured that those who advocate evidence based medicine and criticise use of interventions that lack an evidence base will not hesitate to demonstrate their commitment by volunteering for a double blind, randomised, placebo controlled, crossover trial.

Did you follow that? For a good parachute RCT, half the subjects would have to jump out of a plane wearing a placebo parachute. The authors suggest maybe we enlist doctors who insist too stringently on RCTs over common sense for this dubious honor.

(good news, though, a parachute RCT did eventually get done)

Sometimes good humor is a little too on the nose, like those Onion articles that come true a few years later. The real medical consensus on face masks came from pretty much the same process as the fake medical consensus on parachutes. Common sense said that they worked. But there weren’t many good RCTs. We couldn’t do more, because it would have been unethical to deliberately expose face-mask-less people to disease. In the end, all we had were some mediocre trials of slightly different things that we had to extrapolate out of range.

Just like the legal term for “not proven guilty beyond a reasonable doubt” is “not guilty”, the medical term for “not proven to work in several gold-standard randomized controlled trials” is “it doesn’t work” (and don’t get me started on “no evidence”). So the CDC said masks didn’t work.

Going back to our diagram:

Goofus started with the position that masks, being a new idea, needed incontrovertible proof. When the few studies that appeared weren’t incontrovertible enough, he concluded that people shouldn’t wear masks.

Gallant would have recognized the uncertainty – based on the studies we can’t be 100% sure masks definitely work for this particular condition – and done a cost-benefit analysis. Common sensically, it seems like masks probably should work. The existing evidence for masks is highly suggestive, even if it’s not utter proof. Maybe 80% chance they work, something like that? If you can buy an 80% chance of stopping a deadly pandemic for the cost of having to wear some silly cloth over your face, probably that’s a good deal. Even though regular medicine has good reasons for being as conservative as it is, during a crisis you have to be able to think on your feet.

IV.

But getting back to the media:

Their main excuse is that they were just relaying expert opinion – the sort of things the WHO and CDC and top epidemiologists were saying. I believe them. People on Twitter howl and gnash their teeth at this, asking why the press didn’t fact-check or challenge those experts. But I’m not sure I want to institute a custom of journalists challenging experts. Journalist Johann Hari decided to take it upon himself to challenge psychiatric experts, and wrote a serious of terrible articles and a terrible book saying they were wrong about everything. I am a psychiatrist and I can tell you he is so wrong that it is physically painful to read his stuff (though of course I would say that…). Most journalists stick to assuming the experts know more about their subject of expertise than they do, and I think this is wise. The role of science journalists is to primarily to relay, explain, give context to the opinions of experts, not to try to out-medicine the doctors. So I think this is a good excuse.

But I would ask this of any journalist who pleads that they were just relaying and providing context for expert opinions: what was the experts’ percent confidence in their position?

I am so serious about this. What fact could possibly be more relevant? What context could it possibly be more important to give? I’m not saying you need to have put a number in your articles, maybe your readers don’t go for that. But were you working off of one? Did this question even occur to you?

Nate Silver said there was a 29% chance Trump would win. Most people interpreted that as “Trump probably won’t win” and got shocked when he did. What was the percent attached to your “coronavirus probably won’t be a disaster” prediction? Was it also 29%? 20%? 10%? Are you sure you want to go lower than 10%? Wuhan was already under total lockdown, they didn’t even have space to bury all the bodies, and you’re saying that there was less than 10% odds that it would be a problem anywhere else? I hear people say there’s a 12 – 15% chance that future civilizations will resurrect your frozen brain, surely the risk of coronavirus was higher than that?

And if the risk was 10%, shouldn’t that have been the headline. “TEN PERCENT CHANCE THAT THERE IS ABOUT TO BE A PANDEMIC THAT DEVASTATES THE GLOBAL ECONOMY, KILLS HUNDREDS OF THOUSANDS OF PEOPLE, AND PREVENTS YOU FROM LEAVING YOUR HOUSE FOR MONTHS”? Isn’t that a better headline than Coronavirus panic sells as alarmist information spreads on social media? But that’s the headline you could have written if your odds were ten percent!

So:

I think people acted like Goofus again.

People were presented with a new idea: a global pandemic might arise and change everything. They waited for proof. The proof didn’t arise, at least at first. I remember hearing people say things like “there’s no reason for panic, there are currently only ten cases in the US”. This should sound like “there’s no reason to panic, the asteroid heading for Earth is still several weeks away”. The only way I can make sense of it is through a mindset where you are not allowed to entertain an idea until you have proof of it. Nobody had incontrovertible evidence that coronavirus was going to be a disaster, so until someone does, you default to the null hypothesis that it won’t be.

Gallant wouldn’t have waited for proof. He would have checked prediction markets and asked top experts for probabilistic judgments. If he heard numbers like 10 or 20 percent, he would have done a cost-benefit analysis and found that putting some tough measures into place, like quarantine and social distancing, would be worthwhile if they had a 10 or 20 percent chance of averting catastrophe.

V.

This is at risk of getting too depressing, so I want to focus on some people who deserve recognition for especially good responses.

First, a bunch of generic smart people on Twitter who got things exactly right – there are too many of these people to name, but Scott Aaronson highlights “Bill Gates, Balaji Srinivasan, Paul Graham, Greg Cochran, Robin Hanson, Sarah Constantin, Eliezer Yudkowsky, and Nicholas Christakis.” None of these people (except Greg Cochran) are domain experts, and none of them (except Greg Cochran) have creepy oracular powers. So how could they have beaten the experts? Haven’t we been told a million times that generic intelligence is no match for deep domain knowledge?

I think the answer is: they didn’t beat the experts in epidemiology. Whatever probability of pandemic the experts and prediction markets gave for coronavirus getting really bad, these people didn’t necessarily give a higher probability. They were just better at probabilistic reasoning, so they had different reactions to the same number. There’s no reason generic why smart people shouldn’t be better at probabilistic reasoning then epidemiologists. In fact, this seems exactly like the sort of thing generic smart people might be.

Zeynep Tufekci is an even clearer example. She’s a sociologist and journalist who was writing about how it was “our civic duty” to prepare for coronavirus as early as February. She was also the first mainstream media figure to spread the word that masks were probably helpful.

Totally at random today, reading a blog post on the Mongol Empire like all normal people do during a crisis, I stumbled across a different reference to Zeynep. In a 2014 article, she was sounding a warning about the Ebola pandemic that was going on at the time. She was saying the exact same things everyone is saying now – global institutions are failing, nobody understands exponential growth, travel restrictions could work early but won’t be enough if it breaks out. She quoted a CDC prediction that there could be a million cases by the end of 2014. “Let that sink in,” she wrote. “A million Ebola victims in just a few months.”

In fact, this didn’t happen. There were only about 30,000 cases. The virus never really made it out of Liberia, Sierra Leone, and Guinea.

I don’t count this as a failed prediction on Zeynep’s part. First of all, because it could have been precisely because of people like her sounding the alarm that the epidemic was successfully contained. But more important, it wasn’t really a prediction at all. Her point wasn’t that she definitely knew this Ebola pandemic was the one that would be really bad. Her point was that it might be, so we needed to prepare. She said the same thing when the coronavirus was just starting. If this were a game, her batting average would be 50%, but that’s the wrong framework.

Zeynep Tufecki is admirable. But her admirable skill isn’t looking at various epidemics and successfully predicting which ones will be bad and which ones will fizzle out. She can’t do that any better than anyone else. Her superpower is her ability to treat something as important even before she has incontrovertible evidence that it has to be.

And finally, Kelsey Piper. She wrote a February 6th article saying:

The coronavirus killed fewer people than the flu did in January. But it might kill more in February — and unlike the flu, its scope and effects are poorly understood and hard to guess at. The Chinese National Health Commission reports 24,324 cases, including 3,887 new ones today. There are some indications that these numbers understate the situation, as overwhelmed hospitals in Wuhan only have the resources to test the most severe cases. As of Tuesday, 171,329 people are under medical observation because they’ve had close contact with a confirmed case.

It is unclear whether China will be able to get the outbreak under control or whether it will cause a series of epidemics throughout the country. It’s also unclear whether other countries — especially those with weak health systems — will be able to quickly identify any cases in their country and avoid Wuhan-scale outbreaks.

The point is, it’s simply too soon to assert we’ll do well on both those fronts — and if we fail, then the coronavirus death toll could well climb up into the tens of thousands. It also remains to be seen if vaccines or effective antiviral treatments will be developed. That’s just far too much uncertainty to assure people that they have nothing to worry about. And misleadingly assuring people that there’s nothing to worry about can end up doing harm.

“Instead of deriding people’s fears about the Wuhan coronavirus,” Sandman, the communications expert, writes, “I would advise officials and reporters to focus more on the high likelihood that things will get worse and the not-so-small possibility that they will get much worse.”

She concluded that “the Wuhan coronavirus likely won’t be a nightmare pandemic, but that scenario is still in play”, and followed it up with an article urging people to prepare by buying essential food and supplies.

If we interpret her “likely won’t be a nightmare pandemic” sentence as a prediction, she got the prediction wrong. Like Zeynep, she has no special ability to predict whether any given disease will end in global disaster. But that didn’t matter! She gave exactly the correct advice to institutions (prepare for a worst-case scenario, stop telling people not to panic) and exactly the correct advice to individuals (start prepping). When you’re good enough at handling uncertainty, getting your predictions exactly right becomes almost superfluous.

The Vox article says the media needs to “say what it doesn’t know”. I agree with this up to a point. But they can’t let this turn into a muddled message of “oh, who knows anything, whatever”. Uncertainty about the world doesn’t imply uncertainty about the best course of action! Within the range of uncertainty that we had about the coronavirus this February, an article that acknowledged that uncertainty wouldn’t have looked like “We’re not sure how this will develop, so we don’t know whether you should stop large gatherings or not”. It would have looked like “We’re not sure how this will develop, so you should definitely stop large gatherings.”

I worry that the people who refused to worry about coronavirus until too late thought they were “being careful” and “avoiding overconfidence”. And I worry the lesson they’ll take away from this is to be more careful, and avoid overconfidence even more strongly.

Experts should think along these lines when making their recommendations, but if they don’t, the press should think along them as part of its work of putting expert recommendations in context. I think Kelsey’s article provides an shining example of what this should look like.

Maybe other people got this right too. I’m singling out Kelsey because of a personal connection – I met her through the same probabilistic reasoning forum where I did my cryonics survey years ago. I don’t think this is a coincidence.

[Related: Book Review: The Precipice; Two Kinds Of Caution]

This entry was posted in Uncategorized and tagged , , . Bookmark the permalink.

894 Responses to A Failure, But Not Of Prediction

  1. Mitch Lindgren says:

    Prediction is very hard. Nate Silver is maybe the best political predicter alive, and he estimated a 29% chance of Trump winning just before Trump won. Berkeley professor Philip Tetlock has spent most of his life identifying the best predictors in the world and coming up with complicated algorithms for aggregating their predictions, developing a forecasting infrastructure that beats top CIA analysts, but his superforecasters estimated a 23% chance Britain would choose Brexit just before they did. This isn’t intended to criticize Silver or Tetlock. I believe they’re operating at close to optimum – the best anyone could possibly do with the information that they had. But the world is full of noise, and tiny chance events can have outsized effects, and there are only so many polls you can scrutinize, and even geniuses can only do so well.

    Although you don’t intend to criticize them, this is still phrased in a way that implies Silver and Tetlock’s predictions were wrong. But what should their predictions have been? 100%, since both events actually did happen? Surely not. If we could go back and run the election 1000 times over with small random alterations each time, maybe Trump would only win 29% of the time. But of course we can’t do that, so there’s no way to know.

    • Scott Alexander says:

      I think you’re saying the same thing I am – performing at optimum doesn’t always look like assigning 100% (or even 50%+) to all true statements. Is there a better way I could have written this?

      • Mitch Lindgren says:

        Ah, sorry, it wasn’t clear to me that that’s what you meant. I think what you just wrote in this comment is clearer, albeit perhaps less elegant.

      • matkoniecz says:

        Maybe explicitly compare them with other claiming that Trump has 1% chance of winning?

        “And they did better than many others, for example XYZ who predicted that ABC”.

        • Act_II says:

          I partly agree with this. Forecasters who gave him a 1% or so like The Upshot or PEC pretty blatantly had wrong models (and the excuse of “Sometimes things with a 1% probability happen” rings hollow), whereas I actually chalk up the 29% as a win for Nate Silver’s calibration. But I think the focus of Scott’s post isn’t on blogs like The Upshot. They had a bad model, sure, but at least they tried to acknowledge some uncertainty — their reasoning pattern is still Gallant-like. The post is aimed at pundits (and average people) who tried to make claims with certainty — by rounding down those forecasts or going with their gut or whatever.

          Here’s an example. In fall 2016, my girlfriend got an Uber ride from a guy who turned out to be a big Trump supporter. He talked the whole ride about how he was certain Trump was going to win. As it turns out, he was right! But I’m still not going to put much stock in this Uber driver’s predictive ability — certainly not as much as 538, and not even as much as The Upshot. He didn’t draw his conclusion through principled analysis of the evidence; he just knew what he wanted to be true, declared it fact, and his desires happened to align with reality in this case. It’s not so much about the outcome as it is the process.

          • k10293 says:

            The Upshot was one of the more reasonable models out there. They gave Trump a 15% chance of winning. You might be confusing it with something like the HuffingtonPost model.

          • wiserd says:

            Considering that the news is not just interested in prediction, but in actual political influence it seems relevant to ask; how bad does a prediction have to be before we say it wasn’t make with a good-faith attempt to predict, but rather to influence? i.e. Are we looking at problematic predictions through the lens of mistake theory or conflict theory?

        • LudwigNagasena says:

          Just because a 1% event happened it doesn’t mean that the event had a higher probability.

      • meh says:

        the reference to tiny chance events makes it sound as though they missed a black swan type event. but in reality, a 29% prediction means you think the thing will occur pretty often

      • random_eddie says:

        >Is there a better way I could have written this?

        Yes. You could have characterized these as successful predictions, rather than saying “prediction is hard” and using them as examples of failed predictions.

        Predicting CV19 was not hard. I did it. You did it. Anyone who was paying attention and thinking even a little bit clearly could have done it. What was hard was resisting the massive narrative which was literally built on lies and denial, was propagated by officials and the media, and was self-reinforcing due to status games. This narrative is why so many people went on record making predictions that assigned a zero percent probability to a pandemic.

        You gave yourself a B-, but I think you deserved an A+. Things were uncertain for a long time (and they still are!), but as more and more evidence emerged you continually revised your assessment of the risks, took appropriate actions, and offered appropriate advice. This is exactly what Gallant does. This is what a successful prediction looks like.

        … which in turn is essentially the thesis of your entire post, except you stop short of saying “good predictions are hedged, bad predictions are absolute (whether proven correct or not)”.

        • albatross11 says:

          It would be interesting to see what fraction of doctors, epidemiologists, virologists, etc., took concrete steps to prepare, compared to some comparable set of educated middle-class-and-up professionals. I don’t know how accurate it would be, but a question like “how many rolls of toilet paper were in your house on March 15” would be interesting–it would positively correlate with people actually taking action toward preparing at a personal level.

          • Edward Scizorhands says:

            I think predicting TP hoarding is putting the bar too high. I’m pretty critical of the White House response to all this, but there are dozens of things that someone trying to predict the future would worry about instead, and while you could probably predict panic buying of something, if people were predicting the specific thing that gets panic bought they would put things like hand sanitizer and water.

        • Characterizing them as successful predictions isn’t quite right either. The fact is that we can’t judge from the outcome of one event whether a particular forecast that that event happened was too high or too low or exactly right. Sometimes we can get a sense and update accordingly to say “I think I was over-confident” or “I think I was underconfident,” but usually that is a small update, and never can we say with certainty that a forecast was “right” or “wrong” based on the outcome.

          Relevant: See my comment quoting Tetlock on the “wrong-side-of-maybe fallacy” elsewhere in this comment section.

        • Quixote says:

          I work in risk management at a major bank. By early Feb, everyone in our department had stocked up on non perishable goods, and agreed on the number of cases in the tri-state area when we would switch to working from home (this was on a team basis, the bank wide work from home policy was made via a different set of criteria). So just to say, if you are wondering how various professions did, professional risk mangers got it right.

    • Paul Zrimsek says:

      Well, I don’t know. Based on his familiar New Year’s predictions, if it had been Scott doing Silver’s job, he would presumably have phrased it as “Clinton wins (71% confidence)”– and then come back after the election and reprinted it with a strike-through, meaning “I got that one wrong”. It seems to me that Scott’s approach is just clinging to the edge of what it means to be making a prediction at all, so if there’s any meaningful distinction between what he’s doing and what Silver’s doing, the correct reaction is not “We can’t say Silver’s prediction was wrong”; it’s “Whatever it is that Silver was doing, he wasn’t predicting the election result.”

      • Cliff says:

        The point of that exercise is that if 70% of the predictions that Scott assigns 70% probability actually happen, then his predictions are very accurate.

        • John Schilling says:

          Right. “I got that one wrong” means one thing in the context of calibrating general predictive accuracy, and an entirely different thing in the context of a single actionable prediction. Probably “wrong” is the wrong word to use in the former case, but it’s common usage in a way “the predicted thing did not occur” isn’t.

          From Hillary’s edge in the popular vote count and the particularly close totals in the swing states, Trump 71% to lose looks like it was pretty close to the correct non-wrong prediction.

        • alawisgreen says:

          A nit:

          If 70% of your predictions marked 70% confidence come true, then that doesn’t imply accuracy. It implies calibration, meaning that you are not over-confident or under-confident on average. If an expert is poorly calibrated, it means you can do better than them just by copying their predictions and changing each of them to be slightly more confident or slightly less confident.

          To make an absurd example, you could come up with a list of 60 very likely things (e.g. The S&P 500 will trade at above 100 at the end of the year) and 40 very unlikely things (e.g. an asteroid will destroy the planet on Tuesday), and assign all of them a 60% chance. You would end up with perfect calibration, but the predictions would be useless.

      • Eponymous says:

        We’ve discussed interpretations of Scott’s prediction exercises ad nauseum. In the absence of a measure of how “difficult” the questions are, the only thing we can confidently grade him on is calibration (except for 50%). Labeling predictions as “right” and “wrong” is strictly about which bucket to put them in for the purpose of calculating calibration.

        If we think we know how “difficult” the questions are, we can also grade him on accuracy — the best way would be to have lots of people predict the same set of questions, and use standard scoring methods to compare them. But barring that, we have to use our judgement about how difficult the questions are.

        I think part of the point of Scott’s post is that there’s a third quality we should elevate alongside calibration and accuracy — actually taking the optimal action in response to our probability estimates. (Incidentally, I think NN Taleb makes a similar point).

    • mitv150 says:

      I was thinking this exact thing while reading Scott’s post.

      If I am rolling a die, and I say there is only a 1/6 chance of rolling a six, and then I do, my prediction wasn’t wrong. We know that prediction is pretty close to exactly right, despite it giving a relatively low probability of the eventual outcome.

      Some events are different, however, or at least the timing is. If we magically reran the election 1000 times over, starting from the morning of, Trump probably still wins nearly every time. At that point, almost all of the variability was baked into the cake.

      Giving Trump a 23% chance of winning the morning of the election, by analogy, is like giving the die a 1/6 chance of rolling a six just milliseconds before it stops moving.

      It’s basically impossible to know whether a prediction of a complex event is “wrong” or “right” simply by whether the event happened. Complex world events happen just once. It may be perfectly accurate to say that some event has only a 10% chance of happening, even if it does happen.

      It’s not clear to me how to think about the accuracy of predictions in circumstances like this.

      • Anteros says:

        Uncanny – I wanted to make the same point as you, with the same example of the die, and with your final sentence about not knowing how to think about predictions in circumstances like this.

        The result of the 2016 election tells us nothing about the accuracy of Nate Silver’s prediction. It may well have been the most accurate prediction in the history of the Universe. We simply have no idea.

        • Purplehermann says:

          I kind of disagree with this.
          A few smart people around me thought he had a decent chance from the start, and one actually bet money he’d win.

          • I lost money betting that Trump would fail to win the GOP nomination. That loss really jolted me, not so much because of the money but because I had been literally laughing out loud when I placed the wager. I assumed that Donald Trump winning a major party nomination was such a ridiculous proposition that I was taking a sucker’s cash.

            Having been amazed (and a bit embarrassed), I then opened my eyes/mind and did some listening to some Trump supporters of my acquaintance. By the fall those inputs plus the events of the campaign itself had made me much less certain that he’d lose in November. I was still surprised by that outcome but only mildly, was in my own mind putting his chances up at maybe the 40% range.

            What that whole experience reinforced for me was that proposing that people put some actual cash money down on something they are certain about can be clarifying for them. Even a small amount — it’s not the money that matters it’s the act of wagering. It can cut through gut reactions and make you actually take a look and think about it, at least a little.

      • iprayiam says:

        The problem is the conflation of the ideas about prediction and probability. If you flip a coin over and over, and every time, I give it a 50% chance of heads, you don’t get to say that I had a 100% prediction success at the end of the game. That’s meaningless.

        It’s also the problem with giving Scott a perfect score for getting 7 / 10 70% chance predictions.

        Giving odds to real world occurrences, is a case of the map not being the terrain. For a give observed event, X. X either happens or it doesn’t.

        Giving Trump a 29% chance of winning, basically means that if we picked 100 random universes out of a bag, 29 of them would have a Trump win and 71% would not. That’s nonsensical proposition. Probability works best with means and sample distributions and such, none of which works with a simple bit (yes/ no) event.

        A better way to think about the accuracy applied back to the real world is that giving Trump a 29% chance = “I predict Hilary to win, with roughly 5:2 odds”. Somebody who made that bet would “lose” significantly, but not so bad as the buddy who put down “100:1” odds.

        That;s really the only way to judge a prediction of a singe occurrence in the real world, as a bet with particular odds. Silver or Scott could overall be in the money on their bets, but it is nonsensical to take a single one and say they didn’t miss it because they didn’t put 0% on it.

        • Randy M says:

          If you flip a coin over and over, and every time, I give it a 50% chance of heads, you don’t get to say that I had a 100% prediction success at the end of the game. That’s meaningless.

          But we can say that you perfectly understand the probability distribution of coin flipping, a very useful bit of knowledge should you acquire it in other domains.

          X either happens or it doesn’t.

          But you can invest various amounts of resources in mitigating or preparing for X, and if you can accurately know the likelihood of various categories of events (not easy!) you could optimize your expenditure and minimize risk.

        • eccdogg says:

          More importantly to me with regard to Silver is that from my memory he gave Trump higher odds than what were offered in betting markets.

          So if he were an active trader he would have been buying Trump.

          Also the narrative by Silver leading up to the election was very good in that he wrote that all there needs to be is a 2% difference between the polls and the votes for Trump to win and those happen frequently. He also said that if Trump were to win it would be due to several states in a region all breaking the same way for him compared to the polls.

      • Edward Scizorhands says:

        The comparison to a die roll should be “someone has already rolled a die, and we can gather some partial information about it in order to make predictions about what’s on top.”

        Say these are the faces:
        ⚀ ⚁ ⚂ ⚃ ⚄ ⚅

        and we can only see one face, that is on the side, on the bottom-left corner, and there definitely IS NOT a pip there. That gives us information about what face is on top. The result will not change at all — it’s fixed. But we can try to gather more and more data by glimpsing more and more locations on the die.

      • matkoniecz says:

        Giving Trump a 23% chance of winning the morning of the election, by analogy, is like giving the die a 1/6 chance of rolling a six just milliseconds before it stops moving.

        Seems to be a poor comparison, someone watch dice has access to all relevant data.

        Maybe add “while blindfolded and assessing based solely on sound of a rolling dice”?

        Or “but you do not see dice directly, you rely on information provided by an excited 5 year old”?

      • real_human9000 says:

        My intuition is that the “best” predictor would always be right with 100% confidence, and would accomplish this by taking into account all the relevant data and computing the outcome with absolute certainty.

        We operate in a world without data. We don’t know the slope of the surface or elasticity of the die or the internal state of the die roller’s brain. So we accept the best practical approximation, which is a 1/6 chance for each face.

        Same goes for elections – the best predictor would have nailed Trump’s victory in 2016. Absent that, people like Nate predict based on polls and economic fundamentals. In the long run, his prediction methods may give the best results that can be accomplished using the available data, we of course have no way of knowing. But guessing 23% to Trump hurts his model relative to someone who assigned a higher percentage (even if his model is well calibrated).

        (This all assumes that quantum randomness doesn’t affect things on a macro scale)

      • zipok says:

        Is there not a crucial distinction between a guess at the probability of a certain event, and your confidence in that guess? Both are, at base, probabilities, but of different things. I will happily guess the probability of any event you name with 0% confidence. A mix of a Bayesian and a statistical learning perspective seems warranted here.

        Indeed, one thing that has always confused me about predictions like Nate Silver’s is, “when you put a 29% chance on Trump’s election, how confident are you about that number?” Of course, I fully realize that this is a very difficult question, both philosophically and technically, especially without strong assumptions about your distribution and your statistical model. But a low confidence prediction is really quite useless.

        In other words, my estimator for say the probability of some event is itself a random variable. Its distribution encodes confidence. What does it look like, at least roughly? Can I provide some concentration guarantees, some tail bounds, something?

        If I think back to February, the key issue in deciding how to react to the incoming news was that I had very low confidence in any prediction I might make — at least without very significant time commitment and research. So realistically the only option was “welp, guess we’ll see”. I think some version of this is what happened with a lot of people and the media.

    • Matt M says:

      It might also be worth pointing out that, from what I remember, Silver was giving Trump a better chance than most of his peers were… It wasn’t exactly rare to find very serious pundits who were telling us that Trump had less than a 1% chance of winning…

      • Capt. Adama says:

        Still, I think this species of prediction/forecasting, namely Silver’s probabilistic framing of one-off events is highly overvalued. Practically speaking, confidence values anywhere between 10% and 90% amount to a fairly decent hedge. Hence, anyone who knocked Silver for his 2016 forecast was invariably met with “You don’t understand the concept of probability, do you? Durrr math much?”

        Of course, you can’t run the same election twice, much less 100s of times, so Silver’s “predictions” will always be unfalsifiable so long as he never assigns values of 0 or 100 to any outcome.

        I suppose Silver’s schtick would be more palatable if he weren’t so, I don’t know, exploitative. Maybe I missed it, but I didn’t see him pumping the brakes on any of the oracle talk back in 2008. He embraced the hype. And since then, the 538 setup seems to be that they are explicit predictions when the outcome is good, probability expressions when the outcome is bad.

        • Cliff says:

          Exploitative? Or, wants his business to succeed and doesn’t go out of his way to torpedo it?

          • Capt. Adama says:

            When you allow your brand to mutate into clairvoyant-savant-wizard guy (despite knowing better) while you and your devotees dumb-shame anyone who questions your models… yes, it’s exploitative.

            Notice I didn’t suggest it was a mistake on his part to ride the wave as he did, but merely pointed out that I find his particular patter both unbecoming and unconvincing. I think The Bachelor is smart TV business. I also think it is dumb and exploitative and have no use for it.

        • John Schilling says:

          Of course, you can’t run the same election twice, much less 100s of times, so Silver’s “predictions” will always be unfalsifiable so long as he never assigns values of 0 or 100 to any outcome.

          But we have more data than just the fact that Trump won. We know how close he came to losing, and we know from many previous elections approximately what the range of variance in vote totals and poll:vote deviation is. If Silver assigns 71% to Trump losing, and Trump either wins or loses by say 10% of the vote in the relevant swing states, then that’s pretty close to a falsification of the prediction and I at least would call it such.

          Just like I count the people who said “Trump 95% to win!” as falsified by Trump’s very close win.

          • GKChestertron says:

            Yep, and at this point Silver has made predictions on three presidential elections with a 4th coming up. iirc, he gave Obama a roughly 95% and 90% chance to win in 2008 and 2012 respectively (and some criticized his model in 2012 as being too statistically conservative). This seems like, overall, he expected to get 2.5 of his picks correct, and he got 2 of 3. That seems reasonable, and we can update that again after this election.

            Or another way to look at it, Silver gave (Obama wins 2008), (Obama wins 2012), (Trump wins 2016) a .95 * .9 * .3 joint probability, ie 25%. We can then compare this to the confidence of other forecasters of their picks right before each of those elections, and see who gave this specific set of outcomes the higher probability. The people who gave Trump a 1% chance to win cannot possibly have given this combination more than a 1% joint probability. People who gave Trump a 50% or 90% chance to win might end up looking good, but being wrong or even underconfident on the other two elections could outweigh that depending on the specific numbers.

          • Capt. Adama says:

            I agree, but your typical Silver Bot would probably fight you on this, as the 2016 data would deliver quite the falsification. (Silver, to his credit, admitted to bungling the 2016 nomination, and vowed to take a more statistical approach thereafter. Then the GE happened and it was even worse. The response to that second big miss seems, for the most part, to be retrenchment.)

            That said, I don’t think confidence is intended to be a fairly potent expression of MOV in his models. Silver should say so if otherwise. But I don’t think that is how his models work.

            My understanding is that polling quality, consistency, and momentum play a significant, if not dominant, role in his modeling. So if the expected margin is in a swing state is <5% but the best polls consistently point to a decisive outcome within that margin, the model will assign a win probability that exceeds that margin. But maybe I'm wrong?

            I do like your approach to evaluating these forecasts, but I don't know how you get around cases where you have a fairly inelastic electorate and thus a highly predictable outcome on a very low margin (I ran for a municipal office myself and had to work within precisely this dynamic).

        • MugaSofer says:

          Isn’t this just Bayesianism vs Frequentism?

      • iprayiam says:

        Sure and if it was a horse race, Silver would have lost *less* money than his peers, but he still would have lost money. We don’t call that a win because he hedged more.

        Silver could still go home and cry to his millions hes made from overall betting in the money. Hell he could even celebrate not losing as much or as often as his peers, and celebrate his overall winning performance.

        But on this bet, Silver loses. I don’t see why we treat a prediction of any other single event differently. Statistics is fuzzy in its 1:1 application to “predicting” a single event. Statistics works better for talking about population means. The 2016 election has a population of 1.

        • baconbits9 says:

          Silver only lost money if the market expectations for Trump winning were higher than 29%, or if he bet against his prediction.

          • Dan L says:

            Markets were running ~10% the night before. People playing the 538 line didn’t merely not lose money, they made a killing.

          • eccdogg says:

            Yep my memory is that someone betting using Silver’s model would have bet on Trump given where the market was trading.

            Silver was giving Trump better odds than predictit and ladbrokes.

          • Matt M says:

            I can confirm that I personally thought Trump had a less than 50% chance of winning, but did make money betting on him on predictit because I thought his odds were better than most people were giving him credit for.

          • Anonymous Bosch says:

            I did this too, although it was only a few hundred and I can’t say how much of it was suspicion of mis-pricing versus my tendency to hedge when I have a favored outcome.

            (I also lost all my winnings betting wrong on various special elections throughout 2017. C’est la vie.)

          • iprayiam says:

            You are misextending my analogy. I mean that it’s only comparable to a straight bet with given odd by nature of it being a binary event. Yes a market can form around the odds and you can make money on that in the typical ways, but that becomes removed from my point about probability not being perfect fit for “prediction”

          • Dan L says:

            You are misextending my analogy.

            We’re really not. I recommend you look up how actual, literal horse races work, and the basics of betting strategy in that context. Punchline: it really does come down to personal expectations v. the book values.

            If you instead want to talk about grading based on whether someone gave a >50% chance to the specific outcome that occurred in a specific instance, fine. But that metric isn’t a particularly useful one in the actual practice of forming predictions. Flattening the target odds to an a priori 50/50 split entails throwing out information, and will predictably under-perform probabilistic approaches in distinguishing the quality of competing models.

        • eh says:

          I made money (roughly $70 AUD) based on Silver’s predictions, because the bookies were giving odds around 8.5:1 and I trusted his model more. I was so sure that it was economically sound that I very nearly put $5k on a Trump win, but it was a little too risky for me to commit to.

        • ShemTealeaf says:

          @iprayiam

          But on this bet, Silver loses. I don’t see why we treat a prediction of any other single event differently.

          We call it a win because we can see how much better he did than the prevailing odds. To use your sports betting analogy, Silver predicted that the underdog had a 30% chance of winning instead of a 10% chance of winning, which translates into a bet on the underdog. Since the underdog won, Silver comes out ahead.

          We might also call it a win because the outcome suggests that his model was much closer to reality than the other competing models. To draw an extreme example, let’s say that you and I are trying to predict whether or not I can throw a dart to hit a target. You predict that there’s a .00001% chance that I will hit the target, and Nate Silver predicts that there’s a 49.9% chance that I will hit the target, and I predict a 99.9999% chance that I will hit the target. I throw the dart and just barely hit the edge of the target. Strictly speaking, you and Silver were wrong, and I was right, but the evidence certainly seems to suggest that Silver’s model was much closer to reality than either of ours. If we have to predict the outcome of another throw, we should probably use Silver’s model.

      • albatross11 says:

        Also, Silver initially wrote some articles saying Trump had no chance, and after a couple good turnouts in either polls or primaries (I forget which), he wrote an article saying why he got it wrong (he was relying on his intuition rather than the numbers) and was afterwards very committed to just reporting the numbers.

        Our intuitions about what will and won’t happen are powerful and useful, but they’re likely to lead us astray in the sense of rounding a low probability to zero and in the sense of failing to account for exceptional circumstances. I suspect a lot of experts who got this wrong were falling back on their intuitions rather than running the numbers, more-or-less the way Kahnemann’s work would suggest. Your System 1 is screaming “Not Gonna Happen!!!” so loudly that your System 2’s whispered “How would I compute a probability here?” is never heard.

      • Eponymous says:

        It might also be worth pointing out that, from what I remember, Silver was giving Trump a better chance than most of his peers were… It wasn’t exactly rare to find very serious pundits who were telling us that Trump had less than a 1% chance of winning…

        Yup. There was the guy (forget his name) who said he would eat a bug if Trump won.

        He made an obvious mistake — he assumed errors across polls were uncorrelated. Silver pointed this out at the time, so it’s not hindsight. He made the same mistake in 2012, but “won” that time.

        Clinton was up by 3 points in national polls before the election, so a 70% chance of winning was reasonable just based on this fact and historical performance of polls. She won the popular vote by 2 points, and lost the EC because of a highly unusual distribution of votes across states. The probability of such an outcome is extremely low, and I don’t think anyone predicted it. All the predictions of a Trump win I heard were from people who assumed that some combination of greater enthusiasm of Trump supporters and lying on surveys would mean that Trump would win more votes than polls showed, and win that way.

        So in my book, *nobody* predicted Trump would win *in the way he actually won*. (This includes my guru Greg Cochran.) I’m happy to hear of an exception.

        • and lost the EC because of a highly unusual distribution of votes across states

          That might be true in this case, but all it takes to get that result is for one party’s support to be substantially more concentrated than the other’s. If one party typically wins states 60/40 and the other 55/45, that will do it.

        • Chalid says:

          *nobody* predicted Trump would win *in the way he actually won*.

          FWIW Nate Silver‘s scenario for a Trump win closely resembled what actually happened.

    • Capt. J Parker says:

      Because you cannot re-run the 2016 a statement such as “Trump has a 29% chance of winning” is meaningless. Trump was either going to win or not. Saying an election has a 29% of a given outcome is pseudoscience because this is no way – even in principal – to prove or disprove the 29% was correct. The 29% can only be logically interpreted as a subjective expression of uncertainty in the prediction.

      • matkoniecz says:

        Saying an election has a 29% of a given outcome is pseudoscience because this is no way – even in principal – to prove or disprove the 29% was correct.

        You can do that by comparing model with reality.

        Model predicting that Trump will win because Martians will invade was falsified.

        Model predicting close battle in contested states was confirmed as well-matching reality.

        • Capt. J Parker says:

          The ultimate reality that the model needs to be compared with is the reality of the outcome it was being used to predict. In the case of the 2016 election that outcome was: Trump won. Period. Full stop. That outcome does not carry with it any information that would enable you to determine if “Trump, 53%” was any more or less correct a prediction than “Trump, 85%” say, (or, for that matter “Clinton 72%.” ) If we have no basis to adjudicate between Trump 53% and Trump 85% once the reality of the election is known then the 53% and 85% have no meaning. Neither number is falsifiable so, neither number represents any kind of scientific prediction.

          • anonymousskimmer says:

            The models use statistically determined error bars to generate percent likelihoods. These statistics determining these error bars are usable in more than one event, thus this underlying statistical model can be tested multiple times. This allows people to validate the percent likelihood in a variety of races, the same way Scott tries to calibrate his predictions.

          • Capt. J Parker says:

            anonymousskimmer,

            Saying that a prediction method has a certain likelihood of making an accurate prediction is a totally and completely different kind of statement then saying an event has a certain probability of happening. If people making predictions are really telling us that their prediction has a certain likelihood of being correct then they should say precisely that and not confuse us with language that implies a unique event has a “probability of occurring” (like we see for example in the y axis of Scotts calibration graph) The thing is, your reasoning makes sense for things like weather forcasting, but for unique events like presidential elections if that reasoning was strictly applied then one would think the 2016 election would necessitate dramatically increasing ones estimate of uncertainty for the next election. But, that’s not what is happening. For example, some commentators above are saying the Trump-28% prediction by Nate Silver was “accurate”. And Silver’s team wrote this long piece of nonsense about how they did everything right.

            So, in short the predictors assigned a value with no direct way of checking it’s accuracy then, when common sense hinted that there was something seriously wrong with the value, they just shrugged it off, blamed it on uncertainties associated with the event and promised to continue with what they were doing as if nothing had happened. That’s not science, it’s religion.

          • albatross11 says:

            Capt:

            I think this is just a difference in philosophy. A lot of people here adopt the Bayesian idea (I think best explained by Jaynes in his wonderful book about probability theory) that it makes sense to use probabilities to express beliefs[1]. So when I say “there is a 0.2 probability that it will rain tomorrow” I’m not saying that, say, given a hundred instancess of April 17ths with conditions just like today’s conditions, 20/100 of them had rain the next day. Instead, I’m using 0.2 to express my beliefs about the future, in a form that I could use (for example) in an expected value calculation that might help me decide what to do.

            If you’re using probabilities to express your beliefs or knowledge, then it makes sense to use them even for one-off situations where you’re trying to assess how likely something is to happen.

            The way I understand it, Nate Silver’s approach for predictions is to run lots of simulations. In each one, he simulates (for example) each state’s results based on a probability distribution around the best reported poll numbers, and then sums the results up to get an outcome. And he iterates that a zillion times and when the model comes out Trump 28% of the time, he says “there’s a 28% chance Trump wins.”

            Obviously, this fails horribly if his underlying model is wrong in some way he’s not accounting for in his simulation, but it seems like a pretty reasonable way to try to get a handle on the uncertainties of an election outcome.

            [1] Jaynes saw probability theory as an extension of logic. In _Probability Theory: The Logic of Science_, he developed an argument for what we’d want from an extension of logic to account for uncertainty, and then showed that probability theory meets all his requirements.

          • HeelBearCub says:

            @Capt J. Parker:
            I think you are fundamentally misunderstanding what Silver means when he says “28% chance of winning”.

            @albatross11:
            I think you are also incorrect, in a subtle way.

            When Silver offers predictions, he is doing it primarily off of polling data. Polling data is a view of what the electorate is planning on doing. It is an incomplete, partial and necessarily flawed view, but a view nonetheless.

            Silver then tries to figure out, based on all of his teams vast experience in comparing polls to actual outcomes, what potential compositions of voters could generate the polling data that has been generated. That’s why he always says “If the election were held today, we predict Candidate A would have a 53% chance of winning”. He isn’t trying to predict what voters will do in the future (with one caveat), he is trying to understand how likely it is that the polling data is consistent with Candidate A having the winning vote share. This is a subtle but powerful distinction.

            It’s akin to pulling colored balls out of a bag. You can pull a limited number of balls out, and have good guesses on the composition of the balls in the entire bag. You aren’t “predicting” what balls will get put in the bag. They are already there. You are trying to make a good guess about what is in the bag based on limited information.

            The one caveat to this is with undecided voters. There you have to model how the voters would vote if they were forced to choose today. There again they are leaning on experience, what do voters who say they are undecided tend to do, but I believe he always claims it’s the least accurate part of their modeling. That’s a big part of the reason why he always says their models become more accurate as you get closer to the election, because there are fewer undecided voters in the model.

          • Capt. J Parker says:

            HeelBearCub,
            My gripe with Silver isn’t that he isn’t always right and he can hedge with weasel words like “if the election were held today” that’s all well and good. My gripe is how he crows about how “scientific” his method is when in reality saying Joe Blow has a 63.7% chance of winning is totally unscientific because There Is No Way to Verify Or Falsify the 63.7% number. If you’re making non-falsifiable predictions you are not being scientific. Everyone is so impressed with Silvers process. I’d be impressed if someone could tell me how they would validate the numbers Silver process spits out, especially since those numbers are so tied to a specific instant in time that Silver feels obliged to qualify his predictions with “if the election were held today.”

            albatross 11,
            Use probabilities to express your beliefs all you want. I believe there is a 0.00036% chance Charliez Theron will invite me to dinner (I started out at 50% but have been updating in a Bayesian fashion) But, if you want to test your beliefs scientifically then your beliefs must be falsifiable. If you were willing to use Bayes theorem to update your beliefs about the probability of rain based on observation I bet you will need to incorporate the frequentist meteorological data you claim to not be incorporating into your stated belief.

          • albatross11 says:

            The best way I know to measure correctness of predictions is the Brier score. This is more-or-less the sum of the squares of the errors, where the observed value for each prediction is a 1 if it happened and a 0 if it didn’t. (So if you predicted a .2 probability of rain and then it rained, your score would go up by (0.2-1)^2=(-0.8)^2=.64, and if it didn’t rain, your score would go up by 0.04. Lower scores are better.)

          • Capt. J Parker says:

            albatross11,
            If you start with a number that has no scientific validity because it is inherently non-falsifiable (the one-off yes/no event probability) and then you crunch it through a mathematical function, you end up with another number that is still lacking in any scientific validity. Maybe you succeeded hiding your initial error from other people or maybe from yourself.

            Going back to Silver for a moment, Act_II in the fifth post above implies that a Brier score would give you the “wrong” answer when using it to compare Silver to another 2016 election prediction (s)he was aware of:

            Here’s an example. In fall 2016, my girlfriend got an Uber ride from a guy who turned out to be a big Trump supporter. He talked the whole ride about how he was certain Trump was going to win. As it turns out, he was right! But I’m still not going to put much stock in this Uber driver’s predictive ability — certainly not as much as 538, and not even as much as The Upshot. He didn’t draw his conclusion through principled analysis of the evidence; he just knew what he wanted to be true, declared it fact, and his desires happened to align with reality in this case. It’s not so much about the outcome as it is the process.

          • anonymousskimmer says:

            @Capt. J Parker

            I’m curious how you think predicting the weather differs from predicting an election. The only fundamental differences I see are of scale, and of sampling:
            1) Far more weather predictions are made than election predictions.
            2) Human polling tends to poll new people each time, whereas atmospheric polling frequently uses static sites (I believe).

            The modeling and percent likelihood are presumably quite similar.

          • Capt. J Parker says:

            anonymousskimmer,
            Again, my claim is that if you throw out a number and there is no hope of proving or disproving that number, even in principal, then that number does not represent a scientific prediction. For weather forecasting, if you were in fact saying 20% chance of rain means

            …given a hundred instances of April 17ths with conditions just like today’s conditions, 20/100 of them had rain the next day.

            then that 20% is falsifiable at least in principal. If you said 20% chance of rain means rain will fall on 20% of a given land area in a certain time frame so a stationary observer has a 20% chance of rain falling on themselves then that also is scientifically falsifiable.

            In the prediction “28% chance of Trump” the 28% isn’t falsifiable and therefor it doesn’t pass as a scientific prediction no matter how universally accepted the model is that was used to generate it.

          • HeelBearCub says:

            @Capt J Parker:
            Your complaint about election prediction holds just as true for weather prediction. The weather either is or is not as predicted. It’s just as non-falsifiable. I invite you to prove that their was a 60% chance of rain on a given day.

            If you start pointing me at “Well, if there is rain on 6 out of 10 days they say there is a 60% chance of rain …” I think we have found an issue.

          • albatross11 says:

            Capt:

            Is there any evidence that would convince you that, say, Alice was better at making one-off predictions of this kind than Bob was? Tetlock ran prediction tournaments in which some people and teams consistently outperformed others in just exactly this kind of set-up. I’d say the people who consistently outperformed everyone else (“superpredictors”) are actually better at making this kind of one-off prediction than other people.

          • Capt. J Parker says:

            albatross11,
            I will repeat what I said to anonymousskimmer:

            Saying that a prediction method has a certain likelihood of making an accurate prediction is a totally and completely different kind of statement then saying an event has a certain probability of happening.

            Bob may simply throw darts and have an election prediction success rate of 50%. Alice on the other hand reads a newspaper occasionally and has an election prediction success rate of 65% validated across multiple election predictions. That would not mean that Alice could say “The odds of Trump winning is 65%.” and have the 65% mean something in that context. If Alice said “Trump will win and my success in predicting elections is 65%” I’d be perfectly fine with that because the accuracy of the 65% could be tested. I don’t think that is the kind of statement Silver is making.

          • Capt. J Parker says:

            HeelBearCub,
            You said:

            I invite you to prove that their was a 60% chance of rain on a given day.

            I agree, you can’t prove that statement when taken at face value and I never claimed one could.

          • ShemTealeaf says:

            @ Capt. J Parker

            What if Alice said “The odds of Trump winning are 65%. I’ve predicted a bunch of elections, and previous elections where I’ve predicted that Candidate X would win 65% of the time resulted in Candidate X winning 65% of the time. My predictions are similarly well calibrated at a bunch of other probabilities”? I think that Alice’s claim there gives you much more actionable information than if she had said “Trump will win and my success in predicting elections is 65%”.

            At minimum, it would be nice to differentiate between “Alice predicted 20 elections confidently and got 13 of them right” and “Alice predicted 6 elections confidently, all of which she got right, and she predicted 14 with almost no confidence and got half of them right.” In the second scenario, we can at least ask Alice whether her prediction is confident (in which case she’s a predictor with 100% accuracy) or not confident (in which case she’s a predictor with 50% accuracy), which lets us make a better decision.

          • HeelBearCub says:

            @Capt. J Parker:
            But individual days are one off events in much the same way elections are. The clouds, pressure fronts, etc. are all unique, but also share many similar characteristics.

            We have lots of elections in the US. These are primarily first-past-the-post elections in a country where two primary parties are dominant. We have many, many polls of the electorate before these elections. We know based on how polling data has corresponded with election results how to make good guesses at what will happen in the election.

            Do we have AS MANY trials of that as we have weather observations? No. But that doesn’t make the statistical process any different.

          • Capt. J Parker says:

            ShemTealeaf,
            You asked what if Alice said:

            I’ve predicted a bunch of elections, and previous elections where I’ve predicted that Candidate X would win 65% of the time resulted in Candidate X winning 65% of the time.

            I presume you really meant to ask what if Alice said:

            I’ve predicted a bunch of elections, and previous elections where I’ve predicted that Candidate X had a 65% chance of winning resulted in Candidate X winning 65% of the time.

            That statement is falsifiable and I have no problem it.
            But when Alice says:

            Candidate X had a 65% chance of winning.

            There is no way to falsify that statement so it can’t be regarded as scientifically meaningful.

            HeelBearCub
            I thought we agreed that probabilities assigned to outcomes of one-off yes/no events are meaningless, be they assigned to Trump winning the 2016 presidential election or snow falling on Lexington green on Patriots day 2020. Are you now claiming there is no such thing as a one-off election?

          • HeelBearCub says:

            No, I’m saying snow and elections are the same kind of predictable. Based on observations beforehand you can predict the odds of various outcomes.

            You’re the one who wants to say those predictions are unfalsifiable as if that has some meaning.

          • ShemTealeaf says:

            @ Capt. J Parker

            You can’t completely falsify the statement on its own, but you can judge its accuracy in two ways.

            First, you can treat it as part of a larger predictive record. The statement really means something more like “this election is part of the class of elections where Candidate X wins 65% of the time”. I guess this is really true of any kind of prediction. Even if you predict “there is a 1/6 chance of this die roll coming up as a 5”, you’re really predicting “this die roll is part of the class of die rolls where the die has six sides and is roughly fair”, which can only truly be judged after many trials. However, if you consider the predictor to be competent and acting in good faith (since there are ways to “game” your predictions to appear more well calibrated), you can treat the one-off prediction as being meaningful.

            Second, some kinds of events aren’t completely binary, so they carry additional information that can be suggestive about the chances of different outcomes. If I predict that Team A has a 99.99999% chance of beating Team B, that prediction looks at lot more reasonable if the final score is 100-5 then if the final score is 100-99. You can’t say that the 100-99 score really falsified my prediction, but you can say that in a situation where Team A has a 99.99999% chance of winning, the 100-99 outcome is extremely unlikely.

          • Capt. J Parker says:

            ShemTealeaf,
            Those are good and concise arguments. I’ll attempt to rebut them. In your first example you seem to be saying “If we have a repeatable event whose outcome probability can be measured and then we pretend that a single instance of the repeatable event is really a one-off event we can then attribute meaning to the probability of a one-off event.” Even if true, I don’t think think your manufactured one-off event is applicable to something like the 2016 election. And since my rant is largely motivated by what I see as a totally unjustified lauding of Silver, I’ll add that your example is definitely not what Silver thinks he’s doing in stating his probabilities for election outcomes.

            Your second example where the score is 100-99 might actually make my point. You predicted essentially unequivocally that Team A would win and it did even though the on the field/court/pitch/diamond performance of both teams suggested that the teams were evenly matched. You are a fantastic predictor! You must have one heck of a model that you’re using! My point here is that the support or lack thereof for the accuracy of the 99.99999% probability depends critically on what assumptions you are making about the predictors methods, the game, etc. If it’s validity isn’t falsifiable and is assumption dependant, it’s not likely meaningful in a scientific sense. Please permit me to add that if anyone thinks my logic is strained when I say your high probability in what manifested as a close contest proves your model was good as opposed to disproving your probability then consider that that is exactly what is being said about Silver’s “28% Trump” by many commenters here. Because Silver gave Trump the highest odds in a close race Silver’s process is great! His model is wonderful! Not “his probability was BS because Trump won.”

          • HeelBearCub says:

            I’ll add that your example is definitely not what Silver thinks he’s doing in stating his probabilities for election outcomes.

            How so? Silver is predicting likely vote percentages, not just odds of winning the EC.

          • ShemTealeaf says:

            @ Capt. J Parker

            Regarding the first example, I don’t think the distinction between ‘repeatable’ and ‘one-off’ is quite so clear cut, and it really just comes down to how you’re defining your reference class. The more closely you look, the more everything looks like a unique one-off event, and the reverse is true if you look at everything in the most abstract/general case. We treat die rolls as repeatable events because we don’t usually look that closely at the specifics, and we consider them all part of the ‘die roll’ reference class. If you wanted, you could break that down further and say that you need a separate reference class for rolls made by different people, or on different surfaces, or in different atmospheric conditions, etc. Taken to the extreme, you could gather all the relevant information about the roll and declare that it’s a totally unique one-off event that’s not part of any larger reference class. That’s true in some sense, but it’s obvious that we can draw useful conclusions about reality by considering the larger reference class of ‘die rolls’. The same is true about elections, although obviously to a much lesser extent.

            Regarding the second example, I’ll confess that I’m not really understanding your point. If the score is 100-99, then it’s close enough that fairly common random event (random in the sense that the forecaster couldn’t have reasonably predicted it, like someone stumbling at the wrong time, or one of the players eating some bad shrimp the night before) could have changed the outcome. That’s pretty strong evidence that your 99.99999% probability was vastly overconfident. If you can predict millions of games correctly, then maybe your 99.99999% probability really is justified, and you’re an omniscient forecaster who can analyze even apparently random factors, but a single 100-99 score is evidence against your model. I think it’s just conditional probability: given that your model is true, how likely is the observed outcome? With most reasonable models that generate 99.99999% probability of a win, a 100-99 score would be extraordinarily unlikely. A model that predicts a 49% chance of a win, even though it gets the binary outcome wrong, predicts a much higher chance of that 100-99 score.

    • benf says:

      Silver runs calibration tests on his model and they grade out well in just the way you describe. But it’s way easier to run calibration tests on Congressional races, which are the equivalent of a season’s worth of baseball games, as opposed to a presidential election, which is like the Super Bowl.

  2. gbear605 says:

    There once was a shepherd boy who lived near a small town. One day, he heard rustling in the trees and thought it had a 10% chance of being a wolf, and did nothing, and nothing came of it. On another day, he once again heard rustling in the trees and thought it had a 10% chance of being a wolf, and did nothing, and nothing came of it. On a third day, he once again heard rustling in the trees and thought it had a 10% chance of being a wolf, and did nothing, and he and all of his flock were eaten by the wolf.

    There once was a shepherd boy who lived near a small town. One day, he heard rustling in the trees and thought it had a 10% chance of being a wolf, and cried out “Wolf!,” and all the neighbors came to help but found no wolf. On another day, he heard rustling in the trees and thought it had a 10% chance of being a wolf, and cried out “Wolf!,” and all the neighbors came to help but found no wolf. On a third day, he once again heard rustling in the trees and thought it had a 10% chance of being a wolf, and cried out “Wolf!,” but none of his neighbors came to help because they thought there would be no wolf, and he and all of his flock were eaten by the wolf.

    • Bugmaster says:

      I could be wrong, but shouldn’t the boy and his neighbours be updating their P(wolf|rustle) dynamically based on repeated trials ?

      Yes, I understand that the answer is “the general public is innumerate and so is the media”, but still…

      • gbear605 says:

        How would you suggest they update P(wolf|rustle)? If anything, it seems like it should go down after each non-wolf rustle, which doesn’t help the boy. Whether a wolf is there on any given rustle is independent of being there the previous rustle, so the only evidence that you get from a wolf not being there is that your prediction of 10% might be too high.

        • cmrsalmon says:

          I think the issue here is that the story only allows for two responses to the rustle – cry wolf or not. There are lots of things the boy and the village could have done to manage wolf-related risks, like move the flock away from suspicious bushes, give the boy a gun or a dog, carry out a weekly wolf hunt etc.

          • Robert Jones says:

            I think that’s sort of right, but not exactly right. The village needs to make a trade off between allowing the sheep to graze and protecting them from wolves, and the solution they have found is to assign a boy to raise the alarm when required. If they had the resources to provide better wolf protection or cull the wolves, they would likely already have done that.

            But, the role of the boy is to identify whether there is a wolf, and raise the alarm if there is. The system is too sensitive if it goes off whenever there is a rustle. The rustle should cause the boy to investigate further.

            I’m not sure whether this tells us anything about Covid19.

          • I think the moral lesson here is that sometimes the boy just has to be eaten by the wolf.

    • Leafhopper says:

      That’s a very hungry wolf.

    • Lambert says:

      On the first day, he heard rustling and thought there was a 10% chance of a wolf, so he spent the day collecting some nice heavy stones for his sling.
      On the second day, he heard more rustling and thought there was a 20% chance of a wolf so he got a couple of men from the village to come to the field with him…

      You have to respond proportionately to the threat.

      • Not A Random Name says:

        So after hearing a rustling and observing no wolf on the first day the boy should update his probability of there being a wolf from 10% to 20%?

        I think some other commenter pointed this out before already. No wolf on first day should make you update in favor of there being no wolf. That seems to make sense to me, on first glace anyway. You seem to think otherwise, why’s that?

        • Stephen says:

          My guess is that the rustling in the second instance was 10% more wolf-like. Their point was that, in each case, respond appropriately to the information that you have available.

          • Not A Random Name says:

            Well, my point was that their example clearly was meant to convey that yet as-described it just falls flat. Unless I’m mistaken, which is always plausible.

            Had they included a sentence like “and the second instance of rustling was 10% more wolf-like” in their description I believe their example would be internally consistent.

            In any case, I read them as saying more than what you present as their point.
            Lamberts example goes to show that there are more response options than “always cry wolf” and “never cry wolf”. With that I agree. But then it attempts to show that a more rational response for the boy would be to start at low alarmist levels and escalate from there. With the description-as-given that seems actually irrational, though.

            Maybe a better rebuttal to the OP would have been to point out that with corona we have mounting evidence of the bad outcome coming to pass and that the scenarios described in the OP lack that. That makes them bad analogies. Not sure I hold that view but you could definitely argue that.

        • He is revising his estimate of two probabilities — the probability of a wolf conditional on rustling and the probability of rustling. Each day of rustling and no wolf decreases the former but increases the latter, so might — judging by the behavior described does — increase the probability of a wolf.

          • Not A Random Name says:

            Sure, plausible. But I can explain almost everything if I just get to fill in the blanks with what works for my theory. That’s why I find it best to ignore the unknown in hypotheticals like this.

            So what are the relevant observations we know of?
            1) We know that he observed rustling the first day but no wolf.
            2) We know that he observed rustling the second day.

            I think we agree that because of 1) he should adjust P(wolf|rustle) downwards or at least not up.
            We do not know that or how 2) should affect his estimation. There are many possibilities and (unlike with 1) there seems to be no obvious answer.
            So I say we ignore that, as any choice seems equally arbitrary.

        • Vitor says:

          Rustling on two consecutive days is a different event than rustling on a single day. So the probability could be higher the second day, depending on your model. The rustle/wolf events are not independent, repeated trials!

          On top of that you’re conflating “no wolf attack” with “no wolf”. maybe the wolf was cautious the first day and didn’t attack, but on the third day it was really hungry.

          • Not A Random Name says:

            See my reply to DavidFriedman.

            TL;DR: Plausible, but built upon arbitrary assumptions. When you get to fill in the blanks you can make everything plausible. When you don’t there’s really no evidence for any of this.

    • randallsquared says:

      Yes, this exactly. Taking steps like “stop large gatherings” because of a small-but-real chance of pandemic would have had such partial lockdowns many times this century, and only this time would there have been some benefit (but how much benefit is hard to say). In the meantime, each such partial lockdown would have had its own consequences, economically, politically, and socially. A society that did a partial lockdown (no weddings, funerals, political rallies, etc) for each instance of a 10% chance of pandemic would either be a society where people in general were Gallants, or a society which is poorer, more authoritarian, and less trusting than the US in 2019. It’s not clear to me how the total damage of that scenario would compare to what we now face.

      We do have an existing solution for “small percentage of outsized bad outcome”: insurance. At least some organizations did purchase pandemic insurance; it’s not clear to me why it wasn’t more widespread.

      • Matt M says:

        Indeed. And recommendations like “stop large gatherings!” probably seem like “reasonable low cost interventions” because to most of us, they are. So you have to stay in and stream a movie rather than going to the theater? That sucks, but you can get over it.

        But it’s a pretty big cost to the theater owner. And to all of his employees. Etc.

        And if it becomes common knowledge that any time there’s even a ~10% chance of pandemic, all the theaters will immediately be closed, good luck finding anyone willing to invest in having theaters. A society with such rules will probably quickly become a society where theaters are simply no longer a thing.

        • albatross11 says:

          I think there have been two other instances of something that looked like a serious pandemic threat and got far enough along to justify taking action in my adult lifetime: SARS and the H1N1 flu. (When the dominant strain of Influenza A shifted suddenly.) The result of H1N1 was that even though it turned out not to be so bad, lots of countries and states and local governments put together a pandemic plan, stockpiled supplies, etc. (Zika and Ebola were local threats but couldn’t realistically spread in first-world conditions.)

          I know many events in Asia got cancelled for SARS. I don’t know of any that got cancelled in the US for SARS or H1N1.

          Preliminary steps when you think there’s a 1% chance of a pandemic in your country might be things like making sure of your supply of PPE and (if they’re not already written) to start writing up plans for how to handle it if it hits here, and to ask local health departments and private employers to do the same. There were still large public events going on up until quite late in the expansion of the disease–concerts sporting events, conferences/conventions, etc.

          • craftman says:

            It’s been much theorized that the recent memory of SARS preparation in East Asian countries has led to their [relative] success in the current pandemic.

            I live in the US and am 35 years old, so during SARS and H1N1 I was 18 and 24 years old, respectively. All I remember at the time was these were massively hyped terrible things in the media that all came to nothing. My life didn’t change at all, and that greatly affected my priors when the SARS-COV-2 virus appeared. Where are all the 50-70 year olds who were full-on adults during those pandemics sounding the alarm that we need to take this seriously? Maybe they don’t exist in the US…

            I imagine anyone who is above age 25 at the moment will have strong priors towards action when the next warning comes along.

          • mtl1882 says:

            For some reason, I really get the sense that most 50-70 year olds didn’t think pandemics were really a risk in the modern U.S. Not as a well thought out position, but just as a psychological assumption. I had noticed this prior to Covid. There are of course many exceptions to this, and I’m sure most of young people assumed the same thing. But young people generally feel more invincible and don’t remember the SARs stuff. I had noticed this seemed to be one of the blind spots with adults who thought of themselves as very responsible, careful planners. Including a relative who frequently traveled to Asia on business and used to laugh at how they checked his temperature on arrival and how he took steps to avoid detection so he wouldn’t be put in quarantine (this was years ago, I think pre-SARs, and he just had some 24 hour bug–there was no reason to think he’d been exposed to anything dangerous, but I don’t think he ever considered that he could be one day). I’d occasionally bring up the possibility when polio or 1918 or SARs came up in conversation, and it was just kind of laughed off. They seemed convinced we had it under control with modern medicine, not seeming to be able to process the idea that these things occur because viruses mutate in ways we can’t anticipate and don’t have a fix for.

            My best guess is that this had to do with growing up in the era celebrated for winning the war against infectious diseases. They heard all the time from parents about what things used to be like, and how wonderful it is that this was over. So they associated it with the past. Young people probably had fewer associations.

            Also, just-in-time supply chain stuff seems to have been more emphasized during this era, and just more physically/technologically possible, and would have discouraged thinking too much about preparation that involved “wasting” resources. Again, I’m not saying young people are necessarily any better at this–they may well be even more clueless, but that’s true across all topics. But, on the whole, the people who hold leadership positions and had more direct experience just didn’t seem to be taking it seriously. Information and plans were available, but we didn’t take them seriously like countries more affected by SARs tended to. I think the reaction to SARs and others was oddly calm, because people just didn’t believe it could happen here. For the most part, it didn’t. We got it under control. But that just acted as confirmation for people’s tendency towards denial.

            I do think in the future people who lived through this will be more willing to acknowledge the risk.

        • Michael Watts says:

          And if it becomes common knowledge that any time there’s even a ~10% chance of pandemic, all the theaters will immediately be closed, good luck finding anyone willing to invest in having theaters. A society with such rules will probably quickly become a society where theaters are simply no longer a thing.

          I don’t think this can be right. It reminds me of the argument, which I feel compelled to note that I am not making up, that agriculture is fundamentally not economically viable because if yields are higher than expected, prices drop, whereas if yields are lower than expected, prices rise — but you have low yields, so you don’t get the benefit of the higher prices.

          And it’s true that much financial technology was developed in order to deal with the problem that agricultural yields are unpredictable. But it’s wildly untrue that (1) agriculture is economically unviable due to unpredictability; or (2) unpredictable returns preclude economic activity in general. The first technology to address the problem in the context of agriculture was of course the granary, but the first solution, which remained common long after the development of storage, futures contracts, and everything else, was to starve when there wasn’t enough food. This solution was a five-star success: the feast-or-famine approach was in feast mode often enough that agriculture is an obligatory mode of production for nearly all humans today.

          There are many, many business that use the same strategy. A profitable treasure-hunting operation may have 10 straight years of losses. A profitable movie-producing operation expects to lose money on most films. The concept is known as a “hit-driven business”. Why would we conclude that, if movie theaters are profitable for about two out of three years, and get shut down about one out of three years, that there would be no movie theaters? There definitely will be; the structure of the business will adjust to the periodic closures.

          • albatross11 says:

            Note that the laws allowing health departments to close things down in this kind of an emergency have existed more-or-less for as long as we’ve had a country. They haven’t been used this widely since 1918, but they were also used (for example) to close down public swimming pools when polio was crippling a lot of kids, and they’re used from time to time for local outbreaks of stuff like meningitis or norovirus. (Let three kids in the same high school come down with meningitis and you can bet that school is going to be shut down for awhile.)

            So this is something that *can* happen and that anyone paying attention knew *could* happen. It only happens very rarely because we manage not to have terrible outbreaks of contagious disease very often, thanks to modern sanitation and medicine. And yet, we’ve continued to have theaters and restaurants and convert venues and sports stadiums during this time.

            Those powers have mostly not been used capriciously in the US, because there aren’t a lot of mayors or governors who think “Oh, boy, I think I’ll shutter all the theaters and restaurants and amusement parks and bars and concert venues and sporting events in my state for the next month–that seems like a path to popularity!”

            The issue with prediction we had wasn’t the need to take high-cost actions before it was clear there was a crisis, it was people not working out how likely a crisis was and then taking sensible actions given the probability distribution on how bad things were going to get.

            And I’ll say this again, because I think it’s important: I believe that one of the most important steps in the public health response was getting ordinary people (who don’t listen to virology blogs or worry about properly applying Bayes’ law in daily life) to take the matter seriously and start doing low-cost measures to avoid both giving and receiving an infection. That’s one place where I think that both the media and politicians could have done a much better job. (Though the respectable mainstream sources were offering some useful advice, like “wash your hands often” and “keep 6 feet away from strangers.”)

            One thing pushing back on them doing a good job there is the default setting of “don’t start a panic,” which isn’t crazy, exactly, but if you want to provide a source of information people will trust, you need to report the available information straight instead of fudging it to keep everyone calm. And I think much of the push for this is based on the interests of political parties and leaders–they don’t want the economy to go bad or the voters to get upset.

          • Michael Watts says:

            Laws on the books don’t affect much. There is no reason to adjust your business to something that doesn’t happen. If what happens changes, other things will change to suit, but the fact that they hadn’t done so preemptively is not evidence that they can’t!

          • albatross11 says:

            But public health officials do use those laws for local issues, and have used them more widely in living memory, so it’s not like these are just hypothetical powers of the state that nobody’s ever imagined using. They’re just not used often because we’re pretty good at not needing them.

            This is a bit like the state having the power to declare martial law. Mostly we don’t see this power being used, because the modern US is actually pretty good at keeping order using normal means like police and courts and such. But that doesn’t mean the powers aren’t there, or will never be used.

      • Randy M says:

        The most important part of Scott’s post is the “Cost Benefit analysis” in the Gallant graphics.
        It would have been asinine to do this full lockdown for months every time a new disease was noticed gaining traction. It’s basically the equivalent of letting your crops rot while you go wolf hunting for weeks.

        But it would have been wise to have, say, put hospitals on alert, make sure to procure masks, send out warnings to increase hygiene, and prepare to ramp up as the situation developed.

        • Matt M says:

          I don’t know anything about hospitals specifically, but I feel like a lot of that stuff did happen in February.

          I was reading blogs that were saying “start washing your hands better and more often” back then.

          I tried to buy masks back then but they were already sold out, so clearly someone, somewhere was implementing the “make sure to procure masks” policy as early as it made sense to do so.

          • Randy M says:

            Yeah I don’t mean to say definitively they weren’t. Perhaps we need more slack in the supply chains of emergency supplies. Perhaps this was the most cost-effective response.

            Point is just that Scott is right on about taking low cost measures for low chance events. But of course the actual math is hard and contentious.

          • Conrad Honcho says:

            For one data point, I still get the public affairs emails for my university (major state school in the US south). I went back and searched for emails saying “coronavirus” or “COVID-19.” Here’s the timeline of escalation:

            January 24th: First mention. Suggests halt of non-essential travel to Wuhan.
            January 31st: Recommends people improve hygiene, wash hands frequently, avoid touching eyes and mouth, etc.
            February 28th: Suggest people prepare like they would for a hurricane. (At this point I boarded up my house in case the viral winds get strong enough to break my windows).
            March 11th: Classes closed/moved online.

            I guess that’s an okay response?

          • albatross11 says:

            Yeah, my employer was clearly working from a well-thought-out plan starting more than a month ahead of the lockdowns, doing stuff like making sure everyone who could do their work remotely had the equipment and paperwork and VPN access and such, having the cleaning people disinfect doorknobs and such several times a day, etc.

          • Anonymous Bosch says:

            My employer didn’t shut down until Houston’s shelter-in-place order forced them to, despite my increasingly loud gesticulations about work-from-home starting around mid-February. The day it was announced they hastily created and distributed a one-page memo on telework policy that amounted to “figure it out and don’t slack off!”

            They’re already sending us emails raising the possibility of “unofficially” opening before the order is lifted, which if they insist will likely end up being my “unofficial” constructive resignation.

          • Michael Watts says:

            I tried to buy masks back then but they were already sold out, so clearly someone, somewhere was implementing the “make sure to procure masks” policy

            While this is technically true, it’s not really relevant to the societal response. US masks were sold out in February because people were buying them and shipping them to China, not because people wanted them in the US.

      • sclmlw says:

        What’s the corollary for the current situation, then? On the scale of mild to severe, the case fatality rate of COVID-19 is far below something like Ebola or MERS. Let’s run the counterfactual for a minute, and pretend we’d been able to avert this pandemic early through some of the suggested measures. We all felt the economic pain, but nobody realized the results of the pandemic that never was.

        Fast forward to the next crisis. People talk about executing the same drastic measures to prevent the next pandemic. Everyone says, “yeah, we tried that and it didn’t come to anything.” Of course not. That was the whole point. But people make decisions based on the evidence they collected. It’s harder to even predict the evidence they prevented. So everyone ignores SARS-CoV-3 or whatever. Later we discover it has a MERS-like CFR of 40% with an R0 similar to the current virus. There is a two-week asymptomatic period with viral shedding, such that without those early efforts we rejected, general community spread makes the virus unstoppable. The death toll soars past the Black Death in a matter of months, and we all wait around for wave 2 of 5 to begin.

        Once the severity of the situation becomes apparent, post-apocalyptic bloggers write articles about how we should have known to make nuanced decisions under uncertainty, instead of insisting on hard evidence before making a decision.

        Maybe you don’t need a RCT, but someone has to fall to their death before someone else realizes a parachute might be a useful invention.

        • craftman says:

          I am 35 years old and have not witnessed in my adult life a real pandemic, so when people start suggesting shutting everything down it seems odd. I was only early 20’s when SARS and H1N1 were big, and in the US it all amounted to nothing, and nobody really changed their behavior.

          My point is, I’m with you. We, in the US, didn’t know how bad SARS and H1N1 could have been, so we don’t use those worst case scenarios in our calculus.

          Even hearing that 80,000 people are sick and 3,000 are dead wasn’t enough to trigger an alarm. That’s a little bit more than 0.5% of the population of Wuhan. And it seems like there’s probably 3,000 people dying every day of something in China…

      • Edward Scizorhands says:

        There are steps before “stop large gatherings.”

        For example, encouraging individuals to be prepared, or seeing what would be necessary in order to safely screen all incoming passengers so you don’t scramble to do it at the last minute, or telling people who manufacture PPE “start making as much as you can, we will buy N million units every week for the next M weeks even if this turns out to not happen,” or putting portable hand-washing stations in every airport.

    • Tatterdemalion says:

      It sounds to me as though the boy has figured out that wolves come out of the forest (which always rustles) one day in ten, and is not actually providing any additional information to the village about which days that will be.

      At this point you decide whether you’d rather
      a) keep a constant guard on your sheep
      b) give up on keeping sheep
      c) keep sheep, and accept that one day in ten one of them will be eaten by a wolf.

  3. Bugmaster says:

    I think you’re letting CDC off the hook too easily.

    We can’t perform randomized controlled trials of parachutes, because killing N/2 people is somewhat unethical. However, I am reasonably sure that any new parachute designs are still thoroughly tested. While we can’t throw a live human out of an airplane, we can e.g. throw out a human-shaped dummy wearing a parachute. We can build machines that repeatedly test various operation modes of the parachute without the expense of an airplane ride (e.g. in a wind tunnel). We can hand out parachutes to real humans on the ground, and observe how they deal with the ergonomics. We can stress-test each part of the parachute individually, and so on.

    Even if human RCTs for masks are somehow impossible to rig up (which I doubt, incidentally), what’s the CDC’s excuse for (apparently) not testing masks at all ? There’s a huge difference between saying, “masks are totally useless because we haven’t done RCTs”, vs. saying, “while we have not done RCTs, masks have an X% chance of stopping 95% of mucus particles when worn, according to our battery of non-human tests”.

    • SolipsisticUtilitarian says:

      Agreed. Another reason for not going easy is the outright lying that objectively did occur, when experts were asked for the reason why masks would not work. You can’t claim that you were simply following a scientific principle of cautiousness when you had no problem being very creative with the science to explain to people why they should not buy up masks. It’s one thing to say: “most people will not wear masks properly, so we can’t recommend them”, but saying any of the following is a different beast:

      “virus spreads mostly through surfaces, so masks won’t help” (not true as it was very early known that it spreads mostly through air)
      “the virus spreads mostly through micro-droplets/is airborne, so masks won’t help you” (first part could have been true at that time, but masks will still protect from larger droplets, so they would help)
      “masks only work 20 minutes until they get too wet” (not true, and was never a true scientific consensus)

      All can be found here: https://montrealgazette.com/news/local-news/coronavirus-why-masks-dont-work/

    • Aapje says:

      @Bugmaster

      I’m wondering whether there is a compound or element that:
      – is water soluble
      – can be detected in the bloodstream if inhaled, even in small quantities
      – is harmless in reasonable quantities

      If so, one might be able to use that for testing the efficacy of masks to reducing your own exposure and possibly even how wearing them reduces exposure of others. For the latter, you might have people rinse their mouth with a solution containing the compound, step in an intermediate room, put on a mask and step into the measurement room, where they get to interact with a surface, machine or even person who is then tested for exposure.

      • Garrett says:

        One of the many problems is that the disposable N95 respirators just don’t fit nearly as well as the permanent ones. And fit-testing a disposable N95 respirator always seems strange to me – at best you’ve confirmed that you can at least once get it on correctly. And then promptly throw that one away.

        I recently got fit-tested for a reusable respirator. It’s full-face, the same style worn by fire fighters. Heavy rubber and a bunch of straps I could cinch down tight. And I had to shake my head in all sorts of directions to make sure it wouldn’t become loose at all. Fantastic. Granted, I couldn’t see after 5 minutes, but I didn’t have to worry about anything getting inside.

        Long-term, I suspect that this is the kind of thing we’ll have to go to if we’re worried about healthcare providers getting serious respiratory ailments. The disposable N95s are okay, but not bet-your-life-on-it good.

        • craftman says:

          I received a fit-tested reusable respirator for my work as a college-intern in a copper mine. Surely a profession ER doctor or nurse is worthy of the same consideration. And the rubber is far more comfortable than a paper mask (though it was brutal after 4+ hours without taking it off…)

          • albatross11 says:

            Your respirator is probably designed with a one-way valve to let you exhale. That lets you avoid getting the filter medium wet and makes it less of a hassle to wear, but it means that respirator is protecting you from others, but not protecting others from you. Further, you probably didn’t need to worry about carrying contamination from one place to another in the copper mine, but doctors and nurses definitely need to care about carrying an infection from patient A to patient B via their mask or gloves or whatever.

          • Thomas Jorgensen says:

            but serious protective gear can be built to allow you to, for example, nuke it with UV between patients. UV filter in the face plate, full skin coverage, go into the “airlock” assume the virtu man position, turn on lamps, turn off lamps, proceed.

          • craftman says:

            albatross11 you’re exactly right, I didn’t even consider the non-filtering on exhale. Oops.

            Still, that thing fit like a glove every time with little effort. Filtration media could be popped out, sterilized, and re-used between patients. Just the rubber mask remains potentially infected…why not make some kind of wall-mounted sterilization station where you just stick your face right up to it and it blasts your mask with UV or disinfectant mist…I obviously thought about this for 10 seconds so hit me up with another obvious reason why this is dumb and doesn’t exist, haha.

          • albatross11 says:

            Yeah, I definitely agree that N95 masks are a lousy design–they’re really hard to get a good solid seal on. I can pretty easily get a good enough seal on my P100 mask that I don’t smell smoke, but that seems unreasonably hard with an N95 mask.

            I think a sensible design would be to have a permanent part of the mask (like most P100 masks) that was solid and fitted to you and would reliably give a good seal, and then have the filter medium hook onto the outside of the mask and make a good seal. That way, you’re not trying to get a piece of filter medium with a little rubber on the inside and a paperclip-thickness piece of metal over the nose to make a good seal with your face. You could do something like that with the whole N95 mask clicking into place, and the part of the mask exposed to patients would be the part you would swap out and either clean or discard between patients.

            For the kind of respirator you’re talking about, I think it would make sense in principle (maybe not in practice now with the shortage of good masks) to make a “beak” of N95 filter medium and tape it to make a seal around the outlet valve. Then, your exhaled breaths also go through the filter medium, but you have a little extra surface area so you don’t have to do so much work to get the air out.

          • AlexSpark says:

            The usual concern with medical PPE is with both not contracting illness, and not spreading illness between patients. This is why it has to be disposable – if a patient has coughed on the outside of your mask, you don’t want to be taking that mask into the next room with your medically fragile immunodeficient patient. Nurses are often going straight from eg. helping a patient with explosive diarrhoea use the toilet to helping someone eat or take medication.

            Obviously that’s all going out of the window because there’s a shortage, so a reusable respirator (which can probably be sprayed with disinfectant between patients) is likely a better option, especially when everyone is already infected with the same thing.

      • Bugmaster says:

        You don’t really need to detect the element in the bloodstream (though it would make the test more accurate, of course). You can just strap the respirator to a dummy that “exhales” some particles, and have it “breathe” onto whatever detector you’re using. This way, you can still check for fit (assuming your dummy is reasonably accurate), though admittedly not as accurately as you would on a real human. Obviously you’d need to use multiple dummies, since humans come in a variety of shapes and sizes.

    • Scoop says:

      Not only could the RCTs have been done. It’s nearly criminal that they weren’t done.

      Stopping the spread of disease is a huge concern, even in normal times. If there’s a tool that might significantly help, it needs to be thoroughly tested as soon as it comes out.

      Paper surgical masks have been around, and dirt cheap, for years now. If subsequent trials show them effective at reducing the spread of a wide variety of ailments — meaning we could have had significantly fewer cases of most communicable diseases, every year, for many years now, if we had adopted a norm of wearing masks the second you feel sick — it would be hard to overstate how large a failure it was for medical researchers to miss that.

    • beleester says:

      Scott wrote about this on his post about masks – they do test masks, part of his training with masks was testing if he was wearing it properly by spraying a sweet smell outside it – but it’s apparently somewhat hard to wear a mask so that it seals properly. So the fact that your mask manages to block 99% of particles from passing through in the lab doesn’t necessarily prove that it will be wildly successful at keeping viruses from getting into the noses of people.

      (The problem is probably even worse for homemade masks – I tried wearing a scarf and gave up when I realized that constantly having to adjust it was making me touch my face more than I normally would.)

      The argument for wearing everyone wearing masks relies on there being a dose-response relationship – even if the mask is improperly worn and only stops 50% of virus particles, it’s still helpful because your immune system has 50% fewer viruses to catch. But that’s much harder to test without infecting people.

      (To go back to the parachute analogy, you’ve learned that your human parachutist is twice as heavy as your test dummy. Does this mean that they’ll still get slowed down, just half as much? Or does this mean that the strings will snap and the parachute will provide no help at all?)

    • albatross11 says:

      A major point about masks is that, other than during the actual pandemic when there’s a run on them, they’re about the cheapest measure imaginable to prevent the spread of a disease. Shutting down all theaters puts a bunch of theaters out of business; closing the schools deprives kids of schooling and parents of child care. Telling everyone to wear a mask costs a few cents a day for masks, plus some danger of people messing up their risk-balancing behavior and putting themselves in more danger than their mask reduces.

      This means that even a small expected positive effect for masks is still worthwhile. A 10% probability that masks will significantly decrease transmission is worth advising the public to wear masks and having all people in public-facing jobs wear masks, because if we’re wrong, we’re not doing much harm.

      Similar things can be said for telling everyone to wash or sanitize their hands regularly, or having your cleaning staff spray high-touch surfaces with disinfectant several times a day. The cost is low enough that even if the expected benefit is pretty low, it’s still worth it.

    • matthewravery says:

      Complicating the issue is the question of the size of aerosol droplet required to transmit SARS-CoV-2 vice other things like influenza and MERS. My understanding is that influenza and MERS can transfer very extremely small aerosol droplets, while SARS-CoV-2 may require larger droplets.

      Facemasks don’t stop all things the same. “95% of mucus particles”s is a pointless measure, since mucus particles come in multiple sizes. (One presumes…) What size mucus particle do they stop? Which size mucus particle is required to transmit a disease?

      Masks stop almost all big things and almost no small things, with “big” and “small” being relative to the quality of the mask’s filtration. (In the case of N95, that’s 95% of 0.3 micron particles.) For viruses that can transmit in very small aerosol droplets, the effect of masks could be trivially small. If SARS-CoV-2 requires larger droplets, those same masks could be wildly effective.

      • albatross11 says:

        The way I understand it, N95 masks are expected to stop 95% of particles the size of the smallest airborne respiratory droplets that are believed to carry infection and P100 masks are expected to stop something like 99.7% of particles that size. And that those particles are actually the hardest size to filter–smaller and bigger particles are less likely to get through.

  4. Sniffnoy says:

    Ebola link is broken (you seem to have put in the title rather than the URL).

  5. Deej says:

    “superforecasters estimated a 23% chance Britain would choose Brexit just before they did”

    It’s also just not that small of a chance. Call a coin toss twice in a row and no one’s that shocked really.

    • baconbits9 says:

      These things aren’t supposed to be coin tosses.

      • Stephen says:

        When the world’s best prediction experts are saying there’s a 23% chance that Britain will choose Brexit, it sounds to me like the best model of the real world is a (double) coin toss. It’s fine to say “we should really be able to do better and get close to either 100% or 0%” but unless you have a proposed method for actually doing better across the board, some things are just harder to predict than others. There will always be uncertain forecasts of the future with probabilities like 23%, and a 23% chance is saying just that – it’s a (double) coin toss.

      • notpeerreviewed says:

        These things aren’t supposed to be coin tosses.

        Can you explain what you mean here? Close votes are events with uncertain outcomes, and it seems to me that makes them a lot like coin tosses.

        • baconbits9 says:

          Coin tosses are unconditional outcomes that are well measured (or measurable). Elections are changing events, the odds of winning a presidential election and losing the popular vote has changed over the past few centuries, and there is not much reason to assume that each state vote is independant (ie if Trump wins PA then it is more likely that he wins other swing states.

          Finally the big one is that yelling ‘Heads’ when the coin is flipped doesn’t change the odds of heads coming up, but claiming Trump is 1%, 10% or 100% to win might actually shift the election odds.

  6. InvalidUsernameAndPassword says:

    I met her through the same probabilistic reasoning forum where I did my cryonics survey years ago.

    That must’ve been a meet cute for the ages!

  7. Deej says:

    I think the answer is they didn’t beat the experts in epidemiology. Whatever probability of pandemic the experts and prediction markets gave for coronavirus getting really bad, these people didn’t necessarily give a higher probability. They were just better at probabilistic reasoning, so they had different reactions to the same number. There’s no reason why smart people shouldn’t be better at probabilistic reasoning then epidemiologists. In fact, this seems exactly like the sort of thing generic smart people might be

    I wonder if it’s something else along the lines of being easier to reason from the outside is some instances?

    Shutting down the economy, travel bans etc. have real consequences and very serious consequences. It’s far easier to say you should be shutting down now (or getting the young infected if you’re Hanson) when people aren’t doing that than it is to actually shut down now when it still might be the wrong thing to do.

    There are more issues around alarming people also, when you’re commenting from gov or as an authority figure.

    Also, a major problem in the UK, and no doubt other places too, is that the scientists involved in planning had the assumption that shut down just wasn’t possible in the UK. That wasn’t their field, but it was either not challenged or it was actively agreed with/driven by politicians. They new it would be or was likely to be bad but assumed certain solutions were not possible, made a plan based on that and stuck to that plan for too long. Note: Former head of public health in the UK was dead against this and protesting on Question Time (BBC politics show). And this doesn’t explain lack of action on PPE and ventilators in the UK.

    But overall point is that it’s easier to see what’s going wrong from the outside sometimes. And for some of the people listed I’m pretty sure they – subconciously perhaps – weighed up the cost benefit of their opinions. Good to be an early protestor if you’re right, can quietly slope off if you’re wrong.

  8. OptimalSolver says:

    The Daily Beast complaining that “coronavirus, with zero American fatalities, is dominating headlines, while the flu is the real threat”.

    I wonder how many people have died of regular flu since the start of the pandemic.

  9. Alex Godofsky says:

    I really hate the parachute example.

    First: you know what, I’ll just say, unless I were in a crashing plane and had no other option, I would not use a parachute that had not gone through an RCT. I don’t care how much you insist on “common sense” or “logic”. Actually, there are probably dozens of ways a parachute can fail to work however smart the designers are and I want evidence that those failure modes have been avoided.

    Obviously the RCT will be on dummies, not humans, but I gotta say: I would prefer not to be the first human user of a new model of parachute, either.

    (For what it’s worth, you can drop the R, and really the C as we’re all aware of the control outcome.)

    The other problem is that the two outcomes are really, really obvious and completely opposite: if you jump without a parachute you will die almost every single time, and if you jump with a parachute you will live almost every single time (or else it’s a really bad parachute design).

    Compare masks, where a lot of people who wear masks, especially in the situations where masks are important (when you are sick or around many sick people), will still get sick or transmit their illness; meanwhile, a lot of people who don’t wear masks won’t get sick. There are many different factors and possible confounders.

    The number of confounders on “surviving jumping out of an airplane” is basically zero.

    Does this change the decision problem? Not particularly – especially during a crisis, the equivalent of “in a crashing plane with no other option”. It doesn’t change the correct recommendation. But it should affect our beliefs about the underlying truth, and I have to say I’ve seen a lot of the mask advocates get way out over their skis on the latter.

    (Also, it leads me to believe that the real reason the CDC was bad on masks for so long was institutional inertia with respect to a cached calculation – they reflexively defended their own guidance, guidance which was definitely inappropriate in its new context.)

    • Alex Godofsky says:

      Oh hey guess what? Actually every model of parachute does have to go through a series of extensive test requirements set by the FAA. And those requirements include not only testing a whole suite of individual components for weight capacity and so on, but actual trials with dummies. And most of the trials that aren’t actually “drop the parachute from a plane and see if it works” require the involvement of “an anthropometrically diverse group of individuals from the intended user group”.

      See SAE AS 8015B – Minimum Performance Standards for Parachute Assemblies and Components, Personnel on page A-27 of this PDF.

      So maybe we should just conclude that the BMJ’s joke, while amusing, was also ignorant.

      • Bugmaster says:

        Ok, so why can’t the CDC do the same with masks ? I understand the problem with confounders, but surely they could at least run enough tests to say, “this mask has an X% chance to stop 95% of particles when worn by an average human” ?

        • Alex Godofsky says:

          “This mask stops 95% of particles when worn by an average human” is not the outcome we are concerned about; the outcome we are concerned about is “did I get sick”.

          • stevepostrel says:

            The parachute companies do not use a control group to test effectiveness, which is the point of the example. They aren’t tossing dummies out without parachutes.

          • Bugmaster says:

            Firstly, masks don’t prevent you from getting sick, just as parachutes don’t make you immortal. They just prevent you from dying/getting sick in a very specific way. And I think that some kind of an ideal Platonic mask that stops 100% of particles 100% of the time would indeed prevent you from getting sick via particulate transmission of the virus. You could still get sick from licking infected people or, say, almost drowning in a lake.

          • Alex Godofsky says:

            We know the control outcome without a parachute. Virtually 100% of the time, a small number (countable on one hand?) of extraordinary cases excepted, the control outcome is that you die. We’ve observed this control outcome. We know it doesn’t vary substantially based on the time of day, the weather, etc.

            (Also they probably have tossed at least one dummy out without a parachute, just to make sure it actually gets damaged appropriately / they didn’t make it too sturdy!)

            We don’t know the control outcome without a mask. The majority of people who never wear masks will, in many circumstances, not get the virus. And the fraction of people who don’t wear masks but do get the virus depends tremendously on a whole host of associated factors like “do you work in a hospital” “are you riding the train to work” and so on.

          • Alex Godofsky says:

            Bugmaster: even if you narrowly tailor your definition of the outcome we care about masks – “did you get sick from this specific vector” – filtering 95% of particles is not that outcome.

          • John Schilling says:

            “This mask stops 95% of particles when worn by an average human” is not the outcome we are concerned about; the outcome we are concerned about is “did I get sick”.

            And the outcome we care about with a parachute is “did I die”, which cannot be tested with a lifeless inanimate dummy. All you can do with a dummy is observe some other factors which common sense suggests might be correlated with death of people.

          • Alex Godofsky says:

            Actually, every single time a person uses a parachute, we get to observe whether they died. This corroborates the dummy tests really, really well.

          • John Schilling says:

            Actually, every single time a person uses a parachute, we get to observe whether they died.

            Oh, they almost never die. But that’s just because the lethality of jumping out of airplanes is vastly overstated. The Parachute Propagandists have everyone so afraid of jumping bareback that nobody ever tries it. Really, it’s all in the landing technique – feet first, bent knees, you’ll be fine, but the Parachute Propagandists won’t even allow us to teach people the proper technique in case someone falls out of an airplane without a parachute. I’ll grant you that parachutes will occasionally save people a broken ankle or two, but they’re not 100% on even that. Saving lives? Pshaw!

            And without an RCT, you can’t prove me wrong.

            Sarcasm off, unless I can interest you in a jar of my patent landshark repellant balm – nobody who has ever used it, has ever been eaten by a landshark.

          • HeelBearCub says:

            (Also they probably have tossed at least one dummy out without a parachute, just to make sure it actually gets damaged appropriately / they didn’t make it too sturdy!)

            It’s like people forgot that Buster exists.

          • Alex Godofsky says:

            > Oh, they almost never die. But that’s just because the lethality of jumping out of airplanes is vastly overstated.

            It’s not, though. We also have seen people fall out of airplanes without parachutes, or with clearly defective parachutes. And we have seen people fall from similar heights, etc.

            We have fantastically clear evidence of the relationship between falling and dying and the relationship between parachutes and not dying. This evidence is not based on “common sense”; it’s based on the fact that we have actually observed all of these different events. If we had not observed parachutes working in trials you would be brave or stupid to try one.

            (For that matter, if we had never observed people falling from heights then it would probably be useful for us to at least investigate indirectly rather than simply assuming air resistance is insufficient.)

            We do not have similarly clear evidence of the relationship between masks and not getting sick. We only know the relationship between masks and some intermediate factors that we hope are significant.

          • albatross11 says:

            So instead of saying “this parachute saves people from falling to their deaths IN MICE” we can say “…IN DUMMIES?”

          • Gerry Quinn says:

            Due to air resistance, mice never fall to their death, with or without a parachute.

          • albatross11 says:

            Yet another tragic example of the treatment working out very differently in mice than in humans….

          • John Schilling says:

            It’s not, though. We also have seen people fall out of airplanes without parachutes, or with clearly defective parachutes. And we have seen people fall from similar heights, etc.

            Yes, but that’s falling, not jumping. Furthermore, they know they’re falling without a parachute, so there’s no double-blinding at all. So, still unproven, scarcely better than conjecture.

          • albatross11 says:

            I’d guess there’s a substantial placebo effect here–some subset of patients with existing heart problems will die if they believe you’re throwing them out a plane without a parachute, even if they really have one….

      • notpeerreviewed says:

        So maybe we should just conclude that the BMJ’s joke, while amusing, was also ignorant.

        Your complaint seems to amount to “sometimes satire is a bit unfair.” Yeah, the BMJ article is unfair; no one thinks insisting on high standards for evidence-based medicine is literally the equivalent of insisting on double-blind placebo trials for parachutes. The comparison uses hyperbole to point to a way in which evidence-based medicine can sometimes go wrong.

        • Alex Godofsky says:

          Except that it’s wrong in the other direction, in a way that critically sabotages its argument: we have performed very close logical equivalents to that for parachutes. The standard of evidence we demand for parachutes working is quite high, and we don’t simply admit them because “it’s obvious that they work”.

          More to the point, however forgivable the BMJ’s cheekiness is, I’ve seen quite a few people taking the argument very seriously and thinking the analogy is direct.

          • ausmax says:

            I think the analogy is better than you’re giving it credit for. You’re certainly right that I wouldn’t want to jump out of a plane with an untested parachute design. But given that we’re comparing parachutes to masks, that’s not the comparable situation. The comparable situation is we’re going to throw you out of a plane. You can either wear this untested parachute or no parachute. There are situations where people are going to be potentially exposed to the virus. I feel that they would be quite correct to use the equivalent of the untested parachute in this situation.

          • Bugmaster says:

            @ausmax:
            If you were wearing a parachute at all times, you might be tempted to jump out of the plane just for the heck of it; or maybe you’d be a bit more lax on securing the doors. If you’re wearing a mask, you might be a bit more lax on keeping up social distancing. In both situations, you risk death (or at least exposure) unless your devices actually work the way they’re supposed to.

          • ausmax says:

            @Bugmaster

            It is true that you might do those things, but then again you might not, and if you didn’t you would almost certainly be better off than if you didn’t wear a mask/parachute.

            There are lots of people out there (let’s call them grocery store workers) who are all jumping out of planes regardless of their parachute situation. I don’t know what it’s like where you are, but in my local grocery store only a subset of the customers are wearing masks. I can’t help but think if there had been less misleading advice on this issue, that this particular group of people would be less apt to jump without a parachute. They’re jumping regardless.

    • Stephen Frug says:

      The number of confounders on “surviving jumping out of an airplane” is basically zero.

      Oddly enough, this isn’t true. I don’t know if the factors have been well studied, but there are a surprising number of cases (it was surprising to me before I read about them, anyway, when I also assumed there would be none at all) in which people survive jumping out of an airplane with all parachutes failing.

      To be clear, I’m not saying this is a good idea, or at an RCT is needed or reasonable, or that you don’t get terribly hurt. But it’s possible.

      A few cases:
      • An experienced skydiver named Michael Holmes (if you want to google you’ll need to google “Michael Holmes skydiver”) fell about two miles and survived.
      • Dave Hartsock, a skydiving instructor, taking a student, Shirley Dygert, on a jump, had their parachute fail. Hartsock maniuplated things so that he cushioned Dygert’s fall. She had comparatively minor injuries; he was badly crippled. But they both did live!

      And there are more. Again: not a good idea. But since we’re talking (in part) about making assumptions about what’s true, I thought I’d mention it.

  10. anujdayal says:

    Scott, isn’t a part of your article implicitly assuming (rightly/wrongly is not for me to judge, especially in context of what I’m saying below) that the reaction (of mass lockdowns) to the coronavirus disease is in fact the correct response, whereas some more balanced opinions on this topic seem to be that the jury on that is still out (and I believe will be for a reasonably long period of time)? https://unherd.com/2020/04/jury-still-out-on-swedish-coronavirus-strategy/ the reasons I feel your article assumes so is because in case the Sweden approach is found to be correct, then some of your examples of the more extreme and one-sided early media responses to Covid might be fine, viz. “Why we should be wary of an aggressive government response to coronavirus (it might “scapegoat marginalized populations”)” , I feel the current media narrative which seems to be very certain that a lockdown is the right way to go also needs to be more probabilistic/nuanced. This becomes even more true in countries with low coverage of social security nets where lockdowns can result in deaths as well because people simply can;t buy food or have to traverse huge distances to get to their hometowns : https://theprint.in/opinion/more-than-300-indians-have-died-of-the-coronavirus-and-nearly-200-of-the-lockdown/400714/

    Apart from in countries with low social security coverage, where the problem is acutely humanitarian in the short term, for more developed nations when/if these lockdowns are found to have not completely solved the problem in the course of next few weeks, necessitating intermittent lockdowns that would lead to humanitarian problems in the medium term (my guess is 3 mos onwards). In summary my argument is: a large part of why covid-19 has become such a big issue is the lockdowns (because actual damage from covid-19 is still an unknown including death rates) and we might find the lockdowns were not the right strategy and hence people who thought it was not such a big deal were probably right but their version of events became less probable precisely because it lacked a self-reinforcing spiral that the “panic and mayhem” version benefited from.

    • MereComments says:

      100% agree. I think it’s still way too early to be calling out who got this right and who got it wrong. Right now the MSM is getting excoriated by people I mostly agree with, and as much as I love to see it, I have a feeling that some of those initial takes might being getting a third look six months from now when we’re not trying to assess the carnage from inside the storm.

    • Randy M says:

      then some of your examples of the more extreme and one-sided early media responses to Covid might be fine, viz. “Why we should be wary of an aggressive government response to coronavirus (it might “scapegoat marginalized populations”)”

      Pretty sure this is still wildly wrong. The problem with the aggressive response is the people out of work, not the marginalized populations being scapegoated.

    • Deej says:

      This comment seems spot on to me. Worth Scott addressing in a future post I feel.

    • Atlas says:

      Agreed. It seems too early to me to judge definitively what the best assessment of the risks and optimal responses is/was.

      • Scoop says:

        I think most of these comments miss the actual point a bit:

        While I would agree that it’s too early to say what the optimal response should have been (and whether there was any way to prove that was the optimal response beforehand), most of the linked-to articles argue that COVID-19 wasn’t going to be a big deal, and only stupid paranoid people thought it was going to be a big deal.

        That is definitively wrong.

        • Atlas says:

          While I would agree that it’s too early to say what the optimal response should have been (and whether there was any way to prove that was the optimal response beforehand), most of the linked-to articles argue that COVID-19 wasn’t going to be a big deal, and only stupid paranoid people thought it was going to be a big deal.

          That is definitively wrong.

          But at least my point, and perhaps that of some of the other commenters as well, is that it still seems too early to definitively say that those errors were worse/more consequential than actual past/future potential errors in different directions. (E.g. IHME projections.)

          • Matt M says:

            Yeah. Perhaps in hindsight we’ll look back on “COVID is no big deal and will be no worse than the flu” as yet another noble lie. Something that technically speaking was wrong, but the whole world would have been a lot better off it everyone just believed it…

          • Scoop says:

            I don’t see how the fact that present/future errors may be worse is reason to ignore massive errors of the past.

            Whatever the future proves to be the best way to have dealt with COVID, saying with absolute certainty that it wouldn’t be a big deal is among the most massively wrong predictions ever. And nearly every single member of the world’s elite made it.

            Surely it would help to reduce future errors if we take a look at why everyone made the first error and try to correct for that problem.

    • Purplehermann says:

      Even if sweden “called it right”, the media weren’t running the articles they were because natural herd immunity is the way to go

    • eric23 says:

      If you priority is to “not overwhelm the health system” rather to save lives, then yeah, Sweden’s response looks pretty good. If your priority is to save lives in the long term, then no.

      Sweden does not have a lockdown, but it does have voluntary social distancing and cancelling of large events, which has a significant negative impact on society.

      Most European countries by now have a trend of decreasing new case numbers per day. Sweden is one of the few that does not. That means that in the long term other countries will get to a minimal number of new cases and be able to fully resume public life (except for existing cases and their contacts – these individuals will be quarantined). Whereas Sweden will continue indefinitely with its relatively high number of cases, leading to a much higher death toll, and probably a higher economic toll, overall.

      In general, the sooner and more drastic the lockdown, the less impact it has overall (integrated over time until the virus is eradicated). That is how exponential growth works.

      • Christophe Biocca says:

        That means that in the long term other countries will get to a minimal number of new cases and be able to fully resume public life (except for existing cases and their contacts – these individuals will be quarantined).

        That presumes that the country can identify who has the disease reliably enough that they’ll stop growth from going exponential again. This seems unlikely given:

        – The large fraction of asymptomatic cases.
        – The relatively high R0 under anything but the strictest lockdowns.

        Those two together defeated even Singapore’s contact-tracing efforts, forcing them to switch to a lockdown. If this is inevitable, then Sweden may have more deaths, but it’ll reach herd immunity sooner, than countries that banked on actually getting rid of the disease.

        • John Schilling says:

          – The relatively high R0 under anything but the strictest lockdowns.

          What is the R0 of COVID-19 under conditions of modest lockdown? And what is your definition of a “relatively high” R0, as compared to a modest or relatively low one?

          • Christophe Biocca says:

            Yeah, I guess I’m being weasely here with that statement. I’m taking eric23’s position that the current levels of lockdown seen around the world are sufficient to bring R0 below 1, but Sweden’s relaxed measures aren’t, for granted, and attacking the argument that “stricter measures let you go back to fully-normal quicker, so they’re worth it”.

            That argument would be true if contact-tracing was perfect, but it isn’t. It would still be true if the people we missed would become symptomatic sufficiently quickly that if we missed someone, we’d find out before they infected dozens, but a large fraction of the infected never develop symptoms so we can’t rely on symptomatic cases to flag where we missed people.
            That would still be salvageable if the net effect of contact-tracing would be to decrease the infection rate below 1, but we know the infection rate in a back-to-normal environment would be high, and so we don’t need to miss a lot of cases for the overall spread to continue. And then I bring up Singapore going into lockdown as a concrete example that contact tracing (even starting from a manageable level of infections) isn’t sufficient to keep the virus from spreading, which means that’s not a viable “post-lockdown” plan.

            South Korea’s plan to keep mass-testing people seems to be working though, so maybe I’m over-extrapolating from one example.

        • eric23 says:

          No, Singapore is not the ideal place for prevention of coronavirus. All the way into April it had large public events going on as normal (including malls, casinos, school graduations, etc.) and government policy of discouraging face masks. It may have been the first to apply effective tracking methods, but now those methods are well known and can be applied by any country. So it is by no means the upper limit of what can be achieved without lockdown. Taiwan, with almost zero cases, is the upper limit.

          In any case, Singapore’s tracing methods were enough to contain the virus to a minimal number cases from the first detected case in January all the way until April. Right now there is a major outbreak among foreign workers. I imagine that language/culture differences made case tracking more difficult among Singapore’s foreign workers. But just like a temporary lockdown of one city was enough to eliminate the outbreak in a Korean megachurch (Daegu is now down to zero new cases per day), we can expect that a temporary lockdown plus the existing testing and tracking measures will allow Singapore to eliminate this outbreak too. And then they can go back to the methods that worked previously. And so can any other country that reaches a low enough number of cases that there are resources for effective tracking and testing.

      • The Nybbler says:

        That means that in the long term other countries will get to a minimal number of new cases and be able to fully resume public life (except for existing cases and their contacts – these individuals will be quarantined).

        This presumes elimination, which presumes you can keep the number of cases small enough that you can find all of them and isolate them before they can spread beyond your capability to do so. This does not seem practical, due to the prevalence of asymptomatic cases. You’d need to be testing the entire population regularly.

        Whereas Sweden will continue indefinitely with its relatively high number of cases, leading to a much higher death toll, and probably a higher economic toll, overall.

        Indefinitely? I’m pretty sure it doesn’t work that way.

  11. Chris Phoenix says:

    For myself, I made the bad mistake of assuming the government would be effective in preventing the pandemic from getting a foothold in the U.S. This was a reasonable assumption; I remember SARS, which was stopped even after it went intercontinental; and Ebola, which was stopped near the source despite some of the involved nations being dysfunctional in important ways. I don’t even remember MERS.

    [EDIT: Saying it’s a “reasonable assumption” should not excuse me for failing to consider the probability that the assumption was wrong, and implications if it were.]

    Part of the trouble is we got unlucky with a disease that hides better than many others. Should we have been able to calculate the probability that this disease would have that feature?

    We probably should have given a Populism Penalty to any optimism, for both U.S. and international institutions. But few people really understand how important competent leadership is in a crisis, and I suspect understanding that is anti-correlated with being good at math and probability.

    I’m tempted to blame the current U.S. tragedy on bad U.S. leadership. Certainly the virus would have spread slower if we’d had more testing earlier. And in hindsight, we should have had stronger encouragement to lock down sooner. We might even have been able to hammer-and-dance effectively, which seems unlikely now.

    On the other hand, I see several other Western democracies doing even worse per capita, and Japan and Singapore not flattening their curves. On the gripping hand, it remains to be seen whether we’ll overtake the current front-runners – I’m not optimistic given our size and heterogeneity. US cases have been dominated by the New York bump, and we haven’t yet seen the Red&Rural bump or the Nursing Home bump.

    Whatever lesson we learn here, we should apply to other potential catastrophes. The world has come close to all-out nuclear war several times – and some of them were sheer accident. We still have missiles ready to fly. If you knew there was a 5% chance of accidental large-scale nuclear exchange in the next decade, and you were good at probabilistic reasoning, what would you do? How confident are you that you have enough information to estimate the odds, and that the odds are better than 5% per decade?

    • aristides says:

      I made the same mistaken assumption, and I’ll add that I also made the assumption that the mortality rate was too low to be worried. This was in part driven by early estimates that the mortality rate was only 0.2, no deadlier than the flu. The larger part is I thought I had to only worry about pandemics like SARS that have a 10% mortality rate. Turns out even a 2% mortality rate is crippling.

      • Anteros says:

        I made the same assumption as you, but I think the fatality rate will be only a tenth of 2% (Oxford’s centre for evidence based medicin estimates 0.1 – 0.39 link text)
        It still means the virus is currently the leading cause of death in America – something I have to remind myself of every time I get too sanguine about mortality rates

        • The Nybbler says:

          0.2% is almost certainly too low; it’s at ~0.1% total mortality (not IFR or CFR) in NYC now. There’s a good deal of fuzziness in those numbers, e.g.

          How do you count the “presumed” COVID deaths? Given that positivity among tested symptomatic people is 40-50%, I think counting them all is probably an overcount, but the age structure matches pretty well, so perhaps they are _mostly_ COVID deaths.

          What percent of NYC deaths are non-residents treated in NYC hospitals (more likely than the opposite, I believe)

          But unless deaths stop today (not likely) I think there’s unlikely to be enough fuzziness to support a 0.2% IFR, even if essentially all of NYC is or has been infected.

          (I would put 1% as a likely ceiling on IFR, 0.6% on the low end, given US population structure)

          • Anteros says:

            Agree about the fuzziness for all the numbers.

            I wonder if there’s anything about NYC that would increase the mortality rate (apart from the non-residents thing)?

            I found the Oxford Center’s analysis pretty convincing (tho’ it does fit my biases….) and it was updated today, so they are aware of the NYC situation.

            The most striking argument in the link I gave was that with swine flu, the final consensus on the IFR was that it was less than a fifth of the lowest estimate made during the outbreak. I guess for that to be the case with Coronavirus, there would have to be way more infections currently than are estimated, which if true, would at least be good news.

          • The Nybbler says:

            Note that global IFR is likely to be considerably lower than US or China IFR due to the age distribution of the populations. My guesses are for the US, not global.

    • Matt M says:

      Yes, this is the same for me. I went from “this will probably not be a very big deal” to having a very serious and emotional conversation with my fiance about how we will need to start considering re-scheduling our wedding (which was supposed to be this Saturday) the second I read an article stating that community spread had been confirmed in the US.

      That was the tipping point.

    • Act_II says:

      Yep, I came down to make a similar comment. I have no qualms about blaming leadership, though, at least in the US. We used to be pretty good at responding to infectious disease until this administration dismantled the global health security team — and that’s not even getting into the issues with communication from the Oval Office. I don’t think the fact that other Western countries are having similar problems weakens this argument. In fact, I would even expect the outcomes to be correlated; the groundswell of support for demagogues who care more about strongman posturing than public health has happened across the world, and when too many countries have a weak response — particularly critical ones like the USA — that makes it harder for the remaining countries to control the spread.

      In short, I’m sympathetic to the experts and the media for assuming this would basically play out like swine flu. Maybe, with the kind of global leadership we had in 2009, it would have.

      • sharper13 says:

        What “global health security team” do you believe was dismantled? Are you referring to the single individual on the NSC who no longer works there and the two others who were doing the same job as part of a reorganization, or to something else?

    • DinoNerd says:

      I also started out presuming this was going to be a flash in the pan, with local hot spots and dire predictions, like SARS. I updated to “holy shit; enough people are panicking that I had better do some prepping,” in time that I still have some toilet paper. ;-( I’m still flat out confused about whether I should be especially worried about catching Covid-19, as compared to any of the other hazards of modern urban living; the reported numbers so far aren’t especially scary, expressed as a % of the world population. (Ditto for numbers in my country, as a % of national population.)

      To the extent that I worry about catching it, it’s a combination of my social brain (“everyone else is yelling WOLF, there must be one somewhere”) and reasonably strong evidence that the reported numbers – even of deaths – are wrong, perhaps by orders of magnitude, due to poor testing (and possibly conscious efforts to avoid alarming people, by setting rules for what counts as a reportable CV-19 death in a way that grossly underreports).

      Meanwhile, I’m intensely curious what the data will look like, after the lockdown is over and researchers have had a chance to pick over the statistics. What will we the excess all cause mortality for this period turn out to have been? Will there be a way to distinguish likely-undiagnosed-covid-19 from other causes of death? (We might have e.g. fewer accidental deaths due to the lockdown….) Will we ever have a good idea of 1st outbreak fatality rate for this disease, or whether it really is worse for some demographics (age, gender, blood type, etc.) than others, in the absence of confounders like crowding, smoking, and socioeconomic status. (If it becomes endemic – and my guess is that it almost certainly will – then we’ll eventually have this data for later cases – but those are likely to include lots of subjects already exposed to [an earlier generation of] the virus.)

      I’m not going to hazard a guess about whether the lockdown was/is the best response. We don’t have the data, and probably never will. The more effectively the lockdown works at preventing cases/mortality, the less needed it will look. And we’re always going to have disagreements about the value of lives saved/deaths delayed, compared to other costs.

      • Matt M says:

        We really have to hope and pray that Sweden stays the course and gives us at least one example of “here’s what happens if you don’t have severe lockdowns.”

      • Meanwhile, I’m intensely curious what the data will look like, after the lockdown is over and researchers have had a chance to pick over the statistics.

        What bothers me is the worry that if it turns out, on objective study, that the orthodox response was in some way badly wrong, that the death rate was much lower than believed or that lockdowns were a serious mistake or that the CDC advice on masks made things much worse, most people will probably never hear about it. Except for people who already believed whatever conclusion that information supports.

        How many of the people who, fifty years ago, were warning that population increase would create terrible effects over the next few decades, at least in poor countries and perhaps everywhere, if not dealt with, suffered any serious loss of reputation when their warning turned out to be wrong? How much of the general public revised down its respect for such authoritative claims in consequence?

        • Scoop says:

          Depending on how they phrased their predictions, you can’t really judge them.

          Yes, you can certainly judge the many who predicted that population growth would lead to mass starvation. They were wrong because they didn’t predict that agriculture would grow massively more productive. Or even leave themselves wiggle room. (But, frankly, was there any reason to predict yields would rise so quickly? Did we simply escape mass starvation via very good luck? Were those predictions that justified 90% confidence in 1960? I have no idea.)

          You can’t, however, make any judgement whatever about a 1960 prediction that people in developing nations would be way better off in 2020 without rapid population growth because there’s no alternate world where population growth never happened. Maybe per capita world median income would be two or three times what it is today.

          Climate change would certainly be a lot less of a problem now if the world population had leveled off at 1.6 billion in 1900 — unless, i suppose, economic growth was so rapid that all 1.6 billion created emissions at the rate of modern-day Americans.

          • sclmlw says:

            If I can’t judge someone who continues a trend line out fifty years, predicting disaster because the trend is inevitable, the least I can do is not take them seriously. I certainly don’t think it’s justified to back-calculate current data onto their old model and say, “it would have been better if we’d just listened!”

            Trends continue until they don’t. Individual technologies improve in fits and starts. So extrapolating an inevitable future disastrophe because of current trends +50 years should at the very least carry a caveat that the prediction is low-probability.

          • But, frankly, was there any reason to predict yields would rise so quickly?

            The second edition of The Population Bomb came out after Borlaug got the Nobel Pace Prize.

            You can’t, however, make any judgement whatever about a 1960 prediction that people in developing nations would be way better off in 2020 without rapid population growth because there’s no alternate world where population growth never happened.

            True but irrelevant, because that was not what people were predicting. Ehrlich was at one edge of the orthodoxy, but the orthodoxy wasn’t “we will be better off if we reduce population growth,” which conceivably might be true, it was “we will be much worse off if we don’t substantially reduce population growth.”
            From the predictions of Limits to Growth (Wikipedia)

            Global Industrial output per capita reaches a peak around 2008, followed by a rapid decline
            Global Food per capita reaches a peak around 2020, followed by a rapid decline

          • profgerm says:

            Did we simply escape mass starvation via very good luck?

            As far as the historical record shows, pretty close. Borlaug had lots of failed experiments, and even his successful dwarf wheat was nearly wiped out multiple times. Perhaps someone else, somewhere, was doing similar work and they’ve been totally forgotten by history because Borlaug got there first, but it certainly looks like “one determined guy got lucky.”

          • sclmlw says:

            This still assumes that without the Green Revolution birthrates would have remained at the same rate. It’s extending a trend line off into the future, but that isn’t necessarily so. We know now that people in developed economies dramatically reduce birthrates in response to one set of economic stimuli. Why should we retain the hypothesis that people would continue high birthrates as resources became scarce to the point of mass starvation?

          • profgerm says:

            @sclmlw

            Part of the answer is in your comment: people in developed countries stopped reproducing. Less-developed countries have dropped too, but generally at a much slower rate.

            Historically, did famines significant reduce the birth rate, or did they increase the infanticide rate? I would think the latter but I don’t know if data exists to confirm this.

            There’s also a lag factor; I’m not an expert on the topic but I suspect there would’ve been a time of high reproduction continuing as food limits started, and then the intersection that caused starvations would be a few years later.

            I’d also be more concerned about the civil unrest caused.

            I’m not trying to defend The Population Bomb’s model, but I do think we were just about one unfortunately-timed storm away from the kind of crises that would’ve collapsed Mexico and India, and possibly other Central/South American countries, and thus sent significant shocks through their neighbors.

            Trends continue until they don’t. Individual technologies improve in fits and starts. So extrapolating an inevitable future disastrophe because of current trends +50 years should at the very least carry a caveat that the prediction is low-probability.

            Perhaps this is just a difference in general outlook, but I’d take the opposite interpretation. Because technology improves unpredictably, extrapolating a future catastrophe is higher-probability than not. Betting “a pandemic within the next 50 years” is safe; pandemics are frequent throughout the course of history (pin-pointing SARS being a bust and SARS-2 being serious, now that’s where one would be really impressive). Betting “well, technology develops in fits and starts, so maybe we’ll cure all disease before the next pandemic” is much less so.

          • sclmlw says:

            Betting “a pandemic within the next 50 years” is safe; pandemics are frequent throughout the course of history (pin-pointing SARS being a bust and SARS-2 being serious, now that’s where one would be really impressive).

            I entirely agree with this. This is because it’s not based on extending a trendline up into the stratosphere, but just assumes an event that has occurred with a certain frequency in the past won’t artificially change frequency. Sometimes we do things that can alter those frequencies (such as crowding more people into large cities with lots of international travel, or maybe China will work to shut down wet markets) but that will likely only change when the event happens, not whether it happens.

            Betting “well, technology develops in fits and starts, so maybe we’ll cure all disease before the next pandemic” is much less so.

            This clearly isn’t what I was arguing. I never claimed we were relying on technology to fix trendline issues. I did claim technology is uncertain, so it can easily disrupt a trendline prediction; but extrapolating from that to ‘technology will cure all ills’ is a strawman that ignores my main argument.

            I claimed that declaring a crisis because fifty years from now an exponential growth trend will become a problem is rarely justified. In finite systems exponential growth always reaches an inflection point and the curve becomes sigmoid. Where the inflection happens is unknown in advance, because it can be triggered by a variety of factors – many of which can be known in advance, but many of which cannot (like technological innovation, changes in human behavior, limiting factors we hadn’t considered, etc.). But I rarely see a trendline prediction that even includes the possibility of an inflection point, let alone one that tries to estimate when it will happen as part of the calculation. It’s exponential growth all the way up – which is entirely unjustified.

            Here’s my model: how much I discount a prediction of future catastrophe depends on what type of reasoning was used to make the prediction – frequency analysis versus trendline extension. If you’re asking whether Earth gets hit by another civilization-destroying meteor, the question isn’t if, but when. (Maybe ‘when’ is millions of years so we don’t have to worry, but that’s still a frequency-based prediction.) Same if you’re asking about an earthquake, pandemic, war, etc. But if someone is looking at a slow-moving disaster that’s coming 50-100 years down the road because they saw an exponential growth curve and drew the line out to the right I’m going to significantly discount their probability estimate. I certainly don’t think it’s reasonable to predict people will do the equivalent of watching a steamroller advance upon them at 1 mm/s and not take reasonable steps to avert disaster.

            Trendline prediction’s track record is very low frequency. Ergo the discount.

        • anonymousskimmer says:

          I thought these population effects were dealt with (the green revolution, ethnic cleansing, diseases, famine)?

        • DinoNerd says:

          What bothers me … most people will probably never hear about it.

          What bothers me more is that I may never hear of it.

    • Chris Phoenix says:

      OK, now I’m feeling really stupid.

      I remembered SARS and Ebola. I also remember AIDS. But I did not think about AIDS either when assessing probabilities, or even when writing the above comment!

      The thing about AIDS was that for years, the U.S. government acted mostly as though it cared nothing for the lives of gay men. Which is a reasonable model of their actual collective motivation – anyone remember the “Moral Majority?”

      I simply did not update my government model to say “The current government will act mostly as though it cares nothing for human life, even white rich American life” – even though there was plenty of evidence to that effect.

      If I had really been willing to believe “The government will take exactly as much care to keep your family safe from COVID as Reagan cared about AIDS in 1984” I would have known what was coming. But that would have required me to believe “You are about to lose a major benefit of your privilege.” Which is hard for people to think about – I expect pushback for this if anyone reads it – but really, a privileged person’s unthinking reliance on their privilege is a major source of bias, and we should be talking about it that way.

      • albatross11 says:

        Perhaps one reason why it seems like correct assessments of the risk came more from the fringes than from the center is exactly this insight–do you think the government is going to act to protect you and people like you, or not? Some folks think of that in terms of evil goals, but I tend to think more in terms of capabilities and incentives. I’m sure nobody at CDC or FDA wanted to screw the pooch[1] on testing in a way that allowed undetected community spread for a month or two, but somehow that’s what they did. It wasn’t “we don’t care that a bunch of you die,” it’s “given our internal incentives, institutional culture, and limits on our own abilities, we’re going to let a lot of you die because we can’t find a way to do otherwise.”

        [1] Paging Aapje.

      • Scoop says:

        A. It’s not okay to just slander the hundreds of thousands of people who do public health work for the US government by saying they spent years acting like they cared nothing for the lives of gay men. AIDS, almost from the start, got more funding per patient than nearly any other disease, despite the fact it was never going to be much of a risk to anyone but MSM and drug users who share needles, because people saw that it was uniquely horrible and needed to be treated. Were there some who said it was God’s punishment? Yes, but not many of them were in the government.

        B. “I simply did not update my government model to say “The current government will act mostly as though it cares nothing for human life, even white rich American life” – even though there was plenty of evidence to that effect.”

        We have shut down our entire society to protect lives. We will all be markedly poorer for the rest of our lives. That is not the action of a society that cares nothing for saving lives or for elected officials/civil servants who care nothing for the lives of those they govern.

        • Douglas Knight says:

          a quick google turns up this time series of federal AIDS spending, with millions of dollars: 50, 200, 920 in FY ’83,’85,’87

        • Chris Phoenix says:

          I was talking more about 1981-3 when the problem was known but little action was taken. [Edit: Yes, I mentioned 1984 above, in the context of Reagan. For these posts I didn’t re-research exactly who started caring when – but we definitely lost several years when it was just “the gay disease” and public health response was ~nil.]

          (In your list of people it was problematic for, don’t forget hemophiliacs. By the mid-1980’s half of all hemophiliacs were infected with HIV.)

    • vaniver says:

      We probably should have given a Populism Penalty to any optimism, for both U.S. and international institutions.

      I think this is misdiagnosing the issue.

      First, compare American and European populists; my impression is that Orban and the like have done better than average for Europeans, in part because they’re more willing to lock down travel and otherwise implement harsh measures.

      I think the problem with Trump was twofold: 1) he’s a believer in positive thinking, and 2) he’s a hotel guy. So even though he blocked Chinese travel into the US early ([i]against[/i] the recommendation of the WHO!), his strategy besides that was [i]literally[/i] “wishful thinking”, and turns out that’s a miserably bad strategy when dealing with a virus. And he probably was more in tune with the costs of shutting down travel and inessential businesses than others would be.

      Second, look at the ways in which CDC and WHO guidelines are bad, as discussed in the post above. I remember when Ebola was happening, and what Greg Cochran was writing about in his blog then. Many people have remarked on how Scott Gottlieb, former FDA commissioner, has been saying the right things on Twitter, and is the sort of person you would want running this. But Gottlieb was a Trump appointee! He left last year “to spend more time with his family,” but overall there isn’t too much reason (from my vantage point) to think the non-populists are significantly better at probabilistic reasoning than the populists. [Compare to selecting on probabilistic reasoning or intelligence directly, for example.]

      • keaswaran says:

        Note that Trump is doing it again. He wants to shut down legal immigration but open up domestic businesses. When the vast majority of infections are spreading through local contacts, and when the United States likely has one of the highest infection rates in the world, this really makes no sense from a practical point of view, unless the goal is to stop immigration and open small businesses, and the virus is actually irrelevant.

        • Clutzy says:

          I don’t understand what we are talking about now? The whole point of our domestic tactics is to titrate cases so hospitals are not overrun. We have under-capacity hospitals, so we need to open up to get the hospitals moving and the virus moving through the population so that we can start to approach herd immunity.

  12. Chalid says:

    There has since been a randomized controlled trial of parachutes.

    “Parachute use did not significantly reduce death or major injury (0% for parachute v 0% for control; P>0.9). This finding was consistent across multiple subgroups… However, the trial was only able to enroll participants on small stationary aircraft on the ground, suggesting cautious extrapolation to high altitude jumps.”

    • Chalid says:

      Wow, I forgot the link.

      Parachute use to prevent death and major trauma when jumping from aircraft: randomized controlled trial

      My favorite bit is the caption on the picture.

      • keaswaran says:

        That’s great!

        “The study also has several limitations. First and most importantly, our findings might not be generalizable to the use of parachutes in aircraft traveling at a higher altitude or velocity. Consideration could be made to conduct additional randomized clinical trials in these higher risk settings. However, previous theoretical work supporting the use of parachutes could reduce the feasibility of enrolling participants in such studies. … Third, the individuals screened but not enrolled in the study were limited to passengers unfortunate enough to be seated near study investigators during commercial flights, and might not be representative of all aircraft passengers. The participants who did ultimately enroll, agreed with the knowledge that the aircraft were stationary and on the ground.”

        “We attempted to register this study with the Sri Lanka Clinical Trials Registry (application number APPL/2018/040), a member of the World Health Organization’s Registry Network of the International Clinical Trials Registry Platform. After several rounds of discussion, the Registry declined to register the trial because they thought that “the research question lacks scientific validity” and “the trial data cannot be meaningful.” We appreciated their thorough review (and actually agree with their decision).”

  13. zrkrlc says:

    Referring to LessWrong as a “probabilistic reasoning forum” is the single clearest sign of how mainstream you’ve gotten, Scott. Congratulations (?)

    (I mean this unironically.)

    • John Lawrence Aspden says:

      Oh that is what he meant. I was wondering how to join this interesting-sounding place…

  14. sty_silver says:

    I wrote an email to everyone in my extended family on March 3, warning them about Coronavirus. But that only makes me feel like more of a moron because it didn’t occur to me to do what Wei Dei did and bet money on it. It made me realize that a significant hurdle for EMH is that many people just don’t think about this stuff. I had all of the information to know that it was possible to bet, I legitimately believed that Coronavirus was going to be far worse than commonly held before the stock market crashed and acted on it in many other ways, and yet I didn’t do the thing that would have made me a ton of money. It’s extremely frustrating.

    • John Lawrence Aspden says:

      I put an oxygen concentrator in the post for my parents around March 3rd, but it never occurred to me to bet the stock market would fall.

      I think if it had occurred to me, I’d have assumed that all my clever friends in the City of London who are good at probabilistic reasoning would have already snapped up all the money. It’s kind of a background assumption of mine that the market is efficient relative to me, as the prophet told us.

      Lesson to be learned there, although I’m not sure what it is. Pretty sure that the next time this sort of thing happens the City will be all over it. They’re not stupid.

      • eggsyntax says:

        It’s kind of a background assumption of mine that the market is efficient relative to me

        I make the same assumption, and that’s a better way to put it than I’ve ever heard — I’ll be stealing that the next time I have to explain to someone why I’m not investing in their pet stock. Thanks!

        PS/personal — I’m a fellow Clojurian, so it always makes me smile to see you on here 🙂

      • Gerry Quinn says:

        If you believe it’s impossible for that $20 bill to be on the ground… it won’t be!

    • My son urged me to cancel my speaking trip before it began, and I left March 1, so he had reached that conclusion sometime before that.

      • albatross11 says:

        I went to a conference in late February. I was right on the fence about going, but I had some important things I needed to do there, both personally and professionally. The conference organizers made sure there was plenty of hand sanitizer (I brought my own) and they cleared out and cleaned the rooms between sessions and generally acted like they were taking it as a significant potential threat.

      • patrissimo says:

        At someone’s request I went back and looked at my public predictions, they appear below.

        2/2: Skeptical https://twitter.com/patrissimo/status/1224066769578381312
        [This produced a bunch of interesting feedback.]
        [during intervening 3-week wife preps house & I change mind]
        2/22: A comment reply recommending people buy Bitcoin: https://twitter.com/patrissimo/status/1231330876065107968?s=19
        2/23: R/T predicting significant supply chain impact from coronavirus https://twitter.com/patrissimo/status/1231805677653245952 & poll on who is in quarantine: https://twitter.com/patrissimo/status/1231758629008621570
        2/25: I tweet my annoyance about people on Noahpinion’s twitter being mad that I am comparing predicted covid death tolls to the typical annual death rate: https://twitter.com/patrissimo/status/1232481127488581633
        2/28: My first specific prediction of a 95% CI of 0.01%-30% global infections (which at the time I think was doomsdayish compared to MSM, though not to our Twitter realm): https://twitter.com/patrissimo/status/1233634485788274688
        Sometime in these last few days of February I sold all of my publicly tradeable investments.

        I don’t think I deserve much credit for this, I had the benefit of my wife worrying about pandemic from January (which I scoffed at initially), and Balaji calling me in the first week of Feb and explaining to me in great detail why I was wrong, and then I watched the predicted trends continue. (TLDR of what he said: he personally thought pandemic was likely, he thought even a conservative read would put a significant chance of a pandemic even if not most likely outcome, and most importantly that it was already almost guaranteed there would be massive political, economic, and social impact.)

    • j1000000 says:

      I legitimately believed that Coronavirus was going to be far worse than commonly held before the stock market crashed and acted on it in many other ways, and yet I didn’t do the thing that would have made me a ton of money.

      I am not familiar with financial tools and shorting options — was there really any way for you to ensure you made a ton of money? I understand there are ways a person could do it — Spitznagel and all that. But I mean you or me, as people who want to bet on our intuitions but don’t run hedge funds. I ask because I think this current rally is crazy and would like to bet against the markets.

      The stock market is down only 7% since March 3, the day you sent that email. About half of that time it was dropping and half of it rising, so I’d assume there are easy ways any sort of aggressive shorting (which is the only way you could’ve made a ton of money) could’ve gone very wrong for you.

      I realize SPXU is a bad and naive way to bet against the market and it isn’t meant to be held, but some dumb Wall Street Bets types do buy it and hold, so I sometimes look at that. If you’d bought it the day you sent out your email, you would be down 24% right now.

      Similarly, if you knew in early February that this was going to be bad, you still would’ve had to keep your faith through a mostly positive February as you lost money, then you’d have had to have the balls to stick with it until bottom even when the market had two of the 10 best days in S&P history on the way down, and then you’d have to have sold before it began this rally.

      • baconbits9 says:

        Similarly, if you knew in early February that this was going to be bad, you still would’ve had to keep your faith through a mostly positive February as you lost money,

        I lost some money in early February in my options, but options were extremely cheap then, the total I lost being short from Feb 1st through February 21st was ~ $800, and I had SPY puts that expired worthless on Feb 21st (ie basically the worst possible day to have an put option expire), and I didn’t put another short in place until Monday, the 24th in the afternoon (iirc). That second option position cost ~ $700 and increased ~ 5x in value by feb 28th, and 10x in value by March 9th. The only way you weren’t in a position to make a lot of money with put options late in Feb/early March was to buy them specifically after a few falling days and then sell them after 1-2 rising days.

        • j1000000 says:

          Interesting, thanks. I assume people who kept shorting through bottom the way back up could lose a good deal of their gains?

          Market is back where it was March 9 — are you shorting it again?

          • baconbits9 says:

            I lost a large chunk of my gains, I lost about 65% of my gains (from peak close to now) which was very painful and hopefully educational to watch happen. Still my trading account is up ~1300% from its lows, and ~650% from Jan 1, and my other longer term moves have worked as well.

            Market is back where it was March 9 — are you shorting it again?

            Yes, slowly. I put a small short position 3 days ago, and if the S&P hits ~ 2880 in the near term I will probably double down on that. Options are ~ 3x as expensive now as they were in Feb and another leg down shouldn’t be as steep as the first (though the longer it goes up the more likely it is steep down) so smaller and more conservative positions are called for.

  15. vectorious says:

    The Mongol link for me does not link to anything to do with Mongols, but to a list of books on Bactra.org, none of which seem to be about Mongols – am I missing something?

  16. bungula says:

    Robert Wiblin from 80,000 Hours should also get credit for worrying about coronavirus, back in January!
    https://www.facebook.com/robert.wiblin/posts/886126615045

  17. davidweber2 says:

    “likely won’t be a nightmare pandemic” sentence as a prediction, she got the prediction wrong.

    By the standards of X-risk, this isn’t a nightmare pandemic. This is just a pandemic. It’s not even at Black death levels of civilization destabilizing yet, and doesn’t show signs of being anywhere near that bad. Which is not to say it won’t be scarring for several generations, and a major source of political instability. But there are many reasons why biotech is an important cause area in EA.

  18. Tom Chivers says:

    this is fantastic and gives We The Media something to think about. I try to avoid these pitfalls but god knows I don’t always manage it.

    On a separate point: I dunno if pointing out typos for editing is appreciated, so let me know if it’s not, but you’ve got a “serious” for “series” in there: “Journalist Johann Hari decided to take it upon himself to challenge psychiatric experts, and wrote a serious of terrible articles and a terrible book saying they were wrong about everything.”

  19. JRM says:

    Just logging this publicly:

    My failure was worse. In writing in late February, I predicted 100K deaths in the US with confidence over 50%.(I got a fair amount wrong – I thought the NBA would operate without fans.)

    I’m in groups of people who predict sporting events, and these people are good at this sort of thing and we debated not whether it would be bad but how bad it would be.

    I sold out my position in the market after the first minor drop. And I did *nothing* else for my own benefit. (I warned people, one of whom took me seriously, and he has given me kind words for it.)

    Putanumonit concluded with high confidence that this was going to be very bad on 2/27 and urged action, and I read that, agreed with it, updated to “even more likely to be very bad,” and still didn’t do any of the things he correctly suggested.

    That is… not a good performance.

    • Ninety-Three says:

      What are the things you ought to have done? Because the number one thing I wish I’d done was sold my stock portfolio, and there is no number two. Everything else people talk about (stockpiling rice and hand sanitizer, working from home) is so incredibly minor compared to the massive life consequences of the stock market that I feel your performance is something like 90/100: you might have only done one or two things right, but you scooped up most of the value to be had.

      Edit: I suppose “Short airlines” is arguably number two, but my point is that on an individual level, interaction with the stock market seems to dwarf everything else you can be doing unless you’re Bill Gates.

      • Edward Scizorhands says:

        Put me back in time and I do whatever it takes to make people realize it’s going to be serious, short of lighting myself on fire.

    • John Lawrence Aspden says:

      Not at all. You got the epistemic bit right, and failed the instrumental bit. Way way better than most people, and in fact I’d say you’d done the hard bit.

      Pat yourself on the brain and go work on the ‘correct action taking’ thing that you’re lacking.

      • JRM says:

        Yeah, I did the hard bit. I’m a better predictor than most and have some experience with statistical modeling.

        I have excuses as to why I didn’t do the rest of it, but the biggest thing would be long-term put options on stocks likely to be affected. There was no reason to go halfway and just sell short – options were the way to go to get real money. And to set that account up and do it would be two hours maximum, and I certainly had two hours.

        Not doing the known-correct-action just seems worse to me – it’s a character flaw, not an intellectual flaw. But I appreciate the comments!

        • John Lawrence Aspden says:

          Beware of trivial inconveniences. Learning how to buy a put option is certainly on my to-do list now…

    • Matt M says:

      Jacob’s post tipped me into stocking up on toilet paper and hand sanitizer, just before doing that became difficult, so I give him credit for saving me that headache, at least!

  20. thisheavenlyconjugation says:

    First, a bunch of generic smart people on Twitter who got things exactly right – there are too many of these people to name, but Scott Aaronson highlights “Bill Gates, Balaji Srinivasan, Paul Graham, Greg Cochran, Robin Hanson, Sarah Constantin, Eliezer Yudkowsky, Nicholas Christakis, and Eric Weinstein.”

    Cynical take: this crowd (as in the broader one not just the people listed) includes several people who often form opinions by being contrarian with respect to certain sources. This worked well here because those sources were being bad, but I think it’s easy to imagine a world those sources were saying something slightly different and the predominant contrarian narrative in February was “actually, the real virus is regular flu” or “actually, the real virus is shutting down the economy”. E.g. see Elon Musk’s takes.

    • noyann says:

      I wonder to what extent taking a contrarian view is part of their heuristics, (as opposed to being merely an outcome).

      • albatross11 says:

        Bill Gates has devoted many years of his personal attention and life to health issues, including the potential for new pandemics. So for him it’s certainly not a flash in the pan.

    • ruelian says:

      Well, there is a reason Elon Musk doesn’t get to be on that list. Just saying.

    • e.samedi says:

      Given that the situation is still unfolding, on what possible basis can the people cited be said to have gotten things “exactly right”? Have they been “exactly right” on the economic and social impact? On the longer term consequence of the various responses? It’s not at all clear that people like Gates and Cochran are even correct on the virus itself, let alone the broader impact of it; that is, when they even consider the broader impact which they rarely do.

  21. James Miller says:

    I’ve done a series of five podcasts with Greg Cochran on the virus, the first published on Feb 9, in which we (although mostly Greg as my job was to question Greg as I had no previous knowledge of epidemiology) warned about the virus. Greg thinks the reason he was able to understand the risk while journalists, politicians, and epidemiologists were not is because they but not he are mostly mathematically and historically illiterate idiots (Greg can be blunt) who don’t care that much about truth. Greg immediately pattern matched COVID-19 to the many plagues in history he has studied and easily understands the math behind how the virus might spread. Greg remembered that during the AIDS epidemic and ebola outbreak that epidemiologists cared vastly more about political correctness than reducing the death toll or the spread of the virus and so he has little trust in most epidemiologists.

    • Atlas says:

      Has Cochran made predictions about the number of global or national COVID-19 deaths?

      • James Miller says:

        I think he has given a likely range, but I don’t remember it.

      • AnteriorMotive says:

        In the podcasts he pretty consistently resisted being pinned down to any numbers, saying “there’s too much uncertainty.”

        Obviously that’s not how we roll here on probabilistic-reasoning-forum-offshoot, but when he outlined the uncertainty in question, (or when James was able to press him into finally giving a number) his picture of the coronavirus risk profile seems to have been pretty accurate/level-headed.

      • Pablo says:

        Has Cochran made predictions about the number of global or national COVID-19 deaths?

        James pressed Cochran to give quantitative estimates a few times, but Cochran was generally evasive. I recommend the episodes, though in the end I felt that Cochran’s epistemics left much to be desired (not just because of his reluctance to make explicit forecasts, but also for his failure to admit any error, as when James pointed out that he was misrepresenting Robin Hanson’s position).

    • viVI_IViv says:

      Greg thinks the reason he was able to understand the risk while journalists, politicians, and epidemiologists were not is because they but not he are mostly mathematically and historically illiterate idiots (Greg can be blunt) who don’t care that much about truth.

      I can see this applies to journalists and politicians, but aren’t epidemiologists supposed to be mathematically and hisotrically literate about epidemics? It’s kinda in their job description.

      So how did they fail? Just bad luck, maybe the pandemic was really hard to predict, or complacency due to the lack of non-STD pandemics in living memory, or political bias due to bad institutional incentives?

      • Clutzy says:

        I think, and have a top level comment here, that I think the most likely explanation is political bias and incentives associated. The problem with our current expert class is they very often live in the same bubble as media, education, etc. They all skew in the same direction.

      • Steve Sailer says:

        I’d say that this pandemic has proven objectively difficult to comprehend, much less forecast, so we should be fairly charitable in assessing each other’s failures. For example, time and again I think I’ve noticed a pattern … only to have something else happen that undermines my surmise.

        For example, why hasn’t the widely forecast apocalyptic scenario of overwhelmed hospitals happened very often (at least not yet)? Partly it’s because new infections are down due to the extraordinary social distancing, but it’s also for another reason that few saw coming: the most resource-intensive challenge for hospitals — ventilators and the huge staff needed to oversee them and the high risk of health care workers being infected — have proven less effective than was assumed. So New York doctors have switched to vastly less costly treatments like having patients roll onto their stomachs, which has reduced the need for ventilators significantly.

        But this points out that we are still relatively clueless about the precise nature of the disease we are grappling with.

      • James Miller says:

        Greg has a low opinion of the intelligence of epidemiologists, and thinks this made it extremely challenging for most of them to meet the requirements of their job description. Also, promoting social justice takes up a big part of what epidemiologists think they are suppose to do and this probably reduced the amount of time they had to read history or study logistical curves.

    • Worley says:

      I would beware of how Cochran is judging epidemiologists, as well. Approximately all of them are on the public payroll, directly or indirectly, and so have to be careful not to say anything in public that is non-PC. If you want to test what they think is the truth, ask for their opinions in a way that can’t get them punished. In comparison, stories are coming out that the public health people in the WHO and in contact with those in the WHO were passing around dire warnings a couple of months or more before the WHO went public that this was an international emergency. But what the WHO says in public has to be PC relative to the government of China.

  22. Chalid says:

    I remember thinking this would get pretty bad around Feb 15. I specifically have dated a conversation with a friend to Feb 23, in which we talked about how the spread was probably inevitable, talked about prep stuff, and even discussed buying far out-of-the-money put options on airlines and cruise operators (which would have had enormous payoffs, like initial investment x20 or more) and we agreed that it seemed like a logical thing to do.

    The normal prep stuff got done, and I warned my family to get prepared, but the put options did not. And I work in finance!

    Partly it’s that I’ve built a very strong habit of not making bets outside my particular area of market expertise, and of course that is normally the right thing to do. But this is based on the idea that the market is very good at pricing in things that market participants are paying attention to, and at the time the market was not used to paying attention to pandemics.

    But the other, more important and more embarrassing part was just trivial inconveniences. I hadn’t traded options before so I’d have to file some paperwork to get my account approved for it. Then I’d have to go file more paperwork with my employer and have a conversation with our Compliance department about it (hedge funds typically monitor and limit employee trading for obvious reasons). Then I’d have to fill out more forms and get approval for every bet I actually ended up taking. And I was on vacation at the time so all the hassle of the paperwork seemed even more inconvenient, and so none of it ever got started until the market had actually started crashing and the easy money was no longer there. Of course in retrospect all of it would have taken a few hours at most, spread out over a couple days.

    • Jon S says:

      I’m in a similar-ish boat. On Feb 27th I thought the expected number of US deaths was 450k (though I didn’t predict significant lockdowns or much of a chance of middling scenarios – I predicted a 90% chance of very few deaths and 10% of many more deaths than now seems likely). I also work in finance but never seriously considered making related trades in my personal account.

      I did buy a few months worth of non perishables that I was going to buy later anyway, but bought very little outside my normal purchasing habits. I got a little powdered milk, but didn’t look into it enough to discover reasonable bulk options until well after they’d sold out.

    • Tarpitz says:

      I have myself on record saying to the people I was going on a skiing holiday with in our WhatsApp group on the 8th February that I expected Coronavirus to set off a major global recession, and I remember reading your suggestion as to how to make money from that belief on Reddit and thinking that it sounded exactly right.

      Unfortunately, I couldn’t see any way for me personally to take advantage of the situation: my assets consist of equity in the house I jointly own with my brother (which we both live in), shares in the family business, and an inheritance from my grandmother which I won’t actually receive until some means is settled on of freeing up cash from her equally illiquid assets (a convoluted process involving four siblings with wildly varying personal finances who can’t agree on anything at the best of times). I suppose I could have taken out more credit cards, but taking out extra credit cards to short stocks as an amateur investor is from the outside view so utterly demented that I think I was right not to do it, even though it would have paid off on this particular occasion.

  23. Hoagy says:

    Nate Silver said there was a 29% chance Trump would win. Most people interpreted that as “Trump probably won’t win” and got shocked when he did. What was the percent attached to your “coronavirus probably won’t be a disaster” prediction? Was it also 29%? 20%? 10%? Are you sure you want to go lower than 10%? Wuhan was already under total lockdown, they didn’t even have space to bury all the bodies, and you’re saying that there was less than 10% odds that it would be a problem anywhere else? I hear people say there’s a 12 – 15% chance that future civilizations will resurrect your frozen brain, surely the risk of coronavirus was higher than that?

    I think there’s a negative missing or unduly added here.. it should be 29% or 10% that coronavirus *will* be a disaster, right?

  24. John Lawrence Aspden says:

    This makes me a less confident in Robin Hanson’s prediction markets.

    Could we have asked for a better skin-in-the-game highly liquid proxy for ‘respiratory pandemic’ than airline shares?

  25. zima says:

    If someone predicted that Ebola was going to turn out really badly, I would count that as a failed prediction and that would make their predictive powers with regards to COVID less impressive, just like if someone always predicts that the Republican candidate would win the election, that would make it less impressive if they correctly predicted Trump would win.

    Overreaction can be as bad as underreaction. The most important prepping I could’ve done for COVID would’ve been to sell all my stocks, but if I’d also sold all my stocks when Ebola or the 2009 swine flu pandemics started, I would’ve missed out on huge potential profits. Over the long run of those three pandemics, “do nothing” would’ve done as well as “predict doom in all three cases.” Societally, things like lockdowns and travel restrictions are extremely disruptive, and it’s a good thing we didn’t do those in response to Ebola or the 2009 swine flu because people did not overestimate how deadly those would be.

    The only people whose predictions I would really consider impressive are people who correctly predicted nothing would come out of every other pandemic and potential pandemic in the last 20 years and still saw COVID becoming so disruptive.

    • John Lawrence Aspden says:

      > Over the long run of those three pandemics, “do nothing” would’ve done as well as “predict doom in all three cases.”

      Is that actually true? Surely the correct strategy is ‘On early signs of pandemic, buy an option to profit over the next few months from falls in industries likely to be hit by a pandemic’.

      I can’t believe that wouldn’t be in the money now. I haven’t checked, and I’d be most interested if I’m wrong.

    • Ninety-Three says:

      but if I’d also sold all my stocks when Ebola or the 2009 swine flu pandemics started, I would’ve missed out on huge potential profits.

      Huge? You’d have missed out on six months of average market returns, then realized it was definitely not going to be a pandemic and bought back in. It’s not nothing, but it’s small enough to barely dent the savings you’d have made by selling off in Feb 2020.

      • zima says:

        Yes, the S&P 500 gained almost 25% in 2009 and over 10% in 2014. Missing that would be more than the loss from market peak to today.

        Also, when would you buy back in? Some people who are consistently pessimistic about the pandemic might’ve missed the market bottom on March 23. If you waited two more weeks for clear signs of a turning point instead of buying the dip, that’s another lost 20%+.

        • baconbits9 says:

          The S&P bottomed just as the Swine Flu was breaking out, you would have had to hold all through the 2008 crisis only to pay attention to the swine flu and sell out at 15 year lows. There is a vast difference between the Corona which saw the S&P make all time highs in the weeks after news about it causing havoc in China, and a market that was already 50% down from its high before your pandemic sell signal would have hit.

          If you waited two more weeks for clear signs of a turning point instead of buying the dip, that’s another lost 20%+.

          How is it lost? If you sold out you having lost another 20%, and that is assuming you didn’t participate in the flight to safety and make ~10-15% by switching into bonds for that period.

    • J Mann says:

      Yeah, it’s hard for a non-expert to tell a stopped clock from a canny probabalistic reasoner.

    • DNM says:

      At this point, a 2 week lock down every time a major pandemic first touches the U.S. does not look like such a bad investment. Better yet, testing early and thoroughly. Saying a pandemic every 2-3 years “could” be really bad and acting accordingly can easily end up net positive.

  26. johan_larson says:

    Once this is all over, I expect someone will write a book like The Big Short, about the folks that tried to warn us all about the pandemic real early, and turned out to be right.

    • Anonymous Bosch says:

      I’d read the book, but I would *still* be cautious about revising my estimation of any individual’s analytical prowess based on one call, which may have been a matter of either skill or luck.

      Unrelated link.

    • Edward Scizorhands says:

      When it’s time to line people up against the wall and shoot them for this mess, I hope we distinguish people who made different kids of mistakes, and when they made them.

      Dr. Drew said in January (IIRC) not to worry about coronavirus, but instead the flu, and get your flu shot. That’s good advice! And your odds of getting harmed from the flu were probably greater at that point, and even once the experts realized the pandemic was inevitable, one of their main mitigation strategies was “get everyone a flu shot, so we only have 1 thing on fire at a time.” And while we’d like people to not be deceived by China (and their media mouthpiece, WHO) we don’t want “trusted the wrong guy at first” to be a mortal sin.

      But Dr. Drew also gave some bullshit numbers about a 0.02% mortality rate, which from what I can tell he simply made up. And he kept his bullshit of “don’t worry about coronavirus” going through March.

  27. John Lawrence Aspden says:

    Scott, a question for you directly.

    Chloroquine in malaria-prophylaxis doses is about as safe as drugs get. For decades, most Europeans travelling to malarial areas took the stuff for months, and most Africans and Indians took it continuously, to such an extent that malaria evolved round it. It’s probably ballpark as dangerous as aspirin. It’s dirt cheap. In the UK it’s available over the counter, although you do have to buy it from the pharmacist directly, rather than off-the-shelf. The side effects seem to be mainly associated with large doses taken for a long time.

    It may or may not be effective as a prophylaxis against COVID. I’ve seen lots of weak evidence for, no evidence against, no strong evidence of any kind.

    You’re a doctor, and could prescribe the stuff. You live in an area that was apparently going pear-shaped earlier on, and were apparently sufficiently concerned to wear a respirator.

    How many of your friends and patients have taken prophylactic doses of chloroquine or allied drugs?

    And what is your reasoning?

    • gbear605 says:

      I don’t know about Scott, but a lot of doctors chose not to do something like this because of the limited supply in the US. A lot of people with autoimmune disorders depend on chloroquine and are currently unable to get it, so the jury is very much still out on how a small chance (20% perhaps?) of protecting the people you care for from a small chance of death (1%) compares to protecting people with autoimmune disorders from many of the symptoms of those disorders.

    • Garrett says:

      Technical point: hydroxychloroquine and chloroquine are very similar drugs with similar usages, but “hydroxychloroquine is estimated to be two or three fold less toxic”. Make sure you know which one is being referred to or that you are taking.

    • Scott Alexander says:

      I haven’t had any friends who needed it, and nobody’s come to their psychiatrist for a coronavirus prescription, so I haven’t had to worry about it. Right now I’m pretty bearish on its chances of working, though as you say it’s probably pretty safe as these things go and might be worthwhile anyway if there’s enough for everybody.

  28. John Schilling says:

    What’s the G-name for people who treat clever new ideas as true until somebody provides incontrovertible evidence they are false? I mean “rationalist contrarian” kind of fits, but too many syllables and it doesn’t begin with ‘G’.

    It is nonetheless a real behavior pattern, a very common failure mode for people whose cost-benefit analysis is less than absolutely flawless, and a pretty good reason for adopting a Goofusian bias. I want to see that guy getting his fair share of mockery attention every time Goofus and Gallant are on stage, so maybe a new graphic is needed.

    • Hoagy says:

      Gulliver, perhaps?

    • HeelBearCub says:

      Hear, hear.

      Edit:
      Two things that annoy me about this post, or one thing in two parts.

      No mention of Type-1 vs. Type-2 errors.

      No mention of what should have been done at a population level for each of those potential pandemics.

      For someone who a) wants to reach the singularity, b) worries about dust specs, c) understands that pandemics are hardly the only long-tail event out there, this post does a horrible job of understanding that there are tradeoffs here.

    • The Nybbler says:

      Perhaps “Gangway”, in the sense of the cry meaning “clear the path, I’m coming through”.

    • Geewhizz? There’s at least one prime example on Less Wrong.

  29. Greg55 says:

    Common sense said that they worked. But there weren’t many good RCTs. We couldn’t do more, because it would have been unethical to deliberately expose face-mask-less people to disease.

    As with parachutes, RCT are not the only way to see if masks/parachutes work or not. We can look for a natural social experiment as another option.
    There have certainly been cases of two very similar populations during an epidemic, the only difference being that one was wearing masks and the other was not. With parachutes: one group had parachutes working and the other had parachutes not working. So you compare the infection rates, how fast it spreads etc. between two populations. Of course, you have to take into account the moment into the epidemic the masks are introduced (as they protect against spreading the virus, not against getting it). I do not think it is too difficult to conduct such research even now.

  30. BlindKungFuMaster says:

    I had a workshop mid February where somebody had to give an impromptu talk about the Corona virus. He knew a great deal about it, having by chance just read the RKI-report about it. I told him afterwards that my version of the talk would have been much more alarmist.

    I started prepping soon after, though not very seriously. I don’t own stock, but I also underestimated the probability that Western nations would instantiate a full lockdown a la China, so I wouldn’t have been very confident predicting that an epidemic that hits mostly elderly crashes the economy.

    I’m also on record (just ask my wife 😉 ) saying that schools shouldn’t reopen after the holidays end of February.

    A few weeks back I came across a blogpost by one of my favorite fantasy authors, written February 10:

    “Personally, given the way Chinese authorities bungled the outbreak at the start, and given the alarming tendency of the WHO and CDC to communicate only the most optimistic appraisals of 2019-nCoV, I think this will be the biggest thing to hit humanity since World War II.”

    Very impressive. Should probably read that viral thriller of his.

    • Edward Scizorhands says:

      Science fiction authors are really good at correctly predicting 8 of the last 3 disasters.

      • noyann says:

        SciFi isn’t really about predictions, imo. It’s about exploring possibility space through a story — a conflict playing out, a coming of age, solving a mystery, …
        Of course, some of the author’s ideas can later come true, sometimes because they directly inspire real world scientists and engineers (tricorders are an example, death stars are hopefully not).

  31. Jack V says:

    When I try to look back to how I felt before, I think I can probably summarise it as:

    * If Covid19 starts to spread here, it will probably go exactly like it did in Italy and China because those are the examples we have and we’ve no reason to think our government will act significantly differently. [Correct.]
    * Surely having seen what happened in Italy the government will pounce on new outbreaks coming off airlines and not risk that happening here. [Incorrect.]
    * There’s not much I can do personally other than having a couple of weeks of canned food just in case [Partly correct I couldn’t get the government to take it seriously all by myself but I should have tried!]

    So I feel like I get credit for #1. But I was really complacent on #2, I thought with all their planning departments the government would do SOMETHING, they didn’t want to end up in this situation either, but I was just wrong, this got ignored just like many other important things 🙁

    Indeed, the government were complacent much later than that: about the ides of march when it was obvious to anyone paying attention that it would probably sweep the country without drastic action they were still making jokes and “oh, don’t worry so much” and relying on a not-entirely-applicable model which said something else.

    Reading your post, I wonder if “if then” predictions would resonate with public opinion more: instead of “a pandemic COULD happen”, tying it to an event where public perception and actual fact of likelihood happen to accurately coincide like “if it gets to N cases, we’ll either have a national lockdown or it will get to 75% infections”

  32. argentus says:

    Another unfortunate reality in all this is that people in a position to actually make and institute policy decisions aren’t just dealing with probabilistic reasoning about how effective a given policy or strategy is likely to be. They also have to engage in probabilistic reasoning about how people will respond to a particular policy and how that will impact their willingness to cooperate with later policies.

    Say you put in harsh quarantine measures early because there’s a 20% chance coronavirus develops into a horrible pandemic, and you mostly succeed in mitigating the death rate but you also tank the economy. Is it unreasonable to be afraid that a hefty percentage of people will respond to this with “We tanked the economy over a virus that only caused 10,000 deaths?” and to think those people will be gunshy about committing when the real, unambiguous big one (SARS, MERS, whatever with a 30%+ mortality rate) comes along?

    This has certainly been a concern of mine though I’m in no position to implement policy.

    What if the only way to get people to understand the gravity of the problem is with incontrovertible evidence? What if you can’t get anybody to cooperate with policy *until* they see what’s happening in Lombardy?

    In this case, early warnings are nice, but they are no particular guarantee that policy gets enacted sooner.

    • albatross11 says:

      I think this is another reason to look for low-cost stuff and implement that first. Masks, hand sanitizer, 6 feet distance, and fever checks are all pretty cheap, and could have been put into place fast.

      • Edward Scizorhands says:

        If you know there’s a 10% chance of a house in town catching on fire today, you don’t run out and douse random houses with water. But you do make sure your fire engine is fueled up and you have all your supplies.

      • Scoop says:

        Six-foot distance isn’t really all that cheap. There are very few indoor places where you can maintain it all the time. Maybe if women still wore gigantic hoop skirts, we’d have designed more places to allow so much personal space, but they don’t, so we haven’t.

        • albatross11 says:

          In my normal (pre-lockdown) life, I could manage 6 feet of distance nearly 100% of the time. You can run stores and offices and many workplaces like this, with a few changes, but they can still function. Public transit, including air travel, elevators, and most things you have to stand in line for are harder to fit to that rule, though. Still, I suspect we could get most of our normal society running again with rules that say that everyone keeps 6 feet of distance from everyone else as much as possible, everyone wears a mask, and everyone gets a fever check when they come in and gets turned away if they’re running a fever.

          By contrast, right now, we *can’t* get our society running again with a requirement of everyone getting a negative on a viral RNA test once a week or getting two positive antibody results a month apart, because we don’t have the tests. But if we can get the testing online, that’s another way we could get ahead of this crap. If every workplace requires a negative RNA test once a week and a fever/symptom check every day, and a positive test sends you into quarantine, we can probably reopen most of the world and things will work.

          Similarly, we can’t reopen most of our current businesses and shops and such if everything has to redesign its ventilation system to incorporate the filtering they do in hospitals to prevent spread of the virus. That would probably take a long time and a ton of money to do. (Though this makes me wonder–could you swap in some kind of antimicrobial filtration into an existing HVAC system without having to upgrade the whole system? My not-very-informed guess is that you couldn’t do this for HEPA filtration, but you might be able to do some kind of UV thing at the air intakes or something. But I don’t know much about HVAC systems–someone here probably knows more….)

          • Steve Sailer says:

            The testing, mask and quarantine situations could have been improved dramatically in February. Testing was a botch by the CDC, while pretty much the entire public health establishment had seemingly ideological prejudices against masks and quarantines that took disaster to overcome.

            Also, the number of masks in inventories in the US during the winter were drained by Chinese people in the U.S. going around to American stores and buying them up to ship to China. As far as I can tell, nobody saw that coming. Heck, I didn’t believe it until I saw the evidence.

        • Clutzy says:

          One of the worst aspects is that stores have shortened hours. Costco went from 10-8 to 10-6, Jewel went to a 9-10 schedule instead of 7-2am. I used to do a lot of shopping in an empty store at midnight if I was having trouble sleeping.

    • thoramboinensis says:

      Really important point, but it’s honestly still too generous to the intentions of policymakers. The tacit assumption is that policymakers are ready to do what’s best for the electorate, rather than what’s best for getting them re-elected/promoted. You rarely get points for an epidemic avoided, and don’t get punished very hard for a delayed response (just look at how the media is lionizing Andrew Cuomo right now even though NY had a horrifically bad early response). A lot of decisions by decision-makers start looking a lot more rational when you factor in their perceived inability to act except in response to public demand, as well as their general rational self-interest.

    • Scarbrow says:

      I’ve been thinking along these lines for a while, too, though in another direction.

      My model for politics / policymakers is that they are (relatively) powerless to take harsh actions unless the danger is very real, and quite close, simply because those who would take such measures would be culled (with extreme prejudice, “pilloried” might be a better word) every time they were wrong. Thus, a) Very conservative, on average (don’t act until you’re very, very sure) b) Cannot act too far in advance, because if you avoid the danger, you don’t get the credit unless a big majority agrees your actions were the ones which avoided the danger. And still, you can get burned if your actions are then considered to be either too mild or too extreme, which further incentivizes to act moderately, no matter how great the danger, until and unless there is again the majority consensus of needing such actions (and by that point, you’re usually at the “reactive”, not “proactive” part of the response).

      In short, if there are advance measures with any degree of cost, to be taken under conditions of uncertainty, they won’t be taken at the government / policymaking level. With the possible exception where the costs are born by few, or when it’s become entrenched culture (say, the FDA requisites for new drugs). And even then, expect the costly measures to be single-mindedly directed to ensure it’s not the policymaking authority who takes the blame

      • Edward Scizorhands says:

        While I’ve been very critical of the Trump response, I acknowledge that he was going to be limited by the public’s willingness to respond to unknown threats and take measures that they’ve never experienced in their lifetimes.

        Still, a leader can lead.

  33. FletchFlarFlodex says:

    This article reminds me of the alarm that some people have raised about solar flares. It doesn’t seem like someone will be able to predict the next major solar flare occurrence in time to prepare the electrical grid for it. But since the risk of a large solar flare will remain low right up until it happens, we will do nothing to prepare for an event which sounds like it could knock out the electrical grid for months.

    • Steve Sailer says:

      When I started prepping in the second half of February, I ended up doing a number of things that (hopefully) won’t be useful in the coronavirus crisis, like setting aside 110 gallons of water in my backyard, but which could be very useful after the next Los Angeles earthquake.

      It’s perhaps an odd psychological quirk to prep not for the imminent crisis that you don’t really understand but for the expected one that you feel guilty for not having prepped adequately for before.

      But if the Big One breaks the California aqueduct, I will be raising a toast to my lack of focus from my water stash.

  34. RC-cola-and-a-moon-pie says:

    “The frequent user group said ‘A 12% chance of eternal life for the cost of a freezer? Sounds like a good deal!’” Mr. Pascal would like a word with these people.

    In all seriousness, I would love to see a justification for the idea that some “cryonics” company has a 12% chance of maintaining you frozen for the requisite time. Set everything else you need to 100% and I’d be fascinated to see support for a 12% probability there.

    • Institutions can be long-lasting. We see it in history, a history that has been more violent and unstable then the world is now and will probably continue to be.

      • albatross11 says:

        Surely the lowest probability in that calculation is hat people in the future will be able to reconstruct you at a cost they’re willing to bear.

        • RC-cola-and-a-moon-pie says:

          You’re right, of course. I intended my challenge to isolate one arbitrary component of the analysis, but I realize the way I wrote it makes it sound like I viewed that as the single greatest hurdle.

      • RC-cola-and-a-moon-pie says:

        But Alex, it seems to me that the more relevant question is not whether some institutions may survive long-term but rather whether any particular business is likely to be such a long-surviving institution, and in particular whether a little cryogenics company is likely to do so.

        • The top two providers are non-profit institutions, not businesses.

          • RC-cola-and-a-moon-pie says:

            I appreciate the correction. I’m not sure how responsive it is to the underlying issue, though!

          • cwillu says:

            I appreciate the correction. I’m not sure how responsive it is to the underlying issue, though!

            If you’re wrong, isn’t it at least plausible that you were wrong because of a large number of small corrections like that?

          • RC-cola-and-a-moon-pie says:

            Cwillu: Certainly! If someone identifies further errors, it might absolutely affect my judgment on this question.

    • Steve Sailer says:

      I can barely keep paying a server company to keep a website up. The notion that my descendants (who, after all, will have inherited my semi-competent genes) will keep paying to maintain great-grandpa’s frozen head seems like a fond hope.

      • Matt M says:

        You should crowdsource this. I’d consider donating $5 a month in exchange for a 1/1000th share in Steve Sailer’s Frozen Brain.

  35. David_Hab says:

    Nassim Taleb also predicted the Covid-19 outbreak. He sounded the alarm on Twitter. Apparently his memo informed Trump on the travel ban with China.

  36. alchemy29 says:

    The NPR did really well by my estimation. They simply stated the facts, no opinion pieces or punditry masquerading as news. See a list of their articles since January. They never stated what we should do next, they never mention lockdown. But I do not think that is the job of the media.

  37. Calecute says:

    My theory is that media will never ever tell you to prep for a disaster or anything without the state explicitly signing off on it. So the failure of the media is just a detail, it was a failure of the state and we shouldn’t expect media to do better than they did.

    The fact that some journalists were in fact prepping while they said everything was gonna be fine is evidence that the problem was not one of reasoning.

    • Matt M says:

      Heh, I remember when the panic buying first hit my area… the local news sent out a reporter to give us LIVE coverage from the grocery stores, where there were huge lines, and people were coming out dejectedly and giving interviews saying “there’s no meat or toilet paper or cleaning products.”

      The reporter gave a confident smile to the camera and assured the viewers that there was no cause for concern, the stores all reported that more supplies were coming soon, and that there was absolutely no need for you to go out and try and buy all this stuff.

      It was like a non-humorous version of the “nothing to see here” scene from the Naked Gun…

      • gbear605 says:

        Of course, there now is more toilet paper, cleaning products, and meat. I can go to my local grocery store and buy all of those things. The reporter was completely correct.

  38. Anonymous Bosch says:

    First, a bunch of generic smart people on Twitter who got things exactly right – there are too many of these people to name, but Scott Aaronson highlights “Bill Gates, Balaji Srinivasan, Paul Graham, Greg Cochran, Robin Hanson, Sarah Constantin, Eliezer Yudkowsky, Nicholas Christakis, and Eric Weinstein.” None of these people (except Greg Cochran) are domain experts, and none of them (except Greg Cochran) have creepy oracular powers. So how could they have beaten the experts? Haven’t we been told a million times that generic intelligence is no match for deep domain knowledge?

    These do not have individual links, in your post or Other Scott’s.

    When someone asserts multiple points and doesn’t back up any of them, I check one. If it’s bullshit, I assume the rest is bullshit. So I checked the last one: Eric Weinstein. He has one (1) tweet about the coronavirus in February, explicitly saying he doesn’t have a take. Then in March he started burning down the wires about how he and others were “right.”

    I’m not disappointed in you for chugging the “these dozen articles mean the media hivemind got it wrong” chum. I’ve long since inferred that the person who wrote the Chinese Robbers blog and the person who substantiates their points with Twitter dot com slash shit virus takes are no longer the same person. But I am disappointed that you (and Aaronson) didn’t even feel the need to substantiate this alleged superiority over “domain knowledge” with *cherry-picked* links. It’s just asserted that the media was wronger and you and your buddies were righter. I’ll believe that when you show me all of them being concerned at the same time as the shit virus takes (i.e., no credit for coming around in March).

    To me it seems like you and most other soi-disant rationalists have an outsized talent at being more righteous and *selling* it as being more right.

    • Elena Yudovina says:

      I think this might the Eric Weinstein tweet you’re looking for: https://twitter.com/ericrweinstein/status/1234300424804429824?lang=en

      • Anonymous Bosch says:

        Yes, from March. The “shit virus takes” are from January and the first two weeks of February. Neither Scott should be expecting anyone to suck some general-domain-intelligence dick for hitting the panic button after the stock market had already fallen 10% and the media woke up. Either you were provably ahead of the curve or you weren’t.

    • Hi — I’ve now removed Eric Weinstein from my list, because as you and others pointed out, I was misled by his later self-confidence in criticizing those who got it wrong, and by the fact that others in his circle got it right early on, into imagining that he himself must have. Thanks. The others on my list were sometimes later than I remembered (which partly makes me feel better about taking until late February myself!), but were also earlier than most, in a way that was important for my own personal evolution on this issue. Robin Hanson, for example, blogged on Feb. 14 about how we should consider deliberately infecting people in waves so as not to overwhelm the medical system. I DON’T think that’s the best possible take, but whatever it faults, that played a role in helping to precipitate my own “oh shit” evolution over the following two weeks. Balaji Srinivasan and Bill Gates were both totally right from I believe late January and deserve enormous praise for that.

      • Anonymous Bosch says:

        (1) I’m genuinely glad you did that and admitted error. (Hopefully you appropriately discount his bluster in the future.)

        (2) Full marks to Srinivasan and Gates, but for the rest, late February is at best keeping pace with conventional wisdom, not beating it. Acknowledging that gestalt impressions of “the media” are inescapably subjective: from what I saw they were mostly there by late February too. In fact, late February is where I remember the MSM and rMSM decisively inverting from their tentative priors, probably because that’s when Trump started getting peppered with questions about it and getting defensively Panglossian.

      • vaniver says:

        I was misled by his later self-confidence in criticizing those who got it wrong, and by the fact that others in his circle got it right early on, into imagining that he himself must have.

        I don’t know how things are with Eric–I assume his primary interaction with the public is through Twitter–but I am not sure where to put people like myself, who are deeply enmeshed in the in-person community and were preparing and talking about it in person, but weren’t posting about it publicly. [Like, at least one person has since thanked me for being upstream of them worrying about COVID and preparing for it.]

        Obviously I don’t get “I told you so” credits, because I didn’t tell you so. To the extent I get “I was right” credits, presumably those credits are mostly stored in the food and essentials I stockpiled by acting early. But also if I should be saying “I was wrong”, I should mostly be saying it about “I was wrong for keeping my opinion to myself” instead of “my opinion was incorrect, and I should trace my opinion construction machinery.”

    • Atlas says:

      When someone asserts multiple points and doesn’t back up any of them, I check one. If it’s bullshit, I assume the rest is bullshit. So I checked the last one: Eric Weinstein. He has one (1) tweet about the coronavirus in February, explicitly saying he doesn’t have a take. Then in March he started burning down the wires about how he and others were “right.”

      Thanks for investigating this. (And thanks to Scott Aaronson for admitting and revising error.)

    • albatross11 says:

      Razib Khan and Paul Graham were talking about this early and were very worried about it–I read their stuff regularly and watched it.

    • AnteriorMotive says:

      For what it’s worth, my response to originally reading that list on Aaronson’s blog had been:
      -Balaji Srinivasan, Greg Cochran, Robin Hanson: “I remember that.”
      -Paul Graham, Eliezer Yudkowsky, Sarah Constantin, Bill Gates: “Sounds about right.”
      -Eric Weinstein: “Not what I was expecting, I’d better update in his favor.”

      I think you just got unlucky with your random selection. Before this, I had taken Eric as more of a blustery hothead than a source for rare original insight.

      For completeness sake, I’ve never heard of Nicholas Christakisand.

      • AnteriorMotive says:

        I had to check Eliezer’s twitter to decide whether to put him in the first or second category. I remembered him joining the dogpile once the canaries in the coal mine were proven right, in late Feb, but don’t follow him, so hadn’t seen anything from him earlier.

        And he did not in fact tweet any early predictions.

        Late February is still about two weeks before mainstream opinion converged on “this is a real crisis,” but I guess he also doesn’t get to qualify as a societal Early Warning System.

    • Scott Alexander says:

      I quoted Scott Aaronson because I didn’t have a specific list of names and recognized several of the people on there as genuinely prescient. It sounds like he’s removed Weinstein from his quote, so I’ve removed him from mine.

      Some of these people were actually quite early, others – as you said – came around in late February and early March. I still give them some credit because of how many didn’t. Dr. Fauci, who is a smart guy and *mostly* handled this well, was still saying there was no reason to cancel your cruise vacation as of March 9. I talked to a tech founder who did a survey of other tech founders and found that most of them were still having their employees come into work as of March 15. If you feel like as of late February / early March people had already figured this out, I think you’re living in a bubble of well-informed people. I’m not claiming that the names on Scott’s list are anything more than other people in the same bubble who talked about it loudly and helped set its consensus – see Sarah C’s comment below.

      Re: the media, I’m mostly drawing off of my own experience as someone who lived through this period and consumes a lot of media. I feel like those links were unusually bad but representative (in terms of the sign) of coverage more generally. If you have some stronger evidence that the media mostly got this right, let me know.

      • Anonymous Bosch says:

        If you feel like as of early March people had already figured this out, I think you’re living in a bubble of well-informed people.

        I definitely am, but I was referring to the same bubble of well-informed “people” as the topic of your post; the media, the domain experts, stonk traders, etc., not the hoi polloi. Sorry if twas unclear.

      • eric23 says:

        Dr. Fauci, who is a smart guy and *mostly* handled this well, was still saying there was no reason to cancel your cruise vacation as of March 9.

        He’s limited in what he can say by his boss.

      • xq says:

        I’m not claiming that the names on Scott’s list are anything more than other people in the same bubble who talked about it loudly and helped set its consensus

        Well, you said they beat the epidemiologists. By second week of February there were already epidemiological models saying there would be widespread transmission (with up to 70% of the population infected) without intervention. I follow a lot of epidemiologists on Twitter and my strong impression is that this was the mainstream view.

        To the extent that the people on your list got it right before this point you can make a plausible case that the contrarians beat the experts, but if they got it right after all you can really say is that they figured out the experts were correct before some other people did. That’s a pretty different story and not the one I got from your post.

      • Loriot says:

        I talked to a tech founder who did a survey of other tech founders and found that most of them were still having their employees come into work as of March 15.

        I suppose a survey is more representative than an anecdote, but for what it’s worth, my company told everyone to work from home starting March 1st, and made it mandatory on the 4th. It was a bit weird working from home for two weeks while seeing kids walking home from school and all. For me, the 15th was when the panic went mainstream – schools closed, the supermarket shortages started, the lockdowns started, etc.

    • Eponymous says:

      I don’t know about the rest, but Greg Cochran (who, as Scott said, has both domain expertise and creepy oracular powers) did this podcast on February 9. The first few minutes should be sufficient to demonstrate the claim — he gives a “headline” quantitative estimate fairly early.

      • Eponymous says:

        Actually, I invite anyone who hasn’t heard it to listen to the first 5 minutes of that podcast. Before doing so, consider your estimate of global deaths under the “10% worst case scenario” under current information, and what your prediction would have been back on Feb 9.

        James Miller knew enough to call up someone like Greg to get his take on this new virus in China. Our media and government officials apparently didn’t call up Greg or someone like him. I think that is a problem.

        • xq says:

          Yes, Cochran was saying the right things (I only listened to the first five minutes). But compare to this Vox article from Feb 6. Are they really saying such different things? Both Cochran and Belluz emphasize there’s still a high degree of uncertainty and the worse case scenarios may still be avoided. Both make clear that there’s a plausible scenario that is quite bad with many deaths worldwide.

          The biggest difference I see is that Cochran is quantitative in a way most of the media refuses to be, which is frustrating. But it’s not the case that no one in the mainstream figured out there was a reasonable possibility this could be a huge crisis.

  39. JohnNV says:

    You say the correct response to the threat of a pandemic is to prepare, but it’s unclear what personal preparation one should have taken in Jan/Feb? At least here in rural New Hampshire, our grocery store is still well stocked, so stockpiling food or household goods would have been ineffective. I just got an Amazon package yesterday. I suppose I wish I had gotten a haircut more recently, so I wouldn’t have had to do it myself. I could have bought N95 masks in Feb, but our entire household is home and not coming into contact with other people (except at the grocery store once a week), so they would be of limited value. Looking back, it’s not clear what I should have been prepared for then that I can’t still do today? Is the worst still to come? Do people genuinely think the food supply chain will break down? Or utilities won’t be able to deliver water, electricity, or broadband? If so, wouldn’t the advice be to prepare now? As opposed to “Don’t you wish you had prepared in February?”

    • Matt M says:

      Is the worst still to come? Do people genuinely think the food supply chain will break down?

      Unless the lockdowns are lifted soon, yes, I believe this is a distinct possibility.

    • All the local stores in my area were out of toilet paper for about two weeks, and I heard anecdotes about very long lines at the stores and shortages of other items. So yes, I’m glad I prepped early.(proof here: https://lionoftheblogosphere.wordpress.com/2020/01/28/crazy-low-airfare-prices-to-asia/#comment-249894)

    • Jon S says:

      I think it’s possible there will be temporary shortages of lots of goods, sometimes unexpectedly (e.g. some food plants have recently gone offline when their workers contracted the virus – not enough to cause shortages at the moment, but perhaps some disruptions). I wish I’d bought powdered milk and hand sanitizer in February. Everything else I want to stock up on I’m still currently able to I think, but I’m glad to have done so when infection risk of shopping in a store was lower than now.

      Personal prep may also have included setting up infrastructure to be able to work from home.

      • albatross11 says:

        Hand sanitizer is still unattainable here, as far as I can tell. Apparently this is partly about dumb regulations, though.

  40. I think there’s an important difference that separates masks and parachutes on the one hand and drugs and bone marrow transplants on the other. Mechanical effects like barriers to particles on the effects of speed on the danger of a collision are easy to reason about and you can guess with pretty good accuracy what is going to work. Bodies, on the other hand, are working towards homeostatis with all sorts of weird feedback loops which evolved under selective pressure to not let pathogens mess with it. So we should be far more willing to entertain new ideas about the former.

  41. jmo says:

    I. Your point is mostly spin. The question isn’t who said what when. The question is who, as evidence mounted that this was a big deal, continued to insist that it was not in fact a big deal. I’ve seen a number of people try to cover their tracks by saying, “Yes I said it wasn’t a big deal on March 16th. But the NYTimes also said it wasn’t a big deal.” The fact is of course that the NYTimes said it January 21st and had changed its tune by March 16th. Those who were still claiming, on March 16th, that this was not a big deal were very very wrong.

  42. Elena Yudovina says:

    I want to challenge an assumption here (apologies if it’s been challenged already). This seems to be written from a position of “the costs of action are known and small (possibly negligible?), the benefit of action is uncertain but has a sizeable probability of being huge, we should take the action”. This seems fair when you’re recommending taking a parachute when jumping out of a plane, probably fair when you’re recommending universal cloth mask wearing (assuming risk compensation is insignificant, e.g. because you’re maxed out on risk already), likely fair when you’re recommending publishing more alarmist articles (although I’m not in the publishing business and can’t estimate the costs), but a lot less defensible when you’re talking about things like stay-at-home orders where the costs are definitely not small. It would’ve been nice to see the media push more premonition-of-doom media stories, but I’m not sure I’d be sold on “let’s take bold action that’ll give the US a 15% unemployment rate” actions (https://fortune.com/2020/04/09/ureal-unemployment-rate-jobless-claims-this-week/, sorry about the paywall) before we determined that the correct model for what would happen wasn’t SARS, Ebola, or the flu.

    • Matt M says:

      Yeah. I predict that when this is all said and done, the story won’t be “the media underreacted!” but rather “the media underreacted until late March, at which point they pivoted to overreacting.”

      • Atlas says:

        Likewise/thirded. It seems way too early to me to start deciding whose heads deserve to be put on spikes and whose deserve to be exalted for their oracular prowess.

        For example, it seems that IHME predictions for NYC substantially overestimated the number of hospitalizations/deaths:

        On March 24 Gov. Andrew M. Cuomo appeared at the Jacob K. Javits Center in Manhattan and, as soldiers scrambled to transform it into a hospital, he offered the public a dire assessment.

        Sophisticated scientists, Mr. Cuomo said, had studied the coming coronavirus outbreak and their projections were alarming. Infections were doubling nearly every three days and the state would soon require an unthinkable expansion of its health care system. To stave off a catastrophe, New York might need up to 140,000 hospital beds and as many as 40,000 intensive care units with ventilators.

        “One of the forecasters said to me we were looking at a freight train coming across the country,” Mr. Cuomo said.

        Two weeks later, however, with an unprecedented shutdown of public schools, countless businesses and most of outdoor life, New York has managed to avoid the apocalyptic vision that some of the forecasters predicted.

        The daily death toll has still been staggering, approaching 800 for a third straight day on Friday, and some hospitals continue to teeter on the brink of chaos.

        But the number of intensive care beds being used declined for the first time in the crisis, to 4,908, according to daily figures released on Friday. And the total number hospitalized with the virus, 18,569, was far lower than the darkest expectations.

        The IHME projections from late March assumed social distancing.

        • Joshua Hedlund says:

          The IHME model has been impressively transparent about the assumptions behind the model and explaining the reasonings behind the changes in subsequent updates.

          The hospitalization estimate was based on extrapolations from too little data and was revised down as better data came in.

          These changes in predicted hospital resource use and related gaps in states where demand might exceed supply are fairly large. In this section, we explore the changes in our analytical framework that resulted in these revised estimates of overall lower hospital resource use due to COVID-19.

          First, we have been able to include more up-to-date data for estimating ratios of hospital admission to deaths. These ratios inform model parameters that are used to predict need for hospital beds, ICU beds,and ventilators. In previous releases of our estimates, our ratios were informed by a CDC report with information on early COVID-19 cases in the US –from February 12 to March 16 –and those patients’ outcomes. Based on these data (509 admissions divided by 46 deaths), our overall ratio was 11.1 hospital admissions per COVID-19 death.

          Over the last few days, we have been able to incorporate data sources, including data provided bystate governments, on a substantially larger sample:16,352 hospital admissions and 2,908 deaths related to COVID-19. This allowed us to estimate state-specific ratios where data were available on at least 50 deaths from COVID-19, using random-effects meta-analysis.

          Our estimates released today use the state-specific ratios noted below and for those states without data, the pooled ratio of 7.1 hospitalizations per death (95% CI4.0to 12.7). These lower ratios of admissions to deaths result in predicted peak hospital resource use –total beds, ICU beds, and invasive ventilators –that is lower than previously estimated.

          Whether this initially substantial overestimation is worth a spike or merely the best yet transparent guess aavailable at the time is beyond my intellectual prowess.

          Personally I wonder how much of the collective terror about running out of hospital beds was not really so much based on fully trusting a model that no one had previously ever heard of but rather on the actual reality of seeing that China had rushed to build pop-up hospitals and that Italy was running out of room too and thinking, “well, crap, seems like that could happen here too!”

          • Steve Sailer says:

            The fear that we could run out of ICU beds and ventilators was based on the assumption that ventilators were a quite effective treatment for COVID-19, which turned out to be optimistic: e.g., the IMHE model started with only 1 death per 11.1 hospital admissions, but has since been worsened to 1 death per 7.1 hospital admissions.

            Not surprisingly, doctors have responded by being more reluctant to put patients on ventilators (which are immensely staff time-consuming), patients have been more reluctant to go to the hospital, and the non-infected have been more reluctant to risk infection by going out in public.

    • Purplehermann says:

      I’m pretty sure there is a middle ground between ‘ignore it’ and ‘the sky is falling’.

      Maybe limit sizes of gatherings to 500, ramp up testing ability, restricting travel (which Trump did do, but more and earlier) and testing those who do travel. Ask China for a sample of the virus to start work on vaccines. Just-in-case measures

  43. mobile says:

    Say what you want about the rationality of sports fans, but they have a handle on the distinction between the likely outcome and the likelihood of an outcome. At least in the domain of sports.

    “The Dems have a 75% chance of winning the 2016 Election”. That’s like having a 1-point lead and the ball in a basketball game with one minute left on the clock. You feel good about your chances, but it’s hardly the time to slack off on defense or to make poor shot selections. Those outlier events when your team was almost certain to lose(or win) but then a miracle happened are never far from the front of your mind. The outcome has to be very certain before a sports fan will give up on the possibility of other outcomes.

    • Matt M says:

      Do you think Nate Silver classifies Trump winning as a “bad beat”?

      • mobile says:

        Not at all, and I think he said as much after the election. It was an upset — the kind you’ll see about once a night when there is a full slate of NBA games — but hardly a shocking upset.

        Silver is a sports fan and his site has plenty of sports analysis. I think the “1 point lead and the ball” stat I quoted above came from him.

  44. achenx says:

    If the government orders a lockdown early, and it works to the extent that the disease is completely contained and very few people catch it, is Vox going to print “Thanks President Trump for saving us from a pandemic!”? They are not. Even setting aside partisan fighting, people in general are not going to appreciate preparation for a disaster that works and prevents the disaster from ever occurring. Look at the general interpretations of Y2K. Any politician that had enforced business shutting down (and therefore people getting laid off) when few people got sick would have been run out of town on a rail.

    And the thing of it is, the people are probably right to think that. “Trust me, tanking the economy was necessary to prevent a pandemic that didn’t happen” sounds just the same as “Trust me, buying this tiger-repelling rock is necessary to prevent tiger attacks”.

    Predictions are hard, especially about the future.

    • Ninety-Three says:

      On a similar note, lockdown is heavily dependent on compliance and voluntary measures in the population. Unless you expect everyone in the country to be good probabilistic reasoners, you simply can’t declare a lockdown much earlier than we did, because barely anyone would have respected a “don’t leave your house except for groceries” order issued early February.

      • albatross11 says:

        The other side of that is that the difference between a lockdown and how your citizens normally depends on how seriously they take the situation. Air travel, hotel stays, and restaurant reservations all fell off dramatically before any lockdowns, because people were scared of catching the virus.

      • eric23 says:

        The difference in death tolls between San Francisco and NYC is the difference between a relatively-early lockdown an an inexcusably late lockdown.

      • Scarbrow says:

        Extremely right.

        That also depends on how your local population usually responds to the authorities’ “suggestions”. In Spain, full lockdown was probably needed because any lesser action would have been, if not futile, at least not completely followed (and I live here, in Madrid). But even with that we have our share of people risking €600 fines (on PPP, about $950) for going out on “non-authorized errands” like going to a supermarket further than necessary. Which I suppose puts also a ceiling on the effectiveness of any possible measure, short of full-scale “tanks in streets” military takeover.

    • baconbits9 says:

      If the government orders a lockdown early, and it works to the extent that the disease is completely contained and very few people catch it, is Vox going to print “Thanks President Trump for saving us from a pandemic!”? They are not. Even setting aside partisan fighting, people in general are not going to appreciate preparation for a disaster that works and prevents the disaster from ever occurring.

      I don’t think this is true at all, you only need 1 Italy to be able to say ‘see, it worked’ even spuriously. The fact that the US would have Spain, Italy, the UK and others as significant examples of really bad outcomes would make it very easy to hold the position ‘the lock down worked’ if it in fact did appear to work

    • Edward Scizorhands says:

      This assumes every other country followed Trump’s advice and locked things down at his direction so he can’t say “look at Lombardy, that is the carnage I saved you from.”

      Also, there’s a big universe of steps that could have been taken short of “shut everything down.”

  45. srconstantin says:

    As one of the people mentioned in this post, Scott’s assessment of me is exactly right.

    I did exactly zero armchair epidemiology. I have never written a single thing about COVID-19 epidemiology, except to uncritically copy the estimates of experts. I don’t even have an especially good amateur knowledge of epidemiology.

    Here’s what actually happened: we were all seeing a bunch of news about Wuhan in late February, and my husband Andrew said “what’s the point at which we should get out of town?” (Because it’s common sense that you don’t want to be in a dense city in an epidemic.)

    I looked up the CDC numbers, saw that there wasn’t community spread in the US, and “Ok, we’ll leave town when there’s community spread in the US.” Then, when I saw the first case of person-to-person spread between people who had never been to China — and in Northern California, at that — I said “ok, I guess we gotta go.” I felt silly, but as my husband likes to say, the right time to GTFO is before all your friends are doing it.

    Well, it turns out, exponential growth is fast, and pretty soon we didn’t feel stupid at all.

    And then I had some friends/colleagues who were working on relief efforts for COVID-19, and I helped in the best way I could from small-town New Mexico: by doing literature review and connecting people to each other.

    Exactly none of this depended on *my* prognostications. And in fact I was late to the party compared to people who work on infectious disease for a living; they started working on vaccines as soon as the SARS-CoV-2 genome was published.

    The people on Scott Aaronson’s list weren’t the first to figure out that COVID-19 is a big deal. (That would be epidemiologists, virologists, and…Chinese people, who were living through the worst of the crisis, and making absolutely savage memes back in February.)

    We’re the people Scott Aaronson mentions because we’re the people *he follows on Twitter* who were saying it was a big deal when news outlets were still writing hot takes about racist techies. I think it is fair to say we beat the media, and the official pronouncements of government and intergovernmental agencies. We did not beat the experts. We *actually listened* to the experts.

    • Matt M says:

      Right. As I’ve said above, “heard about community spread within the US” is when I flipped from “this will probably be nothing” to “this might be something big” as well.

      So I guess while it’s fair to say that you don’t need to be an expert epidemiologist to have made some good moves or accurate predictions in this environment, you did need to know enough to at least understand what community spread is, what it implies, why it is important, etc. And that’s hardly common knowledge. “We now have community spread” was covered in the media, but the media’s sense of urgency didn’t update until much later. And I still don’t think the average person even comes close to understanding it.

      • HeelBearCub says:

        “this might be something big”

        When do you think you might flip to “this is something big?”

        • Matt M says:

          It depends. My life has definitely been highly disrupted, although I would argue that it has been disrupted because of state-enforced shutdowns rather than because of the direct effects of the virus.

          I’m a simple man. I’ll probably concede that the virus itself is something big if anyone I know either under 60 and otherwise in good health dies of it.

          • HeelBearCub says:

            I’m a simple man. I’ll probably concede that the virus itself is something big if anyone I know either under 60 and otherwise in good health dies of it.

            That is as perfect a crystallization of the flawed thought process as I could have hoped for.

          • albatross11 says:

            I think a healthy 60 year old in the US expects about another 20-30 years of life, so a lot of 60 year olds dying is actually pretty significant.

    • Elena Yudovina says:

      I think you possibly mean “Scott Alexander” when you say “Scott Aaronson”?

    • xq says:

      Almost all Scott’s links are to articles written in late Jan or very early Feb. I don’t think you get much credit for realizing this was a big deal after mid Feb, the mainstream view had already shifted by then.

    • Anonymous Bosch says:

      Timestamped.

      Links.

      To *any* of this.

    • albatross11 says:

      This is in many ways the best thing about the internet–you can hear from actual experts yourself rather than trusting in the interpretive powers of someone writing for the New York Times.

    • Douglas Knight says:

      But how did you decide that community spread was the threshold? You didn’t get that from the CDC. Maybe the CDC would have said that was the threshold for lockdowns, but their job isn’t to tell the public, so you had to do epidemiology on your own. You relied on the CDC for testing, not for predictions.

      Community spread is a striking choice because it isn’t part of the territory, but part of the map: it is defined as losing track of what’s going on. Without changing actual transmission, contact tracing reduces community spread and community surveillance probably increases it.

      • HeelBearCub says:

        “We’ve lost track of all the places the virus is” seems like an excellent time to start thinking the virus may be spreading uncontrollably.

        I’m not sure I understand your point.

      • srconstantin says:

        It wasn’t a call I put much thought behind. Just seemed like a good Schelling point guess at a time when I didn’t know much about the disease at all. The thought process was something like “if the only people who are getting the disease are people with some contacts in China, that doesn’t apply to my family; but if random Americans are getting it, we’re in theory at risk, and it grows fast, so better safe than sorry.”

  46. Jack Sorensen2 says:

    Your opening paragraph about how much the media blew it strikes me as being a little unfair. In particular, you shit on Vox for one tweet on January 31, but if you search Vox for “Coronavirus” in February you see a slew of headlines taking it seriously; and IIRC, the actual article that that one tweet links to was more alarmist than the tweet itself. So the idea that Vox was downplaying the pandemic this whole time isn’t true.

    Your WaPo example is an opinion piece. WaPo publishes a range of opinions, including from pro-Trump people. If you search for Washington Post coverage of Coronavirus on that same day you get like 20 hits, mostly straight news stories that seem to take it seriously.

    Many critiques of media coverage of coronavirus collapse the timeline and collapse different levels of non-alarmism. I see many of them coming from the right and I think that part of the reason is that the reactions from, e.g., Trump and Fox News were so dismissive, so hostile even to the idea that there’s anything to worry about, and so late in the timeline, that some people want to muddy the waters and make it seem like Vox saying in a tweet in January that there won’t be a pandemic is the same as, say, Rush Limbaugh in mid-March saying it’s the same as a cold.

    • jmo says:

      Precisely! +1

    • Anonymous Bosch says:

      if you search Vox for “Coronavirus” in February you see a slew of headlines taking it seriously;

      There was once a Scott who was *especially* likely to interrogate/criticize a hot take when it played to his existing biases rather than just mindlessly passing it along.

      As best I can tell he died of attention surplus disorder sometime in 2016.

    • It wasn’t just a tweet from January 1st, here’s an article from Feb 13 explicitly comparing it to the flu:

      Some have criticized comparing the coronavirus to the flu because it has a far higher fatality rate and that it distracts from the new virus’s severity. But the fact remains that, so far, the flu has impacted far more people. The CDC estimates that 10,000 people have died from the flu this season, with some 19 million people in the US having experienced flu illness. Data from the CDC suggests that the flu is a greater threat to Americans than the coronavirus. Yet unlike the flu, the coronavirus is new and not well understood, which makes it especially scary to the public, including Silicon Valley’s elite.

      https://www.vox.com/recode/2020/2/13/21128209/coronavirus-fears-contagion-how-infection-spreads

      • Edward Scizorhands says:

        I’ve gone back and forth on whether that article is mocking the people worried about corona or not. I think it depends on how hostile I’m feeling to the media that day.

    • keaswaran says:

      Notably, Scott completely neglects to mention that the media outlet that published the pieces he praises most at the end of this article was… Vox. It’s very interesting that he attributes the bad predictions to Vox (ignoring the author) and attributes the good predictions to the author (ignoring the outlet that published it).

      • Anonymous Bosch says:

        This sort of bad take cherry-picking is akin to burning the commons with Milo-style hot takes. If the media is committed to publishing a variety of voices, then inevitably some of those voices will get things howlingly wrong. Compiling those into screenshots with no overarching theme except “lol media dumb” is a great way to incentivize them not to encourage this variety of perspectives, for no benefit other than looking smug to other people who share your disinclination for nuance.

        • mtl1882 says:

          The problem is that, in general, these outlets don’t portray themselves as “just throwing some competing ideas out there from various voices.” They come on very strong and tend to signal boost each other, then when they screw up in concert they pretend nothing happened. Then they portray themselves as the only legitimate, reliable sources, who have been far more correct than the reckless sources of misinformation who got us here, and obviously must be listened to by all decent people. They don’t acknowledge fallibility nearly enough–if they were less intense about it, I don’t think people would be taking such glee in putting together these screenshots. (I agree this can go overboard and become more about signaling superiority, and that it can present an inaccurate view–the problem is that the mainstream media also does these things, and you can’t only let one side slide). They could easily say they got it wrong, as many did, and that they are still generally reliable. But they are going after others who got it wrong in the same ways, cherry-picking, and then arguing that people should put more trust in them (MSM)–close to total trust as a result. Pretty much everyone had a record of mixed good and misleading or just reckless information. It’s important not to allow a rewrite of events by anyone, but especially of the press itself.

          • Dan L says:

            I think I disagree with about half of what you just said, but it’s hard to tell given how it’s couched in nonspecific generalities. Cherry-picking is bad, but the response ought not to be to make claims both broader and less falsifiable.

          • mtl1882 says:

            Cherry-picking is bad, but the response ought not to be to make claims both broader and less falsifiable.

            Yes, that response is no solution to the broader problem, though it is an inevitable quick retort in an environment like Twitter. But Scott’s argument has much more to it than the screenshot, and it shows these mistakes were widespread. I don’t think Scott’s point lends itself as well to charges of cherrypicking because he is illustrating something more specific. It’s not a “Vox sucks” piece, but he’s using them as representative of a larger dynamic. I do think he should make clearer that the February 6 piece was published on Vox as well, but I don’t think his point was “Vox refuses to post good information and just sucks,” but rather that “Vox published various takes, some downplaying concern with a tone of dismissive certainty, and then acted like taking it seriously was obviously the only right answer.” That’s why he just gives one quick example of each source being wrong–the point isn’t that they were unusually or deeply inaccurate, but flippantly and moralizingly off-base in the same way most outlets were. It would be unhelpfully broad to use these examples to dismiss the media as all nonsense, because of course they also contain a lot of good information. But they are inconsistent in predictable ways, and it is important to know that so you can try and separate good from bad.

          • HeelBearCub says:

            Scott has a history of bagging on Vox in particular, so the charge of bias here has more context than just this post.

          • mtl1882 says:

            Scott has a history of bagging on Vox in particular, so the charge of bias here has more context than just this post.

            Wasn’t aware of that, so thank you for the context. I’m not saying this piece is totally fair to Vox, just that the same argument could have been made without any criticism of Vox. It doesn’t hinge on the matter.

          • HeelBearCub says:

            It doesn’t hinge on the matter.

            The other thing is that this particular kind of … oversight is exactly the kind of thing Scott would call out others for as evidence they were being dishonest in some way.

            Sauce for the goose and all that.

            And the argument, such as it is, does depend to a great extent on looking for evidence of ignorance and then portraying the media as a monolith, in opposition to the few individuals who “got it right”. Given Scott’s basic enmity for the media, and institutions in general, and strong preference at promoting the iconoclastic individual, I think his argument actually does somewhat hinge on it.

          • Clutzy says:

            Scott has a history of bagging on Vox in particular, so the charge of bias here has more context than just this post.

            @HBC

            I think that is a misreading of Scott. It seems to me Scott reads a lot of Vox, and seems to put faith in them generally. But because he reads so much Vox (I derive this from him citing them often in posts), he notices their mistakes a lot.

          • profgerm says:

            @Clutzy

            I think part of the reason Scott focuses on Vox (for better or worse) is an association with “his area.” They’re based in DC officially but they’re reasonably big on reporting on tech/San Fran/etc, and Scott comes across as having a deep emotional connection to the area, so he takes their reporting more personally than perhaps he should.

            Plus, as HBC alludes to with his enmity of media, Vox writers often takes position that are completely antithetical to Scott’s and touch on those subjects that rile him up most. They did run that Voxplainer about monogamy being foolish and outdated; I’m sure he appreciated that.

            He used to cite a lot of Current Affairs, probably not because he trusts their reporting, but he thinks CA has too much popularity for their low quality and the interactions he’s had with Robinson. Though maybe he’s decided reading CA is bad for his health to very little benefit and he avoids it now; perhaps wise to take the same route with Vox.

          • Dan L says:

            @ mtl1882:

            But Scott’s argument has much more to it than the screenshot, and it shows these mistakes were widespread. I don’t think Scott’s point lends itself as well to charges of cherrypicking because he is illustrating something more specific. It’s not a “Vox sucks” piece, but he’s using them as representative of a larger dynamic.

            The kind of systematic media survey that is needed to make cases like this is difficult, expensive, and prone to any number of basic conceptual failings. It should not be surprising that it is consequentially rare, and the best you are going to get without making a concerted effort is a bare handful of anecdotes from a specific individual’s consumption. Be more skeptical about taking them as representative.

            (And that’s even without going into the matter of click-through rates. The whole intro reads very differently if you stop reading at each link to check what’s being summarized.)

            @ Clutzy:

            I think that is a misreading of Scott. It seems to me Scott reads a lot of Vox, and seems to put faith in them generally. But because he reads so much Vox (I derive this from him citing them often in posts), he notices their mistakes a lot.

            I would be very interested in the specifics of how Scott gets his inputs, and generally put a high level of importance on the exact chain of transmission. I take it an unspoken heuristic (and should probably be shouting it from the rooftops) that if I see a retweet by Joe Average linking to a Vox piece by Kelsey Piper, the correct order of attribution is 1) Piper, 2) Joe, 3) Twitter, 4) Vox. Details are context dependent, but there are very few circumstances where social media signal boosting cannot overpower publication quality filters.

      • profgerm says:

        This is an important point.

        I suspect Vox “the organization” would almost definitely claim “it” didn’t have a take, period. Individual contributors had takes, some of which were reasonable and some of which were terrible.

        Someone could try to average it together and say “Vox the organization” doesn’t do a good job of editing/following up/distinguishing contributors/etc, and this is probably true (note that I do not trust Ezra Klein and that probably colors my perception of Vox-the-organization as founder/editor at large (I also find that a silly phrase; it makes him sound like a criminal that Vox is hunting)), but it’s still good to highlight that distinction.

        • albatross11 says:

          I think this leads to a good general rule: When you can, follow individual writers and thinkers, rather than institutions. Kelsey Piper is worth listening to in ways that the median Vox writer isn’t.

          • profgerm says:

            Seconded. Requires more attention, but likely worth the benefits over giving bad actors cover for being associated with a good institution, or missing out on good information from good people due to an association with an institution/others you don’t trust.

      • Chalid says:

        There is a not-Vox-specific pattern of Scott attributing pieces that he’s praising to individuals and pieces that he’s criticizing to organizations. I kind of remember him saying that he felt bad criticizing people by name but I can’t find it.

        And then we go and bemoan outgroup homogeneity bias and wonder how we can change things.

    • WaPo publishes a range of opinions, including from pro-Trump people

      This reminds me of the East German political scene. Few are aware of the fact that there was a “Liberal Democratic Party of Germany” officially espousing liberal ideology(as in classical liberalism) and even a “National Democratic Party of Germany” intended to represent former members of the Nazi Party, the Wehrmacht and middle classes. Outside observers wonder why they bothered. Didn’t everyone see the difference between these parties and the ruling Socialist Unity Party? Didn’t everyone realize there was no real difference between East Germany and other communist one-party states? I think the answer is that some really did believe they lived in a “multiparty” system. After all, the statement “East Germany has multiple parties with distinct ideologies” was true in a literal sense.

      • Jack Sorensen2 says:

        If what I said is like saying “East Germany has multiple parties with distinct ideologies”, then what Scott said is like saying “East Germany is a liberal democracy because look at this political party, the Liberal Democratic Party of Germany, espousing liberal democratic ideals!”

    • Atlas says:

      Thank you. It really irks me when people jump from “one opinion column in one publication somewhere” to “the party line from Pravda.”

      As another example of part of the mainstream media warning about the potential risks early on, here’s the NYT editorial board on January 29th asking “Is the World Ready for Coronavirus?“:

      The coronavirus outbreak, which began in early December in the Chinese city of Wuhan, had as of Wednesday sickened more than 6,000 people across at least 15 countries and claimed more than 130 lives, all of them in China. Experts don’t yet know how contagious, or how deadly, this new virus is. But the growing crisis has inspired panic. Cities around the world are bracing for a potential wave of infections. Stock and oil prices are tumbling. And experts in just about every global industry are fretting over the many supply chains that could be disrupted — from prescription drugs and surgical masks to rare earth metals — if the outbreak grows into an even wider epidemic.

      Given the scope of these anxieties, it’s a wonder more hasn’t been done to prepare for an outbreak like this one. The Centers for Disease Control and Prevention have long warned that the rate at which new pathogens emerge is increasing (partly as a result of global warming). The agency identifies new ones every year, and disease outbreaks — especially those involving viruses that leap from animals to humans, as the coronavirus did — are hardly surprising…

      Global health experts are already cautioning other countries to prepare for China’s containment efforts to fail. Part of the problem is that a quarantine of this size is inherently difficult to maintain. Another major issue is a lack of public trust: Control measures work only if people abide by them. And people are much less likely to follow orders when they don’t trust that the authorities issuing them have their best interests at heart. In Wuhan and elsewhere in China, trust in officials is running low, as some citizens suspect that the government is more concerned with containing bad press than with defeating the virus.

      American officials ought to keep that hard-learned lesson in mind. President Trump said after the first case of coronavirus was confirmed in the United States that the outbreak is “totally under control.” (Since then, such cases have been confirmed in at least five people in four states.) One would like to believe the president’s pronouncements. But Mr. Trump has a history of obscuring facts — especially scientific ones — to score political points. Scientists and the public are right to worry about the potential consequences of such duplicity. What happens if people stop trusting the institutions meant to protect them from natural disasters, faulty medical products or disease outbreaks? If the coronavirus proves especially contagious or deadly, that question will no longer be hypothetical. [My emphases]

      That seems like it deserves some credit to me.

      • Matt M says:

        Ehhhh… not sure I buy it.

        On the one hand, yes, it is technically true that most large media organizations publish pieces from a variety of perspectives, and that typically, they do not publish any official “the organization’s view on X” or anything like that.

        But come on man, we know that Vox has a different worldview than Breitbart. And we’re right about that. In the extreme case, I think it would be entirely fair to say that an organization who publishes 10 “COVID is no big deal” stories for every 1 “COVID is a threat” story has a very different view on the threat of COVID than an organization who publishes 10 “COVID is a threat” stories for every 1 “COVID is no big deal” story, would it not?

        These organizations get to pick who they publish and who they don’t. And if you “stray from their organizational values” too much, they stop publishing you.

        I don’t follow Vox closely, but I know many who do, and as far as I can tell, the consensus is generally something like “Their coverage was despicably awful, except for Kelsey.” Should they get some amount of credit for hiring Kelsey in the first place, and not firing her despite her abnormal opinions on COVID early on? Yeah, sure. That’s better than the alternative. But it’s not better than a hypothetical alternative where everyone they published was a lot more like Kelsey…

        • Atlas says:

          On the one hand, yes, it is technically true that most large media organizations publish pieces from a variety of perspectives, and that typically, they do not publish any official “the organization’s view on X” or anything like that.

          Well, some, like the New York Times, have editorial boards that sort of do that, hence the article I linked above.

          But come on man, we know that Vox has a different worldview than Breitbart. And we’re right about that. In the extreme case, I think it would be entirely fair to say that an organization who publishes 10 “COVID is no big deal” stories for every 1 “COVID is a threat” story has a very different view on the threat of COVID than an organization who publishes 10 “COVID is a threat” stories for every 1 “COVID is no big deal” story, would it not?

          These organizations get to pick who they publish and who they don’t. And if you “stray from their organizational values” too much, they stop publishing you.

          I don’t follow Vox closely, but I know many who do, and as far as I can tell, the consensus is generally something like “Their coverage was despicably awful, except for Kelsey.” Should they get some amount of credit for hiring Kelsey in the first place, and not firing her despite her abnormal opinions on COVID early on? Yeah, sure. That’s better than the alternative. But it’s not better than a hypothetical alternative where everyone they published was a lot more like Kelsey…

          Sure, but people need to cite more evidence than “one article/tweet from Vox was wrong” to convince me that Vox’s coverage of the coronavirus was abominably bad. If you can carefully demonstrate that the net balance of coverage at Vox reflected irresponsible indifference to the potential risks of the coronavirus, fair enough, but so far I’m not sure that anyone has done that.

          • mtl1882 says:

            The issue isn’t that Vox was abominably bad–I don’t think it was that much of an outlier. It’s that Vox claims to be exceptionally good, so even if it just screwed up on average the same amount as other outlets, that’s still a big deal. That’s what is prompting much of the backlash (of course, this is partly driven by culture war rivalries on both sides, not just an objective concern about information quality, but they aren’t separable).

        • profgerm says:

          “Their coverage was despicably awful, except for Kelsey.”

          How many saying that also have a personal or at least non-Vox association with Kelsey?

          I mean, I agree that she was one of few good Vox contributors on the topic (Julia Belluz seemed to do well also) but I think that’s an important factor to keep in mind.

        • albatross11 says:

          IME Vox is an okay source of information (probably better than the average media source) except where the information touches on some kind of culture war issue, at which point their quality falls dramatically.

        • Jack Sorensen2 says:

          I really don’t think this is factually true. I searched “coronavirus site:vox.com” on Google for stories between 2/1/2020 and 2/15/2020, inclusive. Here is what I got on the first page of results, by headline + first paragraph of story:

          Coronavirus cases in China just jumped by 14,000 — but not for the reason you think…Earlier this week, stories and tweets speculating about a downward trend in the number of cases of Covid-19, the new coronavirus disease, cast a ray of hope on what’s been a remarkably worrisome outbreak.

          What happens next in the coronavirus outbreak? We mapped 8 scenarios. … We’re at a pivotal moment in the outbreak of the new coronavirus in China. Depending on whom you ask, we’re either already in a pandemic, meaning there are ongoing epidemics of the virus on two or more continents; we’re hurtling toward one; or we’re on the path to averting a spiraling crisis.

          Why we still don’t know how deadly this new coronavirus is … As the coronavirus continues to spread in China, and countries like Singapore and South Korea face their own local outbreaks, we are at a critical decision point. Should the international community continue to put most of its resources into containing this virus, or should we switch to a focus on national preparedness for and response to a disease that we think is likely to spread globally?

          Why the coronavirus outbreak might be much bigger than we know … In a remarkably calm and candid tone, Singapore’s prime minister acknowledged something difficult: The new coronavirus disease, Covid-19, was likely spreading in his country, even among people who have never been to China.

          China’s draconian response to the new coronavirus, explained by a China expert … Nearly two decades since SARS emerged in 2003, China is again at the center of a battle against a new coronavirus.

          The life-and-death consequences of naming the coronavirus … The new disease caused by the coronavirus outbreak that began in Wuhan, China, in December now has an official name: Covid-19.

          France reports the first coronavirus death outside of Asia … A Chinese tourist died due to Covid-19 — the illness caused by the coronavirus outbreak first observed in Wuhan, China — at a French hospital Friday. The case was the first reported death caused by the virus outside of Asia.

          The first American death is reported in the Wuhan coronavirus outbreak … A US citizen diagnosed with the coronavirus died in Wuhan, China on Thursday, becoming the first American to succumb to the virus, one that has already killed more than 700 others.

          Scientists are identifying potential treatments for coronavirus via artificial intelligence … To battle the novel coronavirus that’s been linked to Wuhan, China, researchers are using artificial intelligence to discover potential treatments, including already-approved drugs and completely new compounds. At the same time, the pneumonia-like illness has only gotten worse. As of Friday morning, the 2019-nCoV coronavirus had taken the lives of more than 600 people and infected more than 30,000, with cases documented in at least 25 countries.

          “No handshakes, please”: The tech industry is terrified of the coronavirus … One of the most prestigious venture capital firms in Silicon Valley, Andreessen Horowitz, recently put up a sign on its door, cautioning eager startup founders and business partners who walk into its offices: “Due to the Coronavirus, no Handshakes please. Thank you.”

          I really don’t think this is a bunch of “COVID is no big deal” stories.

          Honestly the more I think about this, the more it bothers me. Scott writes an entire post about how horrible the media is for allegedly screwing this up, cites vox as an example, cites one (1) tweet, ignores all of the above, and meanwhile nothing about Fox News whose hosts were saying it’s all bullshit and there’s no greater danger to anyone than the flu well into March. On March 9, Sean Hannity said it was a hoax. On March 11, Rush Limbaugh said it was like the cold. When Vox’s tweet happened, there were 7 cases and 0 confirmed deaths in the US (the first death would be 29 days later), and very few cases outside China. When Hannity and Rush said their things, there were 645 and 1205 cases in the US respectively, and Italy had 10,000 and was locking down.

          I like to hate on Vox as much as lots of others here, but the real difference is that guys like Ezra have more integrity than guys like Hannity, and will respond to these accusations with introspection instead of “fuck you what about Hannity”; whereas Fox will ignore and lie about its own actions and then try to portray the left-leaning media in the worst way possible.

          • No handshakes, please”: The tech industry is terrified of the coronavirus … One of the most prestigious venture capital firms in Silicon Valley, Andreessen Horowitz, recently put up a sign on its door, cautioning eager startup founders and business partners who walk into its offices: “Due to the Coronavirus, no Handshakes please. Thank you.”

            Please read the article, this quote is severely out of context.

          • Jack Sorensen2 says:

            On the last article I quoted that Alexander Turok says is “severely out of context”: I selected the stories/quoted a portion of them according to a set rule and not selectively so that I couldn’t be accused of selectively quoting out of context. I wasn’t cherry-picking alarmist stories or anything like that. But I also don’t think this story is particularly dismissive. It portrays some of what people are doing as unnecessary/going too far, but also treats sympathetically the idea that people should be preparing. Would be bad if it was the extent of their coverage, but if it’s the worst one out of 10 then this whole “vox’s coverage was completely dismissive” thing is wrong.

            Link so people can judge for themselves:

            https://www.vox.com/recode/2020/2/13/21128209/coronavirus-fears-contagion-how-infection-spreads

            Anyway Rush Limbaugh said it was like a cold a month after this story ran, so it’s a little weird to split hairs over this.

          • profgerm says:

            Bringing up Ezra specifically is a terrible way to defend Vox; he’s a smug Californian but that does not give him more integrity than Hannity. Not to mention they delete tweets and articles where they “got it wrong,” which is the equivalent of lying about their own actions. Don’t get me wrong, I think Fox is still much worse, but I think Vox sets a much higher standard for themselves and they still consistently miss it. Not in the sense of “perfection is an impossible standard,” but along the lines of “IFL Science but really only care when it’s convenient;” they’re claiming a high standard to provide cover for their mediocre clickbait and outright lies (which is not all, and perhaps not even a majority, of their coverage, but it’s enough to be concerning).

            I even changed my mind on judging Vox-the-organization too harshly (see below), but Klein, Swisher (founder of Recode; defender though not author of that handshake article), and at least a handful of other contributors are the ones that give them the bad name. There are good contributors, and it looks like you quoted several of Julia Belluz’s articles, who I think was doing a great job of handing this reasonably, with an appropriate level of concern, and reporting the science well (which she’s apparently been a science reporter most/all of her career and is, so far as I can tell, leaps and bounds better than the average).

            On the handshake article, what sections of it are sympathetic to people preparing? It calls techies paranoid and racist, then goes on about the rich buying hidey-holes in New Zealand and fortifying their houses.

    • profgerm says:

      >So the idea that Vox was downplaying the pandemic this whole time isn’t true.

      Ehh… I tend to wear my anti-Vox bias on my sleeve but I’d like to build on the other commenter that pointed out Scott not making an organization versus contributor distinction.

      For one, it would be more useful if we had readership numbers. Given the rate they change headlines, delete tweets, etc, I’m sure Vox has detailed information on the popularity of their articles. Vox posts dozens of articles a day by dozens of contributors, and even early on they were posting at a fairly steady rate. If they have one author posting solid articles that get a handful of readers, and multiple authors posting BS articles that get the majority of readers, then the organization as a whole was still doing a poor job with just enough coverage to say “ah, we were going both sides, we covered our ass” even though the general perception is that they were doing poorly.

      This notoriously bad Recode article got lots of attention, especially in hindsight, so how much of “Vox was crap” was due to this one article and Kara Swisher’s defense of it? (For why it’s bad, see actual expert and non-journalist Bajaji Srinivasan).

      Julia Belluz posted good articles containing actual science (1, 2, 3) and I’d enjoy knowing how much attention those received and the impact they had over the… let’s say less scientific and more politically-motivated articles (1, 2, the Recode one above).

      >I see many of them coming from the right and I think that part of the reason is that the reactions from, e.g., Trump and Fox News were so dismissive, so hostile even to the idea that there’s anything to worry about, and so late in the timeline, that some people want to muddy the waters and make it seem like Vox saying in a tweet in January that there won’t be a pandemic is the same as, say, Rush Limbaugh in mid-March saying it’s the same as a cold.

      Good point, and the thoroughness and “sidedness” of the flip was bizarre. I suspect, and I may be projecting, that part of Scott wanting to smack on Vox et al specifically is their general smugness, and that they deleted the tweet to hide the history rather than admit they were wrong.

      • KelseyPiper says:

        My Feb 6th “don’t tell people they’re overreacting” got about as much traffic as that Recode article, and far more traffic if we don’t count Twitter traffic (which was disproportionately from people who were there from Balaji’s takedown of the article). Julia’s January coverage got 20x more viewers than both those combined. Most people who read about the coronavirus on Vox read Julia’s articles.

        • profgerm says:

          Interesting, thank you!

          I had considered Recode one of the more popular… subheadings? verticals? of Vox and expected that that one would’ve gotten more attention. Perhaps my circles just share Recode articles more often, historically speaking, and my perception of its relative popularity is way off.

          I’ll continue to work on focusing on specific authors rather than lumping together the whole organization when possible (both in supporting/paying attention to authors I respect, and not allowing specific authors/subsections to completely negatively color perception of a publication writ large). Treating it as too cohesive and too centered on a couple big and controversial names leads to too much disrespect and distrust towards lesser-known writers doing actual good work.

  47. meh says:

    It wouldn’t have helped anything, but I would have had bragging rights forever.

    This is the worst part of the whole thing. Opportunistic people trying to increase their influence or build their lifestyle brand will try to make an early outlandish prediction to get the ‘bragging rights forever’, and of course suffer no consequence for the 100 times they are wrong. Meanwhile Silver’s expertise is called into question because a *gasp* 29% event happens.

    • Matt M says:

      Yeah, this is definitely a problem in a modern society where “influence” is increasingly the most valuable currency, and where the best way to gain influence is to make a really crazy prediction too early and be right, whereas when you make a really crazy prediction too early and are wrong, everybody just shrugs and says “well nobody could have known.” It definitely incentivizes people to make increasingly extreme predictions, as there’s basically zero reward for predicting anything that’s anywhere close to what the average person is already predicting.

    • The way to do this is to demand pundits keep their predictions in one place, as I do:

      https://alexanderturok.wordpress.com/predictions/

      Archive.org and google cache prevent me from tampering with it. One wonders if it ever occurs to the people at the archive sites to tamper with past pages in order to “predict” things. In the future, if they perfect deepfakes, one could create one and then insert it into an archive and claim it originates before perfected deepfaking existed.

  48. ksvanhorn says:

    Scott’s point about reacting appropriately to a significant-but-less-than-half probability of some very bad thing happening is worth repeating. The way I like to think about it is this: if you estimated a 10% chance of some catastrophe happening, wouldn’t it be prudent to raise the alarm? But notice that if that is your consistent policy, 9 times out of 10 you will be crying wolf. So maybe we should be careful about dismissing the concerns of people who have previously warned of dangers that did not come to pass, so long as the warnings were couched as significant possibilities (preferably with reasonable attached probabilities) rather than as certainties.

    • Conrad Honcho says:

      I’m not sure you need to raise the alarm so much as you just need to be prepared. If there’s a 10% chance of wolf, the correct answer is “buy a gun” but not “fire wildly into the bushes every time you hear them rustle.”

  49. moridinamael says:

    My personal heuristic regarding experts is that you should defer to experts unless (a) the situation is actually subtly outside of the scope of their expertise or (b) the topic concerns something about which personal incentives might be relevant to the expert.

    First, (a). I am an expert on a particular technical thing. You could describe me as a top, world-class expert on this thing. There is, unfortunately, an invisible-to-you boundary that distinguishes “the thing I am an expert at” from “very similar-looking adjacent things where my intuition would be no better than the average professional in my field.” There is another invisible-to-you boundary that distinguishes “things that professionals in my field are likely to be right about” from “very similar-looking adjacent things where the intuition of a professional in my field would be no better than the average smart person on Twitter.”

    This reminds me of how some of the very worst takes in this whole episode came from virologists; virologists are not epidemiologists, any more than expert drywall installers are architects.

    Next, (b). I am not actually talking about corruption when I say “incentives”. I am talking about the kind of Molochian traps that make everything stupid even if individuals are doing almost as well as they can.

    Pretty much every expert in the universe exists in a state of constant frustration. The world is dumb and is not designed to actually take your expertise and process it into the appropriate end result. Experts will deliver the Right Answer to a corporation or government, and the corporation or government will take the Right Answer and process it through a bureaucracy which inevitably manages to use the Right Answer as a kind of prop to justify doing something entirely different for its own bureaucratic reasons. Some experts may anticipate this and shape their delivery of the Right Answer in order to increase the odds that the actual Right Action will be taken after the information is processed by the bureaucracy. This *might* be what happened with the mask recommendation, but I think your own analysis is actually more accurate; they didn’t know makes were helpful and lie about it, they are just not very good at thinking.

  50. brownbat says:

    Scott’s piece hit hard. I put one cryptic tweet out saying this would be bad and liquidated my (unimpressive) stock holdings in February. Not with gusto, but kicking myself the whole time: “Timing the market runs against conventional wisdom you dolt! Who am I to ignore all those people who work so tirelessly to bring us wise conventions?”

    I came off lucky, better than most, but emotionally I’m just as full of regret that I didn’t have greater certainty. My lack of conviction thankfully hasn’t risked anyone’s life, but why didn’t I push my family and friends to leave the market, or not to travel?

    I really like your gloss. It’s not about perfect certainty in uncertain times, don’t wring your hands over that. Focus on how to act under uncertainty.

    I take it that’s not just “always round probabilities upward,” which might seem like the wrong lesson to take from this event. It’s in line with that old saying about how if you’re holding a sapling when you hear the Messiah has come, plant first, then go meet him. Also, though, if you’re heading to a concert when you hear about a possible global pandemic, maybe go stream it instead? I don’t know much about messiahs, but I take it the consequences of being a few days late are pretty different.

    One wrinkle that makes it hard to calibrate here is that “just in time” reactions are so often the right ones, in a lot of more banal situations. They just happen to be catastrophic when wrong. Why weren’t companies just stockpiling cash and medical supplies for a rainy day? That… might not have been the best use of capital for the century between 1918 and 2020.

    “Just in time” reactions defend us against Pascal’s Mugger-lite situations. And committing real resources makes this all so much harder. If we have an extra billion, how do we distribute it between the HILF risks of asteroid prevention, a 200-year sunspot wiping out our thin backup supply of large transformers, and preventing an outbreak of ebola? What resources would you say we should divert right now from containing this emergency to hedge against other high impact uncertain events?

    And N.B., we’re always dealing with some “emergency” that seems biggest in our minds. How many resources would 2018 you pull away from “ensuring the recovery leads to wage growth?” How much would you have defunded Australian wildfire response efforts to prep for this? How much should we divert from THIS crisis to prepare for the next one?

    Ok, ok, that got a bit unfair. You don’t shop for flood insurance while your house is on fire. But there are weird thresholds and tradeoffs at play. When exactly do we stop worrying about precision and just get on with acting under uncertainty? I ask this NOT/NOT as a critique, but because I agree with you, and this just strikes me as the next important question in the series.

    I think your answer might be… “Just get on with it! You do cost benefit calculations under uncertainty. Nobody promised they would be easy!”

    That gets us to a weird dilemma though. If you’re consoling people who were less than omniscient predictors, because that’s to be expected, but urging us to focus on acting under uncertainty, and it turns out that too is also incredibly hard… though much easier if we can at least nail the probabilities… then I worry we might end up going in a bit of a circle.

    Ok, that admittedly sounds like more of a critique, but it’s halfhearted, I don’t know where it gets me. I still get the thorough sense that you’re heading the right direction and have worked this out much farther than I have.

    Thanks for the post, definitely though provoking.

  51. Nevertaken says:

    “We couldn’t do more, because it would have been unethical to deliberately expose face-mask-less people to disease.”

    The very fact that it’s recognized that it’s unethical to expose the control group to the danger of not using masks is evidence that, regardless of what experts said, many really believed that masks work.

    Maybe a good rule of thumb is that when there are no good RCTs because it’s unethical to have a control group, the thing in question should be assumed to work in the absence of evidence to the contrary?

    • Matt M says:

      The very fact that it’s recognized that it’s unethical to expose the control group to the danger of not using masks is evidence that, regardless of what experts said, many really believed that masks work.

      +1

      Anyone who legitimately believes it’s reasonably likely that masks provide no meaningful benefit whatsoever should have no ethical qualms about running such an experiment.

      The fact that we won’t run such an experiment suggests that masks are much more like parachutes than they are like Dr. Frank’s Miracle Snake Oil or whatever.

  52. crybx says:

    I have receipts from when I bought masks on Jan. 23, the day that Wuhan locked down. I also started buying extra food on every grocery trip after that and have a Feb. 10 Costco online order I can point to. I found I couldn’t bring myself to say a single thing in a public place like Twitter though, because coworkers and family members invested themselves hard in convincing me that I needed psychiatric help for anxiety. Additionally, my boss at work shut down a lot of conversations I would have had by acting like talking about the possibility of things getting bad was irresponsible fear mongering and that anyone who bought some masks was practically a murderer.

    Thankfully my partner was supportive and never once said I was overreacting for buying too much food.

    I’m still processing what this all has taught me about relying on my own probabilistic reasoning vs. relying on reading the reactions of people around me.

  53. Aotho says:

    I think there is something fundamental that this article gets wrong.

    I think many people who thought about this weren’t hinging their bets on the virus being contained in China, but rather they decided that western governments won’t take similarly drastic and arguably draconian measures to CCP. That they would look at the mortality charts, notice that not a lot of QALYs or GDP would be lost if no action is taken, conversely lockdowns would highly likely hurt GDP quite a bit, and therefore would choose not to have lockdowns.

    What’s unforeseen then is that they choose to have lockdowns anyway. Why? Do they value elderly lives more than thought? Are they aware of larger adverse effects to GDP of non-lockdown that is not widely known? Are they sufficiently fearing non-reelection that makes them value human lives more? Something else?

    • keaswaran says:

      “That they would look at the mortality charts, notice that not a lot of QALYs or GDP would be lost if no action is taken, conversely lockdowns would highly likely hurt GDP quite a bit, and therefore would choose not to have lockdowns.”

      This seems very strange to me. I’m not sure how you can look at the past two weeks and still say this. We’ve lost substantially over 10,000 American lives so far, which at standard actuarial accounting is worth about $90 billion. If the reactions have prevented 100,000 deaths, then we could afford it taking a $900 billion hit to the economy. But there’s good reason to think that in fact the reactions are saving on the order of a million deaths, which is worth about 33% of annual US GDP. It seems like the lockdowns are a no-brainer, unless you think there’s a way to cut the deaths without having the economic hit. (Most likely, *not* having a lockdown still gives you much of the GDP hit, as individuals choose to stay home in an uncoordinated way, without the mitigation efforts like the CARES act.)

      • Matt M says:

        We’ve lost substantially over 10,000 American lives so far, which at standard actuarial accounting is worth about $90 billion.

        But we aren’t losing “average” lives. We are mostly (no, not exclusively, but mostly) losing lives of people who are already old and/or in poor health.

        • AlexSpark says:

          And doctors and nurses! With significantly higher fatality rates than the general population.

          It’s a pandemic, it’s all hands on deck. So you’ve got 60 year old expert ophthalmologists (who represent a significant store of expertise) doing front line care for covid-19 patients. Losing 2% of your medical experts in the course of a few months is a huge problem, both in the short term and in the long run.

          And “underlying condition” is not the same as “in poor health”. Someone with diabetes or hypertension is probably employed, likely to exercise regularly etc. This is not people so ill they can’t work. This is more than half the population in some places.

          • Randy M says:

            So you’ve got 60 year old expert ophthalmologists (who represent a significant store of expertise) doing front line care for covid-19 patients.

            Do we? I was wondering in another thread if doctors were so interchangeable in an emergency situation.

          • Matt M says:

            “Healthy 60 year old doctors” are not the majority of COVID deaths, or even close to that, and it’s incredibly disingenuous for you to imply otherwise.

            No, diabetes is not a death sentence. But it does lower your life expectancy by a significant amount, and your overall quality of life by some amount as well.

          • AlexSpark says:

            Do we? I was wondering in another thread if doctors were so interchangeable in an emergency situation.

            That example specifically was taken from an article I read. He’d written on his scrubs identifying himself as a first year junior doctor, because that was what he was for all practical purposes in the ICU. Despite being an extremely well regarded expert and highly paid consultant in his field.

            They’re sort of interchangeable. Sort of. Better than a non-doctor, but probably on the same level as someone just out of medical school.

            “Healthy 60 year old doctors” are not the majority of COVID deaths, or even close to that, and it’s incredibly disingenuous for you to imply otherwise.

            No, diabetes is not a death sentence. But it does lower your life expectancy by a significant amount, and your overall quality of life by some amount as well.

            No, healthy 60 year olds are not the majority of covid-19 deaths? Or even the majority of doctors. I didn’t imply that they were, and it’s incredibly disingenuous for you to say I did.

            But a 60 year old doing front line medical care is at hugely increased risk – medical staff are more likely to get severe illness. Healthy medical staff are more likely to die than people who aren’t medical staff. A lot of the examples of healthy young people dying are medical staff. Losing a significant chunk of your doctors is really worrying – they’re hard to replace.

            And plenty of doctors have underlying health conditions. Asthma doesn’t stop you from being a doctor.

            Does a 5 year shortened life expectancy actually matter? Particularly when you’re talking about medical staff. “die after you retire (age 78)” and “die after you retire ( age 73)” doesn’t seem like a big difference. Statistically significant, but not really important. Especially when you’re, IDK, 40?

            The “quality of life” decrease from most of these conditions is really, really minimal. How much does hypertension or asthma actually decrease your quality of life? More or less than eg. a 10k income difference?

            I think putting too much focus (or gaining any real reassurance) from the idea that people dying are mostly already ill or have shortened life expectancy is disproportionate. They’re not that ill, their life expectancy isn’t that much shorter.

            Being male also reduces your life expectancy, and increases your risk of death from covid-19.

          • Matt M says:

            How much does hypertension or asthma actually decrease your quality of life? More or less than eg. a 10k income difference?

            If someone offered you 10k a year to intentionally give you diabetes or asthma, would you take it?

          • AlexSpark says:

            If someone offered you 10k a year to intentionally give you diabetes or asthma, would you take it?

            Yes. Absolutely and without question.

            Moderate, controllable health impacts vs. not having to stress about whether I can pay the rent or afford to take the tube? I’d take the money every time. I’m pretty sure my mental health would improve, and I’d be able to do more with my free time.

            The amount of stuff you actually can’t do because you have asthma or diabetes is really, really minimal. Like, the major impact is that you have to take insulin or use an inhaler. This is pretty true even for more significant disabilities – most of my life would be completely unaffected.

            I mean, I’m a gig economy worker in a country with decent universal healthcare (and you don’t even have to pay the £10 prescription charge for insulin) so that probably changes things, but yeah, I would.

            I think that would be a serious net improvement to my quality of life.

    • mtl1882 says:

      Yeah. I don’t think I was super on top of this issue, but for about ten years I’ve commented to people occasionally that I expected to see a pandemic in my lifetime, probably along the lines of 1918. It seemed obvious to me that in this timeframe, we would see something big, just by chance, if not from antibiotic resistance or other things related to modern life. But I never acted on it, and while I was unusually worried about the issue relative to most people (but hardly thinking of it on a daily basis), when Covid-19 popped up, I didn’t feel much more urgency. I definitely thought there was a chance it could be crazy, and I think I started spending less money and put a moratorium on buying concert tickets or planning trips, figuring cancellation risk was high. But I didn’t say much about it, and got totally dismissed when I did. I’ve felt we were due for a systemic shock for a while, and had really not wanted to take out a mortgage (believing we would soon have a crash, and that the economy could be iffy) and some other things that involve long-term commitments, especially in one location. I allowed myself to be talked into it by a relative (long story), and have been kicking myself ever since. I’ve also had a nagging feeling that I should be saving like crazy, but did not do that as much as I should have, though I’m okay enough right now. I finally canceled my cable recently to help with that, which I retained way too long because people around me reacted badly to the idea and had bought me a TV and all that—very spineless on my part. I knew I had relatives who would help me if disaster struck. I didn’t think the systemic disaster would hit everybody at once, making this less of a back up plan. For the last year, I’ve been telling myself to sell my place and get into the most stable/independent possible situation, but I’ve been overwhelmed for the last few years and just froze. I thought I was being ridiculous, and that was what people were telling me. Now I’m just really mad at myself for not having the strength to act and not just go along.

      But, though I thought the pandemic could wreak havoc and damage the economy by shutting down global travel, I never thought we’d have a shut down like this. I simply assumed the costs would never be accepted, and I assumed the dismissiveness the leadership class reflected the reality that they’d simply decided to downplay the issue and let the deaths happen. I thought that if it got bad, they’d have to shut down global travel and some other such things, and tell the vulnerable to stay inside, advise masks, all of that. I never thought they’d do this. I think I would have prepared more had I thought they would ever shut down the economy so suddenly and completely. And I assume the reason this happened is because they realized the disease was a lot harder to contain and a lot worse than they hoped, in ways that make it hard to cover up, and they don’t know what to do, but fear unrest. But to me that is the craziest part of all of this, and by the time I was really scared, it was too late to act other than to avoid getting the virus itself, which may well be inevitable eventually. I’m far more afraid of the systemic effects than the virus, though I’m certainly not eager to try my luck with the virus.

    • ana53294 says:

      Yes, in my case, I was hoping the Constitution and the protection of our freedoms it offers was strong enough the government wouldn’t shut down the whole economy and lock people inside.

      I thought the maximum a Western government would do was ban large gatherings, encourage filing stuff online, do things like extend deadlines for things (passports, taxes, inspections), quarantine flights, encourage voluntary distancing.

      So, Taiwan and Sweden, not China.

      • Yes, in my case, I was hoping the Constitution and the protection of our freedoms it offers was strong enough the government wouldn’t shut down the whole economy and lock people inside.

        Weren’t these questions already answered during America’s various wars?

        • mtl1882 says:

          Yes, but most of those things didn’t affect people very obviously and directly. I didn’t have much hope in our freedoms being upheld, but I just figured the obsession with economic growth and spending and all that would still be strong enough to make this untenable, plus many people’s resentment of things that very directly affect them. Of course, some people will always be eager to surrender freedoms for any promise of safety, but many won’t.

          I figured most influential people would spend their energy trying to find scapegoats and rationalizations to make the deaths an issue of personal irresponsibility as much as possible. I still think it is likely that a lot of people at the top supporting this are doing so desperate attempt to pause us prior to collapse while they attempt to find a miracle, rather than accepting a hit to the economy because they are so worried about saving lives. I’m not saying they don’t care at all about whether people die and are evilly plotting to sacrifice, just that they pretty much see everything in terms of the economy and use wishful thinking about other concerns. Some level of collapse was going to happen even if we remained open, awkward and visible trade-offs were going to have to be made–control over “the narrative” was going to be lost. I think many are still in denial and telling themselves that they can make things go back to normal if they keep things in what they imagine is suspended animation. And I think some of the public is playing along in part because of the same kind of thinking—a hope that they can avoid any kind of net loss in the long run if they stay put and let experts try to figure it out.

          • Yes, but most of those things didn’t affect people very obviously and directly.

            Please read up on the home front during WWI and WWII.

          • mtl1882 says:

            Please read up on the home front during WWI and WWII.

            Good point–I need to read more about that. For some reason I assumed you were referring to post-9/11 Patriot Act/TSA/etc. type issues related to Iraq and Afghanistan, but “various wars” should have sent me in another mental direction.

        • Edward Scizorhands says:

          Note that ana isn’t American. She’s Spanish.

    • Purplehermann says:

      People freak out when they see death counters ticking higher, faster, daily.

  54. John Thomas says:

    I’m singling out Kelsey because of a personal connection – I met her through the same probabilistic reasoning forum where I did my cryonics survey years ago. I don’t think this is a coincidence.

    Because nothing is a coincidence.

  55. Atlas says:

    First, a bunch of generic smart people on Twitter who got things exactly right – there are too many of these people to name, but Scott Aaronson highlights “Bill Gates, Balaji Srinivasan, Paul Graham, Greg Cochran, Robin Hanson, Sarah Constantin, Eliezer Yudkowsky, Nicholas Christakis, and Eric Weinstein.” None of these people (except Greg Cochran) are domain experts, and none of them (except Greg Cochran) have creepy oracular powers. So how could they have beaten the experts? Haven’t we been told a million times that generic intelligence is no match for deep domain knowledge?

    I think the answer is they didn’t beat the experts in epidemiology. Whatever probability of pandemic the experts and prediction markets gave for coronavirus getting really bad, these people didn’t necessarily give a higher probability. They were just better at probabilistic reasoning, so they had different reactions to the same number. There’s no reason why smart people shouldn’t be better at probabilistic reasoning then epidemiologists. In fact, this seems exactly like the sort of thing generic smart people might be.

    Two more leading members of the Legion of Doom:

    Nassim Taleb was sounding the alarm by January 26th.

    Anatoly Karlin (who I think I first learned about the coronavirus from) was warning that the coronavirus couldn’t be quarantined and could be a huge pandemic on January 24th.

    • baconbits9 says:

      Taleb is certainly an interesting case, he wrote that a pandemic like this was likely in The Black Swan 14 years ago, but his media attention is far less right now than it was during the 2008 crisis.

      • romeostevens says:

        A strong vindication of the particular point that experts have trouble reasoning about high impact small probability events, and that the blindspot is related to their confidence bounds being too narrow to include such outsize events. I think this gives us concrete heuristics for knowing how much to trust experts in the future.

    • BlindKungFuMaster says:

      Karlin also wrote “The coronavirus will kill millions and crater the world economy” on Feb.24.

  56. dumky2 says:

    Another striking illustration is this FiveThirtyEight survey of experts mid-March, asked to predict cases March 29th in the US:
    https://fivethirtyeight.com/features/infectious-disease-experts-dont-know-how-bad-the-coronavirus-is-going-to-get-either/
    Prediction: around 19k
    Actual: around 140k

    I suspect this is likely due to memory bias (the US didn’t face viruses the same way that Asian countries did in recent year, we recall overreactions rather than under-reactions from those) and exponentials. What is surprising is how narrow the confidence intervals they claimed.

  57. Atlas says:

    Goofus started with the position that masks, being a new idea, needed incontrovertible proof. When the few studies that appeared weren’t incontrovertible enough, he concluded that people shouldn’t wear masks.

    Gallant would have recognized the uncertainty – based on the studies we can’t be 100% sure masks definitely work for this particular condition – and done a cost-benefit analysis. Common sensically, it seems like masks probably should work. The existing evidence for masks is highly suggestive, even if it’s not utter proof. Maybe 80% chance they work, something like that? If you can buy an 80% chance of stopping a deadly pandemic for the cost of having to wear some silly cloth over your face, probably that’s a good deal. Even though regular medicine has good reasons for being as conservative as it is, during a crisis you have to be able to think on your feet.

    Nassim Taleb had a great demonstration of this point in Fooled by Randomness:

    The general press floods us with concepts like bullish and bearish as these mean to refer to the effect of higher (bullish) or lower (bearish) prices in the financial markets. But also we hear people saying “I am bullish on Johnny” or “I am bearish on that guy Nassim in the back who seems incomprehensible to me”, to denote the belief in the likelihood of someone’s rise in life. I have to say that the notion bullish or bearish are often hollow words with no application in a world of randomness – particularly if such a world, like ours, presents asymmetric outcomes.

    When I was in the employment of the New York office of a large investment house, I was subjected on occasions to the harrying weekly “discussion meeting”, which gathered most professionals of the New York trading room. I do not conceal that I was not fond of such gatherings, and not only because they cut into my gym time. While the meetings included traders, that is, people who are judged on their numerical performance, it was mostly a forum for salespeople (people capable of charming customers), and the category of entertainers called Wall Street “economists” or “strategists” who make pronouncements on the fate of the markets, but do not engage in any form of risk taking, thus having their success dependent on rhetoric rather than actually testable facts.

    During the discussion, people were supposed to present their opinions on the state of the world. To me, the meeting was pure intellectual pollution. Everyone had a story, a theory, and insights that they wanted others to share. I resent the person who, without having done much homework in libraries, thinks that he is onto something rather original and insightful on a given subject matter (and respect people with scientific minds like my friend Stan Jonas who feel compelled to spend their nights reading wholesale on a subject matter, trying to figure out what was done on the subject by others before emitting an opinion – would the reader listen to the opinion of a doctor who does not read medical papers?).

    I have to confess that my optimal strategy (to soothe my boredom and allergy to confident platitudes) was to speak as much as I could, while totally avoiding listening to other people’s replies by trying to solve equations in my head. Speaking too much would help me clarify my mind, and, with a little bit of luck, I would not be “invited” back (that is, forced to attend) the following week.

    I was once asked in one of those meetings to express my views on the stock market. I stated, not without a modicum of pomp, that I believed that the market would go slightly up over the next week with a high probability. How high? “About 70%”. Clearly, that was a very strong opinion. But then someone interjected “But, Nassim, you just boasted being short a very large quantity of SP500 futures, making a bet that the market would go down. What made you change your mind?”.

    “I did not change my mind! I have a lot of faith in my bet! (audience laughing). As a matter of fact I now feel like selling even more!” The other employees in the room seemed utterly confused. “Are you bullish or are you bearish?” I was asked by the strategist. I replied that I could not understand the words “bullish” and “bearish” outside of their purely zoological consideration. Just as with events A and B in the preceding example, my opinion was that the market was more likely to go up (“I would be bullish”), but that it was preferable to short it (“I would be bearish”), because, in the event of its going down, it could go down a lot. Suddenly, the few traders in the room understood my opinion and started voicing similar opinions. And I was not forced to come back to the following discussion.

    Let us assume that the reader shared my opinion, that the market over the next week had a 70% probability of going up and 30% probability of going down. However, let us say that it would go up by 1% on average, while it could go down by an average of 10%. What would the reader do? Is the reader bullish or is he bearish?

    Accordingly, bullish or bearish are terms used by people who do not engage in practicing uncertainty, like the television commentators, or those who have no experience in handling risk. Alas, investors and businesses are not paid in probabilities, they are paid in dollars.

    Accordingly, it is not how likely an event is to happen that matters, it is how much is made when it happens that should be the consideration. How frequent the profit is irrelevant; it is the magnitude of the outcome that counts. It is a pure accounting fact that, aside from the commentators, very few people take home a check linked to how often they are right or wrong. What they get is a profit or loss.

    As to the commentators, their success is linked to how often they are right or wrong. This category includes the “chief strategists” of major investment banks the public can see on T.V., who are nothing better than entertainers. They are famous, seem reasoned in their speech, plow you with numbers, but, functionally, they are there to entertain – for their predictions to have any validity they would need a statistical testing framework. Their fame is not the result of some elaborate test but rather the result of their presentation skills.

    • baconbits9 says:

      Let us assume that the reader shared my opinion, that the market over the next week had a 70% probability of going up and 30% probability of going down. However, let us say that it would go up by 1% on average, while it could go down by an average of 10%. What would the reader do? Is the reader bullish or is he bearish?

      Hey, he is describing the stock market right now, how prescient!

  58. Gumpalonia says:

    What are the odds of this being just a another case of corruption? Everyone kinda trusted WHO and many gave it credibility because it was a global institution and did not share the same incentives for bias as a nation state would have. I mean, has WHO:s current leadership ever taken a position that would something else than the current official position of china? Stuff like refusing to use the word “Taiwan” in public or supporting opening of wet markets are a bit weird positions to take for a truly global institution.

    Other things that might have caused the media and most pundits to fail is all the time and energy that was invested on critizising Trump. What are the odds that you are open to accepting that a bigger threat is on the horizon if you believe that a destructive dictator has risen to power and needs to be stopped? What if they were unable to panic because they were already in a panic?

    Also one could question if the current rightist and leftist mental models of the world are really that open to taking new emerging threats seriously, since they are already convinced on what the real threats are, and thus focus more on “activism”.

    • Everyone kinda trusted WHO and many gave it credibility because it was a global institution and did not share the same incentives for bias as a nation state would have.

      Speak for yourself.

      I mean, has WHO:s current leadership ever taken a position that would something else than the current official position of china? Stuff like refusing to use the word “Taiwan” in public or supporting opening of wet markets are a bit weird positions to take for a truly global institution.

      Next we’ll learn the UN Human Rights Committee doesn’t care much for human rights…

    • AlesZiegler says:

      WHO declared a global emergency on 30 January.

      • Gumpalonia says:

        Also the position it took on travel restrictions seems really weird: Support restrictions from Wuhan to other parts of china but discourage restrictions from Wuhan to other countries. Also memes

        • keaswaran says:

          Did they discourage restrictions on travel from Hubei to other countries? My understanding is that they always recommended targeted travel bans – they only rejected blanket bans of countries when other areas with similar infection rates were still allowed.

    • Edward Scizorhands says:

      I’ll accept someone who was fooled by WHO and China for January (although our intelligence community wasn’t). But now you have to explain February and March.

    • Loriot says:

      What are the odds that you are open to accepting that a bigger threat is on the horizon if you believe that a destructive dictator has risen to power and needs to be stopped?

      If anything, anti-Trump bias made the left *more* insistent on sounding the alarm, since Trump positioned himself in the CV denialism camp.

      I recall most of the early media coverage being along the lines of “look at what happens when a clueless idiot is in charge during a crisis”.

  59. dark orchid says:

    I’m from the UK and the whole Brexit prediction thing makes me wonder if it wasn’t just one group of people (more or less Blue Tribe, although this being the UK they vote for the Red Party) going “I’m really confident that no-one I know would vote for Brexit”. I mean sure, Tetlock and the likes will have been more sophisticated than that, but in the half a year or so before the referendum, you could see something building up in the non-university-going social classes if you knew where to look. Especially if like me, you have friends and relatives in those classes.

    • Jon S says:

      Superforecaster here, though I stopped participating in Tetlock’s research prior to the Brexit vote. I think that played a much smaller role than you’re imagining. 29% would have been a pretty reasonable estimate if you just looked at polling if I recall correctly – maybe you’d have been slightly higher?

      • dark orchid says:

        I’m definitely not a superforecaster and I don’t read too much into getting this single one right, but I did switch to “I really don’t want it to, but it’s going to happen” about a week before the referendum. Most of my work colleagues thought I was insane.

        I’m sure there’s accurate polling out there, but all the polls in the kind of media that it’s ok to discuss at my workplace have been massively biased in a single direction for a while now – brexit vote, Trump vote, US midterms, several UK general elections. I don’t know if they’re being dishonest and writing what their audience wants them to hear, or whether there’s lots of people their polling just doesn’t reach (if I had to guess, I’d say the latter).

        • keaswaran says:

          What polls are you looking at? Polling I saw on Brexit, Trump, US midterms, etc., was always within a few points of the final result. It just so happened that for Brexit and Trump, the polling was always 2 points on one side and the end result was 2 points on the other (in Trump, the popular vote result was actually in agreement with the polls! just that errors of four or five points in several states put those states on the opposite side of 50-50 but canceled out overall).

          • Douglas Knight says:

            Brexit polling changed a lot. Over six months leading up to the election, a lot of people moved from undecided to leave.

            Normal polling is much simpler because it’s the same parties every cycle, but this was a new choice, so it’s much harder to interpret the undecideds.

        • Jon S says:

          Polling averages (ideally adjusted for the kinds of things that 538 adjust for) are pretty accurate, up to historical margins of errors of polling averages. News outlets like to cherry pick individual polls that they’re able to hype up.

          The last few election cycles, polling averages have been slightly more accurate than average (2012 they were off by more than normal I think, but nobody really pays attention when it doesn’t change the outcome).

  60. Clutzy says:

    I think you are downplaying another, highly likely scenario for the media’s response: Malice/Opportunism.

    The best explanation I would suggest for the media’s timeline was that they downplayed the risk when they thought that was politically good for them, and started hyping the risk when they thought that was politically good for them.

    I complained about this is early March. There was plenty of alt-Media that gave me enough information to convince me to personally buy rice and wipes. There was not enough legacy media confirmation (at the relevant times that people could have meaningfully done things on the nationwide level) that I had the confidence to short the market.

    The theory of politics is useful because it works on all levels. Conformity and signaling the ingroup are both known quantities. The political calculations are quite easy to deduce, etc. And this also incorporates most of the experts, because they belong to the same social class.

    • albatross11 says:

      ISTM that the more plausible model is that media types are mostly not very good sources of information on something like this, and that mainstream media types are just going to parrot the official line–and that’s mostly about as much as they can do, really. A J-school grad who barely passed her one mandatory stats class isn’t going to be critiquing epidemiological models.

      • Clutzy says:

        Does not compute with what we’ve seen. No public health experts aside from perhaps the imperial people have been as “11” as the media has been ever since it switch stances on a dime.

        • Matt M says:

          I think the issue is that the media only has “11.” When the balance of expert consensus was “this probably won’t be a big deal” their reporting was “This is absolutely nothing to worry about and anyone saying you should worry is a bad person that deserves derision.” And as soon as the balance of expert consensus switched to “this is worth implementing some sort of preventative measures” their reporting was “This requires us to shut down everything immediately and anyone who opposes that just wants your grandma to die!”

          • Clutzy says:

            They seem quite capable at not being at 11 when politics so demands. The Joe Biden story is being treated quite tamely.

  61. brownbat says:

    N.B. and a bit of a tangent — I’ll toss out a complicated example for Scott’s list of people who got this right. One of the best calibrated people on Twitter, judging by late January / early February tweets on coronaviruses in Asia, was @IAMIRONMAN7.

    Aside from prescient early warnings about the looming medical and economic crisis, he mostly seems to tweet… Bible verses and Avengers memes? That is not strictly mutually exclusive with well calibrated predictive capacity. But it’s not the sort of signalling typical of the other folks in Scott’s list.

    Maybe there’s something to be said for smart generalists who know how to aggregate and filter info from both specialists and particularly prescient outsiders. I guess that’s the point of a lot of Tetlock’s work.

  62. keaswaran says:

    I’m surprised to see people still repeating the meme that complaints about racism were misplaced. Isn’t it clear that our response to the virus has been severely damaged by its racist nature? The United States imposed two major travel bans – starting on Jan. 31, they banned travel to the United States by foreign nationals that had been in China over the previous 14 days, and starting on March 15, they banned travel to the United States by foreign nationals that had been in the Schengen area over the previous 14 days. Isn’t it now obvious that both of those bans were major failures prompted by racism? In both cases, they did nothing more than shut one half of the barn door after the horses were already gone. The outbreaks in Seattle and California were seeded by travel from China in January, and the outbreak in New York was seeded by travel from Europe in February, and I believe in both cases, US citizens were major parts of the seeding.

    As far as I can tell, all the travel bans did is cause mass crowding at airports on March 13-14, and this is precisely because of the racist nature of the bans (technically, it’s a xenophobic nature, since it discriminates officially by nationality rather than by race, but the distinction between racism and xenophobia isn’t very clear at all).

    In the case of the Europe ban in particular, things would have worked much better if they had either imposed an immediate ban on *all* travel (by US citizens as well) or else just let the market pull back on flights. But because of Trump’s long-stated goals of putting racist travel bans in place, they ended up doing the worst thing by seeding major international airports with long lines of probably-infected people.

    • The outbreaks in Seattle and California were seeded by travel from China in January, and the outbreak in New York was seeded by travel from Europe in February

      And those were the only outbreaks that were ever going to happen, right? It would have been impossible for any to be seeded by travel from China after January…

      • Anonymous Bosch says:

        At the end of the day a “travel ban” that only applies to one country’s nationals is useless. If anything a Chinese citizen visiting the US was less likely to be a vector than an American citizen visiting China, because the latter returns home after one-to-many proximity with millions of potential asymptomatic transmitters.

        Trump’s edict was purely political theater in any universe except one where American citizenship confers immunity to COVID. (To say nothing of the five-week gap between it and any other federal action.)

        • Trump’s edict was purely political theater in any universe except one where American citizenship confers immunity to COVID.

          There are greater costs to stranding an American citizen in China than preventing foreigners from entering the country. Your comment is the logical equivalent of saying “it makes no sense to shut down non-essential business X if essential business Y is not also shut down.”

          • Anonymous Bosch says:

            There are greater costs to stranding an American citizen in China than preventing foreigners from entering the country.

            That the cost of doing only half a travel ban was less than a full one is not compelling when it also renders a near-null benefit.

            Closing down travel to China on both sides may well have been too costly (since by then it was also already in Korea, Japan, various cruise ships, etc.) and I leave to others the exercise of constructing alternate histories favorable to their points. Mine is simply that if you don’t close it both ways, the benefit is reduced to the point where you may as well not do it at all.

          • baconbits9 says:

            That the cost of doing only half a travel ban was less than a full one is not compelling when it also renders a near-null benefit.

            You have offered zero support for this position, the East Coast is have a vastly harder time than the West Coast.

          • Anonymous Bosch says:

            You have offered zero support for this position,

            I offered what I considered to be fairly clear deductive reasoning for considering one-sided travel bans worthless from a disease perspective.

            You, by contrast, are offering inductive, post hoc reasoning based on one data point (the East Coast has it worse) with near-infinite inputs (among them the relative timelines of state/city lockdowns, the differing demographics of the coasts, etc.)

            To draw a parallel to another Golden Age Scottpost, this is a bit like replying to someone criticizing the minimum wage with a post hoc argument about unemployment not falling after place X raised it (when unemployment, too, is an outcome with near-infinite inputs).

          • I considered to be fairly clear deductive reasoning

            Yeah, not seeing that.

          • baconbits9 says:

            I offered what I considered to be fairly clear deductive reasoning for considering one-sided travel bans worthless from a disease perspective.

            You can repeat it as often as you want, but the travel ban effected travel to the West Coast more severely and the outbreak has been vastly worse on the East Coast. Your ‘deductive’ logic would imply that a single infected person getting into the US is equivalent to zero restrictions on travel into the US indefinitely.

          • That the cost of doing only half a travel ban was less than a full one is not compelling when it also renders a near-null benefit.

            You don’t seem to have offered any argument in support of that claim. If half a travel ban cuts in half the number of people coming from an infected region to the U.S. it should cut roughly in half the number of subsequent infections coming from people coming from that region. The costs, both political and human, of preventing Americans from returning to the U.S. were substantially higher than the cost of preventing foreigners from visiting the U.S., so it made more sense to keep out foreigners than to keep out returning Americans.

          • HeelBearCub says:

            If you can’t reliably screen, quarantine and contact trace people coming in, it makes very little difference whether you let in N or 1/2 N. You’ll reliably get local infection and community spread.

            It’s like putting leaches on someone with sepsis on the premise that you are removing some of the bacteria in the blood.

          • Edward Scizorhands says:

            I don’t know why someone would think a travel restriction applied to Americans has to “strand Americans in China.” You still let Americans back in, but quarantine any that have passports indicating a visit to China in the past 14 days.

            In fact, once you announce this policy, you immediately will get a big drop-off in Americans doing any casual travel to China, because they don’t want to get caught in that.

          • albatross11 says:

            My sense is that by the time anyone was seriously talking about banning foreigners coming from China, Europe, etc., it was mostly too late.

            If you were going to do this, Edward Scissorhands has the right idea: you’d need to add some kind of quarantine.

          • Edward Scizorhands says:

            And there is good reason to treat your own citizens better besides plain chauvinism (and chauvinism is fine). You know where your own citizen lives. You can check on him easier with a cop visiting his house to verify a stay-at-home deal. You can enforce punishment against him easier in case of violation. You can count on him, on average, to place a higher value on American safety than a foreigner. It’s more conceivable that an American would want to stay at home for 2 weeks than that a foreign visitor would want to sit in a hotel for 2 weeks.

            I don’t know exactly what the line should have been for each of (a) foreigners coming to the US and (b) American returning home. I can buy them being significantly different. But no enforcement at all on Americans was the wrong call.

          • If you can’t reliably screen, quarantine and contact trace people coming in, it makes very little difference whether you let in N or 1/2 N. You’ll reliably get local infection and community spread.

            That’s an argument against any travel ban. Whether it is correct depends on whether the number of contagious people coming is or is not substantial relative to the number already here.

            But I was responding to the claim that distinguishing between foreigners and returning Americans was racist — that “a ‘travel ban’ that only applies to one country’s nationals is useless” because keeping out half the people had “a near-null benefit.” (quotes from two posts by one poster).

          • Anonymous Bosch says:

            But I was responding to the claim that distinguishing between foreigners and returning Americans was racist — that “a ‘travel ban’ that only applies to one country’s nationals is useless” because keeping out half the people had “a near-null benefit.” (quotes from two posts by one poster).

            The quotes are mine, but the term “racist” before the em-dash isn’t, it’s the guy two posts up. I think Trump’s (literally) half-assed travel ban had more to do with politics than race. He wanted the appearance of doing something, but only things that wouldn’t panic the stock market or possibly threaten his China trade deal, which excluded anything that could’ve made an actual difference.

          • HeelBearCub says:

            @David Friedman:

            That’s an argument against any travel ban.

            No, it is most certainly not. It’s an argument against a half travel ban, something that accomplishes nothing. If I said that closing the bottom half of a Dutch door was no use in keeping flies out, you can’t say that this is an argument against closing the entire door.

            There are times and places when a full travel ban may be exactly what you need to do.

            Unless you think that quarantines are ineffective as a rule, it’s very clear that travel bans, in a general sense, may be called for. When you don’t have some other way to stop the spread of an infectious disease, you sometimes have to impose a barrier of time between when you have been in a place where you are likely to have been in contact with infection and the general populace.

          • That’s an argument against any travel ban.

            No, it is most certainly not. It’s an argument against a half travel ban, something that accomplishes nothing.

            So your claim is that keeping ten thousand people, a hundred of whom are contagious, from arriving in (say) New York, substantially reduces the problem, but keeping five thousand, fifty of whom are contagious, doesn’t?

            Back before there is significant herd immunity, wouldn’t you expect the number of infected people at time t+20 to be roughly proportional to the number at time t? If so, if X is the number of people in NY already contagious, then the number infected at a later time, prior to the point at which a large fraction of the population is either infected or immune, will be AX if you let nobody in, A(X+50) if you let in five thousand, A(X+100) if you let in ten thousand.

            So how can you argue that keeping all of them out accomplishes something, keeping half out accomplishes nothing?

          • Anonymous Bosch says:

            So how can you argue that keeping all of them out accomplishes something, keeping half out accomplishes nothing?

            By using the words in casual conversation where they are shorthand for “this thing would matter several orders of magnitude more than this other thing” rather than literally all or nothing.

            In this case it’s the difference between a 50% effective NPI (the NPI being a travel ban) and a (100-X)% effective NPI, where X is your estimation of the rate of people who would successfully evade a travel ban applying to both countrys’ citizens. If you’d like to continue jousting you can propose an unrealistically high X value but this seems like an awful lot of effort at not getting the point.

          • HeelBearCub says:

            @David Friedman:
            I know you understand math, and exponential growth also.

            Tell me, oh exceedingly wise professor, how many extra days have you bought yourself by halving the number of entrants if the number of infections doubles every 3 days?

            It’s not as if we haven’t been going over this lesson ad infinitum recently.

            Whereas, if there is no one currently infected in the community, the current number of infections is zero. How many doubling does that take to get to 100?

          • Edward Scizorhands says:

            I was going to argue that the American going to China is a bigger risk (to the US) than a Chinese visiting the US. But then I wasn’t sure. The Seattle case, which happened before the travel ban, came from an American citizen. Italy happened because a Germany citizen took the virus from China and then it into Italy. That’s, like, natives being 50% more dangerous.

            This isn’t exactly like screening radiation, where a bunch of particles are flying at you stochastically and blocking 50% reduces your risk 50%. Some particles are much more dangerous. Other particles will react based on the blocking. If there is business between Chinese Company C and American Company A, and Chinese Company C is unknowingly a hot spot, they’ll send a Chinese citizen instead of an American citizen.

          • @ HBC:

            If you are assuming that people coming in are the only source of infection, then stopping all of them is indeed much better than stopping anything short of all of them, but your doubling argument is unnecessary. Is that your assumption?

            If you assume that there are a substantial number of contagious people already here, and if you think the only relevant question is how many days difference one policy or the other makes, then keeping all of them out has only a minor effect for precisely the reason you have just argued. If, for instance, X in my equations is 50, then no immigration gives you 50 contagious people, restrictions only on foreigners gives you 100, and no restrictions gives you 150. By your criterion those are all about equally bad, since the best is only a factor of 3 better than the worst.

            On the other hand if your criterion is not how many days extra you have but how many people are at any instant infected, then my calculation goes through.

          • HeelBearCub says:

            @David Friedman:
            No, your argument is specious nonsense.

            The fact that diseases can grow quickly and exponentially is precisely the problem. If disease growth is linear to those being admitted, well, I don’t know that you have an infectious disease, but you definitely don’t have a pandemic.

            If you can’t reliably screen and quarantine, but do not yet have known community spread, then imposing a complete travel restriction might be warranted.

            If you have in place the means to reliably quarantine those who are ill from those who are not, then allowing limited travel, to keep the load on your procedures under its maximum, does something. If you already have unchecked community spread, are not screening and quarantining, the differences between N cases let in and 1/2N cases let in is the doubling time of the disease.

            Your criteria of measuring “how bad it is” by the number of known cases is exactly the mistake Trump made when he said “we have 15 cases that will soon be down to zero”.

          • Scoop says:

            “There are greater costs to stranding an American citizen in China than preventing foreigners from entering the country.”

            Compared to the cost of allowing the entry of a disease that shuts the entire nation down, the cost of making citizens who wish to return undergo quarantine is not that large.

            The proper response — in hindsight, I certainly wasn’t advocating it at the time — was quarantine for anyone entering the US (from anywhere, not just China) on the day WHO confirmed human to human transmission. And then relax the quarantine restrictions when places proved themselves disease-free (or you got a foolproof test for the disease).

          • If disease growth is linear to those being admitted, well, I don’t know that you have an infectious disease, but you definitely don’t have a pandemic.

            You are misreading my claim.

            Disease numbers are linear in number initially infected, as long as most people are neither infected nor immune. That’s linear as a function of number infected, not as a function of time. And it is just as true if growth over time is exponential.

            Assume growth is exponential, most people are neither infected nor immune. My claim is that:

            Number infected(t) = (Number infected (to))(B^(t-to))

            That’s linear in number earlier infected, exponential in time. It isn’t precisely true because the number who can get infected is falling over time, but it’s close to true as long as most people are neither infected nor immune.

          • viVI_IViv says:

            @DavidFriedman

            The goal of the government should have been to contain the outbreak, or failing to do so, delay the peak long enough to buy time deploy mitigation measures.

            A half-hearted travel ban that only halves the number of initial carriers does nothing to achieve containment and only delays the peak by 3 days or so. Hence it’s close to useless.

            A ban that is 98.5% effective, on the other hand, delays the peak by 36 days, and more importantly allows the government to attempt effective containment by contact tracing and screening.

          • HeelBearCub says:

            @David Friedman:

            Disease numbers are linear in number initially infected, as long as most people are neither infected nor immune.

            Say what?

            Disease growth numbers become asymptotic as you get to herd immunity rates. That’s the opposite of what you are saying. Diseases can grow fastest when the number of infected/immune is low.

            Of course any exponential curve looks linear in the early phase,
            if you also graph later growth. But that’s an illusion.

            The most you can say is that, if you are early in the infection, such that the number of cases you would let in without a ban swamps the number currently infected, then the effect of a complete ban may save many doublings. But letting in 1/2N still only saves one doubling, and 1/2N still swamps the number of currently infected.

          • Edward Scizorhands says:

            While delaying a peak for 38 days is obviously what we want, delaying the peak for 3 days is not completely useless. I remember all the graphs showing that starting social distancing even 1 day earlier can make noticeable changes to the curve.

          • Scoop says:

            @vivi-iviv

            A half-hearted travel ban that only halves the number of initial carriers does nothing to achieve containment and only delays the peak by 3 days or so. Hence it’s close to useless.

            Isn’t three days earlier action what separates NY from CA? Not perfect but definitely significant.

          • @HBC:

            As best I can tell, you have still not figured out what I was saying, hence are making arguments that, true or false, have nothing to do with mine. Having explained it once, I don’t see much point to doing so again.

            Disease growth numbers become asymptotic as you get to herd immunity rates. That’s the opposite of what you are saying.

            On the contrary. That is why I wrote “as long as most people are neither infected nor immune”, explicitly restricting my claim to a point at which you were not approaching herd immunity.

          • Swami says:

            “Isn’t three days earlier action what separates NY from CA?”

            I would say three days is pretty minor compared to disgusting subways, and congested densities.

          • HeelBearCub says:

            @David Friedman:

            Disease numbers are linear in number initially infected, as long as most people are neither infected nor immune.

            [I am] explicitly restricting my claim to a point at which you were not approaching herd immunity.

            You claim infection rates ARE LINEAR when people ARE NOT approaching herd immunity.

            Whereas I am saying the only time in an uncontrolled infection progression we start to see linear progression is AS WE APPROACH herd immunity.

            We are saying the opposite thing. And I have no earthly idea why you are trying to say a geometric progression is the same as linear one.

          • The Nybbler says:

            @HeelBearCub

            If a disease is progressing exponentially, the number of people with the disease at any given time is linear in the number of initial infections. So if 4 people are initially infected, you get a two-doubling-period head start on the exponential curve compared to one person being infected.

          • HeelBearCub says:

            @TheNybbler:
            I don’t think that’s what his point is. I already addressed that. The absolute worst case is that the number currently infected is much smaller than N. In that case so is 1/2N. In three days we are at ~N anyway. In cases where N is smaller than the number infected, it matters even less.

            The number of infections being linear as a function of previous infections is … sorta pointless. You need smaller and smaller slices of time to observe that linear growth. Trying to remove time from the equation doesn’t make sense.

            There are M infections. There will be M+x infections. If you add N infections there will be M+x+N infections. OK that’s linear.

            But who cares?

            That linear growth is completely swamped by the growth over time. Adding a fixed number of infections at some point has essentially no effect compared to what happens over time.

          • You claim infection rates ARE LINEAR when people ARE NOT approaching herd immunity.

            You still are misreading me.

            “Infections are linear” means they are a linear function of something. You keep assuming I am saying they are a linear function of time, when I have, I thought, made it clear that is not what I am saying.

          • HeelBearCub says:

            @David Friedman:
            As I said in the reply to Nybbler, infections being linear to previous infections is essentially pointless.

            That would be why adding N or 1/2N infections doesn’t matter. The geometric rate of progression is essentially unchanged over time, regardless. When you have uncontrolled infections, time is what matters. Until you put in place measures to control the infection, everything else is noise.

            So I still have no idea what point you are trying to make.

        • Garrett says:

          One counterpoint is that it’s generally considered a violation of international law and various treaties and whatnot to prevent a US citizen (or Person or National or whatever, but you get the idea) from entering the US. The President has vast powers, recently confirmed by SCOTUS) to bar foreigners from entering the country.

          So the argument would be that Trump should have broken the law to bar US citizens from entering the country, which is also insane. Alternatively, he should have stopped all international flights, which is more likely to have been legal.

          • Anonymous Bosch says:

            So the argument would be that Trump should have broken the law to bar US citizens from entering the country, which is also insane. Alternatively, he should have stopped all international flights, which is more likely to have been legal.

            CBP can’t prevent US citizens from entering the US, but it can detain them on entry on a “reasonableness” standard. He wouldn’t have needed to ban anyone, just a brief quarantine like what was imposed on the returning cruise ship passengers in February. According to multiple people I talked to, though, CBP was only even bothering to give extra warnings when someone was coming directly from Hubei and even then it was just a vague flyer.

            This is, of course, grossly specific Monday-morning quarterbacking. Given the scant reliable information about the virus in late January, I’m sure there would’ve been a great hue and cry from the media if he had taken such a drastic measure, although he’d be coming out roses on the other end now. (I also think Trump’s established willingness to assert wholly illegal executive orders and narrow them into a Roberts-shaped hole as the appellate cases progress makes this something of an isolated plea for legal modesty; by the time anything was decided the quarantine period would be over and it would be moot.)

            But ultimately I don’t blame Trump that much for failing to do even the stuff entirely within his power, like pausing international flights, in January. The shit he was talking in February and March while failing to update until the stock market cried uncle is much more concerning.

          • HeelBearCub says:

            @Anonymous Bosch:
            It’s worth also noting that Trump didn’t even take any material action with the ban he enacted.

            The timeline of that day is also important when it comes to Trump’s claim that this was a bold step. The travel restrictions actually came after the three major carriers who have service to China — American Airlines, Delta and United — had already announced earlier in the day that they would halt that service. As USA Today noted at the time, the move by the airlines “all but rendered White House action moot.” The White House had floated potential travel restrictions earlier in the week, but this was something the airlines jumped on first.

            Nor did Democrats take particular umbrage at the actions taken:

            And going back to the claims about opposition to the move, some Democrats actually seemed to praise it. Reps. Nita M. Lowey (D-N.Y.) and Rosa L. DeLauro (D-Conn.) on Feb. 4 commended Azar for the administration’s initial attempts to contain the virus, including mentioning quarantines and the “significant travel restrictions.”

            “We strongly support your recent decision to declare a public health emergency in response to the ongoing outbreak of the 2019 novel Coronavirus,” they wrote.

          • Edward Scizorhands says:

            OMG. It’s like the reverse of authorizing governors to re-open if they want.

    • Randy M says:

      Isn’t it now obvious that both of those bans were major failures prompted by racism?

      No.
      It reads, based on your argument, more like a government having a higher bar for keeping out foreigners than keeping out its own citizens. Given that the latter have an actual claim on the government’s protection, this seems reasonable.

      • keaswaran says:

        Given the exponential nature of this sort of epidemic, a ban on a large fraction of visitors does very little, compared to a ban on all visitors. The costs of bans are linear in the number of people banned. The benefits are highly non-linear. If you’ve decided that banning the Americans is too high a cost to bear for that extreme benefit of cutting off all travel, then the only way that banning foreigners could be worth the tiny benefit of cutting off half of travel is if the rights of foreigners count only an extremely tiny amount compared to the rights of Americans. But that seems to go against the Constitution, which only distinguishes between the rights of citizens and non-citizens in a few specific places.

        • Randy M says:

          Your calculation differing from the administration doesn’t indicate racism to me, especially given the regions banned.
          Yes, it would have been more reasonable to quarantine and test US travelers upon their return. Government incompetence is no surprise, particularly now. Giving it an increasingly ill-defined boo-word, though, isn’t terribly helpful.

          To be honest, I wasn’t considering legal residents in your complaint. A legal resident citizen of another nation kept out of their home would be understandably upset by this and possibly have some sort of case.

          Foreigners traveling for business or tourism do have basically no claim on the country, imo, and complaints about being kept out by xenophobia aren’t going to get a lot of sympathy when we’re nearly confined to our homes.

        • Edward Scizorhands says:

          if the rights of foreigners count only an extremely tiny amount compared to the rights of Americans

          America should absolutely be prioritizing the rights of people who live here.

          There are also many reasons we can be easier on people who live here.

          This doesn’t excuse doing no enforcement on people who live here. The absolute bare minimum is giving people a card that says “if you feel sick in a few days, call this number.”

    • John Schilling says:

      Isn’t it clear that our response to the virus has been severely damaged by its racist nature?

      How do you get “severely damaged”? The travel bans may have been too late to do much good, and maybe you can even convince me that they were “racist”, but strictly on the pandemic-mitigation front they don’t seem to have done any harm and I don’t think they displaced any plausible alternatives that would have done more good.

      • keaswaran says:

        > strictly on the pandemic-mitigation front they don’t seem to have done any harm

        Look at the photos from O’Hare Airport on Friday, March 13, and see if this sort of hastily imposed travel ban did any harm on the pandemic-mitigation front.

        https://www.bbc.com/news/world-us-canada-51895246

        • albatross11 says:

          Yeah, this raises another issue that’s important to factor into your predictions about the world, and your policy prescriptions: much of the time, whatever policy is implemented will be implemented in an inept, hamfisted way.

          A travel ban from some country can be implemented in a sensible way–give everyone as much notice as possible, allow some exceptions that are going to be necessary and make sure there are enough humans with decisionmaking authority to grant the exceptions, etc. And similarly, mandatory screening for symptoms or virus testing or quarantine could be implemented in a sensible way. But that requires competent people implementing the plans, with enough time to plan and prepare and enough resources to do them well.

          One aside, though–going through passport control into the US and many other countries is usually stupidly crowded and slow. There’s not any inherent reason for this–they could probably ask whatever questions they needed to on the plane or as you got off the plane, they could certainly hire enough clerks to scan everyone’s passport, etc. And yet, just about every time I come into the US from overseas, I’m stuck in a long dense crowded line that’s pretty-much optimal for spreading disease. And very often, that’s also true coming into other countries. Since they weren’t set up to avoid long delays and crowding and lines when they were just scanning a passport and asking you a couple perfunctory questions, they surely couldn’t manage to avoid them when they needed to add a fever scan and a few questions about your health and whether you’ve been around any sick people lately.

          • Edward Scizorhands says:

            give everyone as much notice as possible

            Mostly agree, except for this. You never say “we will make restrictions tighter in a few days,” especially in a confusing way. Because it makes people flee for borders right away. The whole goal is to slow down travel, not speed it up.

            So you announce new screenings on people coming in, and you implement it within an hour or two of that. This requires competence, and not your son-in-law coming up with the plan earlier that day.

          • One aside, though–going through passport control into the US and many other countries is usually stupidly crowded and slow.

            But not all countries. Last time I arrived in the U.K., I concluded that the U.S. should subcontract passport control to them. It took very nearly no time at all.

    • Controls Freak says:

      imposed an immediate ban on *all* travel (by US citizens as well)

      That’s gonna be illegal. Like, Constitutionally illegal. If you do that, you’re going to have a lot more people than just the anti-racists petitioning the courts to say that it’s illegal, and the courts are going to rule in their favor.

      They basically took your plan, carved off the parts that would be illegal to do, and did what was left. It seems like if we follow your plan in a legal way, we get to racism. Sorry, but you’re the racist now, dawg.

      • Scoop says:

        Quarantines are not illegal. Anyone can come, but they have to spend three weeks in quarantine.

        Also, our entire response has been unconstitutional, and no one has fought it. There is no way that governors (or presidents) have the right to ban large groups, especially church services. The constitution guarantees the right to assembly, not the right to assembly unless some governor has declared an emergency.

        • HeelBearCub says:

          I’m pretty sure the right of the government to restrict personal freedoms in the case of a genuine threat to the health of the public is fairly well established in constitutional law.

          But if you’ve got some relevant precedent, I’d like to know.

          • Anonymous Bosch says:

            There’s very little that’s truly on point for this (CFNV v. Louisiana and Jacobson v. Massachusetts come close) but from a pragmatic standpoint of “Constitutional is whatever John Roberts says it is” rather than my own view if I were on SCOTUS, I just don’t see where you get 5 votes to overrule a state lockdown provided it applies equally to religious and secular public gatherings. Things like penning up protestors in “free speech zones” miles from an event have already had SCOTUS’ blessing for a while and that’s without a state of emergency.

          • Scoop says:

            I’d love to see a theoretical legal argument justifying a governor’s ability to deny liberties that are guaranteed without restriction by the first amendment.

            I fully understand the practical argument that things might go better for the state as a whole if people are temporarily denied their liberties, but that’s not a legal argument.

            I’m not a lawyer, let alone a constitutional scholar, but from my layman’s view it seems like a challenge to pretty much all the stay-at-home orders would take about a minute to win in any court that took the constitution even remotely seriously.

          • The Nybbler says:

            I’d love to see a theoretical legal argument justifying a governor’s ability to deny liberties that are guaranteed without restriction by the first amendment.

            And how will you make an argument, if the courts are closed, Mr. Anderson? Get arrested for a quarantine violation, you just disappear into the system at least for the duration.

          • Anonymous Bosch says:

            I could make one, but without a background in Constitutional law I’d just be citing a bunch of meaningless case names at you. It is a feature of Anglo common law that the literal words are rarely informative because they sit atop a mountain of sometimes conflicting precedent, and the sort of radical textualism you describe has never really been the order of the day in America.

            For example, “incorporating” the Bill of Rights to apply against the states as well as the feds is a (relatively speaking) recent development; it’s still not a universal rule although most of the important ones have precedent by now. The first time this happened was in Gitlow v. New York and you’ll note that while the court held that the First Amendment did theoretically apply, they also held that a law criminalizing anarcho-communist literature didn’t qualify as an infringement.

            In your specific case lack of a qualifier in the assembly clause hasn’t stopped the courts from blessing TPM (time/place/manner) restrictions which is why a protest of sufficient size can be broken up if you don’t seek approval from the sheriff or city council. (If they stonewall you and refuse any time or place, then that would be a colorable 1A claim, but again, this is the degree of restriction they allow under normal circumstances, and while the case law on “emergencies” is pretty thin, it’s also pretty generous to the states as far as their police powers.)

          • Matt M says:

            in any court that took the constitution even remotely seriously.

            I’m afraid I have some bad news…

          • Scoop says:

            @AB

            It is a feature of Anglo common law that the literal words are rarely informative because they sit atop a mountain of sometimes conflicting precedent, and the sort of radical textualism you describe has never really been the order of the day in America.

            That’s in cases when the law, or the constitution, is not crystal clear. Most of it is followed to the letter. The constitution says we should elect representatives every two years, the president every four years and senators every six. And we do. And you’d probably get disbarred for suggesting in court that we shouldn’t get hung up with radical textualism when the president would rather put off the election for a few years.

            It’s only when things are murky — “due process of the law,” “cruel and unusual punishment” — that I’ve ever heard anyone make any serious argument against just doing what the law says.

            I guess you could make the argument that the first amendment only talks about Congress not making any laws that yada, yada, yada, but, as you note, states generally have to abide by the first amendment these days. And it’s pretty unambiguous about people having a right to gather.

            As for the stuff about denying permits for rallies, I’ve always thought the justification for that hinges not on stopping the assembly but on the rally being on public land (and thus denying others the right to use it at the same time.) I have never heard of any court upholding orders against people assembling on their own land, for, say a BBQ.

        • Controls Freak says:

          keaswaran asked for a broad ban on US citizen entry into the US. That would be illegal.

          • Scoop says:

            I feel like most people who say “no entry” really mean “no entry, unless you undergo a full quarantine that guarantees you’re not carrying the virus.”

            But I suppose I could be wrong.

          • albatross11 says:

            Yeah, I’m pretty sure a law or directive saying “US citizens currently living in France may not return home” would not survive a challenge, whereas “US citizens currently living in France must undergo a 14-day quarantine upon returning home” would be upheld.

    • DinoNerd says:

      *sigh* Xenophobia, maybe. Idiocy and security theater – probably. But if the motivation had been racism, in what universe are Europeans a different race from Australians, Canadians etc.; Thai et al. a different race from Chinese; and Africans (sub-Saharan or Middle Eastern) not targets of a “racist” measure.

      • keaswaran says:

        Quite fair. It’s really xenophobia rather than racism. But those two mentalities often feed off of one another in slippery ways. And my main point is that too many people seem to see the specter of “people complaining about racism” as something worth more comment on than xenophobia itself, which is an actual problem.

    • albatross11 says:

      I’ll admit, I don’t see how calling the travel bans racist adds any light or insight at all to the situation.

      a. It’s pretty obviously not true[1].

      b. It mainly serves to load up the travel bans with negative emotional associations.

      c. It secondarily serves as an attractor for some derail of the conversation into whether Trump is really racist or merely evil.

      Now, if you want to argue that the travel bans were ineffective, yeah, I suspect you’re right. But you won’t get any smarter on the factual question of how effective they were by amping up the emotional or tribal appeals.

      [1] The ban on flights from overwhelmingly white parts of Europe is supposed to be an instance of Trump’s racism? And previously the ban on flights from China that didn’t include any other Asian countries? This doesn’t make any sense.

      • Anonymous Bosch says:

        For the record and to avoid to be that guy who only punches in one direction, I agree “Trump’s racism hurt our response” isn’t a terribly cogent point despite (1) agreeing that Trump is probably more racist than a replacement-level President (2) agreeing he’s shit the bed on our coofid response more than a replacement-level President

        (Nate Silver really should get around to constructing VORP-but-the-P-means-President)

      • HeelBearCub says:

        In what way were the travel bans, as implemented, effective at preventing uncontrolled/untraced transmission from either Europe or China? Thousands of people travelled, with no screening at all, by dint of being American.

        If you grant that they were not effective, then why would they be implemented in this fashion? Sure, there is the basic impulse to be seen as “doing something”, but why would this be seen as “doing something”?

        There is a long, long, long history, everywhere, of viewing foreigners as the source of contagion. These bans did nothing to meaningfully stop broad scale transmission of the virus. All they did was keep foreigners from coming in to the country.

        • Scoop says:

          “These bans did nothing to meaningfully stop broad scale transmission of the virus. All they did was keep foreigners from coming in to the country.”

          You cannot assert that as any sort of fact. Their failure to be fully effective in no way means they were not partially effective.

          If they prevented a significant number of infected individuals from entering the US — and they almost certainly did — they reduced the number of starting points for the disease and, thus, probably reduced the total number of cases and deaths.

          Should we have also forced US citizens to undergo quarantine? Obviously, but the travel bans probably were not useless.

          • keaswaran says:

            In the early phases, a contagion grows exponentially once it is at a non-zero value. Allowing travel adds a linear term as well as the exponential one. Very early in the process, that linear term is significant. But once you’re more than a few days into the doubling, the linear term is basically irrelevant compared to the exponential one, even if it is technically non-zero.

          • albatross11 says:

            If indeed there were very few cases in the US and many cases in {China, S Korea, Italy, etc.}, then imposing any of {a travel ban, mandatory RNA testing on entry, mandatory fever and symptom check on entry, mandatory quarantine on entry} could help by not introducing a bunch of new cases into the country. However, my impression is that the European travel ban in particular was probably quite a bit too late–we’d had community spread going on in several US regions for weeks by then. And the China ban was probably late (it had spread to other countries by then) and also was leaky (you could get here from China if you flew via another country).

          • Swami says:

            That was a great response, keaswaran. I never thought of it like that.

          • EchoChaos says:

            That also depends on the R0.

            With an R0 that is quite low, the linear part could give you days or even weeks of doubling, but with a high R0, it will be completely swamped.

            Nobody really knows the genuine R0 of COVID-19 (lots of guesses out there), but it seems to be at the level where 50-100 new cases early on really did advance us several doublings.

            Which is why the largest transit hub to Europe is also the largest nexus for the USA.

          • albatross11 says:

            I think the way we implemented nationwide shutdowns and bent the hell out of the exponential curve makes a case that slowing the expansion of cases down, even just by a few days, can make a real difference.

            If we were just looking at an exponential and our actions could do nothing to change it, then yeah, travel bans couldn’t help much–the ultimate result would be the same. But if we were suddenly moving as fast as we could to slow the spread down and try to drop R_0 below 1 or at least below 2, in hopes of avoiding having a Northern Italy style meltdown in the hospitals in NYC, then an expensive action that stopped a significant number of new cases being introduced made sense.

          • Clutzy says:

            In the early phases, a contagion grows exponentially once it is at a non-zero value. Allowing travel adds a linear term as well as the exponential one. Very early in the process, that linear term is significant. But once you’re more than a few days into the doubling, the linear term is basically irrelevant compared to the exponential one, even if it is technically non-zero.

            Sort of. The problem with travel is that travelers very often travel in ways that cause lots of spreading. They are on a plane. Planes are C19’s favorite. Then in a big airport like LaGuardia, LAX, Ohare you have to take a mini subway to get out of the international terminal. Sometimes then you take the subway/light rail away from the airport. Then you, a traveler are probably going to want to go to either an attraction like a museum, or an office building, because why else are they a traveler?

            So, sure, an infected traveler is only a linear addition, but he’s also likely to be an addition that spreads more than your average infected citizen. Plus, because of the kinds of activities they engage in, even uninfected travelers are going to act as force multipliers for the virus.

          • albatross11 says:

            I would feel a lot better if we’d at least implemented fever checks and a quick set of questions like “have you been ill recently” and “have you been around sick people recently,” for both Americans and foreigners, with the wrong answers leading to a polite insistence on quarantine in a nice paid-for hotel room until we can manage to get someone here to test you for the virus.

          • baconbits9 says:

            In the early phases, a contagion grows exponentially once it is at a non-zero value. Allowing travel adds a linear term as well as the exponential one. Very early in the process, that linear term is significant. But once you’re more than a few days into the doubling, the linear term is basically irrelevant compared to the exponential one, even if it is technically non-zero.

            This is a flawed way of approaching the issue, it works if the only possible way of slowing the contagion down is travel bans and its either full travel ban or no ban and it also ignores how people move.

        • webnaut says:

          @HBC did you see that clip – the American woman in Japan during the cruise ship debacle who admitted lying on television about being tested negative for Corona and the State department stuck her on a commercial flight back to the US. The infected Americans were supposed to be sealed off from the other passengers.

          My jaw hit the floor when I watched that. https://www.youtube.com/watch?v=9J6FdOmpA8Y

          It was an extraordinary admission because it proved on video the State department didn’t confirm the list of repatriated from the cruise ship and the list of tested from the Japanese Health Ministry were related. Somebody should have resigned over that.

          • Edward Scizorhands says:

            I’m not surprised that people, when confronted with an uncaring bureaucracy, would rather lie than do the right thing. Admit something and the low-level drone puts you in a windowless room for 8 hours, because fuck you that’s why.

            I’ve heard many people say that their “screening” was just being asked if they had been to a hotspot, and if they say “no” they continue with their lives so they say “no.”

            If you want the public’s cooperation, you need the public to trust you.

          • Matt M says:

            Yeah, I’m pretty much at the point where I no longer see “lying to the government” as a moral harm, ever, in any case.

          • webnaut says:

            @Edward Scizorhands, Matt M

            I would hope people would be more civic minded. I went to a local hospital and donated my spare masks – it didn’t make my life easier to do that and I could have made money online selling them. I feel a certain amount of contempt for the lady and think that reasonable.

            My real ire is directed at the State Department managing the operation – they should have double checked the lists from the Japanese Health Ministry and compared them to the repatriated instead of taking people at their word – that was just unintelligent. The infected – I think they were sealed off with plastic on the aircraft – but both infected and uninfected went into quarantine on an army base when the plane landed and if I remember the other passengers were commercial so they could have easily been infected by one of the liars sitting next to them.

      • viVI_IViv says:

        [1] The ban on flights from overwhelmingly white parts of Europe is supposed to be an instance of Trump’s racism? And previously the ban on flights from China that didn’t include any other Asian countries? This doesn’t make any sense.

        It’s like the “Muslim ban” which didn’t actually affect most of Muslims in the world. Orange Man Bad.

    • webnaut says:

      A racist would be happy to claim we should have been more racist sooner and more consistently?

      I agree with how nonsense it was for people at airports to get clumped at the worst possible time – I feel like that should have been anticipated by policy makers.

  63. adamishere says:

    Was the early coronavirus writing even about coronavirus, though?

    To me, very few of the articles listed were really about coronavirus. I know that’s not an intuitive take when the articles are headlined things like “there’s not going to be a coronavirus pandemic”, because it really feels like that’s what it’s about. But, the media (A) aren’t epidemiologists (B) aren’t probabilistic (C) probably aren’t great at learning a logical system and extrapolating out the consequences. And I think the response to that is to actually write about Something Else, where coronavirus becomes a metaphor / allegory for Something Else.

    And Something Else is usually about your position relative to others.

    So, “Is coronavirus going to be a pandemic” =

    “Are the [other] media prone to unduly panic?”
    “Am I on Team Adult or Team Panic?”
    “Is my character/brand/tone cool and sceptical or is it alarmist?”
    “How do I situate myself relative to the political poles and their stake on the politically important question of Wether It’s A Pandemic”

    ^ that’s the kind of question that Vox et al. were writing about.

    I think it’s a structural issue where passing factual information makes less financial sense than editorialising; editorialising is a political process; and people aren’t interested or don’t know much about the fact, which is a subject of new and sudden importance, so they don’t have the staff for it.

    • albatross11 says:

      Yeah, a huge amount of low-quality internet writing is me writing about my personal hobby horse or my favorite bit of meal-ticket clickbait, but somehow relating it to whatever else is in the news today. Ostensibly, I’m writing an article about X, but really, I’m writing it about my hobby horse, and X is just an excuse to talk about it.

      So maybe I’m writing about the latest YA book, or about the recent Ghostbusters movie, or Trump’s latest offensive tweet[1], or this weird SARS-like thing that’s arisen in China, or whatever. But really, I’m not writing about any of those things, so I probably haven’t spent much time thinking deeply or digging into what I say about them. The latest clickbait article about the unbearable whiteness of hockey isn’t about hockey and isn’t going to be a good source of information about hockey, in the same way that a talk radio rant about how the boy scouts have been co-opted by the gay agenda isn’t actually a good source of information about the boy scouts. Quite often, I so much want to make my point about my hobby horse that I’ll get details wrong, omit important things, or even make things up to make a better argument. (You can see this in news stories all the time when you pay attention.) And people who read that and think they’re being informed about hockey or the boy scouts or whatever are mainly just absorbing misinformation and lies in a format that they find emotionally satisfying, which probably makes the information go down more easily.

      [1] This has replaced “unrest in the middle east” as the world’s most reliable evergreen topic.

  64. AllAmericanBreakfast says:

    I’ve been developing an alarm bell for the next pandemic. It’s a model to assess early evidence for whether it’s pointing to an economy-crashing pandemic or something serious but not world-shaking.

    I don’t know why Scott thinks the market didn’t crash til mid-March. Here’s a selection of articles from the NY Times explaining the stock market from the week of Feb. 20th to Feb. 27th, 2020.

    Feb 20th: Why the Stock Market Isn’t Too Worried About Coronavirus (whoopsie!)

    Feb 24th: U.S. Stocks Plunge as Coronavirus Crisis Spreads

    Feb 26th: Stocks Fall Again as Coronavirus Spreads

    Feb 27th: Coronavirus Fears Drive Stocks Down for 6th Day and Into Correction

    All of these articles clearly, confidently attribute the stock market decline to Coronavirus.

    • Scott Alexander says:

      I wrote:

      “The stock market is a giant coordinated attempt to predict the economy, and it reached an all-time high on February 12, suggesting that analysts expected the economy to do great over the following few months. On February 20th it fell in a way that suggested a mild inconvenience to the economy, but it didn’t really start plummeting until mid-March – the same time the media finally got a clue.”

      It sounds like you’re talking about the relatively mild fall on February 20. Given that it later fell much, much more than that, I think it’s fair to say that underestimated the extent of the problem.

      • Anonymous Bosch says:

        It sounds like you’re talking about the relatively mild fall on February 20.

        He is very clearly talking about the fall throughout the week of February 24-28?

      • sdax says:

        Scott, this is silly. The equities markets fell something like 15% in the week February 24-28th. It *is* a mystery why it held up so well in early February (preference cascade?), but there is no way the sentence you wrote (“On February 20th it fell in a way that suggested a mild inconvenience to the economy, but it didn’t really start plummeting until mid-March”) makes any sense at all.

        If you want an answer to “when did the market react?” then it is basically here: link text

        And the answer is about February 24th

      • AllAmericanBreakfast says:

        The S&P 500 fell from 3373 points on Feb. 20th to a low of 2237 on March 23rd. It had fallen 50% of that distance (to 2805) on March 10-11. Saying “it didn’t really start plummeting until mid-March” is just not factually accurate. As I point out, the NY Times already identified the stock market correction as COVID-linked on Feb. 27th. They had a clue at least by Feb. 24th, and definitely by Feb. 27th.

        The basic gist of your piece is that the stock market more or less accurately represents our state of world knowledge.

        For that to be true, we must have been learning information from Feb. 20th-27th that justified an 11% drop in stock prices. Can anyone point to specific data that came out during that week that seems so significant?

        And after that happened, what further information did we learn from Feb. 27th-March 23rd that justified a further 20% or so drop in stock prices?

        And remember, there’s a difference between predicting stock prices and predicting that the pandemic was going to be a world catastrophe. The latter is a lot easier than the former.

        It’s so easy and obvious, frankly, that this was a case of
        a) Misguided optimism trying to shore up political support or “prevent panic”
        b) Everybody watching each other waiting for permission to act
        c) Conservative policy in orgs like the WHO for the threshold for a pandemic declaration

        For me to believe that the market prices accurately characterized our world state of knowledge about the course of this pandemic, you’d really have to show me what data the investors were basing their judgments on. We knew all the fundamentals well in advance of Feb 20th – COVID’s more deadly than swine flu, more transmissible than SARS, it’s all over the world, they shut down an 11-million person city over it, we’re low on basic medical supplies even in the industrialized world, and we’ve got no proven drugs.

        The only thing that I could see investors wondering is whether we were just going to let all the old people die. Once it became clear that we were going to actually shut some things down, they sold their stock.

        But that, again, is a separate question from whether the pandemic was going to be a world catastrophe. I just can’t see what we learned intellectually in that span of time that justified that flipped reaction.

        • Anonymous Bosch says:

          As with my earlier comment, I will recognize that my impression is a subjective gestalt as an active observer (or at least I’d like to think so) and merely offer it on that basis:

          You have to think of the 🌈correction🐻 in two phases: the impact of the coronavirus, and the impact of our response. Earlier in the thread I criticized Trump for not taking the threat seriously “until the stock market cried uncle” and that is my impression: that the first week or two of correction was mostly factual reality about the coronavirus soaking in, and then at some point it became more about the markets drawing some conclusions about what Trump’s hardheaded “all is well” response portended for a disease with such a :monkas: growth rate.

          The perspective of equities traders can be narrow in certain respects, but you can’t accuse them of not understanding exponential growth.

          • Scoop says:

            “Trump didn’t take it seriously until the stock market cried uncle”

            Isn’t that when he should have started taking things seriously? Isn’t the stock market the collective wisdom of many thousands of the world’s very brightest minds, all of whom are paid to predict what will and won’t be a big deal? Doesn’t it make more sense to trust the wisdom of that crowd than the handful of scientists who thought this was a big deal before then?

            And it wasn’t all scientists who thought this was big before Feb, recall. Fauci was still telling people it was ok to go on cruises in early March.

            I’m not married to the idea that things aren’t big till the stock market says they are, but it’s at least a defensible position.

          • Anonymous Bosch says:

            Isn’t the stock market the collective wisdom of many thousands of the world’s very brightest minds, all of whom are paid to predict what will and won’t be a big deal?

            Maybe. But unless you’re a very precocious child, you should know from experience that doesn’t stop them from occasionally being systemically and catastrophically wrong (or in this case, just a bit late). Part of that is the fallibility of man, like bog-standard herding, and another part is the increasingly purified incentive of maximizing revenue in the next quarter.

          • Edward Scizorhands says:

            Isn’t that when he should have started taking things seriously?

            The government has access to non-public information. Ask Bill Barr.

        • keaswaran says:

          On February 20, there were 4 known cases in Italy and 5 known cases in Iran. On February 27, it was 645 in Italy and 289 in Iran. So I think before that week, it was thought that this was still like SARS – human-to-human spread in China, spread in Singapore and South Korea that had been detected and successfully isolated, and individual travel-related cases popping up in other parts of the world.

          On Feb. 20 it may have been plausible for many people to think this would continue to play out like SARS – East Asia would solve their problem, everyone else would watch airport arrivals and quarantine them effectively, and within a few weeks everything would stabilize and gradually go away. By Feb. 27 it was clear that this wasn’t happening, since community spread was very clear from the public data in Italy and Iran, and probably also clear from genetic data in the United States and elsewhere.

          Incidentally, I think this is the week that I realized this – I was giving talks at three universities over the course of that week, and I remember that at the beginning of the week I was treating it like a normal trip, but by the end of the week (when I was coughing and had had a little fever) I was greeting everyone with elbow bumps and had planned to talk to my students about it on my first day back in the classroom. It’s possible that my argument now that this is the week people should have reasonably realized it was going bad is actually just a post hoc attempt to justify my own reaction (which may have been driven by the stock market dives that week just as much as by the news stories and numbers coming out of Italy and Iran).

          • AllAmericanBreakfast says:

            I think you are right. I found a day-by-day chart on Wikipedia of new cases by country.

            It shows that Feb. 20th and the few subsequent days really were the tipping point when it became clear that we were seeing international community spread. The first day when there were 10+ cases in 3+ countries outside China was Feb. 21st (in South Korea, Japan, Italy, and Iran).

  65. notpeerreviewed says:

    If you had asked me, in late January, what I thought the probability was that COVID-19 would become a global pandemic causing hundreds of thousands of deaths, I would have said something like 80%. Just because I turned out to be right doesn’t mean that was a good guess; maybe there really was still a high probability of containing the outbreak at that point. But anyway, my problem – the reason I didn’t react with more foresight – isn’t that I insisted on too high a standard of evidence; it’s that even when I guessed correctly, my default sense of normalcy prevented me from thinking seriously about the implications of what was about to happen. It was a failure of imagination.

    • keaswaran says:

      I saw a few Facebook posts and comments I made in late January and early February that accepted the likelihood that the disease would be a global pandemic and eventually settle into an endemic seasonal disease like the flu. However, I didn’t fully imagine what the early phases of that process would look like (i.e., March and April), so I didn’t anticipate what sorts of reactions were reasonable for me and for the world to start taking. I was thinking that a 2% fatality rate is high, and would mean millions of deaths, but relatively low risk to any particular individual, but I didn’t do the statistical value of a life calculation to realize that this very clearly justified a vast economic shutdown that would knock a dozen percentage points off the world economy to prevent the worst of the damage during the transition process.

  66. AlesZiegler says:

    I´d like to point out that major flaw with that Vox article is that it is asking “What went wrong with the media’s coronavirus coverage?”, as if “the media” would be some meaningful entity. SSC might also be considered as part of “the media”. They should instead ask simply what went wrong with Vox´s coronavirus coverage and do some soul searching.

    • webnaut says:

      I deleted my comment but I did actually tell people in the SSC reddit that Coronavirus was going to blow up and my sense was they weren’t taking it seriously. My pet theory is the subgroups of society are too synchronized because of the internet and the synchronization is making us blinder than we would otherwise be.

      Trump, Brexit, Coronavirus – these are big misses and I expect more of them in the future.

  67. eqdw says:

    The Vox article says the media needs to “say what it doesn’t know”. I agree with this up to a point. But they can’t let this turn into a muddled message of “oh, who knows anything, whatever”. Uncertainty about the world doesn’t imply uncertainty about the best course of action! Within the range of uncertainty that we had about the coronavirus this February, an article that acknowledged that uncertainty wouldn’t have looked like “We’re not sure how this will develop, so we don’t know whether you should stop large gatherings or not”. It would have looked like “We’re not sure how this will develop, so you should definitely stop large gatherings.”

    I have spent a lot of time thinking on the subject of “what would have been reasonable for a person to think if that person didn’t obsessively read the depths of the internet like I do?”. This is a hard thing to do, because it involves emulating not-knowing something. My gut instinct is to call attention to media failures in the face of this pandemic as an example of how they are so incompetent it borders on malice, but that’s not really fair, because I don’t know what the journalists actually did know when they wrote their articles. But I think I can identify three categories of error that they made. I think they are 100% morally responsible for these failures and I think that holding them accountable with punishments up to and including things I can’t say on this blog are appropriate given the severity of the current situation. I don’t expect this will ever happen, but this is to frame my thought processes.

    1) I am not so concerned about them “saying what they don’t know”. I would rather they _don’t_ say what they don’t know. There was a lot of reporting in January and February about how this plague is totally fine don’t worry about it. They were wrong. Meanwhile I was reporting to everyone who would listen, hey 20 million Americans are gonna die, we need to do something. I appear to have also been wrong. What is the difference? When the media said hey don’t worry about it, they said that very confidently. When I said hey, worry about it, I said it explicitly as “hey there’s massive uncertainty here and I’m just delivering the facts but you should know that this thing is happening in China and if it keeps its current trajectory it will happen here so we should probably all start prepping”.

    I know this seems like kind of a petty concern (my first predicted criticism is “it’s the media’s job to say things with confidence or else why would anybody trust them?”) but I think it’s a serious concern, precisely because of the chaos and ambiguity of the circumstances. I know it’s a meme, and I’m not trying to make a wider political argument, but just pointing out: four weeks ago, the official line out of almost every government agency and almost every media outlet I can think of, was that masks don’t work and you don’t need them. Meanwhile, as of Monday, it is now a crime punishable with six months in jail for _anybody_ in my city to be outside _anywhere for any reason with two very small exceptions_ without a mask.

    Data evolves and we learn new things. It’s entirely possible that we didn’t have data supporting mask use until this week. But there’s a significant difference between “hey we don’t have strong evidence that masks work so maybe it’s not needed” and “hey dumbass why are you buying masks don’t you know they don’t work”. Our media was reporting with the second tone, and that means that the credibility authorities just craters every time they change their tune. There’s a creepy “we were always at war with the maskless” vibe to it, and I’m not the only person who reacts negatively to this

    2) This is kind of the same as (1), but not only were they stating falsehoods with way too much confidence, but they were also stating falsehoods with smugness and that makes the problem worse. To roll with the mask example, it would be one thing if we had a bunch of articles that said “we have no evidence masks work so don’t buy them”. That would be overly confident but not smug. But instead we get a million articles like “we have no evidence masks work and so if you buy one you’re a crazy idiot who believes in pseudoscience and everyone should mock you”. Now I get it that the driving force here is nothing more nefarious than “gawker-style writing sells” but I’ve been opposed to gawker-style writing for so long that I’ve had a hit piece written on me about it (which apparently is no longer online so you’ll have to take my word for it). In the wake of current crisis, this stops being socially erosive and starts being a critical threat to peoples lives. Because it wasn’t just that the media was reporting that you shouldn’t/don’t need to prepare for this. It’s that they were levelling the full force of weaponized smugness to put everyone in a situation where they perceive a strong level of social pressure not to prepare for anything. If they reported exactly the same things that they did, but without the smugness, I would have much less problem with it

    3) It is clear, to me at least, that most of these journalists do not think their job is “tell people true facts”. They don’t even think their job is “tell people true _stories_”. They think their job is to be social engineers. They think their job is to decide what people should think and then make them think it, by saying whatever needs to be said in order to make them think it. This is why we get things like “the real risk isn’t the pandemic, it’s anti-asian racism”. There is absolutely no reason under the sun why, if you were truthfully and sincerely reporting that “hey this plague isn’t a big deal”, that you would couple it to an anti-racism narrative. But of course what is going on here is that they have already decided in advance that their job involves a moral obligation to push anti-racism, and as a consequence they completely destroy all credibility regarding reporting on the plague in order to push anti-racism. Now, I’m not trying to say that they shouldn’t push anti-racism (I mean, I think they shouldn’t push anything but I am not pro-racist). What I’m trying to say is that they can have a goal of “accurately report true information” or they can have a goal of “push support for social policy” but they can’t do both; each goal necessarily conflicts with the other. If these people could have put their social engineering agendas down for one second to just rally and be like “hey, it’s crisis mode, different rules apply now”, that would be a lot more understandable and foregiveable. They wouldn’t even have to stop pushing their anti-racism entirely! It’s just that there’s a tone and emphasis difference between what they reported (“Racism is really really really bad and we’re not going to let these crazy conspiracy theorists who think there’s a disease in Wuhan use that to justify their hatred”) and what I’m saying (“Hey there’s a disease is Wuhan and it’s really spooky and we should all prepare, but just keep in mind that this thing spreads so fast that it’s definitely in non-asian populations now and so taking actions aimed at Asian communities, in addition to being kind of racist, would not be helpful”)

    • For another small example along your lines, someone has already quoted a NYT piece which, along with some serious warnings about the epidemic, threw in a note about how it was the fault of global warming (I exaggerate literal meaning but not tone).

      • eqdw says:

        Ironically, on the one hand, this plague has done more to fight global warming than anything else we have ever done, and on the other hand, if the people who say global warming is an X-risk to humanity actually believed that, they would be ok with measures as drastic as “global plague” to avert it

        • Anonymous Bosch says:

          As someone who does think global warming is a (minor, tail-end) x-risk, this is an time-compressed object lesson in “if you wait until it’s obvious to the man on the street, it’s too late.” I don’t think drastic measures are advisable or necessary right now, but if we continue the rest of this century without even bending the emissions curve (assuming this year will be an outlier), then by 2100 you will definitely be seeing governments doing some crazy authoritarian shit and their populations being frightened enough to back them (wars over resources/migration can have that effect).

          I hold out some hope that the world can be the equivalent of Taiwan or South Korea and implement modest measures now to stave off drastic ones later. For instance, with fossil fuel prices at a historic low and extrinsic reasons to discourage travel, I think now is as good a time as any for governments to look at carbon taxes.

  68. ajfirecracker says:

    I wish people could do cost-benefit reasoning with uncertainty about other things as well, e.g. tonic water (unclear benefit, negligible cost)

  69. JJ says:

    Now Goofus and Gallant discuss God

  70. Xanderkish says:

    Do you have any good resources criticizing Johann Hari’s book? I concluded I didn’t trust it, but I’d like some comprehensive information to give me a more informed opinion on it.

    • Atlas says:

      I would also be curious to hear more about this, because I’d previously assumed, from the testimony of e.g. Sam Harris, NPR and Glenn Greenwald, that Hari was a respectable writer. (Indeed, Greenwald had Hari on his podcast recently to discuss the “mental health impacts of the coronavirus.”)

    • sicromoft says:

      I’d love to hear more about this as well.

  71. elspeth diana says:

    freezing your brain is risky. that’s why we use cryoprotectant.

  72. kalimac says:

    There are other reasons to eschew cryonics besides thinking that a 12% chance that it’ll work is too small to bother with. (Besides the fact that my own guess as to the likelihood of it working is a microscopic fraction, not anything like 12%.)

    And that it that I don’t -want- to be frozen for an indefinite period and then wake up in a strange future.

    Ever read a science fiction story called “Ozymandias” by Terry Carr? It’s in Again Dangerous Visions. That pretty much is my answer to why I’m not interested in cryonics.

    • And that it that I don’t -want- to be frozen for an indefinite period and then wake up in a strange future.

      Why not? If the world started descending into madness, how bad would it have to be before you would refuse further medical treatment?

      Would a future where they are able to resurrect you be that bad?

      • kalimac says:

        I don’t consider a speculative experimental therapy with only a microscopic chance of working to be worth it to preserve a life which is ending anyway. I’ve even taken the DNR order, because that’s not worth it either, for similar reasons. And if the reason for going is that the world is descending into chaos, all the more reason not to go.

        “Would a future in which they are able to resurrect you be that bad?”

        As I said: Ever read a science fiction story called “Ozymandias” by Terry Carr? It’s in Again Dangerous Visions. That pretty much is my answer to why I’m not interested in cryonics.

  73. Two relevant stories:

    Many years ago I observed a highly intelligent elderly man ignore evidence of an approaching forest fire until he went out and saw it himself — at which point he took rapid and prudent actions to get his family somewhere safe. Thinking about it afterwards, I remember a distinction, I think in something Richard Posner wrote, about the distinction between fluid and crystalized intelligence. Fluid intelligence figures out a solution to a problem. Crystalized intelligence looks at the solution figured out in the past and uses that. As people get older, they accumulate more past solutions, have less time in which to benefit from finding new solutions, shift from fluid to crystalized intelligence. If you have lived somewhere for many years and never observed any hint of a forest fire, it’s natural to assume it isn’t happening until you have clear evidence it is.

    About two months ago I was part way through a speaking trip in Europe. My younger son, communicating via email, urged me to cancel and come home in order to avoid the risk of what seemed to be becoming a serious epidemic.

    My initial response was that he was over reacting — especially given that my next talk was to be in Ljubljana, an interesting city I had never visited. Thinking about it, I remembered the first story and concluded that I was making the same mistake my father had made, relying on crystalized rather than fluid intelligence. I had traveled over much of the world, never had to cancel my plans to avoid an epidemic, hence took it for granted that it wasn’t necessary this time. I cancelled my last two talks, booked a flight from Lisbon to Paris and another from Paris to SFO, got home on March 13th and have been self-quarantining with my family ever since.

    The leading experts media writers go to are likely to be established people, hence late in their careers, hence old. Anthony Fauci is 79.

    • AlesZiegler says:

      Now I recall that you wrote that you had a scheduled talk in Prague. I assume that it was cancelled, since lockdown here started around the time of it. But I totally forgot to check that out, despite my original plans.

      • Ex post, I don’t think I could have gotten to Prague — the border closed after I decided to cancel but before (I think) I was scheduled to fly there. But at the point when I cancelled, the people in Prague where still planning to host the talk.

        The people in Ljubljana were planning to hold the talk in a mostly empty room and live stream it. The talk (more a panel) in Lisbon had already been canceled. I ended up doing five talks out of a planned eight.

    • kalimac says:

      And Donald Trump is 73.

      Are you saying that Fauci’s cautionary warnings are likely to be wrong? It seems to me that he’s the one who is trying (at least when the President is not looking directly at him) -not- to ignore the forest fire.

      • I am saying that people like Fauci are likely to make the same mistake I was making, failing to update their view of what was happening and what to do about it in a novel situation, and that the media largely depend on such people. I haven’t followed the timing of what Fauci said when to know if it is true of him in particular.

        • kalimac says:

          Undoubtably there is, as you note, an age-related tendency here. But that factor can easily be drowned out by others relating to general nature of intelligence. Experience suggests that the number of formerly highly flexible people whose minds become rigid with age is far outweighed by the number of younger ones who never were that flexible to begin with. To accuse, even if speculatively and only by implication, an older man of high reputation and continued competence, merely on the grounds that he’s 79, does not have a good look.

          • albatross11 says:

            kalimac:

            It may indeed have a bad look. But personally, I don’t care so much about whether it has a good or bad look as whether or not it is true.

          • I accused both myself and my father of the particular error I was describing, although in both cases the error was corrected before it had serious consequences.

            I did not accuse Fauci of anything other than being old.

    • webnaut says:

      “About two months ago I was part way through a speaking trip in Europe. My younger son, communicating via email, urged me to cancel and come home in order to avoid the risk of what seemed to be becoming a serious epidemic.”

      I asked somebody here to email you about this actually (prompted by an RSS update you were travelling to Europe while China Flu was ringing the bell), you may not have received it but I’m glad you made it out of there safely! 🙂

      What you’re saying about age rings true – people are starting to notice all the frontrunners for president were born in the 1940s or 1950s. We should all add the question “do I need crystal or liquid” to the metacognitive stack – ironically triggered for you to do liquid from a crystallized memory.

      Gwern is also fine. I worry terrible things could happen if he runs out of oolong tea though. I don’t know if you’re into loose leaf tea but it’s really a joy in a pandemic.

  74. If we interpret her [Kelsey Piper’s] “likely won’t be a nightmare pandemic” sentence as a prediction, she got the prediction wrong.

    No, this is what Tetlock calls the “wrong-side-of-maybe fallacy.”

    From Superforecasting:

    [T]he wrong-side-of-maybe fallacy.

    If a meteorologist says there is a 70% chance of rain and it doesn’t rain, is she wrong? Not necessarily. Implicitly, her forecast also says there is a 30% chance it will not rain. So if it doesn’t rain, her forecast may have been off, or she may have been exactly right. It’s not possible to judge with only that one forecast in hand. The only way to know for sure would be to rerun the day hundreds of times. If it rained in 70% of those reruns, and didn’t rain in 30%, she would be bang on. Of course we’re not omnipotent beings, so we can’t rerun the day–and we can’t judge. But people do judge. And they always judge the same way: they look at which side of “maybe”–50%–the probability was on. If the forecast said there was a 70% chance of rain and it rains, people think the forecast was right; if it doesn’t rain, they think it was wrong. This simple mistake is extremely common. Even sophisticated thinkers fall for it.

    Even sophisticated thinkers who have already read that passage and written a review on the book!

    (I also suspect you made a similar mistake when talking about Zeynep Tufecki. I am not sure what exactly you mean by “her batting average is 50%”, but it