Joint Over- and Underdiagnosis

Today I had several more terrible lectures on ADHD.

In one of them, I was informed that America is medicalizing normal childhood mischief and loading anyone who gets worse than a B+ up with Ritalin or amphetamines as part of the pathologization of everyday life.

In another, I was informed that ADHD is shamefully underdiagnosed and most of the children who need stimulants most are going without them and failing school unnecessarily, so we need better screening programs and more efforts to seek out potential sufferers of the condition.

So I asked one of my attendings, Dr. L, which one it was. Are we overdosing ADHD? Or underdiagnosing it?

He answered that we are both overdiagnosing and underdiagnosing ADHD, the same as every other psychiatric disease, and then explained this so it made perfect sense and I was embarassed for not realizing it before.

Suppose that 3% of the population has ADHD.

Suppose that of people with ADHD, 50% of them realize they have ADHD like symptoms and go to a psychiatrist to get checked out.

Suppose that of people without ADHD, 10% of them falsely believe they have ADHD and also go to a psychiatrist to get checked out.

The Conners Continuous Performance Test is a commonly used test that evaluates children for ADHD. It is found to have a sensitivity of 75% and a specificity of 73%. In theory our system is based on faith that a trained psychiatrist can do better than a neuropsychological test; in practice they probably do much worse. Let’s give them the benefit of the doubt and say this is an excellent psychiatrist who outperforms the test handily and has both a sensitivity and specificity of 85%.

We can see that of every 100 people, 3 will have ADHD and 97 won’t. 1.5 true patients and 9.7 false patients will show up for psychiatric evaluation. The psychiatrist will diagnose 1.275 true patients and 1.455 false patients with the condition, and prescribes stimulants according to the diagnosis.

So we have three things that, surprisingly, all happen at once:

1. We have an excellent psychiatrist who outperforms the tests and is right 85% of the time.
2. The majority of people who are on Ritalin, shouldn’t be.
3. The majority of people who should be on Ritalin, aren’t.

Number two sounds a lot like what we mean by “overdiagnosis”, and number three sounds a lot like what we mean by “underdiagnosis”. So even with a pretty good psychiatrist acting honestly, we expect ADHD to be both overdiagnosed and underdiagnosed at the same time.

Even in conditions that do not quite satisfy the “majority” part of (2) and (3), we might still expect it to be true at the same time that a sizeable chunk of people diagnosed with the disease don’t have it and a sizeable chunk of people with the disease aren’t diagnosed.

If this seems counterintuitive, it is just another example of the annoying world of medical sensitivity and specificity statistics, which are constantly tripping up even the most experienced doctors. See also the infamous mammogram problem.

Once I understood this joint-overdiagnosis-and-underdiagnosis problem, several other candidate situations immediately leapt to mind. Antidepressants are almost certainly both overprescribed and underprescribed. So are opiate pain medications.

This entry was posted in Uncategorized and tagged , . Bookmark the permalink.

172 Responses to Joint Over- and Underdiagnosis

  1. Princess_Stargirl says:

    After all the Bayes theorem examples I have worked through I too am very sad I didn’t immediately apply the logic to ADHD. (also I sort of think stimulants are good for everyone who can avoid abusing them, so I am not overly worried about over-diagnosis and was not likely thinking clearly).

    Why do you have to make cry Scot?

    jk.

    This post was very good and I am going to share it with a ton of people soon

    • Randy M says:

      I too am going to share it with a ton of people, the ten or so Facebook friends I have.

    • anon1 says:

      > also I sort of think stimulants are good for everyone who can avoid abusing them
      Fair enough for adults who have a choice in the matter, but ADHD is mostly diagnosed in children, who don’t really, and forcing mind-altering drugs on people is sufficiently uncool that there had better be a good reason.

      • Adelene says:

        Indeed.

        I was on ritalin from ages 7 to nearly 17. If I go more than a week or two without taking some sort of stimulant (caffeine works) my brain turns into a completely nonfunctional puddle of goo, which might just be the natural course of things for me, but I can’t help but wonder if things wouldn’t be a bit different if my brain had had a chance to mature without being taught that stimulants were just a thing that would always be there.

        Plus, y’know, the obvious consent issues. PTSD is no fun either. 😛

        • Eric Rall says:

          My experience actually points in the other direction. After a period of stimulant use, I tend to get in a groove of good work habits that let me sustain high focusing function after tapering off stimulant usage.

          YMMV. We may have different patterns of drug reaction in general, and I started medication later in childhood than you did, went off it sooner, and didn’t take it continuously while I was being medicated (my doctor recommended taking Ritalin only on an as-needed basis: skipping doses on weekend, vacations, etc).

  2. Tarrou says:

    Now imagine how the actual statistics look with random schoolteachers, hypochondriac-by-proxy parents and “counselors” “diagnosing” ADHD and badgering doctors or pill-shopping.

    • Tuna-Fish says:

      And on the other hand, all the parents who see the excessive medicalization of people around them and thus are very reluctant to accept that their children have ADHD or even that it’s a real thing.

  3. Andrew says:

    I’d argue this is more of an issue in psychiatry where disease categorization is not as well understood and diagnosis has weaker tools and is less clear. In other areas of medicine, sensitivity and specificity are much better, especially when done in stepwise fashion. In these cases underdiagnosis is because of health system utilization/access failures and overdiagnosis tends to be due to overscreening.

    All this points to us needing to understand the brain better. As for anarcho-tyranny, I guess same goes for political economies. As for the rest of medicine, we should expand health services and reduce our care practices to those that are evidence-based…

  4. Protagoras says:

    Depressing that even without the problematic confounding factors, less than stellar results are to be expected. Especially as there surely remain plenty of confounding factors to make things worse; there do seem to be a lot of people involved with some pretty entrenched irrational biases in various directions as well.

  5. Cyan says:

    Wait wait wait. What’s the gold standard here? What are the observables that by definition dichotomize the population into ADHD and non-ADHD? Without this, we can’t even begin to talk about sensitivity and specificity of (presumably more cost-effective but noisy) tests.

    • Scott Alexander says:

      I don’t know for sure, but usually in these cases it’s a structured clinical interview, which is a very specific interview with a very heavily trained psychiatrist or psychologist who thinks very very hard about whether the patient’s answers imply they meet DSM criteria for the disease.

      • Paul Torek says:

        It would be cool to have inter-rater agreement numbers for these things.

      • Anonymous says:

        From what I’ve understood, it varies tremendously. Some people just ask some questions and hand out pills. Some people go so far as to do an EEG. When I was diagnosed with ADHD they did very thorough behavioral testing, which I personally would consider the gold-standard.

        I took the WIAS, and they looked at the ratio between my working memory scores (which were around the population average) and the rest of my IQ (which was above the population average, but apparently average for the lesswrong-o-sphere) The ratio indicated my working memory was much worse than the rest of my scores would predict, which indicates ADHD, but the psychologist was unsure if that warranted diagnosis because while I did show the selective working memory deficit, my working memory was technically average. (I’ve pretty much spent my entire life coming up with compensation strategies)

        I also took the TOVA, which tests response inhibition, in which I did so poorly that the responses could not be scored. It was very humiliating. All you have to do is press a button when a symbol come sup, and not press a button when a different symbol comes up. I couldn’t do it. That was the test which really settled the diagnosis in the psychologist’s mind. (Before the TOVA and the digit span, they were kind of teasing me about my scores and wondering why I bothered to come in, which was both weirdly ego boosting and frustrating at the same time – I was afraid I wouldn’t get help at first)

        I also took Wisconsin Card Sorting, Stroop test, dislexia screening, and so on, which I passed with flying colors, screening out various other disorders.

        I’m a little surprised you don’t know for sure… aren’t you a psychiatrist? (that’s not intended as rudeness, just curiosity as to why they don’t train psychiatrists to know these things)

        • Creutzer says:

          Is EEG actually useful in diagnosing ADHD? I only know about it being done for the purpose of checking whether you’re liable to develop epilepsy if they give you stimulants.

          • Anonymous says:

            Yes, there are systematic EEG differences, although personally I’m philosophically inclined to think behavioral tests are what really matter.

    • Anonymous says:

      I outlined what happened during my diagnosis in reply to Scott’s comment.
      (I should probably have replied to your comment, instead of Scott’s – sorry!)

  6. Scott F says:

    And that’s assuming a superlative psychiatrist! Add in the moralising around the “character flaws” that are its symptoms (not paying attention to authority figures, forgetting dates / appointments / deadlines, turning up late, not being able to get stuff done, etc), the fact that most medications that manage ADHD will supercharge a more normal brain, season with a little Molochian optimising for school performance (not quite throwing your children into the fire, but still suggestive), and then back all this up with a section of mainstream psychiatry that cheerfully and correctly admits we are overdiagnosing it, and I’d be surprised as hell if even 50% of people with ADHD seek treatment or get a diagnosis.

    (Recently, a very intelligent someone was troubleshooting their procrastination and lack of motivation, mentioned that they found stimulants effective, and then dismissed the possibility of ADHD … by giving a textbook example of hyperfocus.)

    A selfish part of me almost hopes that the terrible ADHD lectures continue until you reaches your breaking point, and we get to see Righteous Scott unleash his true power. In the meantime: have you read or heard much of Dr Russell Barkley on ADHD?

    • Error says:

      not paying attention to authority figures, forgetting dates / appointments / deadlines, turning up late, not being able to get stuff done, etc

      A doctor said I was borderline ADD as a kid (no -H), but I was never medicated for it. This list reminds me that I should really consider getting checked out again.

      Are any of the various online tests for ADD any good? The CCPT Scott mentions doesn’t appear to be available via google.

      not paying attention to authority figures

      This seems out of place to me. What do authority figures have to do with attention, except perhaps for ADD making it harder to compel someone to focus on your tasks over theirs?

      dismissed the possibility of ADHD … by giving a textbook example of hyperfocus

      Does treating ADD reduce hyperfocus proportionally to attention problems? Fear of that is one reason I’ve never gone to get checked out; I like hyperfocus and would be sad if I couldn’t do it anymore.

      • Ozy Frantz says:

        I’m not sure if treatment reduces hyperfocus, but if it does the effects of stimulants on ADD are short-term, not long-term. You can take the meds when you need attention and skip them when you want hyperfocus.

        • Error says:

          Ideally I want both at the same time. 😛 That is, I want to be able to hyperfocus on demand.

          That’s still good news to me, though. For some reason I thought ADD drugs were like antidepressants, where you (seem to be?) stuck doing them long-term or not at all.

      • Eric Rall says:

        My experience is that treatment actually improves hyperfocus.

        The main effect of treatment for me are 1) more mental energy with which to pay activation costs, and 2) reduced effects of fatigue (physical or mental). The net effect is that I can more easily deliberately enter hyperfocus, more easily make a conscious decision to break it, and I can sustain it for longer without feeling burned out.

        • Scott F says:

          Seconding these effects, with the caveat that a conscious decision to break hyperfocus went from “impossible, would never happen” to “difficult but possible”.

    • Berna says:

      Your remark about ‘character flaws’ reminded me of this awesome post: Errors vs. Bugs and the End of Stupidity. TL;DR: don’t moralize, fix the underlying problem. (But do read the article, it’s really good.)

  7. Cyan says:

    I’m a Bayesian statistician by trade, but my degrees are in other fields; as a statistician I’m almost entirely self-taught. I find myself reassured by the fact that it was intuitively obvious to me how a condition could be over- and under-diagnosed at the same time.

    • Steve says:

      Yeah; I thought of the answer before Scott got to the part about asking his attending, and I was extremely gratified that the time and effort of reading the sequences and trying to apply probabilistic reasoning hadn’t gone entirely to waste.

      Quite the feel-good post.

  8. Mary says:

    Both parents and teachers agree that when ADHD students are put on drugs, their grades go up.

    Look at the grades. In reality, they don’t. Both parents and teachers are deluded about the effect.

    • cheesesandwich says:

      Citation?

      • CaptainBooshi says:

        I don’t know for sure, but I imagine she’s talking about stuff like this study or this study. I recalled reading about something like this last year, so I went looking, and these were the first things I found with a google search. I don’t know how much other research has been done.

        The first is a Canadian study based off of Quebec. Apparently, at one point, they had a huge increase in medical insurance covering prescriptions, so a whole bunch of kids went on ADHD medicine all at once, and the investigators couldn’t find any academic improvement for kids who went on the medication in the long-term. In fact, they found some evidence that they did worse if they went on the medication.

        The second study is actually a follow-up study to the big MTA study done in the 90’s by the NIH. While the initial study showed that there were significant improvements, the follow-up studies appear to show that the improvements had completely disappeared after three years, and once again (if I’m reading the paper correctly) some evidence they did worse in the long-term.

        The Quebec study also proposes 3 possibilities for why kids on ADHD medication might do worse than they would otherwise: that being diagnosed with ADHD might have negative consequences like social stigma or being put into special education; that the medication just stops disruptive behaviors that would normally get them extra attention and help, but don’t actually improve capability; or that the medication itself might have unknown side effects, especially if it’s not given or taken correctly.

        I’d like to emphasize that I don’t actually know enough about this subject to be useful, or even judge how trustworthy these studies are. I just knew I had heard about claims like these recently, and figured a bit more hard information would be useful.

        • drethelin says:

          With regards to extra effects: my girlfriend has said that Ritalin made her feel like the angel of death was behind her at all times.

    • Levi Aul says:

      Presumably, when non-ADHD students are put on (those) drugs, their grades also go up. This isn’t even because the drugs always make you really-super-focused; this is because a regular average person is still kind of lazy, all things considered, and the school system is built around the motivation level of neurotypical people. Someone who is just as motivated as a somewhat-motivated person will usually outperform the majority of their peers.

    • Princess_Stargirl says:

      Ahouldn’t giving neurotypical students stimulants improve their grades. At least somewhat? Given that many of the students given these drugs are neurotypical the test scores should go up?

      I am not even trying to argue that student who actually have ADHD will improve.

      • Anonymous says:

        “Ahouldn’t giving neurotypical students stimulants improve their grades. At least somewhat?”

        http://www.nber.org/papers/w19105.pdf

        Apparently not. Why should it? If you become focused on something *too* easily, isn’t that just as bad as it being hard to focus? In order to do well, you not only have to focus on the right things, you also have to focus on multiple things without neglecting any of them too much. That’s assuming that all ritalin does is make people focus more, when it’s entirely possible it does other shit, too. A friend of mine had a lot of trouble eating enough food on ritalin, for example.

        • Jaskologist says:

          It should assuming that higher grades are associated with higher than average concentration ability.

          I’d wager, for example, that such drugs would have helped students studying for the Confucian exams back in the day.

          • Anonymous says:

            It should assuming that higher grades are associated with higher than average concentration ability.

            I mean, not necessarily. Higher than average might be quite beneficial. Ritalin-fueled might be significantly higher than just “above average,” to the point where it could even be negative. I dunno, and anyway it’s not that crucial a point given that Ritalin certainly does something more complicated than give “+5 concentration” or whatever.

            I’m also not sure whether “concentration” is exactly the way to say it. It’s a positive descriptor that we use to describe people who put effort towards the tasks we think are important. Increasing “propensity to focus” in general might not just lead to behavior we’d call “concentration,” and could in addition lead to “obsessiveness” or some other more negative descriptor.

        • Princess_Stargirl says:

          I know modafinil is different from ADHD drugs (though I don’t think anyone really understands what modafinil does exactly). But I take modafinil and have never been diagnosed with anything. My work quality has gone way up. If ADHD drugs are anything like modafinil I think they should help most people?

          I have very limited experience with most stimulants.took Adderral once and Cocaine at a coupel parties. So I really don’t know what they do for studying etc.

  9. Douglas Knight says:

    Yes, it can be true that most people who need the drug don’t get it, and simultaneously, most people who get the drug don’t need it. But is that what the first two lectures said?

    There’s usually a trade-off between false positives and false negatives and you just have to accept that. You can make it to the optimal frontier and you can tune the parameters to move along it, but once there, you cannot have both over- and under-diagnosis compared to other frontier options.

    Maybe you aren’t at the frontier. Maybe more people should be sent for diagnosis, but the excellent psychiatrist should diagnose a smaller proportion of people. But both lecturers complained about the number being sent to the psychiatrist. They cannot both be right that the same number should move in opposite directions.

    What I’m saying is that the attending dodged a real question. Maybe he told you something more important, but he also pretended a real question was a false question and did you no favor therein.

  10. Anonymous says:

    Opinion: I think de-regulation could take care of this. The real question is “do stimulants help or not”. A big part of the problem is that people say “okay, let me go in for evaluation and blindly follow what the good doctor says”… and that’s when you get small mischievous children / people with disorders other than ADHD / people who just have low IQ getting misdiagnosed and taking stimulants.

    If the attitude was more “Hey, try this coffee. Does it help? Yeah? Okay, let me test whether it actually helps. Okay, looks like it does, here you go, just use this whenever you feel the need to” combined with a battery of cognitive tests I think that would be better than the current system, where small children religiously take pills. You might get a few more addicts, but you’d get way fewer people who are simply misdiagnosed, and those who didn’t need the stimulants would naturally use them less.

    Unlike a lot of other disorders ADHD is one of those problems where patients *do* often have privileged insight into their mental processes and understand precisely what the issue is, even if they don’t know what it’s called.

    (disclosure – I self-diagnosed with ADHD, went to a psychiatrist, had him say “yup, that’s ADHD”, acquired medication, and it helped me. Over the course of a year, I use only 2 months worth of prescription. If de-regulation became a thing, I’d probably experiment more with things like modafinil to see if it has a better benefits-to-bad-side-effects ratio on my attention span.)

    • Anonymous says:

      Re: parenthetical: Why don’t you go to your psychiatrist and say: “I’d like to experiment with modafinil”?

      • Anonymous says:

        1) I’m procrastinating going to the psychiatrist again and I haven’t been able to get organized enough to fit it into my schedule… I suppose that’s technically a symptoms of my disorder.

        2) The psychiatrist’s time is expensive, and that’s a big deterrent.

        3) I’m not sure how careful I need to be here. Asking about “experimenting” with Schedule I drugs might give off the wrong impression. (Although I hope the fact that I’ve generally been very conservative and have so far been radically under-filling prescriptions puts off that suspicion). Plus, many doctors don’t like patient’s who play at being doctors, and I guess I haven’t figured out his personal feelings on patients experimenting. I did undergo very thorough psychometric testing though, so at least my diagnosis should not be under question. (That’s another thing relating to diagnostic accuracy- from talking to other people with ADHD, a lot of them never actually did quantitative psychometric evals. I don’t understand why.)

        But I do plan to do just that, eventually. Just gotta make time for it and figure out how to phrase it.

        • somnicule says:

          I’m not sure if you’ll have similar problems as me, but my psychiatrist seems to think that Modafinil is being hyped online by pharmaceutical companies and isn’t all that useful when I asked about it to relieve some of the fatigue that comes with my antidepressants. In hindsight he might have misunderstood me, thought I was asking about it for antidepressant effects.

        • Anonymous says:

          Yes, doctors don’t like it when their patients talking about “experimenting,” but… you are asking to switch from scheduled to unscheduled.

          [technically modafinil is scheduled most places, but it is on a much less significant schedule than amphetamine or methylphenidate. In America, only schedules 2 and 3 matter in practice.]

        • Protagoras says:

          I’ve used modafinil, and my experience was that it didn’t work as well as dextroamphetamine (which is what I’ve used most for ADHD), and I seemed to build up a tolerance over time (which didn’t happen with dextroamphetamine). So I guess I’m inclined to think that maybe the reason somnicule’s psychiatrist said modafinil is overhyped is because modafinil is overhyped.

          • Anonymous says:

            How did your doc respond to you asking for it? (assuming you got it via prescription)

            I’m mostly drawn to the fewer side effects aspect. When it comes to brain enhancement for mostly functional people, I tend to think less is more.

            Also – subjectively gauging effectiveness with stimulants is even trickier than regular placebo effects, since the stimulant effect on reward circuitry artificially boosts your perception of how well you are performing.

          • Protagoras says:

            Doctor suggested it. I was still a grad student at the time, and psych services at my school was excellent. I haven’t really figured out myself what to do about not having access to anything similar since finishing school several years ago.

        • Cliff says:

          Modafinil is not a schedule I drug. I buy it from India, no problem

  11. Levi Aul says:

    This isn’t actually over- and under-diagnosis (i.e. a system at equilibrium with two test-based faults). This is over-diagnosis (a fault in the test) and a self-selection bias (a fault in the population).

    Right now, almost nobody gets tested for ADHD, because A. it’s not an available idea in culture compared to “you’re lazy”, and B. it’s kind of a shameful thing to have (because, being unavailable to most other people, you’re basically assumed to be proudly waving a flag that says “lazy” on it), and so there are e.g. no famous people doing ADHD awareness.

    Because of this, there’s a selection pressure causing only those who are really, really motivated to bother getting themselves (or their children) tested. Motivated, in this case, by the desire for ADHD medication.

    So, on the “actually bothers to get an ADHD test” side, you’ve got drug-seekers, Munchausens’-by-proxy victims, some kids who teachers thought were strikingly hard to deal with, and a (very) few people who noticed some neurological symptoms, looked them up, determined that ADHD best suited their symptoms, and then decided they should probably ask a psychiatrist about that.

    On the “doesn’t bother to get an ADHD test” side, you’ve got lots of people who aren’t even aware ADHD is a thing; many people who can’t be convinced it’s a thing even after being told by a doctor that they exactly fit the symptom profile; and, of course, the people who think they probably have ADHD but are too unmotivated to go through the long struggle to see a therapist, get a referral to a psychologist, wait on a waiting list, speak with them five times, and finally get the drugs they’ve been sure they need for years.

    (Also on the people-who-aren’t-aware-they-have-ADHD side: a lot of people who self-medicate with–and then become addicted to–stimulants like meth and cocaine. If you have any interaction with Vancouver’s numerous crackheads you will come away with the impression that, whatever else they have going on, they’re certainly unmotivated, scatterbrained, and need to pace a lot. Those are not supposed to be things you are when you have a high dopamine level!)

    So, the system is not at equilibrium: a lot more people should be coming in for ADHD screening (maybe everyone, even; it seems to be about as common as nearsightedness in children, and we make them all get tested for that.) Only once that is true would the sensitivity and specificity of the test have anything to do with the number of people living suboptimal lives.

    After that, there’s another problem: right now, the test could be returning “yes, they have ADHD” 100% of the time and it would still look vaguely correct to the people performing it, given that everybody is strikingly happy with their diagnosis. Bringing in people who haven’t self-selected as “really wants ADHD medication” would make the faults in the test clear immediately.

    • Anonymous says:

      Are you talking about America?

    • Nornagest says:

      …drug-seekers, Munchausens’-by-proxy victims, some kids who teachers thought were strikingly hard to deal with, and a (very) few people who noticed some neurological symptoms, looked them up, determined that ADHD best suited their symptoms, and then decided they should probably ask a psychiatrist about that.

      You’re seriously overplaying the self-selection angle here. Adult ADHD probably isn’t that rare, but it’s thought of as atypical enough that it’s usually qualified as such when people talk about it; most ADHD diagnosis happens in children and adolescents. That means that most cases are going to be initiated by a parent or teacher, and that means that both drug-seeking and considered self-diagnosis are probably negligible. Munchausen by proxy, meanwhile, is quite rare: Wikipedia mentions 451 documented cases, which is certainly an undercount but is still a very small number indeed.

      That leaves “some kids who teachers thought were strikingly hard to deal with”. Broaden “teachers” to authority figures in general, and you’ve probably got a fairly good description of what’s going on, but this isn’t anywhere near as uncommon as you’re making it out to be.

      See, I don’t think the motivations work out quite the same way with kids as when you’re managing your own health. ADHD is a well-known disorder. Given that your kid’s acting inattentive, which is more flattering to you as a parent: that they’re suffering from neurological problems through no fault of yours or their own, which can be mitigated at least to some extent by giving them synthetic amphetamines, or that they’re just a lazy little shit?

  12. Matt C says:

    I’d be interested to hear your thoughts about what ADHD “is”. From what I understand it is a disorder defined solely by its symptoms. If you tick five of the checkboxes you have it. If you tick four you don’t. It is drawing a line down a continuum.

    I have gotten into arguments with one friend in particular who denies this is true. He says there’s a specific, binary distinction between having ADHD and not having it, and the apparent fuzziness of the disorder is only because diagnosis is often done poorly. Curious to hear what you or other commenters think.

    • Hainish says:

      But isn’t this the case for nearly any psychiatric disorder? Say, depression?

      • Matt C says:

        I dunno about nearly any, but yes, some. Some physical disorders too. I don’t think it’s necessarily wrong to draw a line and say everything to one side is a disorder, and everything to the other is acceptably healthy.

        However, there does seem to be a popular idea that ADHD is different, that there is an underlying condition that you either have or don’t.

        My friend, for example, says that people with ADHD have different brain structure under a brain scan than people who don’t. I looked into this a little and did not find the evidence convincing, at least not in a way counter to my idea of what ADHD is. He also says people who have ADHD react to stimulants in a fundamentally different way than people who don’t have it. I haven’t investigated this, but I am skeptical.

        I could be wrong. Thought Scott or others might have a strong case to make one way or the other.

        • Ozy Frantz says:

          In college, my friends would informally diagnose people with ADD by giving the people some of their Adderall prescription and telling them to try to take a nap, which is at least anecdotal evidence in favor of that theory.

          • Hainish says:

            Hm…the one time I tried Ritalin, I turned into a zombie (hello, psychomotor retardation!), but I’m not sure it would have made me sleep any better.

          • Matt C says:

            A little. It shows my friend’s idea is common, but isn’t strong evidence that it is correct.

            I am curious, though. Did different people have strongly different reactions when they tried this? What effect was supposed to be a positive indication of ADD/ADHD?

          • Adelene says:

            As another point of anecdata – I have an ADD diagnosis, and while I think I’m better described as autistic I do clearly have some substantial executive functioning/focus problems either way; I sleep just fine on even large amounts of caffeine, and caffeine has been known to put me to sleep if I’m sufficiently tired when I take it.

          • Ozy Frantz says:

            Matt C: Yes. Some people had ordinary stimulant reactions to it, which meant they were totally incapable of sleeping; others could nap fine. The people in the latter category all got diagnosed with ADD when they went to the psychiatrist for it.

          • Matt C says:

            @Ozy: Thanks for both remarks, interesting.

      • Mary says:

        By definition! All psychiatric disorders are checklists of symptoms, because when they aren’t, we stop calling them psychiatric disorders. Why is epilepsy no longer a mental illness? Because it’s no longer defined as thrashing about uncontrollably but by the brainstorm that produces it. Why are certain forms of catatonia no longer called mental illness? Because we have identified the thyroid deficiency behind them. Etc.

        • Matt C says:

          Hmm. I don’t think this is true, or maybe I expressed myself poorly in what I’m trying to get at.

          ADD/ADHD is a set of symptoms that everybody has. Everyone is inattentive sometimes, and most people are hyper at least sometimes (well, most kids, anyway). The symptoms are not considered disorderly in themselves, only in their frequency/intensity.

          Like someone else said upthread, this is similar to depression. Most people have some depressive symptoms, sometimes.

          Something like schizophrenia is different. I don’t know that much about schizophrenia, but I don’t think the symptoms overlap much with what we think of as normal behavior. Most people don’t have auditory hallucinations, at all, for example. You either have schizophrenia or you don’t.

          My friend, and other people, argue that ADHD is a distinct condition, rather than a zone within the normal range of human behavior.

          I am skeptical. ADD/ADHD looks to me like the same kind of personality variation you see in other character traits, but we happen to pay extra attention to this one because it is especially relevant for school and work.

          This doesn’t mean I think ADHD isn’t “real”, but it would change some things I think about it if I turned out to be wrong.

          • Protagoras says:

            Scott is obviously the local expert, but I was under the impression that a lot of people have hallucinations at one time or another, they’re just not normally a problem. Also, I’m pretty sure schizophrenia doesn’t always involve hallucinations. So your clear case may not be as much clearer than ADD/ADHD as you suggest.

          • Matt C says:

            This would be really interesting if true. I’ve never had a hallucination (*), and never heard anyone talk about having them except when using drugs.

            (*) I have ever heard voices when falling asleep, but I think of this as straddling the border between dreams and waking, and not properly a hallucination.

          • Nornagest says:

            ADD/ADHD is a set of symptoms that everybody has. Everyone is inattentive sometimes, and most people are hyper at least sometimes

            I feel unempathic sometimes. Doesn’t make me a sociopath.

          • Matt C says:

            I think it is possible that sociopathy is also a zone within the spectrum of normal human behavior, and not a distinct binary condition. I’ve met people that seemed to me to be moderate sociopaths.

          • Susebron says:

            This would be really interesting if true. I’ve never had a hallucination (*), and never heard anyone talk about having them except when using drugs.

            IIRC, they’re often short enough that people don’t really notice them. Maybe they think they hear or see something for a fraction of a second, and then it’s gone and they just dismiss them. As for more noticeable hallucinations, I think the main causes are things like sleep deprivation or dehydration.

          • Ozy Frantz says:

            Anecdata: I’ve certainly had auditory hallucinations and I’m not schizophrenic. (Mostly imagining I hear the door opening or someone calling my name, I check it, no one is in the house.)

    • Scott F says:

      There is pretty good evidence that there’s a specific binary difference between ADHD and non-ADHD; the metaphor that makes the most sense to me is thinking of your frontal lobes as a leg.

      There are variations in how well someone can use their leg; there is also a very binary “your leg is broken / your leg is not broken” distinction. Stimulants are splint or casts: strap them onto a broken leg and you can walk on it, sort of. Strap casts on a healthy leg and it can support even more weight, but limits the flexibility. So in the same way that a cast affects a broken leg in a fundamentally different way to how a cast affects a normal leg, stimulants have that fundamentally different effect.

      The paradoxical effect of stimulants in ADHD brains is very surprising. One theory for why so many ADHD patients have enormous sleep issues is that they lack the ability to focus on falling asleep; there is always a sleepover party and their frontal lobe isn’t strong enough to declare ‘lights out’.

      Another very interesting thing that stimulants do is improve the woman’s ability to reach orgasm – in order to climax, you need to focus on building up to it. No focus, no climax 🙁

      (The “broken leg” metaphor has a lot of good points: you can develop ADHD symptoms from traumatic brain injuries to the frontal lobe, very common in contact sports. In this metaphor, there is a genetic “bones don’t form properly, leg is always broken” disease that parallels ADHD.

      • Douglas Knight says:

        Could you point to this evidence?

        • Scott F says:

          This may be slightly technical but it is a good overview.

          • Matt C says:

            I found that paper pretty difficult going, but I appreciate the link, and also the earlier comment that I missed at first. Thanks.

            I wonder how these studies would look if done on kids who have hyperactivity but who are not diagnosed ADHD. (Hypothetical question, I assume this isn’t much focused on right now.)

      • jsalvatier says:

        I agree that stimulants having different effects on ADHD vs. non-ADHD would be very strong evidence for a binary sort of thing. The claim I’ve normally heard is that stimulants give ADHD patients more focus and less focus for other people, but it turns out this is just a dosing effect and everyone gets more focus from low dose stimulants. Do you know of any real differences?

        • Douglas Knight says:

          Most people experience modafinil as a weak form of amphetamine, but ADHD people often experience it is qualitatively different, doing some things amphetamine does, but omitting important others. Have you tried it?

    • jsalvatier says:

      I was diagnosed with adult ADHD a while ago because I sought it out. I did some research on this topic, mostly be reading ADHD In Adults (http://smile.amazon.com/ADHD-Adults-What-Science-Says/dp/1609180755?sa-no-redirect=1), and I didn’t see anything to suggest that it was at all binary, which was something I was looking for. I couldn’t even find anything that seemed to distinguish it from just having low Conscientiousness.

      My guess is that ADHD doesn’t really cut reality at the joints, but its useful because having a named disorder is the only way we get to use medication and therapy to treat it.

  13. ari says:

    Once I understood this joint-overdiagnosis-and-underdiagnosis problem, several other candidate situations immediately leapt to mind. Antidepressants are almost certainly both overprescribed and underprescribed. So are opiate pain medications.

    And some criminals go free while some innocent people go to jail, sometimes legitimate messages go to your spambox while some spam gets to your inbox, sometimes you make advances on someone who turns out not to have been interested at all while other times you don’t even though they would have been, sometimes you see faces in the clouds while other times you don’t notice a human behind a bush or a dirty window, sometimes you get phantom vibrations while other times you miss a call, etc..

    I don’t like the idea of saying that in situations like this there’s “both overdiagnosis and underdiagnosis”, at least not based *just* on the fact that some elements of the set you’re looking at get misclassified one way while others get misclassified the other way. It just applies to too many things.

    Say you have an ADHD test that gives back a 0-100 ADHD-ness score, and you have to decide a cutoff for where to actually diagnose people with the disorder. You also have a supercomputer that can simulate the social consequences – it calculates a function from the cutoff to the utility gained by setting the cutoff at that point. Say the cutoff would have been at 50 earlier but you find, examining the graph of the function, that its maximum value – the maximum resulting utility – is where the cutoff is at 60, and that the utility function is concave. In that situation you would know that you would be doing better if it was harder to diagnose people with ADHD, and you would be doing worse if it was easier to diagnose people with ADHD: There’s overdiagnosis, and not underdiagnosis. There might still be some people who should have been diagnosed but weren’t even at the lower cutoff, and that’s sad, but if you wanted to do the right thing, you’d raise the cutoff.

    Similarly in other situations. Our visual system isn’t perfect, but it’s set to have a cutoff for “human!” (as well as “threat!”) that maximised inclusive fitness in our ancestral environment, our spam filters aren’t perfect but they’re tuned to make the users keep using your service, our justice system isn’t perfect but it’s tuned… well, mostly to fit biases that make it succeed politically but work less well in practice, but oh well.

    If the graph had multiple peaks, and you turned out to be in a valley between peaks, I would even say that you’d be justified in saying that there’s *either* overdiagnosis or underdiagnosis, because tuning the cutoff either way would improve things. But that seems unlikely in practice.

    I suppose if you went in one step deeper, you’d consider ROIs, and at that point you could reasonably have both “improving specificity would be worthwhile” and “improving sensitivity would be worthwhile” as facts that can independently be true or false, depending on how much it costs to improve either and how much you benefit from it. I don’t know about medical school, and it didn’t come up directly in my schooling in CS, but I’d hope that at least business schools go over this stuff.

    • Ialdabaoth says:

      I think it’s a matter of degree. If less than 50% of the positives are real, and less than 50% of the real candidates are detected, I think that’s a substantial problem.

      • Anonymous says:

        What is a “problem”? To the consequentialist, the only problem is that you are taking a suboptimal option. It is a problem if your balance of true and false positives could be better, or if you fail to make worthwhile investments in improving your technique. But it is also a problem if you try to improve the technique just because 50% sounds like a big number.

        • Matthew says:

          50% is relevant because if your rate of both type 1 and type 2 errors is greater than 50%, you’re doing worse than you would be just by randomly assigning a diagnosis or non-diagnosis using the (presumed) base rate and 2d10. Worse than chance sounds like a problem to me.

          • Anonymous says:

            Yes, if you define things so that the example is stupid, the example is stupid. And yet the example is taken from the original post, where it is not stupid.

          • Adelene says:

            > you’re doing worse than you would be just by randomly assigning a diagnosis or non-diagnosis using the (presumed) base rate and 2d10

            Uh, no.

            Say ADD occurs in 2% of the population. Of 10,000 people, 200 will have ADD.

            In a scenario like the one Scott described, you might see 95 of the ADDers and 105 of the non-ADDers get ADD diagnoses – less than half of the ADDers are diagnosed and less than half of the diagnosees have ADD.

            But if you just pick 200 random people from the population, only 4 of those people (2% of 200) will have ADD, and the other 196 ADDers go undiagnosed – clearly much worse than the described scenario, even though the described scenario isn’t good enough to get most ADDers diagnosed or have most diagnoses be accurate.

          • RCF says:

            I think you should check out this page: http://en.wikipedia.org/wiki/Confusion_matrix

            Ialdaboath said: “If less than 50% of the positives are real, and less than 50% of the real candidates are detected…”

            The former, TP/(TP+FP), is known as the precision. The latter, TP/(TP+FN), is the sensitivity. Type I, fall-out, or false positive rate is FP/(FP+ TN). Type II, Miss, or False Negative Rate, is FN/(FN+TP).

            So, Ialdaboath said that if the precision and sensitivity are both less than 50%, there’s a problem. You’re defending that by saying that it’s possible for Type I and Type II error rates to be both greater than 50%, and the test to still be useful. That’s a non sequitur.

          • Adelene says:

            Yup, I didn’t say anything about whether there’s a problem or not. My point was solely about the math: 50% isn’t a magic number, especially when you’re dealing with an unbalanced data set.

            (If my point there still isn’t clear – imagine finding a way to accurately guess lottery numbers 10% of the time. 10% is certainly less than 50% but you’d still end up very rich.)

        • Jaskologist says:

          Failing to help people who need help is a problem. Harming people by giving them “help” they don’t need is also a problem. This is true whether or not we are currently capable of doing a better job with an acceptable ROI. If we aren’t, well, now you have three problems.

    • Mary says:

      it’s set to have a cutoff for “human!”

      This may amuse
      http://freefall.purrsia.com/ff1600/fc01585.htm

  14. social justice warlock says:

    Not all the relevant examples are medical. I was reminded of Athrelon’s recent attempts to explain to me his version of the far-right concept of anarcho-tyranny. At first this didn’t make sense to me – how could there be anarchy and tyranny at the same time? Athrelon was able to walk me through the logic, which it turns out is the exact same as above. Imagine the government as trying to “diagnose” the situations where it needs to use force, and over- and under- diagnosing them at the same time. He will make this into a blog post soon, and I will link you to it.

    I.e., anarcho-tyranny is any government whose policies you dislike, or as it’s usually called, “tyranny.”

    • Silver2195 says:

      Something like that, although what people who talk about “anarcho-tyranny” generally really mean is that they think the government is too harsh towards white people and too lenient towards non-white people. Actual statistics showing that the criminal justice system (in both the US and UK) is not particularly favorable to non-whites are met with the standard neoreactionary argument that Campbell’s Law means anecdotes trump statistics.

      • James James says:

        Silver2195, could you provide some of those stats?

        Silver2195 and social justice warlock, yes, it is saying that the government doesn’t have its priorities right. It seems to me to be poor priorities if a government prosecutes people for using the wrong units (https://en.wikipedia.org/wiki/Thoburn_v_Sunderland_City_Council), but takes three days to put down riots (https://en.wikipedia.org/wiki/2011_England_riots).

        The term “anarcho-tyranny” is implicitly in favour of order, implicitly against anarchy. You, however, may agree with these priorities.

        • Andy says:

          Prosecuting someone for using the wrong units is easy when all parties recognize the authority of the government to do so. Quelling a riot without simply killing everyone involved or making it worse is a much more complicated affair, and the two aren’t usually handled by the same people. I am calling bullshytte on using these two examples as an example of poor priorities, as they aren’t at the same time – 10 years apart! – and not using the same assets.
          I do, however, eagerly await the logic behind anarcho-tyranny. It looks fascinating.

          • Mary says:

            How about arresting fathers from trying to take back their minor daughters from an organized band of rapists? How about arresting girls for public drunkenness when you find them in the company of aforementioned band? How about finding the time to declare a couple that voted for a party are unfit foster parents while the band is raping hundreds of underage girls?

            Rotherham is an anarcho-tyranny.

        • Silver2195 says:

          http://lmgtfy.com/?q=racial+disparities+in+sentencing

          Mary is right that Rotherham does look an awful lot like “anarcho-tyranny” actually happening, though.

          • Azathoth123 says:

            Nice trick there with expecting your debating partner to Google the evidence for your side.

            That way even if they do as you suggest and find a site describing racial disparity in sentencing, and poke holes in it’s statistics, you can accuse them of strawmanning.

          • RCF says:

            +1 on criticizing lmgtfy cites. It’s the ultimate extreme of a Gish Gallop, and with an extra dash of condescension. I’ve reported it for being neither kind nor necessary. And my position is that the proper response to a Gish Gallop is to take the first response and treat any refutation of it as being a refutation of them all. In this case, the first link is http://www.sentencingproject.org/template/page.cfm?id=122 , which simply asserts that black people are imprisoned more often than white people, and makes no attempt to show that black people do not commit more crimes. In addition, it does a poor job of citing its sources. And from now on, it’s perfectly legitimate for anyone disagreeing with something you say to google the claim, take the first response, refute it, and treat your claim as refuted.

      • B says:

        There’s no need to bring race into it as an explanatory factor, and given its toxic nature as a topic in the US, imputing such motives just shows ill will re: your opponents in this debate.

        Anarcho-tyranny manifests just as much in finance where reporting limits for transactions only ever get reduced, to the point where you’ll, hyperbolically speaking, have to bring AML docs to the deli in a few years but front-running is perfectly legal as long as you do it using a computer (“HFT”).

        Finance is not exactly the land of the NAMs.

        • social justice warlock says:

          Paleocon/alt-right attitudes towards finance have an ethnic component as well.

          • B says:

            Yes. Many things to be debated there, in all directions, but:
            Grampa (generationally & ethnically speaking; luckily not mine personally, AFAIK) had a bit of a criticality excursion there and I guess I’ll leave that one to other people, and peoples.

        • Ariel Ben-Yehuda says:

          I’m not sure how HFT is front-running ‒ the typical problem with front-running is a broker stealing from his clients, while HFT mostly compete with automated trading algorithms.

          • Silver2195 says:

            Read Flash Boys if you’re interested. Basically, HFT automates the front-running.

            Edit: I should have been clearer. HFT isn’t inherently front-running; that’s just what it tends to be used for in practice.

          • Douglas Knight says:

            It isn’t front-running. Yes, I have read flash boys.

          • B says:

            Yes, that’s a valid criticism as far as terminology goes; HFT isn’t front-running in the sense that it’s not a violation of fiduciary duty to one’s clients like classical FR was.

            But IMO, it’s semantically justifiable to describe it as FR:

            You used to be only able to front-run your own clients since that’s how information flowed in actionable time frames. This has changed.

            FR is fitting as a term here if you see the “timing attack” as the core of the issue of front-running, which the name itself seems to imply; if the violation of fiduciary duty is the core of the issue, maybe insider trading would be the better analogy: Acting on unfairly obtained data in a way to overtake & hurt legitimate activity.

            Of course, it’s all legal, but that’s exactly the point that makes it relevant to the anarcho-tyranny debate.

      • no one special says:

        what people who talk about “anarcho-tyranny” generally really mean is that they think the government is too harsh towards white people and too lenient towards non-white people.

        It seems that what might be a useful term is overwhelmed by group association with crazy “America is a white country” racists.

        I’d like a term that lets us talk about why everyone can break the speed limit, but people get arrested for letting their kids play in the park. Or why some people can smoke pot in their own homes without fear, while other people get stop-and-frisked for walking down the street in their neighborhood.

        Anarcho-tyranny might be a good term for that, but using it will even more strongly signal association with a group that I disagree strongly with. 🙁

        • Mary says:

          What’s worse is that people will make that association in bad faith to avoid talking about issues they don’t want to talk aobut.

      • Leonard says:

        Although your statement is loosely true, you make it about race when it is not. This is either your ignorance of (or blindness to) racial differences in criminality, or bad faith.

        Anarcho tyranny is the idea that the government is too lenient with criminals, while it is too strict against law abiding subjects. (Note that there is a huge difference between “criminal” and “black”, contra your assertion that your enemies conflate them.)

        There is often a causal link between the “anarchy” and the “tyranny”. It is because our rulers refuse to control criminals adequately that they crack down on everyone else. Unchecked victimization causes fear, and with politicians’ jobs on the line, something must be done. Thus, for example, criminals kill people with guns; the state refuses to kill criminals any more, and imprisoning them is very expensive. So all must lose their gun rights. Arab Muslims are our frenemies du jour, but the government refuses to apply any more scrutiny to them than anyone else, much less simply stop granting them visas. So every American must to submit to wasteful and degrading searches at airports, and every American is now subject to 24/7 electronic surveillance by the NSA.

        • Anonymous says:

          That is a very different set of examples than James James’s example of prosecuting people for using imperial measures, which is typical of the usage I see.

          • Azathoth123 says:

            Those are also examples of anarcho-tyranny.

            You may want to think about why it seems to you that people using the wrong units is sufficiently bad to warrant prosecution.

          • Anonymous says:

            What leads you to believe that I believe that?

          • Azathoth123 says:

            If you don’t then what do you have against the “prosecuting people for using imperial measures” example?

          • Anonymous says:

            The difference between Leonard’s examples and the imperial measure example is that Leonard proposes that his examples of tyranny are connected to the anarchy, in contrast to James James’s examples, which seem to be two unconnected complaints about government.

      • drethelin says:

        Actually cops pulling over black guys for driving a car in the wrong neighborhood but not eliminating black gang violence is a perfect example of anarcho-tyranny. Somehow racism is enough to get a black guy pulled over but it’s not enough to get them not to have wide-spread, widely known networks of drug salesmen on street corners? In fact, the problem of white guys not getting arrested for smoking pot and black people getting arrested is ALSO anarcho-tyranny. It’s about capricious and unreliable law and order.

    • MugaSofer says:

      Not quite – most people don’t call a government they think is to lenient “Tyranny”.

      Of course, they don’t call it “Anarchy” either – I think the term is deliberately confusing/contradictory.

      • Andy says:

        The crux seems to be that the tyranny is too much on the wrong people.
        The term makes sense to me even though I think it’s completely wrong, and the thinking behind it is utterly stupid.

      • Illuminati Initiate says:

        Not sure about governments they see as too lenient, but I have heard people refer to government policies that reduce their power to hurt others as “Tyranny”, i.e reactions to child rights type stuff and providing needed medicine against abusive parents’ wishes.

        Of course tyranny originally just meant a government that came to power illegally, which has nothing to do with how it’s used today.

        • Azathoth123 says:

          > reactions to child rights type stuff and

          What kind of “child right” type stuff? Is your claim that children are mentally competent to exercise adult rights?

          > providing needed medicine against abusive parents’ wishes.

          Who decides what medication is needed? Who is more likely to care about a child. His parents, or some government bureaucrat? Or would you consider Justina Pelletier’s parents an example of abusive parents?

          > Of course tyranny originally just meant a government that came to power illegally, which has nothing to do with how it’s used today.

          A government that came to power illegally has just established a precedent that it’s possible to overthrow the government and succeed. Thus to avoid suffering the fate of the government it just overthrew, it has a strong motivation to be tyrannical in the modern sense.

          • Illuminati Initiate says:

            Of course I don’t think children are mentally competent enough to exercise all adult rights. But they certainly should be able to exercise some, for example, the right to “free speech”, which they are routinely denied. Obviously children need to have their actions to some extent constrained by their legal guardians but I think there should be legal limits to how restrictive it can get. For an example of the people who want to keep their power to harm children, see some of them’s (grammar?) website here http://www.parentalrights.org/ (I AM NOT ENDORSING THAT WEBSITE I AM SAYING THEY ARE BAD).

            As for the medicine thing I think their should be guidelines as to what type of medicine and when young children should be required to take. The status of various vaccines as mandatory for should be decided on a case to case basis*, and any medicine that has a reasonable chance of being life saving and/or long-term debilitation-preventing should be given regardless of what the guardians say. Older minors should have some level of say in their own medication. Obviously I don’t know all the details of how this should be set up, I’m not a bureaucrat.

            I didn’t know who Justina Pelletier was so I just Googled her. From my quick glance I don’t have the information to judge that specific case and can’t really find enough. But these parents are certainly abusive: http://www.ohio.com/news/local/amish-family-flees-to-avoid-chemotherapy-for-girl-with-cancer-1.448261 . As are parents who refuse to give their children important vaccinations.

            As to overthrowing governments, if there is a group with enough power and motive to possibly try then there is motive to act like the modern meaning of tyranny. “Precedent” seems largely irrelevant. Also I would like to point out that many government throughout history have acted like that despite being in power for a long time and not being very threatened. And almost all governments started out as tyrannies, exceptions include some peaceful secessions (like what Scotland just tried).

            *Edit: “case to case basis” meant between different vaccines. Though actually exceptions should be made for vaccine allergies.

          • Illuminati Initiate says:

            Looking at the Justina story again the hospital was certainly in the wrong there, but something seems fishy about that whole story… I don’t understand why the hospital felt like they needed to take her away to treat psychosomatic illness? And if there is zero evidence of abuse why was this allowed to continue? There does not seem to be much good sources on the case. It seems like something is being left out.

            I actually do have a problem with forced psychiatric drug use regardless of age or mental competence (except maybe in very extreme circumstances, though as a consequentialist I could say that about anything), and think they should be treated differently from “physical” (ugh that sounds so duelist) medicine. I missed the psychiatric drug part when I first read the story. Though in this case I think whether or not to take psychiatric drugs should have been up to Justina rather than her parents or doctors.

      • peterdjones says:

        If they had some objective way of better separating the too much from the too little, they would be on to something. Otherwise it’s just a high status way of “I should be able to do what I like, but you shouldn’t be able to what you like”.

        • Azathoth123 says:

          If “what I like” happens to be “use whatever units I want” or “drink raw milk” and “what you like” is “go around rioting and smashing stores”, I’d say there is a very good case that I should be able to do what I like and you shouldn’t be able to do what you like.

        • RCF says:

          It’s quite a jump from “You are unable to articulate an objective standard differentiating these two categories” to “You are categorizing merely on naked self-interest”.

  15. Nestor says:

    The Spanish saying “No son todos los que están ni están todos los que son”, which is often applied to mental asylums but works in many contexts, is apropos.

    “Not all who are here should be and not all who should be here are”

  16. blacktrance says:

    While in the literal sense it’s possible for ADHD to be simultaneously overdiagnosed and underdiagnosed, I think that “Is ADHD overdiagnosed/underdiagnosed?” is a disguised query that really means “Should the standards for ADHD diagnosis be stricter or laxer?”.

  17. LRS says:

    When we began discussing the crime of rape in my criminal law class during my first year of law school, my professor kicked off the discussion by saying something along the lines of “rape is the most underreported crime and the most overreported crime.”

    I’m sure Scott is grateful that I’ve brought up this controversial gender-related issue in the comments of a psychiatry post.

  18. Ozy Frantz says:

    Possibly contributing to the overdiagnosis/underdiagnosis problem:

    Obtaining a diagnosis is pretty hard: it involves skills like leaving the house, making appointments, showing up to appointments, talking to people, not staying in bed all day, having money, etc. that many people with mental illnesses are bad at. So people who don’t have mental illnesses have more of the skills required to get a diagnosis than people who do.

    College disability services approach the worst-case scenario, in which they have successfully managed to filter out all the disabled people and accommodate only the fakers.

    • coffeespoons says:

      College disability services approach the worst-case scenario, in which they have successfully managed to filter out all the disabled people and accommodate only the fakers.

      This is interesting – could you expand on this?

    • Error says:

      Aren’t most ADHD people noticed as kids? Those obstacles are typically solved by parents paying for and dragging the patient to appointments, I think.

      College disability services approach the worst-case scenario, in which they have successfully managed to filter out all the disabled people and accommodate only the fakers.

      That’s horrible. -_- Probably true, though.

      (ETA: Checking my understanding, you’re suggesting that most people with such problems don’t make it to college, hence aren’t among college disability services’ patients, right?)

      • Ozy Frantz says:

        No, I’m saying that IME of college disability services most people who are sufficiently disabled to deserve accommodations are not capable of convincing the college disability services that they deserve accommodations, because it requires things like “being persuasive” and “having documentation” and “managing to make an appointment”. (Note that my experience is mostly with people on the ADD/depression/anxiety side of things, rather than the learning disability side; I don’t know if dyslexics actually manage to get properly accommodated.)

        A fair number of mentally ill people make it to college. Partially because you can make up for mental illness by being smart, partially because (as you said) parents make executive functioning issues easier to deal with, partially because college is hella stressful and so something that was not a big issue in high school becomes a huge issue in college.

        • Error says:

          I stand corrected. Thanks.

          partially because college is hella stressful

          I’m not objecting, but I’ve never understood this. College was less stressful for me than grade school, not more. Grade school was hell.

          (I ended up dropping out of college, but it was because of crippling akrasia, not stress)

        • anon1 says:

          Yes! I managed to make appointments and everything, got into therapy, and whoops turns out I was too anxious to get across to my therapist that I had anxiety problems (with pretty serious academic consequences, like starting problem sets at midnight the day before they were due and being way too scared to ask TAs/professors/anyone who wasn’t floundering as much as I was to explain anything) so what’s the point? Though I think things are much better (in some regards) on the physical disability end of things, like can we set up this lab so you can do it from a wheelchair.

          • RCF says:

            I feel like the way mental health support is offered is like having a support desk for people in wheelchairs on the third floor of a building with no elevator.

        • papermachine says:

          In my limited experience (n=500 or so) dyslexics on one particular college campus get somewhat accommodated, in the sense that I observed between 10 and 15 people with official documentation.

          Accommodations included designated note takers and/or recorded lectures, verbal clarification of quizzes and exams, and extra time allotted for each.

        • mareofnight says:

          Perhaps it depends on the college? Anecdata for my college is people who get as far as asking for help usually can get something – but then, my sample doesn’t include people too shy to have made friends and such.

      • Hainish says:

        “Aren’t most ADHD people noticed as kids? Those obstacles are typically solved by parents paying for and dragging the patient to appointments, I think.”

        Not if you’re primarily inattentive, female, and get good grades most of the time. Esp. not if your parents are unfamiliar with psychology/psychiatry.

  19. Matthew O says:

    Before I started substitute teaching, I used to think that ADHD was just a social construct.

    Then I started substituting in some special ed classes. Whoa boy. Let me tell you, ADHD is real. It is utterly, utterly, dreadfully real.

    You know what my amateur non-psychologist opinion thinks they could really benefit from, in addition to Ritalin or Adderall? Some combination of:
    1. Silent yoga. Holding poses for up to a minute silently. That would help attention and physical fitness.
    2. Silent meditation while standing (to avoid accidentally falling asleep. Because I have noticed that with these kids, the minute they stop being hyperactive and crazy, they crash). Try to teach them to passively introspect on their own thoughts non-judgmentally and come to a better understanding of how their own thought processes work. That would also train their attention.

    Even getting the kids I’ve worked with to stay on-task at these things for 10 seconds at first without hooting like an animal (a strangely common behavior, I’ve noticed) or picking on each other or blurting out random thoughts or fidgeting around would be a miracle. And maybe over time you could steadily increase the length.

    • I have an ADHD diagnosis. It may be incorrect, and I am a sample of one, but silent meditation is scary for me. I can’t do it and I cry and panic about not being able to do it. (Confounders: I am also clinically depressed. And I have an extremely complicated and angsty relationship with religion.) Meditation where I am allowed to say words over and over works better.
      But, saying we should treat ADHD by making people stand still seems like saying we should treat short-sightedness by making people stare at far-away objects. I understand that neuroplasticity exists in a way oculoplasticity doesn’t (which is why just I had to make up that word) but it’s still… making people do a thing they can’t do? Maybe what you are talking about is very very gradual increases in demands, in which case it would work.

      • Anonymous says:

        When I want to be able to do something I can’t do, I do a little bit of it at a time until I can do it.

        Is there another way of learning a skill?

        • drethelin says:

          If someone is missing a leg, they are not in need of practicing the skill of having two legs. They are in need a prosthetic leg.

          The point of ADHD as diagnosis is that no amount of practice will give them the ability to sit still in silence for a minute. It’s not a thing they can gain through practice.

          • Douglas Knight says:

            How do you know what is the point of a diagnosis?

          • Scott F says:

            I want to strongly emphasise how correct drethelin is here. Although, it is less like practicing the skill of having two legs and more like practicing the skill of walking. If you have two legs but aren’t good at walking, then lots of practice walking will probably help. But if you are missing a leg, practicing walking is useless. You need a prosthetic leg, and then you need to practice the different skill of walking with one prosthetic leg, and when you haven’t got your prosthetic leg on you still won’t be able to use that skill.

          • Adelene says:

            Also – speaking as someone with substantial executive function issues who’s functioned unmedicated for the last decade – there is a ‘get around by hopping on one leg’ skill, but it’s still not walking and learning either sort of walking skill won’t help with it. (Being able to sit quietly for a few minutes might make a kid easier for a teacher or parent to put up with, but it’s not actually a skill that helps the kid accomplish anything – not even listening, since they’re spending so much of their attention on suppressing their impulse to move around that there isn’t enough left to do that well at all.)

          • Nick T says:

            How can someone who has some trouble walking find out whether they’re missing a leg or just need to practice walking more?

        • “How can someone who has some trouble walking find out whether they’re missing a leg”
          I know you were speaking metaphorically, but I feel the need to point out that there are some very easy ways to discover how many legs you have!
          Anyway, suppose we live in a hypothetical world where an inept wizard accidentally cast a magic spell which made it impossible for all people to count their own legs and nobody knows how to break the spell. Let’s further suppose that the spell makes it impossible to count anyone else’s legs for them, either. So people have to guess how many legs they have based on things like “how well can you walk?” “have you had any injuries which could plausibly have caused you to lose a leg?” and “do you benefit from a prosthetic leg?”
          In this hypothetical world, people guess badly. There are people who have walking problems (maybe caused by arthritis or cerebral palsy or CFS or EDS or any number of things) but who nevertheless have two legs, who go around with a prosthesis that constantly gets in the way, and every time they ask their doctors if they should try walking without the prosthesis, they are told “I know you are struggling now, but you would be even worse off without this.” There are people who have one leg AND have another walking problem (like arthritis etc) and who get a prosthesis, notice they still have trouble walking, and get rid of the prosthetis and are even worse off. There are people who beg their doctors for prosthetic legs and are dismissed as attention seekers. There are people hopping around everywhere on one leg because it never occurred to them that one of their legs might be missing, and assuming the fact they get around so slow is because they are somehow morally deficient. There are people who fake having one leg so they can get a prosthesis to sell on the blacjk market. There are people who genuinely need prosthetic legs but are poor and end up selling them on the black market anyway. There are people who are pretty damn sure they need a prosthesis but can’t walk to a doctor’s appointment. There are people who try one kind of prosthetic leg but would be better off with a different kind, but give up after the first kind failed. There are people who tried one kind of prosthetic leg, found it didn’t work and desperately want to try another kind but live in a country where only one kind of prosthetic leg is approved. There are people, both those who need prostheses and those who don’t, grinding up prosthetic legs and snorting them to get high. (This is a universe with wizards, I see no reason for it not to also be a universe where you can get high on legs.)
          In the long term, the solution is to learn how to break the spell that makes leg-counting impossible. Short and medium term solutions are harder, but “You have trouble walking? I know a solution to that? WALKING!” is the opposite of helpful.

          Also, I would like to end with a quote from one of my favourite Doctor Who episodes
          Elderly woman: Doctor Constantine?
          CONSTANTINE: Mrs Harcourt. How much better you’re looking.
          HARCOURT: My leg’s grown back. When I come to the hospital, I had one leg.
          CONSTANTINE: Well, there is a war on. Is it possible you miscounted?

      • RCF says:

        If people with ADHD can’t sit still for one minute, then it should be rather easy to tell the difference between them and neurotypical people. I find this to be an extraordinary claim.

  20. rkj says:

    Hmm.

    If you’re willing to pay a higher cost (more time, more effort, more money), there are ways to the size of one or both errors. For example, you could require multiple independent diagnoses or you could try to change the pool of people getting diagnosed (as others have suggested).

    But I suppose you first have to decide on how big of a problem these errors are and which error is worse. Is there any consensus on that?

  21. anon says:

    As someone who thinks he has ADD and was diagnosed with ADD as a child, a big barrier to getting medication has been psychiatrist fees. But if you have money, finding a doctor who will prescribe medication after one or two visits where you recite a bunch of symptoms seems easy.

    This effect could also just be due to my being poor and having a 125 IQ and actually having ADD.

  22. Shmi Nux says:

    I recall seeing a graph like this once (reconstructed by memory): http://i.imgur.com/cLFqglZ.jpg

  23. cassander says:

    I’ve been taking stimulants for almost two decades now. I have received a variety of diagnoses from a variety of doctors, ADD, Aspergers, Autism (I suddenly notice they all start with A). I tend to describe myself as “a little Autistic” because people are familiar with the term and because I have a vague notion that ADD isn’t taken seriously. The truth is, though, that I have no idea what I am and don’t much care. I know stimulants make me more functional and I know I respond very differently to certain stimuli than normal people, and I know psychiatrists/psychologists are quacks, but still helpful.

    I like to compare modern psychology to medicine 50-60 years ago. Like shrinks, doctors in the 50s had some miracle drugs, but lacked fundamental understanding of how the body really operated. for example, DNA was only proven to be responsible for heredity in 1952. That does not mean, however, that Penicillin didn’t work. I absolutely believe in better living through chemistry. shrinks are even further from understanding than 50s era doctors were because the brain is more complicated than the rest of the body put together and harder to study. We need to resist the impulse to categorize, because categorization requires far more knowledge than we have. I do not have ADD, Autism, or Aspergers, those things probably don’t exist. I am someone who responds positively to stimulants and negatively to Bupropion.

    • Protagoras says:

      Huh. I also respond positively to stimulants and negatively to Bupropion. I’d gotten the impression that responding negatively to Bupropion was extremely rare. And probably it is, it’s just that anecdotes like this stand out for me because of my personal experience. And probably we shouldn’t be too hasty to think responding positively to stimulants and negatively to Bupropion is a meaningful category, as we know people are over-hasty category-generators.

      • Hainish says:

        ” I’d gotten the impression that responding negatively to Bupropion was extremely rare.”

        Really? My impression is that responding negatively to Bupropion is practically a cliche. (It made me angry and near-violent the one time I tried, and it made a friend suicidal.)

      • cassander says:

        positive to stimulants and negative to Bupropion is at least a category firmly grounded in empirical reality. maybe it’s diagnostically useful, maybe not, but it’s certainly better than someone else’s vague notions of why I do what I do.

  24. Cliff says:

    Unfortunately there is no such thing as ADD or ADHD, these are just words used to put a medical diagnosis on personality traits. Do these people benefit from Adderall? Quite possibly, but let’s not pretend they have a disease. After all there is no condition that causes ADD or ADHD. So what does it even mean to say someone does or does not have ADD or ADHD?

    • Nornagest says:

      You’ve just defined away the entire concept of personality disorders.

    • Ozy Frantz says:

      Let’s taboo “disease.” Do you disagree with:

      1) There are people who are vastly more inattentive and impulsive than ordinary people are and who cannot change it.
      2) These people have more difficulty with ordinary life than most people do.
      3) This is probably a product of their brains somehow because all personality traits are the product of brains.
      4) These people are usually identifiable to people who know what sort of traits to look for.
      5) They may have a considerably easier time dealing with ordinary life, with few to no drawbacks, if they are given medication, accommodations, and/or behavioral therapy.

    • Eric Rall says:

      They are words that describe a cluster of behaviors and traits that appear to often occur together, and which tend to respond positively to certain treatments (not just medication, but also coaching of coping techniques). Giving a name to the cluster helps identify people who are particularly likely benefit from those treatments.

      • Cliff says:

        The point is that there is no way to definitively say who has ADD/ADHD and who does not because the definitions are made up and have no relation to any disease state or known physical cause. How can you even talk about false positives and false negatives? When 10 psychiatrists diagnose the same person they often come up with 10 different diagnosis.

        It’s like if you have a medical diagnosis for “jealous personality” or “sensitive personality” or “shyness”. Is that going to be under or overdiagnosed? Is that even an intelligible question? It’s subjective. Even if a person does not respond well to medication, that doesn’t mean they don’t have ADD since it is only defined by its symptoms. Vice versa if someone is benefited by drugs they don’t necessarily have any disorder. Plenty of shy people benefit from alcohol, or ecstasy or cocaine or xanax, it doesn’t mean they have social anxiety.

        • Matthew says:

          I used to be shy and have social anxiety.

          Now I am not shy and have social anxiety.

          Some other people are shy and do not have social anxiety.

          These are, in fact, distinct things.

        • Ozy Frantz says:

          “When 10 psychiatrists diagnose the same person they often come up with 10 different diagnosis.”

          Citation please?

          • Scott Alexander says:

            Well, the research I know about says that there is inter-rater reliability of over 80% for the most common psychiatric diagnoses, which is equivalent to reliability in most other medical fields. But I’m sure Cliff is working off equally rigorous research which he will be happy to cite for us.

          • Cliff says:

            No, I was working off of my memory of things I have read, probably mostly journalistic summaries of studies. However, when the disorder is defined by a certain arbitrary number of symptoms listed in a book, which can only be determined by interviewing the patient and asking them if they have those symptoms, you should be able to get a very high level of inter-rater reliability, shouldn’t you? That does not really address my point.

          • Ozy Frantz says:

            If inter-rater reliability is entirely unrelated to your point why did you bring it up in the first place?

            If I understand your model correctly, it would argue that before the germ theory of disease smallpox would be made up, because it would not be related to any known physical cause, and therefore we could not diagnose people with smallpox and it would be meaningless to talk about underdiagnosis or overdiagnosis. Is this true?

          • Cliff says:

            I didn’t spend a terribly long time putting together my comment. It did seem related to me at the time but in retrospect not so much.

            Anyway, no. Smallpox would obviously not be made up (not sure if they did or could come up with the idea of that disease without germ theory?). But if you could only diagnose it by asking patients about their symptoms and there was actually no way to know whether someone had that or a completely different disease or nothing at all, then YES. It would make little sense to ask about over or under-diagnosis.

    • Scott F says:

      Frontal lobe damage as part of traumatic brain injury is a condition that causes all the symptoms of ADHD.

      ADHD itself is more than just a collection of symptoms: it is a label for a theorised brain deficit centred on the frontal lobe, that causes executive dysfunction exhibited as [list of all those ADHD symptoms], and is strongly genetic.

      I think society at the moment actually has a huge problem with pretending that ADHD sufferers don’t have a disease. People seem to want to cling to calling us lazy and unmotivated, in a way that they don’t try to cling to calling anxiety sufferers uptight or depression sufferers whingers.

      • Cliff says:

        A “theorized” brain deficit because no one can find a real one? Is it more or less inheritable than other personality traits? I personally do not believe people with ADD/ADHD are “lazy”, that was not my point. My point was that this discussion of over and under-diagnosis is pseudo-scientific nonsense at this point.

  25. Douglas Knight says:

    Here are three hypotheses that come up repeatedly in the comments and people seem to treat them interchangeably, while I don’t think they tell much about each other:

    1. ADHD (or anything) has neurological correlates
    2. it has a bimodal distribution, rather than being an arbitrary tail
    3. long term change is not possible

  26. Ted says:

    I look forward to the day when people seek psychiatric treatments as often as they seek dental treatments.