Why Doctors Think They’re The Best

Ninety percent of drivers think they’re above-average drivers, ninety percent of professors think they’re above-average professors etc. The relevant studies are paywalled, so I don’t know if I should trust them. Our recent discussion of therapy books would make more sense if ninety percent of therapists believed they were above-average therapists. I don’t know about that one either.

But I am pretty sure ninety percent of doctors believe they’re above-average doctors. Here are some traps I’ve noticed myself falling into that might help explain why:

1. Your patients’ last doctor was worse than you. Think about it; if somebody has a good doctor, they’ll stay with them, and you will never see that patient. If somebody has a bad doctor, they’ll go see another doctor instead. That other doctor might be you. So your current patients’ last doctor will be worse than average. But this is where most of your chance to compare yourself with other doctors comes from: “my patient’s last doctor misdiagnosed them, but I got it right” or “my patient hated their last doctor but says I’m much better”. See also You Are Not Hiring The Top 1%.

2. Your patients love you. Similar to the above: if your patients love you, they will stick with you. If your patients hate you, they’ll leave. So over time, your practice will consist of patients who are very happy with your style of practicing medicine. Suppose some patients like very practical/efficient doctors, and others like very touchy-feely doctors. A practical/efficient doctor will collect a set of patients who prefer practical/efficient doctors and who praise them for how focused and sensible they are. A touchy-feely doctor will collect a set of patients who prefer touchy-feely doctors and praise them for how caring and personable they are. It will be obvious to the practical/efficient doctor that they’re satisfying their patients’ preferences much better than the touchy-feely doctor down the road would, and vice versa.

3. Patients often come to you, but never leave you. When a patient transfers to me, I have to review the case, confirm that this is an appropriate case for me, talk to the patient about why they left their last doctor and what we can do differently, and then keep seeing that patient. When a patient leaves me, I usually have no idea. It usually looks like the patient saying “I’ll make an appointment for next month”, and then not making that appointment. If I’m not careful, I never notice this; it’s a dog that doesn’t bark. Even if I do notice, patients stop coming all the time for all sorts of reasons. Sometimes they forget. Sometimes they feel better and decide they don’t need you. Sometimes they move out of state and don’t tell you. Sometimes they lose insurance and can’t afford you, or change insurances to one that doesn’t cover you. If you’re really on top of things you try to call these people and ask what’s going on, but sometimes they don’t answer and sometimes they lie (it’s really awkward to tell a doctor that you’ve fired them). So it naively feels to me like patients switch from the doctor down the road to me all the time, but nobody ever switches from me to the doctor down the road. I’m sure it’s not true, but that’s how it feels.

4. You’ve probably successfully treated most of your patients. Now pull all of the above together. Suppose a patient has a chronic disease like depression or diabetes. If you treat it successfully, they will love you and stay with you; if you fail, they will switch to another doctor (and you will never hear about it). Ten years later, you wake up and notice that most of your patients are success stories. But your patients usually describe their previous doctor as a miserable failure. Selection bias is a heck of a drug.

5. You know what you know, but you don’t know what you don’t know. Suppose each doctor makes errors at the same rate, but about different things. I will often catch other doctors’ errors. But by definition I don’t notice my own errors; if I did, I would stop making them! By “errors” I don’t mean stupid mistakes like writing the wrong date on a prescription, I mean fundamentally misunderstanding how to use a certain treatment or address a certain disease. Every doctor has studied some topics in more or less depth than others. When I’ve studied a topic in depth, it’s obvious to me where the average doctor is doing things slightly sub-optimally out of ignorance. But the topics I haven’t studied in depth, I assume I’m doing everything basically okay. If you go through your life constantly noticing places where other doctors are wrong, it’s easy to think you’re better than them.

6. Your victories belong to you, your failures belong to Nature. Sometimes I get a really difficult case, something nobody else has been able to figure out – and I absolutely nail it. I ride the high for days. I feel like a miracle-worker. Other times I get a difficult case nobody else has been able to figure out, and I can’t figure it out either. I don’t worry too much about it – some things are beyond the ken of modern medicine; obviously nobody can treat untreatable stuff. This is especially true in psychiatry. There are so many things we don’t understand, so many cases that are just inherently unresponsive to any kind of treatment we know about – that it’s hard to beat yourself up over any given failure. When you fail, you just say “I guess that was one of those treatment-resistant cases”. Maybe if I was a slightly better doctor I would have been able to figure out a way to treat it – but I will never know, and that’s not a natural way to think.

7. You do a good job satisfying your own values. Everybody wants to make people healthy and save lives, but there are other values that differ between practitioners. How much do you care about pain control? How much do you worry about addiction and misuse? How hard do you try to avoid polypharmacy? How do you balance patient autonomy with making sure they get the right treatment? How do you balance harms and benefits of a treatment that helps the patient’s annoying symptom today but raises heart attack risk 2% in twenty years? All of these trade off against each other: someone who tries too hard to minimize use of addictive drugs may have a harder time controlling their patients’ pain. Someone who cares a lot about patient autonomy might have a harder time keeping their medication load reasonable. If you make the set of tradeoffs that feel right to you, your patients will do better on the metrics you care about than other doctors’ patients (they’ll do better on the metrics the other doctors care about, but worse on yours). Your patients doing better on the metrics you care about feels a lot like you being a better doctor.

Some of these seem to generalize; do other jobs where customers select someone to work with have the same problems?

This entry was posted in Uncategorized and tagged , . Bookmark the permalink.

147 Responses to Why Doctors Think They’re The Best

  1. RC-cola-and-a-moon-pie says:

    And this isn’t comparative but the fact that many illnesses are of such a duration that they spontaneously get better or go away on a time frame that can often seem like a successful response to treatment has to inflate perceptions of (absolute, not comparative) efficacy on the part of many kinds of physicians. Even though these kinds of things are controlled in the trials it must be hard to keep constantly in mind in daily practice.

    • MawBTS says:

      Yes. Health regresses to the mean. If people go to the doctor when they’re particularly sick, they’ll probably feel better soon regardless of what the doctor did or didn’t do (unless they’re dying).

      The medical profession was useless for centuries: a 16th century doctor could have done nothing to help almost any health complaint you can name. People still paid, though. The medicine seemed to be working.

      • A lot of medicine in thr 16th century and earlier was nutrition, based on the theory of humors. Has anyone gone over it in detail to figure out whether, on average, the advice based on that theory was good?

  2. RLM says:

    And all this is amplified by your reputation and network of friends. They eventually get a feel for what sort of person you are and only present you with opportunities that it seems like you will like / can handle. They silently ignore you for other opportunities and chose others who are better suited. (Think things like running conferences, dealing with sensitive issues, etc.) So, just like silently losing patients, you also silently lose some opportunities while being presented with many opportunities which you know that you’re better qualified to handle than other doctors you know. It feels like you’re the one people turn to and think of to solve the hard problems, and you know that your friends would be worse at solving these problems than yourself!

  3. Nicholas Weininger says:

    Managers are subject to a lot of these same dynamics and probably prey to similar overestimation, especially those of us who manage skilled professionals with a lot of choice of managers. One saving grace for us is that employees can typically give much better-informed and considered feedback than I imagine patients can since being an employee is a less episodic condition. Soliciting useful feedback from one’s reports is one of the most difficult and important tasks for a manager dedicated to self improvement.

    • Simon_Jester says:

      It occurs to me that the converse of this helps explain the Dunning-Kruger effect among managers… but also serves to protect it. Bad managers tend to be very effective at making it prohibitively difficult to tell them they’re doing a bad job. Managers who are only holding onto their positions through favoritism likewise, since the people you’d be reporting their mistakes to tend to be the same people who favor them.

  4. Markus Karner says:

    Scott,

    the fact that you wrote this, in my mind, makes it much more likely that you’re an above average doctor.

    • edanm says:

      And this is of course just another form of selection bias. Not surprising that the kind of people who read Scott’s blog, tend to like his writing / thinking about things. Of course we’re going to think “oh, how sensible a post – only above average people would write something like that!”.

      Except for the untold masses that find this kind of thinking wrong, and would consider it a perfect example of why Scott is obviously only average. (“Clearly he’s average – the kind of doctor who is smart enough to be above average *knows* that they’re above average. E.g. the blub paradox – if you’re at a higher level, you *know* that everyone else is below you, it’s only looking up that you have trouble.”. Or something to that effect.)

      • benf says:

        Point number seven overcomes that. “Good” relative to what? It takes two nominals to make a real – you need to stipulate a measurement and then compare two cases to make a judgment about ordinality.

  5. theodidactus says:

    I’m in law (or at least I will be, when I drag myself out of this last semester). I think all the systems you describe above except maybe (6) apply to the legal profession as well as the medical profession, so if you’re right the field should work the same way…I’ll tell you what it’s like once I’m fully integrated.

    I hope the dynamic is less persuasive than in medicine…most of the time when I “get it right” it’s because I quadruple-check everything on the assumption that 90% of lawyers (or law students) are better than me.

    • Calicozack says:

      As a practicing lawyer, I see a few differences. Most cases are working toward some of kind of resolution, so you don’t have chronic clients who just constantly need legal advice. I think this makes it less likely for clients to switch attorneys; they’re more likely to complain to you if they have problems (or leave bad reviews that you can see). When a client does switch off you, you know because you have to sign a stipulation for substitution of attorneys, and their new attorney may request your file. Also, the adversarial nature of law means you have lots of opportunities to directly compare yourself to other attorneys.

      On the other hand, I can kind of see a parallel to number 6 in that it’s easy to blame problems on the client. By definition, everyone who needs legal advice has some kind of dispute they couldn’t resolve themselves. They failed some obligation or broke a law or just couldn’t get along with the other party (there are exceptions, but you see what I mean). So they tend to be flaky jerks who don’t take responsibility for their mistakes. That makes them easy scapegoats when they end up with a bad resolution. If they have bad facts it’s their fault, if they complain about communication it’s because they never returned your calls, if they’re unhappy it’s because they had unreasonable expectations and didn’t listen to you.

      • Simulated Knave says:

        As I used to comment to newer lawyers when I worked for Legal Aid: if our clients were stable people with their lives together they generally wouldn’t be our clients.

        • theodidactus says:

          Frankly this is why I like the profession, especially my little corner of it. If I wanted to work with stable clean-cut people I’d have gone into some corporate sector. Real humans are way more interesting.

      • theodidactus says:

        It’s also worth noting that my dream jobs are all government positions, so if I land one of them, I won’t really be able to contribute to this analysis.

        The adversarial nature of the profession is something I hadn’t considered…and yes, that does probably change how on views their own position in the great chain of being.

        Also, ultimate failure and success leave human-readable feedback. If your patient relapses or something weird happens, you can always chalk it up to being idiopathic or something, but with law, when you lose on a purely legal issue, the judge usually tells you exactly why you lose. Now you might not agree with their reasons (and their reasons might not actually make any sense) but they’ll be right there, and written down.

        Of course, when a jury is concerned there’s a bit of “nature” at play.

    • Simulated Knave says:

      As a practicing lawyer in Canada: this all applies. Dad’s a dentist, and this applies to them too.

      There’s a practical reason for it: ethically, if you’re a crappy doctor/lawyer/dentist/other professional you’re supposed to STOP. If you’re not up to the job, you’re supposed to send them somewhere else. By definition, you think you’re good, or you wouldn’t do the job.

      There’s another point, too: it’s really easy to be above average in a lot of professions, because the poor examples are VERY poor indeed. 90 percent may genuinely BE above average.

      • theodidactus says:

        to quote a wise man I met once, it’s easy to half-ass it because most people don’t ass at all.

      • Bram Cohen says:

        You mean 90% may be above the mean, which is true. It is of course impossible for 90% to be above the median. Same goes for the fraction of people who outlive their life expectancy.

      • unreliabletags says:

        You could be low on the scale for your profession globally, but still contributing usefully in a simple specialty.

  6. Rack says:

    Even though by and large educators don’t get to pick their students, we can certainly still fall prey to biases similar to many you’ve listed, Scott, if we’re not careful and honest with ourselves.

  7. shakeddown says:

    This is also why the bay area dating market – which only has something like 10% more men than women, IIRC – feels a *lot* more skewed than that.

    Assume everyone rates people on the same attractiveness scale, but people have different standards. Since there’s a noticeably higher number of men than women, every women with moderate standards will have already settled down (unless she just broke up), while women who are only willing to settle down with the hottest 1% stay on the market for a while. Since the women with moderate standards are only on the market for a short time before finding someone and getting off it, most of the single women you meet will have unreasonable standards.

    (Note that women don’t have the same issue – since a mostly-random 10% subsample of men are already single just by the numerical assymetry, women looking for boyfriends don’t have a very strong selection bias working against them).

    • Michael Watts says:

      Since there’s a noticeably higher number of men than women, every women with moderate standards will have already settled down (unless she just broke up), while women who are only willing to settle down with the hottest 1% stay on the market for a while.

      Nope, the dynamics are completely different. Since there’s a noticeably higher number of men than women, the value of all single women skyrockets, preventing them from marrying. The hottest 1% of men still have severe problems.

      Elaine Ou wrote this up pretty well: https://elaineou.com/2017/05/28/gold-diggers-in-the-gold-rush-era/

      • shakeddown says:

        That’s one contributing mechanism, but the selection bias mechanism is stronger – perceptions of market value affect different people in different ways, but it’s selection bias that makes it so that you mostly encounter the relationship-averse women.

        • Michael Watts says:

          The selection bias mechanism is not stronger. Recall the quote that prompted the piece:

          if you’re in your mid-30s, only something like a quarter or less of girls growing up in the Bay Area are married.

          There is no strategy that will allow you to avoid meeting mostly relationship-averse women, because the environment causes them all to be relationship-averse. The women are trapped in a feedback loop; the rarer relationship-minded women are, the more valuable — and therefore less relationship-minded — every woman gets.

        • Plumber says:

          @shakeddown > “…it’s selection bias that makes it so that you mostly encounter the relationship-averse women”

          In the late 1980’s and early ’90’s when I was in my early 20’s in the San Francisco bay area (my time spent mostly in Berkeley, Oakland, and San Francisco) the press did pieces on “single heterosexual women outnumbering single heterosexual men” and how difficult it was for women to find husbands (like similar stories for Manhattan), but in my experience the women my age didn’t want to go steady at all, only one to three nights “stands”.

          As it was the local girls I grew up with mostly moved away to go to college and find work, and were replaced by girls who came to town to go to the University so since more girls go to college/university than boys you’d think the odds would be favorable but none wanted long term relationships.

          When I did find a girl who wanted more than three nights (my future wife) was she was a law school student in her late 20’s when I was in my early almost middle 20’s, and she didn’t consent to a legal marriage until we were in our 30’s and I had a union job that provided medical insurance to spouses.

          I don’t think sex ratios have anything to do with it, women in their 20’s just don’t want marriage and they haven’t for decades, my mother did get married (in 1967) when she was 20 but she soon divorced my Dad, and few of her peers married so young, one of my grandmother married at 16 in the early 40’s, but she and my grandfather lived apart until the war was over, my other grandmother married in the 1930’s and divorced in the ’40’s, so the pattern for decades is that most 20-something women either don’t want marriage, or they don’t want it for long.

          If you want a wife and children look for ladies no younger than their late 20’s, and expect to need to pay for fertility treatments, even just being a steady boyfriend is unlikely with an early to mid 20’s woman.

          The older you both are (until the age of 50) the better are the odds she’ll consent to a long-term relationship.

          • Simon_Jester says:

            Given that the pre-modern status quo was based on the premise that any woman who didn’t find a husband or stable fiance by her early 20s was an “old maid” whose social status would rapidly decline towards nil and who had few if any means of supporting herself…

            Well, that meant a lot of artificial incentives to force women to start long-term relationships whether or not they liked the prospects of those relationships. We would not reasonably expect a modern woman to marry a drunk or a reckless-spending idiot. Even if she herself met those descriptions, her marrying another person with the same problems would likely not do her any favors or be in her long term interests. But in the 19th century such men might have had relatively little difficulty finding a wife- because there were women who were at much greater risk of simply starving to death without a husband.

            Dating markets and hostage situations should be easier to tell apart. Now they are, and a significant fraction of women choose “single” over the available male prospects for long-term mating.

            I’ve also heard this discussed in the context of communities where single motherhood is common. Sometimes a stable long-term relationship with one of the available men is quite simply a worse prospect than no stable long-term relationship at all, other than occasional shopping among the available prospects to see if anyone worth bothering with has shown up.

            Dating isn’t a sorting algorithm where everyone wants to get a mate with a high score and settles for less. A “one” or a “two” may not be merely a mate with low social status, attractiveness, or income, but one with issues and behaviors that will make your life a living hell for as long as you stay with them. Singlehood may be rationally preferable.

          • Aapje says:

            @Simon_Jester

            who had few if any means of supporting herself…

            This is false. Premodern women often worked as (literal) maids, teachers, governesses, seamstresses, etc.

            any woman who didn’t find a husband or stable fiance by her early 20s was an “old maid” whose social status would rapidly decline towards nil

            The facts don’t match your claims. Hajnal showed in 1965 already that Western Europe was distinct for having a late marriage culture since at least the 17th century. In quite a few periods/places, the average age for first marriage of women was above 24, which presumably means that a very substantial number of women married in their late twenties or their thirties, which means that their social status can’t have been “nil.”

            Well, that meant a lot of artificial incentives to force women to start long-term relationships whether or not they liked the prospects of those relationships.

            Again, false. The data shows fairly large numbers of people staying single until quite late in life, so clearly they had a choice (note that the Catholic church had an explicit alternative to marriage for women: a nunnery). Free room and board was also not an uncommon part of certain work relationships as well (like live-in servants).

            Corbyn Morris remarked with dismay in 1751 that many ladies and gentlemen preferred relatively comfortable bachelor/spinsterhood over the harder family life:

            The discouragement to matrimony in London is a grand operating cause of the diminution of the christenings and consequently of the exceeds of the burials. The unmarried ladies and gentlemen in this city of moderate fortunes, which are the great bulk, are unable to support the expense of a family with any magnificence, and therefore cannot intermarry together without retiring from high life, and submitting to relinquish those pleasures of the town to which their appetites have long been raised; they therefore acquiesce in celibacy; each sex compensating itself as it can by other diversions.

            Of course, this may simply be an early instance of the modern life that we see around us, not reflected in rural places. Nevertheless, your claim that women were artificially forced to marry horrible men is greatly exaggerated. In so far that it is true that people were pushed into marriage much more than today, surely a lot of it was due to non-artificial causes, like a lack of reliable birth control, general low productivity/poverty, the effort required to run a household, etc.

            Besides, men where also pushed into marriage. For example, many were not taught to run a household, which is the other side of the coin to women not being taught job skills that earned as much as male job skills. Most modern people, used to supermarkets and washing machines, seem to be unaware of the impossibility for a single person to have a full job and run a household, in the past. The typical narrative frames the female gender role as limiting women’s choices, but completely ignores how men’s choices were limited and how they were pushed into marriage. The lack of concern over men marrying horrible women is indicative of a general lack of interest in the well being of men.

            Sometimes a stable long-term relationship with one of the available men is quite simply a worse prospect than no stable long-term relationship at all, other than occasional shopping among the available prospects to see if anyone worth bothering with has shown up.

            Sure, but the other side of the coin is that people may have an inflated sense of their own qualities, where those people would be happier than they are single with a person who will have them, but feel entitled to someone of high quality, who doesn’t want them.

            This sense of entitlement is not necessarily equal for each gender and may have changed over time, not necessarily still matching the stereotype.

            A “one” or a “two” may not be merely a mate with low social status, attractiveness, or income, but one with issues and behaviors that will make your life a living hell for as long as you stay with them.

            Sure, but if high social status, income, etc correlates with dark triad traits, people who have strict demands about the former can implicitly exclude perfectly nice people who don’t have any severe deficiencies.

            It’s a very common complaint by men that being a perfectly nice person whom women are gladly friends with, is not sufficient to be attractive as a mate, which if true, is not very consistent with the idea that these men are scorned for making women’s lives ‘a living hell’.

            Singlehood may be rationally preferable.

            I agree, but I also think that many people talk themselves into severe dissatisfaction with (potential) partners over things that merely cause the relationship to be less optimal than it could be.

    • Freddie deBoer says:

      There’s some Berkson’s Paradox floating around in there.

    • As Michael Watts points out, this would lead you to expect more and earlier marriage when the opposite is the case. The problem with reasoning from regional sex ratios is that people select into and out of these areas. The guys who accept a job in the Bay Area know the sex ratio exists; they will be the type of people who don’t care as much.

      • Also, while an abundance of men will make getting married easier, it will also make waiting for marriage while engaging in short-term relationships with attractive men easier.

        • Aapje says:

          If those short-term relationships increase women’s sense of entitlement (to the top tier of the guys they date short term), then ‘waiting for marriage’ can become permanent…or at least until the biological clock starts ticking.

  8. chaosmage says:

    Despite the objective test of whether the code will run, and the fact the difference in ability between programmers can be much more dramatic than the difference in ability between psychiatrists, there are similar effects for programmers.

    The code I write is easy to read and uses intuitive data structures, everyone else uses weird solutions and writes them poorly. My own innovations are clever responses to unusual circumstances, everyone else’s are obviously weird. Every time I have to fix someone’s code it is terrible. I never hear about my own code being terrible and in need of fixing, every time I need to touch my own code again it is because it is so good my customers want to make another use of it. If someone uses my software it is because it is good code, if they use competing software it is because of their clever marketing. If I copy code from Stackexchange I’m being efficient, if someone else does it they couldn’t do it themselves. If I use an obscure language or library I’m leveraging my unusual skillset, if someone else does it they’re prioritizing their own convenience over maintainability. If someone like John Carmack is just better than me at programming in every way it is because they are in a special heroic reference class that it would be unfair to compare me to but that I obviously aspire to.

    • B_Epstein says:

      Thanks so much for writing this. Way too many of those apply to me, all while I sit here and think “oh lucky us engineers and programmers, we have objective standards”.

      • chaosmage says:

        Great! We can go more meta. Here’s why we’re all above average SSC commenters.

        If I use SSC lingo I am right on topic, if someone else does it they are signalling ingroupishness. If I use an obscure reference I am being clever, if someone else does it they are being weird. If I write a long comment I am putting in work, if someone else writes a long comment they are taking up space. If I write a short comment I am being concise if someone else does it they are being lazy.

        If my comment gets no responses nobody found fault with it, if someone else’s comment got no responses it was boring. I will soon forget it however, while I can basically quote from memory that one in 2015 that made Comment of the Month that proved to you all I am awesome and is a big part of why you all quietly respect me. I don’t recall anybody else’s Comment of the Month.

        I focus my commenting on what are clearly the most important SSC posts while the ones that I don’t comment on are the ones that don’t matter so much. Everyone else seems to chase the most controversial posts which can only be because they’re looking for trouble.

        If reading this is hurtful to you I am on top of my biases and uncovering painful truths, if reading someone else’s comment is hurtful to me I might kindly not report them.

        • quanta413 says:

          When I go meta, it’s because I see through to the deeper truth of what really matters. When someone else goes meta, it’s a desperate attempt to avoid admitting their poor grasp of the object-level discussion.

    • magehat says:

      As a programmer who occasionally suffers from imposter syndrome, it’s funny how these statements basically reverse.

      E.g. if I have to use Stackexchange I feel like I couldn’t have done it myself, but someone else is just doing it to save time. I feel guilty about using unconventional solutions because it feels like a hack, but if I see someone else’s it’s because they must be smarter than me. Etc.

      Make sense, as bias works both ways.

    • Error says:

      I never hear about my own code being terrible and in need of fixing

      I do! But it’s usually from my own comments.

      Back when I did PC repair, there was a similar dynamic, though. We’d routinely get machines that the local Geek Squad had epically failed to fix. Of course, we never saw the ones they did fix, and ultimately I’ve no idea if they were actually any good.

    • moonfirestorm says:

      Every time I have to fix someone’s code it is terrible. I never hear about my own code being terrible and in need of fixing, every time I need to touch my own code again it is because it is so good my customers want to make another use of it.

      I’m not sure I buy this in a 9-to-5 job environment. I regularly fix bugs I introduced and adjust features that I wrote.

      You run into more of other people’s bad code than your own bad code, but if you’re working in the same area of code you can become very aware of the problems that you introduced.

      It also doesn’t bring up the failed attempts of programming, where you get 6 hours into a revision and realize it’s not going to work because of restrictions you didn’t know about when you started, and now there’s nothing left to do but revert the whole thing and find another route. That’s pretty good for keeping your ego in check.

    • Loriot says:

      > I never hear about my own code being terrible and in need of fixing.

      This is not consistent with my experience. I’d be surprised if there are any programmers who *don’t* hate their own past code. All it takes is working on the code again after an interval of a couple weeks and you’ll be cursing your past self with the rest of them.

      • acymetric says:

        I don’t hate my old code so much as I am simply confused by it (in a “why did I do that this way” sense, not a “what the heck does that code even do” sense, although I suppose sometimes the latter as well).

      • Noumenon72 says:

        Well, nothing’s true of every personality. I spend most of my time when I write my code thinking about structure and maintainability. When I come back to it I’ve usually been given a gift from my past self to understand it. When I’m confused it’s just, “Oh, I should have structured this so it would be in the first place I looked”, when I see a problem it’s just “Oh, let me make a note not to do it this way again”.

        I’m not saying my code’s better but I maybe just have a more self-forgiving attitude. My code is my baby and even my very first Android app, I go back and look at its code so I can be proud of it, not to get mad.

    • Murphy says:

      Every time I have to fix someone’s code it is terrible. I never hear about my own code being terrible

      Really?

      It’s a standard joke on programming forums.

      “What fucking asshole wrote this! undocumented, shitty code, incomprehensible functions, filled with bugs” [checks the source control] “Fucking past-me, what a fucking asshole!”

    • jensfiederer says:

      As a programmer myself, I have to disagree.

      Most good programmers are appropriately humble, because they have witnessed so many of their own senseless bugs. I’m generally considered pretty good (my boss and another programmer who is a friend argued because my boss did not consider me a genius but only “the most competent person I have ever known”, which I actually considered flattering) but I have made many mistakes that would turn your hair white to look at them, assuming you have hair. An expert programmer is capable of making far more mistakes than your novice!

      • nancy says:

        Yup. I was speaking at a career event for 14-year-old girls in the 80s (when a female coder like me was a relative rarity). I accidentally came up with a line that I still love:
        “Programming is the perfect career for someone who can’t get anything right the first time.”

    • Jonas says:

      I read your examples and they sound like you’re applying different criteria to your own code compared to other people’s code. I read Scott’s examples, and they sound like they’re about obtaining different and systematically skewed data about your own performance relative to that of others’.

      If you don’t do pair programming, you see all the false starts and buggy half-baked intermediate versions of your code. When you look at the ready-to-ship work of your colleagues you don’t see the flaky intermediate versions, leading you to undervalue your own performance. A culture of testing and talking about bugs suggests it’s still there, however.

      When you write code as part of your employer’s application, it’s clean, consise and to the point. When you read other people’s libraries, whose multifaceted demands and modularity requirements you only know the half of, the code is too abstract, uncohesive and diffuse, or just plain old poorly organized.

      When you browse Stack Overflow and only read the accepted answers, the rest of the world artificially looks more competent and knowledgeable than it is on average, because you’re not sampling the average. Comparing yourself to selectively the best makes you aware of your own ignorance. Or, depending on the question and your personality, makes you believe that you could’ve known whatever everyone else knows had you chosen to read the tutorial and/or fucking manual, ergo in terms of potential you’re at least as good as everyone else.

      If you keep getting promoted so long as you perform competently in the role you get promoted out of and you don’t attribute the stoppage of promotion to a competence ceiling, you get the Peter principle (https://en.wikipedia.org/wiki/Peter_principle). If you keep getting fired for incompetence and you don’t attribute the stoppage of firing to a change in demands, you’ll think you just had to discover the right niche for your skills and personality.

      When you read The Daily WTF you simultaneously marvel and recoil in horror at the awful code people out there write, you realize that you’re not as terrible as… whatever the hell causes those train wrecks.

      I don’t really know. There are programmers who are worse than me and there are programmers who are better than me. Some are better at algorithm design but have less knowledge of specific technologies and vice versa, et cetera. I don’t know how seriously skewed my data about relative performance is (although see above for guesses). I care a lot more about improving my own set of skills in an absolute sense than any ranking competition.

  9. Unnamed says:

    #5 (“You know what you know, but you don’t know what you don’t know”) is the Dunning-Kruger effect. The ability to recognize flaws in your performance is often related to the ability to perform well, so in the places where we are unskilled we are often unaware of it.

    #7 (“You do a good job satisfying your own values”) is also covered in Dunning’s research, in the paper Ambiguity and Self-Evaluation: The Role of Idiosyncratic Trait Definitions in Self-Serving Assessments of Ability.

  10. bsrk says:

    It is good that you focused on your own faults, and not the faults of others. It is rare, a honest attempt at sincerity. What a beautiful sight it is!

  11. JulieK says:

    Does this mean it’s not the case that 90% of everything is cr*p?

  12. Etoile says:

    I wonder if there are also clustering effects – where the good/bad doctors are.
    Is there geographic and socio-economic clustering, where the best diagnosticians and the ones who think go to big prestigious systems and hospitals, who rise to the tops of hierarchies, etc.?
    The quality of medical professionals in my area – as measured by schools gone to, niceness of offices, nuance of knowledge and discussion on medical topics – definitely varies by the neighborhood you go to.

  13. James Green says:

    >If I’m not careful, I never notice this; it’s a dog that doesn’t bark.

    About a month ago I noticed for the first time in two years of living here that there are no loud barking dogs in my neighbourhood. I was in a very good mood for the rest of the day after noticing that.

  14. Radu Floricica says:

    I’ve thought about it from the perspective of a huuge missed opportunity. There is a lot less feedback from patients than it seems reasonable – mostly because when a patient doesn’t show up again, there is almost no way to tell if it’s because he’s happy or unhappy. Both are equally valid possibilities. So you prescribe a treatment and… nothing. I’m not talking about double blind scientific studies here (although in many cases they seem trivial to implement over existing medical practice), but simple elementary feedback.

    And the solution seems to be stupidly simple: have an independent organization (the government? insurance company? private?) finance a call center that simply calls back some of the patients and asks them a few questions. You can have multiple options, decided by the doctor or the clinic. If he preregisters each consultation, he can publicly display a rating score. The patient can ask for his feedback to be anonymous. Etc. But point is, any move in this direction would be a huge step forward.

    • LICA says:

      This is actually a fairly competitive space with a fairly long history, but it has really been kicked into gear in the last decade with reimbursements being affected by HCAHPS scores. There are quite a few companies that provide exactly the service you’re describing specifically tailored for healthcare – Press Ganey is probably the largest if you’d like an example.

      The biggest obstacle that I’ve run into is actually getting physicians to look at the feedback. They have a very long list of reasons (many valid, others less so) that the feedback is inaccurate or otherwise misleading.

      So much so that I’m thinking about how (with Scott’s permission) I might use this article at my own facility to push back against the unwillingness to engage with patient experience feedback.

      Source: I work in a patient experience office at a major academic medical center.

    • DinoNerd says:

      As a consumer, I’m currently swamped in bad surveys, many of them most likely designed to demonstrate a high “net promoter” score for whatever the survey is about. With Google in particular asking me to give them free content about any place I visit, I rarely answer any of these, but more importantly, I don’t trust their results. There’s also the phenomenon of various service places giving their staff bonuses – or worse, not retaining them – based on the difference between a perfect rating, and one just a hair lower (and thus IMO more likely to be real). Net result, I wouldn’t trust the results of any such surveys. And as an ex-patient, I probably wouldn’t waste my time answering them. (I also get so many spam calls – some taped advertisement – that I might not even answer the phone.)

  15. bodo says:

    At least anectdotally this seems definitely to be true. In my experience doctors are, or at least project an image of being, extremely sure of themselves. A colleague who worked in a hospital for many years (not as a doctor) recently told me: “Doctors study medicine, but they also train in narcism.”

    • gkai says:

      I think it’s probably needed to be effective with the vast majority of patients. I don’t think many people would prefer a doctor making suggestion and speaking about multiple possible outcome and how he could be wrong, compared to one which tell you have this condition, do that and you will get better. Especially as second approach will often improve outcome just by placebo effect.
      The first approach can work with technically-inclined patients who prefer not delegate reponsability, even regarding their own health. Those patients exists (but I believe they are a small minority). I also think that not all doctors are able to comfortably switch to first approach with those patients, even if they were able to fresh out of school: the second approach has become a second nature after being exposed to so many patients demanding it…

      • Garrett says:

        Story time.

        I recently had a middle-aged male patient who called 911 because they felt weak, dizzy and were sweaty all over – signs of heart problems, possibly a heart attack. I got on-scene and did an EKG. It wasn’t an obvious heart attack, but something strange which I hadn’t previously encountered. I recommended that they be seen in the ER immediately.

        Then I asked: “do you want us to take you to the hospital?”

        What was an attempt to ensure consent from the patient turned into a 10-minute discussion with the whole family. The husband suddenly thinks he needs his wife’s permission – I got the feeling he thought she had medical training (maybe she did). This is a case which might have been time-sensitive. When you start passing out and your heart is doing something funky, you don’t know how much time you have before the condition becomes “irreversible”. It could be minutes. It could be decades.

        So now I skip that question and ask: “which hospital do you want to go to?” Because taking that time could have an impact on a patient’s outcome. Might that come across as narcissistic? Sure. Have physicians had the same experience? Probably.

        I suspect a big part of it is that most patients don’t have the relevant knowledge to be able to ask useful questions. Imagine your doctor tells you that you have high blood pressure and then asks you: which class of anti-hypertensive medication would you prefer? Just looking through that list I discovered a few categories I didn’t know, and that some drugs used for other conditions can be used for this as well. Instead, they take the quick route of “this is what we’re going to do” and leave it to the patient to follow treatment or not. It’s not ideal, but it’s what we have the time for.

        • Simulated Knave says:

          I used to ask people in jail with no money if they wanted to apply for legal aid.

          Some would say no. Because they wanted to hire someone. They had no idea how and no money, but they wanted to hire someone.

          So you then got to talk them back INTO applying for legal aid…

      • caryatis says:

        There’s a book about this, which I thought was very good and would recommend to all doctors: Your Medical Mind, by Jerome Groopman abd Pamela Hartzband. A good doctor will take a patient’s preferences about how much information to get and how many decisions to make into account (along with, of course, the circumstances such as a potential emergency.)

        I still get angry when I think about the time I asked a doctor what he was about to inject into me and was told, “don’t worry, it won’t hurt.” Don’t be that guy—but also, don’t assume that every patient wants to know.

  16. calvin321 says:

    I always start with disagreements over what success looks like. People are performing well to their personal standard of what ‘good’ means.

    In driving there are various risk appetites on getting to your destination safely, quickly, avoiding traffic, minimum distance, scenery, etc. And the various personal preferences on how to best balance those options.

    For health the objective would be something like maximizing quality of life, quantity of life, guaranteed next few years, minimizing risk of catastrophic results, etc. To make matters worse knowing what is the minimum effective dose when the science in health is very hard to know and most published research is false.

    So there is variance on the target, uncertainty on the methods, and a large degree of differences on how to carve a path on prescribing solutions.

  17. AJD says:

    The other side of the equation from the patient’s perspective is equally as biased. Go on the Nextdoor app and ask somebody to recommend a doctor of any specialty. Within a mere hour there will be 40+ responses, each as fanatically enthusiastic about their recommendation as every other. There will be some overlap in a local area, but each and every person will be absolutely certain, and sometimes belligerently so, that their personal physician is not only the best they have been to but arguably the best globally. It almost doesn’t matter what kind of professional you request, or even non-professional like a handyman or a dog groomer, customers and patients are largely of a kind.

    The attachment that grows between a service provider and a customer becomes familial. And the recipient of the service becomes loyal beyond all reason. I’m sure that there is psychological terminology and theoretical explantation for this attachment. But I am not a psychologist. To me, it is both of curiosity and a phenomenon that I seek to leverage in my work as a strategy consultant and executive coach.

    Like most patients, these fanatically attached folks may have no basis of comparison and no standard for judgment. For example, as a patient, unless I happen to be a physician in the same specialty, the odds are that I have no real way of assessing whether my doctor is any good. How can I know whether I got better because of treatment, time, or just luck? But I attribute it to the doctor. I could just as easily attribute it to prayer, or to the proper alignment of the planets and stars. There is almost no way to be objective as a patient.

    And the role of individual personality and taste, as you point out, is prominent. Some patients like very much to have a full understanding of the science, thinking, diagnostic process and so forth. They benefit from a doctor who includes them from start to finish. Others simply want to be reassured and to feel cared for and known. Others yet again prefer a more paternalistic approach in which a doctor simply says “Trust me. You’ll be fine”. Moreover, along with each of our individual preferences I suspect that each of us believes our preferences are the correct ones.

    So short of very robust longitudinal studies comparing medical outcomes across individual physicians treating like populations, it is a fool’s errand really to attempt to make these determinations isn’t it?

    • caryatis says:

      I think it’s not so much that I’ve thought about whether my guy is the best and decided he is; it’s more like I (want to believe I’m) great, therefore my doctor/dog walker/car/TV must also be great. To admit that my doctor is mediocre, but I don’t care enough to change that, is psychologically too close to admitting I am mediocre.

  18. johan_larson says:

    The reference to the “You Are Not Hiring the Top 1%” article got me wondering. How would one go about finding the top 1% of programmers, even in theory? Before you stand all the 4.4 million computer programmers in America. They have all agreed to do whatever you tell them for the next eight hours. What do you want them to do?

    • thisheavenlyconjugation says:

      Not a well defined question.

    • John Schilling says:

      Before you stand all the 4.4 million computer programmers in America. They have all agreed to do whatever you tell them for the next eight hours. What do you want them to do?

      I want each of them to give me a ranked list of the best programmers they have personally worked with.

      And maybe a brief evaluation of each, but the start of the search is seeing what names come up most often, and then what names are given most often by the people whose names come up most often, etc.

      • David Speyer says:

        “then what names are given most often by the people whose names come up most often, etc”

        Congratulations, you have invented Pagerank.

      • The Nybbler says:

        I think what you’ll get by doing this (aside from a lot of programmers grumbling about doing performance evaluations) is “locally” top programmers. The lowest-quality programmer on the Tensorflow team may be better than the highest quality programmer at TaPro Business Programming Associates, but the latter person is going to get mentioned and the former is not.

        • John Schilling says:

          But who does the highest-quality programmer at TaPro call out as the best? If TaPro is entirely incestuous, that’s easy enough to spot from the network graph, and then probably you just ignore them. Otherwise, the best programmers the top guy at TaPro knows, are the ones who were good enough to jump ship and get jobs someplace better – like Tensorflow. But the top guy at Tensorflow isn’t going to be pointing to anyone at TaPro.

          You probably do want to identify the clusters and reduce the weighting on intra-cluster references as those will be disproportionately driven by laziness and social politics. And any real-world implementation, as you note, does have to deal with the motivation problem – the hypothetical posits someone who can motivate every coder in the USA to show up for an interview, so we can handwave a bit of paperwork on top of that. Referral bonuses are the usual way to accomplish this IRL, but I’m open to better alternatives.

    • Spiritkas says:

      TL;DR – The top 1% thing is just marketing hype to try to get better applicants for less money spent on recruiting. It wasn’t meant to be taken literally.

      There is the already mentioned issue of there not being a ‘best’ in a broad field such as programmers as it depends on your company’s needs. It is unlikely there the top 1% candidates would somehow be universally good in all possible programming languages.

      As a rough stab in the right direction towards actually getting some folks from the top 1% at the thing you want them to do; I’d suggest entrance exams. Some companies in other industries such as finance have had these for a while. Then you might be testing preparation or cheating ability, it’d be prohibitively expensive to run an incredibly valid examination or skills demonstration.

      Not to mention your current pool of employees who would be judging might not be in the top 1% themselves since you’ve been fooling yourself the entire time. Could you design such a test or recognise such a candidate if you’re only a top 25% person or even a top 5% person yourself?

      I think the prohibitive costs and extended hiring process typically cost more or are perceived to cost more than simply going with the best person you can. By using the more marketing oriented idea of ‘we hire the top 1%’ of x, y, or z candidates, you could actually get better quality candidates as those who doubt themselves might be discouraged from applying. A strong student with good scores ‘might’ get into an Ivy League school if they applied, but since there is a reputation of perfect test scores and top rankings from prior education…a huge number of people are discouraged from applying.

      It is a mix of how much you can invest in recruiting vs what tactics a company can use. If they have to wait 6 months and spend $50,000 going through candidates after candidates hoping to get at some preposterously literally defined top 1% then that’s likely a loser of a company which wont get anything done. If they can be reasonably sure most of the time to get someone from the top 10% of candidates through reputation and picking the best of the bunch/using headhunters who have reliably produced such candidates in the past…then they can get a position filled quickly with someone who is good enough.

  19. sethgodin says:

    Scott, this is one of your best ever. Thank you.

    I’m wondering if it’s different for creatives:

    In many professional fields, including medicine, there are factors at work that remind busy practitioners that they are the very best in their field.

    However, for creatives, the opposite forces are often at work:

    1. Since most of our work is purchased a la carte, and since there’s far more supply than demand, most of the feedback we receive is rejection.

    2. Since the work we do involves widely available tools (like a keyboard), the group of people who believe that they can also do the work (or improve on ours) is very large.

    3. Since many of us have a transient base of fans (music lovers like many musicians, not just one), there’s a great deal of churn in the fan base.

    4. Since negative criticism is easier to spread than positive feedback, most public criticism of our work is negative. On the other hand, people who are fairly satisfied say nothing.

    5. Because we work in novelty, our existing customers are often hesitant to return, because someone else (anyone else) can offer more novelty than we can.

    6. Because creative magic is truly breathtaking, the audience (and us) are chasing a once in a lifetime moment. Those, by definition, are rare, and so most of our interactions don’t meet that standard.

    To be fair, there’s a tribal/fan bias at work as well, and that counteracts most of the factors above.

    • Aapje says:

      5. Because we work in novelty, our existing customers are often hesitant to return, because someone else (anyone else) can offer more novelty than we can.

      6. Because creative magic is truly breathtaking, the audience (and us) are chasing a once in a lifetime moment. Those, by definition, are rare, and so most of our interactions don’t meet that standard.

      I have to disagree with these. Lots of fiction writing now consists of book series. Lots of movies are now franchises. The first Harry Potter book sold 120 million, the second 77 and then sales stabilized at 65 million for the later books. That is immense loyalty.

      I think that a major reason for failure by creatives to capitalize on this is more that it’s hard for them to be equally creative within an already constructed universe/straight jacket, then to invent a universe; rather than that the audience wants that much novelty. I see a lot of creatives complain about their audience expecting more of the same, while they want novelty or struggle to be creative within those confines. Those that do try to cater to their audience then often don’t have their heart in it. If each new entry deteriorates in quality, more and more people will drop out & lots of people will be disappointed, resulting in lots of negative criticism, for good reason.

      Another reason for negative feedback is that sales has an internal conflict: if you are better at increasing expectations, sales go up, but disappointment goes up as well. So there is a strong tendency to kill the goose with the golden eggs, by over-hyping products, at a cost of lesser satisfaction and people being less willing to buy again.

      In my perception, the major exception to high customer loyalty for products that people actually enjoy a lot, is hypes, where people get a status boost or other network benefits from buying into something. Then novelty is crucial, because hypes work likes waves, where people gain status from buying into at the right time and lose status for not dropping out at the right time. Note that both depend on people’s existing status. The top status person (can afford to) initiate hypes, gaining status, while the medium status person benefits more from signing on to hypes with decent momentum. The top status person also has to drop out sooner not to lose status, while the medium status person can stick with it until later into the hype life cycle.

      The people that adopt things too soon and/or stick with it when it is no longer fashionable are those foul geeks/nerds. :/

    • gleamingecho says:

      It seems to me that if you’re *in* a creative profession (i.e. you can make money off your creations), you’re already great. Being in the top X% of creatives is like being in the top X% of professional baseball players. Just being in the profession makes you pretty great.

      FWIW, I’m not a creative.

      • Aapje says:

        There are a ton of people who make no or some money of these jobs, with no clear line between them and those who earn enough to live on.

    • Kiwanda says:

      What is a “creative”? I think I have an idea, but the term seems insulting to that set of people, a hundred times bigger, for which creativity is also a key part of their work.

      • nancy says:

        It’s yet another upsell in the naming of professions. It covers designers, writers and other people whose work product is considered the result of creativity. Of course there are many jobs that are best performed when using creativity.

        Back when I wrote code, I was called a “programmer,” but today I’d be called a software engineer –or developer, if you aren’t hyping as much (and that job felt plenty creative). The job I have now, graphic designer, was called “commercial artist” when I was in high school. Now I am one of those “creatives.” You can see where the people in these jobs wanted to rename the job to sound more “professional.” And it somehow worked. Blecch.

  20. alwhite says:

    I think there’s also cultural stuff influencing this. Schooling for MDs really pushes this exceptionalism. You can even see differences in job selection. There’s a huge divide between hands on medicine (physical therapy, occupational therapy, nursing, etc…) Hands on medicine is dominated by women, so there’s some kind of bias going on there. PTs have doctoral level training but it’s taboo for them to call themselves doctor. It’s the MD who has the final say on patient care and is the authority, even though the PTs and OTs do most of the work. PTs and OTs receive a lot of grief from MDs giving wrong advice, but they aren’t allowed to correct or complain to the MD who gave the bad advice because the MD is established as the authority.

    I think there’s some really powerful cultural thing going on that is enforcing this as well. My guess is that this cultural script is the predominant factor over the ones you’ve listed but I don’t know how to prove that in any way.

    • Garrett says:

      > PTs have doctoral level training but it’s taboo for them to call themselves doctor.

      I believe this involves the confusion between “medical doctor” and “PhD”*, and the need for a term for the person who has ultimate authority and responsibility for a patient’s care. This matters most in cases where all members of the conversation aren’t known to each other. The “doctor” has the authority to issue medical “orders”. I would *love* to watch people freak out at someone with a PhD in nursing demand to be called “doctor” in a hospital. But between the potential risks of various sorts and the professionalism involved, it isn’t done.

      > It’s the MD who has the final say on patient care and is the authority

      Yes. Because they have that authority legally, as well as the responsibility which goes along with it. A quick search for online malpractice insurance rates shows that a self-employed PT can get a year’s worth of malpractice insurance for $300. For physicians you are looking into the thousands. For surgeons into the 10s of thousands. That’s a lot of responsibility. Under my State’s physical therapist act, physical therapists may only provide treatment upon referral from a physician (lots of details omitted). Likewise, physicians can be sued for missing a diagnosis and the associated complications. Physicians are expected to have a full set of medical knowledge in order to evaluate the risks and benefits associated with all treatments and their interactions. At the same time, a professional physician should be willing to listen to suggestions (or criticism) from any other allied health professional or source. Yes, including the janitor.

      > even though the PTs and OTs do most of the work

      This only views “work” as it related to patient contact time. But any time something is being done on a complex system, almost certainly the “planning” and “implementation” steps are separated. Not just in medicine but in a lot of other fields like engineering. The engineers responsible for office building construction do a lot less work than the iron workers or electricians, but they bear overall responsibility for the safety and usability of the structure.

      * Yes, I know that the academics had it first. We’re in need of a term which specifies people who are at the top of their practice area (physician, surgeon, dentist, etc.) with wide medical authorization which isn’t already used and would be accepted by the public. Good luck.

      • alwhite says:

        Everything you’re saying is proving my point. You’re arguing that MDs are more special. No wonder they think they’re the best. Everyone argues that they are.

  21. jonabar says:

    I think the seventh point is the key one. I often think about this for the driving example that always gets mentioned. People who weave in and out of traffic, run lights, drive with their knees, etc. think that they’re better than average drivers, because they’re the ones engaging in the riskiest behaviors and (most of the time anyway) succeeding, whereas the people who drive very conservatively, always signal, always yield, never cut the corner on a turn, etc., also think they’re better than average, because they’re the ones who are best at controlling the impulses that would otherwise encourage them to do risky behavior. If there’s a quality-efficiency spectrum, everyone is going to think their place in it is best.

    • Adam says:

      Exactly. I have caused zero accidents; in that respect, I am demonstrably above-average. In other respects, not so much.

      It is entirely possible for every driver to be above-average in some measure.

    • Noumenon72 says:

      I love congratulating myself on coasting exactly to a red light without having to brake, picking the exact moment to speed up in a traffic jam so I’m not the last slow person, and changing lanes so I can keep a constant cruise. Other people are counting how many cars they passed or how much fun acceleration they felt. I bet I don’t even know what they think of as their driving utility function.

  22. Xammer says:

    I’ve encountered another bias: a doctor suggested me two surgeries for the same symptom, and the surgeries went without a hitch, and so they considered them a success. The thing is the symptom didn’t pass away but they completely forgot about it (several times) and especially that it was the reason for the surgeries (I think their mind independently assigned a different reason for each surgery).

  23. wfenza says:

    This is the Dunning-Kruger Effect. The most important takeaway of the Dunning-Kruger research is that almost everyone thinks they are slightly above average at everything. People in the bottom 5% think they are slightly above average. People in the top 5% think they are slightly above average. So it’s not surprising that doctors all think they are above average. Everyone in every profession does.

  24. chaosmage says:

    This is a great example of why it is good to look at what cognitive biases feel like from the inside. Scott has already done something like this is in https://slatestarcodex.com/2014/12/13/debunked-and-well-refuted/ but the present post fleshes it out much more.

    I have now fleshed the same subject out even more on LessWrong and argue this is much better than abstract descriptions like “confirmation bias is a tendency to preferentially seek out confirming evidence”: https://www.lesswrong.com/posts/ERWeEA8op6s6tYCKy/what-cognitive-biases-feel-like-from-the-inside .

  25. metacelsus says:

    The relevant studies are paywalled, so I don’t know if I should trust them.

    Come on, Scott! The link to sci-hub is right there in your sidebar . . .

    • Telomerase says:

      Yeah, let’s not pretend that sci-hub doesn’t exist until the drone hits them. That’s a critically important resource, if for nothing else than to remind us how ineptly we’ve set up the research-information system. Taxes pay for the parts that they shouldn’t, screwing up incentives… but then they DON’T pay the last few million dollars to distribute the information.

    • Xammer says:

      Why does everyone talk about sci-hub and not Library Genesis?

    • thevoiceofthevoid says:

      One time, our university library put up a set of two posters on easels near the front entrance. The first: “Place a sticker on the amount you paid for textbooks this semester.” (bins: $0-$100, $101-200, up to $1000 plus. Or something like that.) The vast majority was in the $0 category, with a bit of a bimodal distribution (the other peak around $400). The other poster had “Please write any comments here about the proposal for the university to move to a free-textbook model”, with a couple Expos. This was dominated by comments to the tune of “hey, people who put your stickers at not 0, have you heard of LibGen? Here’s a URL!”

  26. Mouth says:

    Driver skill might have a steep distribution and small standard deviation. Skill differences among the 90% who consider themselves above average might be small. This would make it difficult to determine if you were above average or not. Small amounts of bias could be enough to cause the survey result.

    If the first decile contains drivers that are grossly worse than average, the average would be skewed. 90% of people could be above average. Even if true, I imagine replacing average with median on the survey would still result in 90%.

  27. Telomerase says:

    You probably ARE one of the best psychiatrists, because you don’t uncritically believe in your cult.

    Most real doctors are just in cults as well… maybe not most internists, but surgeons, egad.

  28. Peter Gerdes says:

    Many of these apply to teaching. Especially for college courses where students have a choice of teachers.

  29. IvanFyodorovich says:

    As a scientist, I feel like I have the opposite effect. I know how many of my ideas fail to pan out and how much stuff my lab screws up. Since ideas that went nowhere and technical failures don’t usually end up in papers, I don’t see other peoples’ screwups. I just see their successes, and it can make me feel bad.

    At least I’m old enough to know that everyone else is also screwing up (and that lots of papers are cherrypicked nonsense), but new graduate students often don’t know this and get discouraged.

  30. Seth B says:

    The quality of many of these interactions are match specific (i.e. patients stay with doctors who are good matches for them).

    The good-news upshot of this is that more than 1% of doctors can be a top percentile doctor for their set of patients. In the extreme case, suppose there is no such thing as patient-independent doctor quality, and that doctors and patients are both very heterogeneous in the services they provide and value. Then it is possible (again, not realistic, but possible) for every patient to have the best doctor for them in the world.

  31. Simulated Knave says:

    I mentioned this in a reply, but there are two things:

    First, it is very easy to be above average. The worst professionals are much, much worse than the median. Given a bit of difference in values, everyone genuinely may be above average for a particular average.

    Second, given professionals’ assorted obligations to refer out for matters where they’re not capable, you have to think you’re good or you can’t practice.

    • gleamingecho says:

      First, it is very easy to be above average.

      I’m no statistician, but my assumption would be that in a random draw, you’re just as likely to draw someone below average as you are to draw someone above average.

      • DinoNerd says:

        Only if the distribution is symmetrical. (This assumes we’re talking about the arithmetic mean, which is the most common definition of average, and the one Simulated Knave has to be using for their comment to make sense.)

        • Simulated Knave says:

          Indeed.

          Trust me, there’s a lot of real incompetents out there, and they’re a LOT worse than everyone else.

  32. craftman says:

    #1 seems like the same mechanism by which all car insurance companies claim “average customer saves x%”. The people who are shopping around are more likely to be paying “too much” for their coverage given their situation.

    And if I shop around and don’t find cheaper insurance, I’m certainly not paying more (given same level of coverage) so there is a lower bound of $0 on people included in the bin of “saved on car insurance”.

  33. Rafael Trindade says:

    So how to avoid falling in this trap? How to better access what you don’t know, what you are not?

  34. rahien.din says:

    These are good reminders of important traps. It is essential to be humble. Medicine is hard.

    But much of this is totally normal.

    It’s important not to forget the patient here – their perspective is the most important one. The patient-physician relationship is central to all aspects of care. So it makes sense that patients would select for a physician that they click with. Clicking with your doctor feels better, but it also genuinely improves care. Most people, by definition, have routine/treatable maladies and fairly normal personalities, so they’ll click with most doctors. Putting that all together, most patients will find and stick with a doctor they like, and will get treated successfully. If they ever change doctors, something wasn’t working and their new doctor is probably a better fit.

    So 1, 2, 3, and 4 are all… fine. To the extent that doctors feel this way, these are probably correct assessments. The patient seems happier with you than with their previous doctor? Good for them that they found a better fit! Common treatments treat common problems? As expected! Patient happy with successful patient-physician relationship and sticks with it? Seems reasonable!

    We must consider what “above-average” means in this circumstance. If “above-average” means “better quality than most doctors,” of course that’s wrong. But if “above-average” connotes “expecting to click with all their patients,” then sure, that’s possible. Every key is “above average” in terms of its ability to open its lock. That doesn’t mean the key is of higher quality per se, only that the relationship between key and lock is successful.

    Also, medicine is hard. Most of the time, there is not a single best treatment pathway – there is simply the treatment pathway you are able to offer to the patient, based on their needs and desires and values. So you really can only satisfy your values and your patient’s values – both in the sense that value satisfaction is the entire goal, and in the sense that it’s impossible to act except in the service of your own values. And, there are genuinely some people who we aren’t yet able to help. Some failures really do belong to nature. If we knew what we didn’t know, we wouldn’t not know it – what standard are you holding physicians to?

    So 5, 6, and 7 are… also fine. If all errors were prevented, and all treatment failures were due to errors, then sure, all patients would be cured – but that doesn’t seem currently realistic.

  35. Jack says:

    These statistics get bandied about to suggest that people misperceive their skills. But evaluating a doctor’s perceptions of their skills is hard, so instead we ask them to rate themselves relative to an imagined median doctor. (Simulated Knave pointed out the problem with using an imagined average doctor above.) If most doctors think they’re above the median, bam! We have proved they misperceive their skills!–

    –Is the argument. But it proves nothing of the sort. It can instead be that doctors misperceive the skills of the median doctor. And since doctors have no idea “how good” the median doctor is, this seems likely. Different mechanisms can lead to misperceiving one’s skills and to misperceiving the imagined median doctor, and the two misperceptions lead to different problems, so it would be useful to keep them separate.

  36. Nearly Takuan says:

    I do score really high on most of those timed multiple-choice programming quizzes. The results generally congratulate me on being “in the 96th percentile” or whatever. But I suspect the quizzes are dishonest, poorly calibrated, or both. Some of the questions I get right because I’m good at meta gaming multiple choice questions. Others I get right despite the bad phrasing of the question and the fact that multiple answers were technically “right”, I just happened to also figure out which of the right answers they were thinking of as the only right one. I award myself bonus points for those, but I should really be second-guessing the accuracy of my score more than that.

    Outside of skills tests, I also do get a lot of… affirmation bias? at work. At each of my past several jobs, I was hired because I was bringing certain skills and experiences nobody else on the team had yet, and they forgave me for not knowing the stuff everybody else knew, or having the domain/industry knowledge of what the company exactly is even doing. I hardly notice the stuff I don’t know because, we’ll, it’s just an ordinary part of onboarding. The team helps me learn a bunch of stuff and from there I get to basically take it for granted. But meanwhile I’m teaching 5-6 other people with job titles more impressive than mine (and vastly higher salaries) about stuff they’ve apparently never seen before.

    Also, when I get a new job and see the codebase I’ve inherited to work with, it’s easy to make fun of all the obvious stupid mistakes that were made along the way, and curse the terrible design, poor documentation, etc. that allowed things to get this way. This is the unsatisfied treatment-resistant patient I came to save, and as I set about optimizing the code base and swaying my fellow devs to suit my own preferences, I perceive success. But I only occasionally have to find out what happened to the codebases I left behind.

    And, yeah, my values aren’t the same as those of other people in my profession. I get into arguments on github, stackexchange, and twitter at least monthly. I work with a bunch of people whose values differ vastly from mine, and I usually defer, sometimes compromise. When stuff ends up being slightly unsatisfactory, I tell myself I’ve done everything I could, but the Nature of a badly-designed project is to be treatment-resistant. On the rare occasion I persuade someone to do it my way, it’s an unqualified success! —at least, by the values I care about measuring. Worth noting that “things I can persuade other devs to do” is not even close to a random sampling of “things I wish other devs would do”, and there’s probably positive correlation between persuasiveness and goodness of a concept in this sphere. Maybe the remainder of my ideas are truly terrible, and I only don’t know this yet because my coworkers protect me from discovering my own idiocy.

    • janrandom says:

      Thank you for writing a version on software development. To me, this seems to be more authentic and in the spirit of Scott’s post than the more witty account by chaosmage (which was also good as measured by the replies it got).

  37. jerryb0222 says:

    I came across this last month. The studies focus was Wisconsin, but my experience with doctors and healthcare are similar no matter what US state. In my 60 years the competence of doctors has declined significantly, especially in the last 30+ years.

    https://www.jsonline.com/story/money/business/health-care/2019/12/04/wisconsin-doctors-could-reduce-costs-while-improving-care-study-says/2601902001/

    The study found that an estimated 35% of the physicians provided below average care and an estimated 21% provided care that was more costly than the average for their counterparts.

    Like every other occupation, do want someone who graduated at the top of their class or the bottom?

    • Midge says:

      Do we want doctors who graduated at the top of their class or at the bottom? All else being equal, top, but not all else is equal.

      Some of the most appropriate medical care I’ve received was from people who I have every reason to believe weren’t above-average doctors — doctors in seedy, low-rent clinics or people not doctors at all (nurses, physiotherapists). I’ve had the good luck to have acute but simple problems any sentient being with a prescription pad could fix, and the bad luck to have something even top-of-class doctors tend to not recognize for what it is except by happenstance (happenstance in my case being non-doctor providers who by sheer luck had firsthand experience with signs otherwise easily dismissed).

      The study you cite looks at “accepted quality measures for 26 medical conditions”, presumably common conditions attracting the bulk of medical spending, efficient treatment of which is of course vital for good public health and overall savings. I don’t intend my anecdotes to contradict the importance of the physician population having general competence, as measured by treating those conditions — greatest good for the greatest number at the lowest cost, after all. But any specific patient, facing a specific condition, might want having his specific condition addressed at a price he can afford even more than he wants a doctor competent by general metrics. Competency by general metrics should usually lead to efficient treatment of a patient’s specific condition, but especially as patients get more desperate, they could get less choosy on that score.

  38. wilskm says:

    I wonder why/if this carries over into other, non-medical (or medical-adjacent) aspects of their lives. I’ve worked with a lot of clinicians on studies who have next to no background in methods or statistics but have an unshakeable confidence that whatever they’re doing *amazing* work (it is often not).

  39. rui says:

    The last point kinda takes the premise literally when I was interpreting it charitably. If that’s the dominant reason, then there’s no bias, no paradox, it’s all good. 90% of therapists/teachers/etc can be better than the “median”, if each is using a different measure of better, and it makes sense that each is a bit fine tuned to their own measures.

  40. toastengineer says:

    Or see if you can get on the free wifi, or get on a library computer, at your local university. They’ll almost definitely have access.

  41. PhaedrusV says:

    I’ve heard that article about how 90% of drivers think they’re better than average referenced all over the place to show that people don’t understand how bad they are at something, but I think that the point is invalid. Driving is such a complicated skill and people so obviously value different driving behaviors that it’s entirely possible that 90% of people are better-than-average drivers if they each get to choose what makes a good driver.

    Hell, I know one guy who thinks he’s one of the best drivers in the world because he drives the speed limit in the left lane, and thereby helps keep people from speeding and getting into trouble. I wish I were joking.

    See, me, I know I’m a better-than-average driver because I drive around the 98th percentile on speed, and lane change constantly, and thereby I’m very considerate and don’t hold people up and keep them from where they want to go. Right? Right.

    As far as the bigger point, you’ll see the same thing anywhere where “better” isn’t clearly and quantifiably defined. I bet 80% of doctors are better than average if they get to choose the metrics they’re each judged by. This isn’t a Lake Woebegon problem, it’s just a lack of clear standards for “better”.

    • Plumber says:

      @PhaedrusV says:

      “I’ve heard that article about how 90% of drivers think they’re better than average referenced all over the place to show that people don’t understand how bad they are at something, but I think that the point is invalid…”

      Well put me in the 10% or less that knows they’re a terrible driver, and I know why.

      I hate it.

      I’ve logged a lot of miles over the years and I find driving boring when it isn’t terrifying, I often pay too much attention to flipping stations on the radio or playing with my cellphone while driving trying to get some music to make my time spent driving less irritating.

      I’ve offered to pay co-workers to drive me and my car to a BART (our local subway) station near my home and let themselves off there so I have a shorter drive home.

      Why don’t I just take public transportation instead?

      I have a wife and kids and a job with too much overtime, plus a 7AM start time which means I’d have to leave home before 5AM and walk a mile with a flashlight to catch the train, and I hate how crowded the going home train is and I hate dodging feces, needles, and sleeping bags walking to work in San Francisco.

      So I drive.

      And I hate it.

      Because I do it so damn much!

      So I drive badly!

      • I think I’m a below average driver, especially for long drives, because I get bored and pay attention to the conversation with the person next to me, or the conversation in my head with an imaginary person or the radio talk show host, or possibly with a scene I’m plotting for something I’ve been writing.

        Which is one reason why, for our long summer trip, my wife does almost all of the driving.

    • Paul Zrimsek says:

      The Dunning-Kruger narrative also implicitly assumes that people have accurate opinions about the quality of other people’s driving. This flies in the face of everything we know about cognitive bias.

  42. zardoz says:

    I sort of have to push back on that Joel Sposky post about “You are not hiring the top 1%” a bit.

    Basically, Joel’s idea is that less skilled candidates are more likely to be interviewing. Therefore, 99% of the candidates you will tend to see are serial interviewers who aren’t very good– sort of the metaphorical Wandering Jews of the job market, cursed to interview forever and never get the job.

    I don’t buy it. I buy that worse candidates do more interviews than good candidates, but I don’t believe that they do 100x as many inteviews.

    So why does every organization claim that they are hiring the top 1% of developers? Well, first of all we should step back and realize that talk is cheap. Nobody wants to say “we hire shitty engineers.” That would demoralize the people working for the organization, and make it harder to recruit good people. But objectively, salaries do vary greatly for software engineers. On the low end you have things like rent-a-coder or contract gigs. On the high end you have people making 500,000 a year or more.

    Joel sarcastically asks if all the bad software engineers are at General Motors. Having worked in enterprise software, I think the sarcasm is misplaced. There is a lot of boring software in the world that needs to be maintained. A lot lot lot. And it almost never makes the news. The big Fortune 500 companies employ a lot of people who are the software equivalent of hedge wizards. They probably can’t summon a greater demon, but they can probably integrate Vertica with your MS Access database.

    There is also no bright line between high-wire software engineering and everything else. There is a long tail of software software-adjacent professions like System Administrator (now rebranded as DevOps), Business Analyst (now rebranded as Data Science), Inforamtion Technology, Quality Assurance Engineer, etc. These people may not be the best at pure programming, but they have other skills and abilities. Sometimes these people will apply to pure software engineering positions. Usually they inteview poorly, but sometimes not.

    Finally, different companies value different things. Most big companies will hire fresh college graduates. Smaller companies usually want to hire people who can hit the ground running. Some companies are looking for a specific technology. And so on. As someone who has given a lot of interviews, my sense of things is that a pure IQ test would weed out maybe 75% of the people you don’t want to hire, but for the other 25%, you’d still need to ask more traditional interview questions. One question that comes up surprisingly often is: does this candidate actually want to work on what we do?

    • Jiro says:

      One question that comes up surprisingly often is: does this candidate actually want to work on what we do?

      I hate this. Because the answer is “any job that allows me to pay my rent and buy food is something I ‘want to do’.” Most people don’t have the luxury of taking jobs for self-satisfaction.

      • zardoz says:

        As an interviewer, I can tell you that desperation is never a good look.

        Desperation is also kind of hard to understand, considering how good the job market has been for software engineers throughout the last decade. “We need to help the poor software engineers who only make 3x the median income make 4x instead” will not get you a lot of sympathy from the normals. (Although there was a brief moment during the Obama presidency when it seemed like Google was about to merge with the White House…)

        • Aapje says:

          Perhaps that software engineer prefers to write books or play music, but programs to pay the bills, attract a partner, etc…

          The very fact that (certain types of) programming pays so well actually strongly suggests that it is not the preferred job for quite a few people who do it professionally, requiring high salaries to attract people, unlike book writing and music playing, where many people do those jobs despite low salaries. Conflating good pay with attractiveness of the work itself is a pretty big mistake, IMO.

          Note that ‘never a good look’ goes both ways. The idea that people should enjoy the work itself because the pay is good is ‘never a good look’ when it comes to employers, IMO. It betrays a pretty fundamental misunderstanding of not merely how salaries work, but even how capitalism works.

          • The Nybbler says:

            Lots of people write software for free. Even people who write software for a living. The difference is if you (for a usually-corporate value of “you”) want software for your benefit, written your way to your schedule, you’re going to have to pony up the cash. Otherwise you’re stuck with whatever happens to be out there.

          • Aapje says:

            Whilst true, the software people write for free is often different from the software that people get paid to write. Shifting those preferences requires payment.

            It’s like how many people write novels in their spare time, but very few write technical manuals for free, so companies that want to ship a technical manual with their product have to pay for it.

          • nancy says:

            You: “how good the job market has been for software engineers throughout the last decade.”
            Me:
            (1) You must not know any developers over age 60.
            (2) Eight years ago in Oregon, I knew numerous people in software who ended up changing careers because of the lack of jobs! These were folks with 10-15 years of experience, graduate degrees in math, and similar — and even good social skills.

          • zardoz says:

            I think maybe you guys misinterpreted what I was trying to say. I’m not saying we’re looking for people who will work 80 hours a week or who don’t have any interests outside work. Personally, I clock out almost every day at around 5:30pm to go home to my family. I’m just saying that we want people who are reasonably engaged in what we’re doing and who enjoy doing the work.

            So if you want, you can consider this an interview tip. Tell the interviewer a story about how you always wanted to work on what they are working on. If you don’t really feel that way, then you can fake it. (But I can tell you from experience that you will probably regret faking it, if you get the job and you don’t like it!)

            (1) You must not know any developers over age 60.

            I do know some older developers, but not many.

            People talk a lot about age discrimination, but I think a big reason we don’t see many older developers is just that the industry was much, much smaller back a few decades ago. And since most older people tend to stay in the careers they started out in, there just aren’t many older developers to begin with.

            (2) Eight years ago in Oregon, I knew numerous people in software who ended up changing careers because of the lack of jobs!

            That’s an interesting data point. I wonder if the situation would still be the same today. Remote work has become a lot more acceptable in the last few years.

          • Aapje says:

            @nancy

            I’m closely related to a 70 year old programmer, who keeps working due to workaholism. This person had a shitty career due to misfortune (like entering the job market during a severe recession), poor life choices (like getting a PhD) and low social skills, yet was able to get a break after switching to programming. He’s underpaid as a programmer (although he mostly works from home, which is a nice perk), but has a good salary compared to his previous (temporary) jobs and more importantly, has stability.

            @zardoz

            I think maybe you guys misinterpreted what I was trying to say.

            I think that I understand perfectly. You feel entitled to have people like the work, because you pay a lot. You also had bad experiences with people who liked the money more than the work.

            However, you fail to recognize that paying more, normally lowers the average satisfaction with the work, as people with high job satisfaction are most likely to accept a lower salary. This is basically how capitalism works. Salaries are intended to make people do things they often wouldn’t do otherwise.

            The particular jobs you offer may be exceptionally interesting and then you can afford to be picky. Or you may just be very good at filtering for dishonest people, who seem genuine to you.

            Tell the interviewer a story about how you always wanted to work on what they are working on.

            “As a kid, I dreamed of automating your business processes.”

            I think that for most programming jobs, this is bad advice.

  43. Plumber says:

    Well @Scott Alexander I KNOW I must be in the top 10% of people doing my job because I scored in the top ten percent of those who took the TOTALLY RELIABLE OBJECTIVE TEST that enabled me to get my job so I MUST BE AMONG THE BEST! [/sarcasm]

    Except that the scores are public record and I’ve worked alongside enough temporary hires who were in the lowest 10% of the test taker scores and that did the job better than me and most of us who got permanent jobs via the tests.

    I got my job because of them but the tests are bullshit..

    There is only one “aptitude” test: Actually doing the job.

    But we don’t do that instead we ration opportunity with multiple choice exams.

    Because people want to believe in “merit” instead of luck.

    I’ve seen enough.

    I don’t believe there’s many “correlations” with “aptitude tests” and on-the-job performance.

    There is no merit to “meritocracy” because what it measured as “aptitude” is bullshit.

    An aristocracy based on lies.

    Also, while the physicians are younger and better looking, to this patient the nurse-practitioners seem far more competant.

    Sergeants know more of war than Captains.

    • The Nybbler says:

      There is only one “aptitude” test: Actually doing the job.

      That’s certainly the most reliable test. But I’m sure it’s possible to devise a test for plumbers which could make some distinctions. At least between a competent journeyman and someone who doesn’t know NPT from an NPC. Almost certainly a practical test. If you can hire and fire quickly, it may not be worth it (just put the new guy on the job for a couple of weeks and watch him), but often that’s difficult or expensive.

      I’ve never given or taken a multiple choice exam in a job interview.

      (Also in my area, NPs seem to generally be younger and better-looking than the doctors. The latter in my own biased opinion because they’re more likely to be women)

      • Plumber says:

        @The Nybbler says:

        “…I’m sure it’s possible to devise a test for plumbers which could make some distinctions. At least between a competent journeyman and someone who doesn’t know NPT from an NPC. Almost certainly a practical test…”

        If I just judged by the tests I took I’d say you were right, they really seemed like what I would have come up with myself as a means to measure plumbing knowledge, with some other stuff (arithmetic, reading comprehension) that was similar to the PSAT test I took in high school (so IQ-ish) but in working alongside guys job performance just didn’t match up with test scores, and for some reason black guys especially worked better than they tested, I don’t know why but I experienced it often enough.

        “If you can hire and fire quickly, it may not be worth it (just put the new guy on the job for a couple of weeks and watch him), but often that’s difficult or expensive

        True, I think a lot of it is wanting a test that works well, I just don’t think they do.

        “I’ve never given or taken a multiple choice exam in a job interview”

        Many times for me to qualify to be interviewed.

        “Also in my area, NPs seem to generally be younger and better-looking than the doctors. The latter in my own biased opinion because they’re more likely to be women”

        In the hospital I go to most of the physicians now are young women, the nurses are usually older women, with the nurse-practitioners older tall women (though I did have a male NP who just retired).

        I know it’s a bit of a trick, older and tall just looks more authoritative than short and/or cute.

        • Chris Phoenix says:

          for some reason black guys especially worked better than they tested, I don’t know why but I experienced it often enough.

          The search phrase is “stereotype threat.”

    • zardoz says:

      You’re living in a very different world than most blue-collar workers. Typical home improvement contractors in the bay area hire a lot of illegals who couldn’t take any test, even a third-grade English test. Landlords often try to fix things themselves, including plumbing, and they often make a mess of it. For example, I had one landlord try to fix a leaking toilet seal by putting in grout around the toilet. Three times. No, it didn’t work, any of the times.

      We’re hardly living in some kind of empire of standardized testing. It’s even more or less illegal for companies to give an IQ test to candidates.

      I’m guessing your union wants to limit the number of members, so they come up with a test that’s more difficult than what is really needed. That’s unfortunate, but it doesn’t invalidate the idea of testing.

      • Plumber says:

        “…We’re hardly living in some kind of empire of standardized testing”

        Standardized tests are how I got my last several jobs so it seems that way to me.

        “It’s even more or less illegal for companies to give an IQ test to candidates”

        I’ve had several employers insist on a multiple choice “psychological exam” in order to get hired by them, typically many of the questions were the same questions slightly worded differently, and most amounted to “Do you think it’s moral to steal from us?”, if that’s not a test of intelligence (and/or willingness to lie) I don’t know what is.

        Other tests, while they had some job knowledge amounted to much the same as the PSAT which is supposed to result in score ranking much like IQ tests, just change the label of what’s tested from “intelligence” to “aptitude” and “compatibility”.

        “…I’m guessing your union wants to limit the number of members, so they come up with a test that’s more difficult than what is really needed…”

        Not just the union, but the City and County of San Francisco, with tests to be their plumbers very similiar to the union “journeyman turnout” test (no surprise why, most of the City’s supervisors who devised the tests were union apprentices once upon a time), and when I took it it really seemed a good test to me, but afterwards when I was assigned to work with “temporary exempt” guys I would look at their (public record) test scores to judge how likely they were to get permanent city jobs (based on interviews a supervisor may pass over some higher test scored individuals to hire a lower scored one, but not many, the city used to just hire based on test scores, but it was found to work badly). Time and time again I’d find good workers with bad test scores (disproportionately blacks for some reason). It really seemed that interviews without reference to the tests were better at selecting good employees.

        In a way there was a test, even for the temporary exempts, which was to get interviewed for a temporary position and allowed to take the test for a permanent job as it involves a lot of “doting every i” on-line (which is incredibly annoying), and providing documentation of experience of experience (both of which I found far more difficult than the written and hands on tests).

        “That’s unfortunate, but it doesn’t invalidate the idea of testing”

        The idea of testing?

        Maybe not, but the tests?

        Yeah I’d say if thr tests aren’t better at selecting good candidates than interviews they aren’t much good, you could save time an effort with a lottery (so the supervisors don’t have to interview everyone) instead of tests.

      • TJ2001 says:

        Often as not – these “Immigrants” are coming in with significant unionized trade experience… Mexico is a heavily industrialized first world nation. The difficulty for us is understanding how their work is organized…. 100% of employees in companies are unionized by law. But – qualified/papered independent contractors are widely available for a low price. So what ends up happening is that the “Company employees” act as supervisors to the “Independent contractors” who do the actual work.

        Anyhoo..

        The thing about working professionally in a trade is that you learn “How we do it here” by working with that company.. You don’t learn it by paper testing.

        One reason conventional paper testing doesn’t work as well as interviews is that Reputable companies self-police their work to ensure they provide “Workmanlike quality work” which follows standard professional conventions. The major concern for the crew leader in this case now becomes whether someone is a good match for the company culture in terms of personality and work style – aka will they do the work “Our way” and not embarrass our company while working under our name.

        • zardoz says:

          To be clear, I wasn’t saying that all of the illegal immigrants are unskilled. Some of them are skilled, and some of them aren’t. I was just saying that none of them could (or have to) pass a test.

  44. Artischoke says:

    >It will be obvious to the practical/efficient doctor that they’re satisfying their patients’ preferences much better than the touchy-feely doctor down the road would, and vice versa.

    To generalise this argument a bit, we all have somewhat different yardsticks of what makes a good [doctor/driver/parent/human being/etc.] than the person next to us. We also try to be the best according to our own yardstick and self-organise accordingly. In addition, since we know what we know and don’t know what we don’t know, our yardstick is already shaped in correspondence with our strenghts and weaknesses. It is also shaped by our needs and desires – the touchy-feely doctor probably values it a lot when he receives care and consideration for his feelings from others, learned to give such care and consideration to others and has surrounded himself with people who respond to that in his personal life. What you value and what you represent and fight for all go together.

    So of course most peoply think they are better than average in the majority of dimensions they care about. And unless you think that you have found THE objective way to evaluate a good [doctor/driver/parent/human being/political opinion to have] there is nothing wrong with this perspective. My take is that it boosts self-esteem and is a net positive. But it helps to keep in mind that other people probably operate with other yardsticks than you with equivalent justification. Walk in another persons shoes and all that.

  45. bagel says:

    Oh absolutely, software is rife with that attitude. For basically the same reason.

    Unlike on programming homework, in the real world there’s usually either zero ways to do something or many, but rarely just one. Your choice of how to do it is some unpredictable sum of your own training and experience and style, and the resources you have.

    “The last programmer sucked” is such a cliche that it’s the butt of many jokes, including at least two Dilbert comics (I can only find one: https://pbs.twimg.com/media/CvDv_uFXEAAtn7b.jpg, but the other is about the PHB complaining that nobody praises the last programmer.)

    And I even know that at least twice (out of a dozen internships and jobs) I’ve been hired to fix the last guy’s code. it wasn’t even necessarily because they didn’t have good ideas, but simply because the ideas were implemented poorly (which, it turns, was as much due to standard startupy moving goalposts as bad code practice).

    I even once worked in a lab where the prof in charge was adamant, even zealous, about only using bash and C++. He did this to vaccinate his codebase against programmers – students – who know they know best. He acknowledged that some problems really were better solved other ways, but he couldn’t teach everyone everything in order to maintain it after the author was gone. It turned the enterprise from an art project, two steps forward and one step back, into a slowly ratcheting but indomitable force.

  46. Null42 says:

    A bit off-topic, but it seems logical to me psychiatrists and therapists would specialize by personality type. Scott seems wonderfully qualified to deal with the problems of programmers, etc. with his ultra-rational style–he can probably show you where in the syllogistic chain your thoughts break down when doing CBT. He’s probably also familiar with the particular problems of people in that group, like men having trouble finding a partner and social anxiety. As such I can see why he would want to practice in the Bay Area.

    I don’t see why we don’t have more ‘psychiatrists for nerds’, ‘psychiatrists for creatives’, and so on. I kind of wonder how a psychiatrist for rural southern people would get around the stereotypes of their profession–talk about Jesus a lot? Use examples from the Bible? I’m curious.

  47. naath says:

    Mean average or median? Of course it is impossible for more than 50% of people to be above the median; but the mean? yes. If the bottom 1% are sufficiently bad. Indeed media coverage of the bottom 1% is probably disproportionately high, leading to a skewed view (downwards) of the mean “goodness” of doctors.

    The pool of “all doctors” from which this average is to be determined includes homicidal assholes (Harold Shipman was one man, but he got a fair amount of coverage in the UK) or profiteering liars (Andrew Wakefield) and these people are clearly dragging the mean average of “how good is your doctor” way down. I mean, you aren’t actively seeking to kill your patients, so you clearly aren’t the worst out there right?

  48. morris39 says:

    How Effective are MD’s
    Some branches of medicine have become incredibly effective since the middle of 20th century. That excellence includes work in the ER, the OR and some diagnostics which I think tends to seriously distort medicine as a whole to be effective. The advances came from accumulated knowledge and technology and maybe some other things. However if alt medicine is included, much (most?) is just demonstrable deception. But within conventional medicine there is virtually nothing, with exception of fields mentioned above, that is not just palliative.But there is consensus, best practices etc. Surprisingly there is little interest in research which might advance medicine e.g. organizing principle of the animal body, the implications of the internal eco system, almost to the point of deliberate intent.
    So why the universal acceptance of effectiveness ? Maybe a sort of mass conspiracy like the x steps to succeed at whatever which never happens.
    My guess is that no one here will agree with the above opinion or present a credible counter.

    • TJ2001 says:

      Define what you mean by “Medicine” vs “Alt medicine”….

      What specifically is working and what is not by your criteria.

      For example – in Psychology… Many universities studies don’t reproduce worth a hill of beans – yet Professional practicing psychologists generally help their patients get through whatever it is they need help getting through…. It’s pretty shocking to openly and honestly admit that a properly administered “Placebo” is often strikingly effective….

      And I really think “Palliative care” is under-rated… The more I learn about medicine – the more I find that many many “Curatives” actually just break into whatever “Wrong response” the body is doing and “Resetting” some switch that allows the body to cure itself…

      For example – the miscellaneous fungal “Mycin” type antibiotics are really weird things.. The latest reading I have done on them points to them disrupting the mucous layer that a bacteria produces as protection – which allows the body’s immune system to directly attack the bacteria… Aka they “move the furniture out of the way” so the body can heal itself.. Along the same lines, I am amazed at the effectiveness of nasal steroids to empower your body to heal itself…

  49. morris39 says:

    By alt medicine I include broadly, naturopathy, ancient Chinese medicine, herbalism, chiropractic etc, all of which are strongly attacked by MDs.
    Effective medicine includes mechanical interventions e.g. surgery, wound/burn treatment, toxin removal, vaccines and the like. These are truly impressive compared to earlier ages.
    One aspect of research which seems very odd is the lack interest in the animal body autonomic system, almost intentional.That system is remarkably competent. It works without our conscious knowledge and is poorly (if at all) understood). A possible conflict here is the consensus view that the brain, just one organ among many runs the show. That could stretch to the “blank state” view of our being.

    • TJ2001 says:

      My friend – when you look at things “Attacked by modern medical associations”…. Be careful to recognize that sometimes professionals attack competitors outside of their associations as a marketing/PR strategy. For example – many DO’s (Doctors of Osteopathy) practice techniques which are very much like chiropractic techniques in addition to prescribing medication and enjoying all the standard privileges and licensing as conventional “Medical doctors”…. These techniques can be strikingly effective when practiced correctly.

      A little tickler… If you notice the symbols associated with medicine – one of the most common is the “Caduces” a sort of pole with two snakes on it… That is a traditional symbol associated with Hermes, the greek god of commerce – not medicine. Just saying….

      https://en.m.wikipedia.org/wiki/Caduceus#Confusion_with_Rod_of_Asclepius

  50. Yosarian2 says:

    7. You do a good job satisfying your own values

    This one is huge in the education field, and one of the reasons most teachers think they’re better than average. “How did none of my students learn X in middle school”.

  51. Plumber says:

    Subscribe