Shamelessly stolen from my hospital’s Journal Club: Huhn et al (2014) graph the Efficacy Of Pharmacotherapy And Psychotherapy For Adult Psychiatric Disorders, and it looks like this:
Before anything else – we kind of have to assume that in each case they’re getting a representative sample of the best drugs/therapies for the disorder. In practice, there is this weird equivalency for most things: most common antidepressants work about equally well, most common antipsychotics work about equally well, et cetera. I don’t know as much about therapy, but I get the impression the same thing goes on there too. So probably it’s not much of a stretch to expect that the efficacy of whatever they studied at least kind of translates to the effectiveness of whatever real treatment you’ll get from your own psychiatrist. At the very least, even lossy and compressed information like this will tell us something.
The effect sizes are mostly around 0.5, with a few much higher and a few much lower. This is common for these sorts of studies. See for example Leucht et al, Putting the efficacy of psychiatric and general medical medication into perspective, which also finds psychiatric effect sizes average around 0.5 and finds this is about equal to average effect sizes in other fields of medicine – thus debunking the popular claim that psychiatry is less effective. Leucht and a few other authors from that piece are also involved in this one, which doesn’t surprise me much.
I do however admit my statistical ignorance in exactly what is going on here. Effect sizes are a good way to compare two unlike domains – for example, I recently noted that leading physicists are about as smart as NBA players are tall. This paper is within that tradition. In fact, if we wanted, we could describe psychiatric medications as about one-sixth as effective as NBA players are tall. This is perfectly honorable. The height of NBA players is a tough bar to live up to.
But I don’t have a good intuitive feel for what it means to use standard mean differences along a non-normally distributed variable – as psychiatric diseases no doubt are. And I’m not sure where they’re even getting their distributions from. When they say schizophrenia meds have an effect size of 0.52, are they talking about the distribution of the general population, with almost everyone near zero and a few schizophrenics way off to the right? Are they talking about the distribution of how schizophrenic particular schizophrenics are, which for all I know might be a bell curve but which is probably very different depending on how you took your schizophrenia sample? I really don’t get this and it’s preventing me from getting a good feeling of exactly how comparable these numbers are to each other.
If we just assume they’re allowed to do what they’re doing, their graph looks about how I would expect it to look. Most psychiatrists always figured that the psychotic disorders were more susceptible to medication and the anxiety disorders to psychotherapy. But three surprises stand out.
First, this graph shows that drugs are more effective than therapy in treating borderline personality disorder. That’s the opposite of the conventional wisdom, which says that some drugs can decrease impulsiveness in this population but that the definitive treatment has always been Dialectical Behavioral Therapy. But it looks like their borderline psychotherapy “meta-analysis” had a sample size of 20 patients (I would hate to see what the individual studies had!) compared to thousands of patients for most of the believable results. So I wouldn’t place too much faith in this anomaly for now and would continue to recommend psychotherapy for borderlines.
Second, this graph shows that drugs are more effective than therapy for insomnia. Now, we use drugs instead of therapy for insomnia, but conventional wisdom had always been that this was very sad, and there was great therapy available for insomnia if only somebody would provide it. But here the therapy looks mediocre at best. On the other hand, the sample size is “NI”, which I don’t know what it means but doesn’t sound promising. Also, now that anticholinergics probably cause dementia, every single sleeping pill now officially has terrible side effects.
Third, in all conditions drugs seem more effective at preventing relapse than at stopping acute episodes. My “clinical experience,” which is the fancy word doctors use for anecdotal evidence, was exactly the opposite. I now realize I probably faced a lot of selection bias – the patients who do well on their drug and don’t relapse might never see me again. Also, I have a feeling that a lot of the people who come back to me a month later and say “Well, your drug must not have worked, I’ve relapsed again” probably weren’t taking the drug correctly or at all, something which these studies probably enforce better than I can.
In general, the table seems to support psychotherapy being better than drugs for a lot of things. This would not be too surprising if true – their list is heavily tilted to the kinds of conditions therapy works well on – but a caveat is necessary.
The psychotherapy trials were generally of lower quality. Part of this has to do with the culture of psychotherapy research, but more has to do with the underlying territory – giving people “placebo psychotherapy” is more complicated than giving people a sugar pill and a lot of studies don’t bother. Also, in psychotherapy, it tends to be the patient’s therapist recording results more often than corresponding pharmacology studies use the prescriber to record results. That eliminates another layer of blinding.
This has serious ramifications. The study finds that “low-quality psychotherapy trials in general had a higher effect size (SMD = 0.74) than high-quality trials (SMD = 0.22), p < 0.001". Those high effect sizes for psychotherapy aren't looking so good now, are they?
Actually, reread that one more time. Effect sizes for the low quality trials are triple those for the high-quality trials. If you ever wanted proof that it’s way too easy to inflate positive findings if your science isn’t really exceptionally good, there you go.
The most important domain where pharmacotherapy trials are worse than psychotherapy trials is publication bias. The paper suggests that this is because psychotherapy’s lack of sufficient blinding and control groups makes publication results unnecessary. In other words, psychotherapy research isn’t even good enough to have publication bias, because publication bias at least requires you to be rigorous enough to occasionally turn up a negative result to suppress. Ouch.
> every single sleeping pill now officially has terrible side effects.
Other than being somewhat habit-forming and not available in the United States, what’s wrong with zopiclone?
Likely carcinogenic, may cause general immune system damage.
what about melatonin?
ctrl-F melatonin
What about ambien, other than possibly performing oral sex and forgetting about it?
Personally, I get really annoyed at the younger generation of psychiatrists’ reluctance to prescribe benzos. They are the only thing that fixed my panic attacks, they did so without noticeable side effects, and I found it not very difficult to taper off them, even after daily use for several months. Moreover, they work immediately and can be used “as needed.” Right now I take a valium once every few months if I feel unusually freaked out or have really bad insomnia, and otherwise I don’t need anything. Much preferable to taking an SSRI every day (which is not, in any case, an option for me, since SSRIs give me insomnia, necessitating that I take some kind of “downer” with them).
Many doctors I’ve encountered seem so obsessed with avoiding addiction that they are reluctant to use tried-and-true, effective medicines and instead favor more experimental medicines with less predictable side effects and inferior efficacy. Honestly, it would have been safer and more effective for my doctor to prescribe a stiff drink every night (the ancient form of benzos) than for them to give me abilify, risperdal, seroquel… all of which had bizarre and extremely unpleasant effects.
I have had a similar experience. I have been prescribed mirtazapine, trazodone, antipsychotics…never a benzo or z-drug.
(I’m not saying those are ineffective for sleep problems — I’m saying I’ve never been given medications widely known to be)
Or, for that matter, how about sodium oxybate (i.e. prescription dose-controlled GHB)?
I’ve never prescribed this drug myself. It’s only indicated for very serious sleep disorders like narcolepsy. I have seen people use it as a sleeping pill, and I’m sure it works, but this seems like using a nuke to crack eggs.
It’s potentially addictive, so much so that it’s a schedule 1 controlled substance, it’s fatal in overdose or conjunction with alcohol, and there’s a lot of vague research that it’s neurotoxic and impairs memory and learning ability, possibly permanently.
Effectively, GHB-receptor drugs all pretty closely match the side-effect profile of alcohol. They’re addictive, you can overdose on them, they have horrible cross-reactions, chronic use is bad for your brain, you can go into benzo-like withdrawals from them, etc.
But the effects (and side-effects) of GHBergics are heavily dose-dependent, and the few studies that have been done seem mostly concerned with the dose levels that induce general anesthesia (presumably because that’s how people treat it as a drug of abuse.)
When GHB is self-administered as a “party drug”, much lower doses are used, which apparently effect a “mild euphoria” that sounds very similar to the effect profile of “having a glass of wine with dinner” (trip reports describe it as alcohol minus the loss of motor functioning.)
I’m optimistic that a GHB prodrug (which sodium oxybate is) might be able to be used at the serum-equivalent “party drug” dose in the way melatonin is currently used: taken a few hours before bed not to knock you out, but to shift your brain toward a “neurotypical person-who-has-been-awake-too-long” state. Except, of course, for working on people melatonin fails to help.
This reminds me of something you earlier said, where you found it weird that narcoleptics could remember things that happened to them in dreams without innately and automatically recalling it was a dream.
I am narcoleptic, and I have sometimes recalled things that happened in dreams without instantly knowing they were dreams, and I figured that was how it was for everybody… not just because that’s how it was for me, but because I’ve seen several things in media about that happening and nobody ever remarks “that is weird, you should see a doctor if you couldn’t instantly tell it was a dream”. The one that most immediately comes to mind is that Friends episode where Phoebe is really, really mad at Ross for something, refusing to explain why for most of the episode because he should know, and then when she finally does explain it she starts about how it was when they were playing chess on that frozen lake and okay MAYBE this was actually a dream.
So, is this still a strange trait of narcoleptic people that, for some reason, people put into TV shows and books etc as if it were a thing most people can relate to? Or what’s going on?
Can’t speak for anyone else, but that’s happened to me a moderate number of times, and I assume I’d know if I had narcolepsy.
People who develop narcolepsy often had many sleep-weirdness symptoms before they developed the full-blown condition, that is, the problematic symptoms. So I wouldn’t be surprised if you are pre-narcoleptic without knowing it. Also, most people have no idea what narcolepsy is, so fail to self-diagnose, even when they are sick enough that they go to the doctor.
(I’ve had memories that everyone else disagreed with, that I concluded were dreams, but I’ve never been sure that was the origin of the memories. In particular, they never had surreal details for simple assessment.)
The post Beat mentioned, 8th paragraph.
(two above anon here)
Sounds implausible. I haven’t experienced any other symptoms listed for it in the top four google sites. I’m also just above the age range when onset is supposedly most likely.
Anyway, it’s not like I have no concept of the difference between dreams and reality. Usually I wake up aware of my dreams as dreams. But once in a while I’m in a negative mood for a while about something that turns out to be only a dream, or I’ll be presupposing something in conversation that when I reflect on it turns out to have been from a dream.
That’s odd; I just had a dream in which I was insisting to myself that of course I’d been on a plane, hadn’t I flown to [some country]?
It took me a while after I woke up to go “Hang on, that was a dream, you’ve never been to [some country]!”
Also happens to me, but not more than once every few years. (I once went to my aunt’s house for Christmas and realized that it looked completely different than I remembered.)
trazadone and mirtazapine?
not very anticholinergic.
Or hydroxyzine.
This is kind of taking the discussion off on a bit of a tangent, but:
“… now that anticholinergics probably cause dementia, every single sleeping pill now officially has terrible side effects.”
I applied some quick and dirty google fu on this, and I’m wondering how confident they are that it’s the anticholinergics that increase the risk of dementia, as opposed to the organic conditions (or a subset of such conditions) that are being treated by these drugs that are correlated with dementia? (The article is behind a pay wall for me, and none of the secondary sources that discuss it seem to raise that possibility)
Short answer – this is the crux of the issue, you are correct to bring it up, and there’s no good way short of unethical randomization to be sure – but insofar as they’ve investigated it, it doesn’t look like this is what’s going on.
They’ve done sub-analyses by reason the drugs are prescribed – for example, people who are prescribed the same drug for sleep and for bladder problems show similar effects. There’s an apparent dose-response relationship. And there’s a very plausible mechanism – acetylcholine is heavily involved in dementia, so much so that the most common treatment for dementia is cholinergic drugs (ie the opposite of anticholinergics).
So I’d say we’re still not sure, but that there is some effort to rule out confounders and what there is hasn’t been good for the confounder hypothesis.
I’ve never read anywhere that anticholinergics actively cause dementia (in the sense of making someone who wouldn’t suffer from dementia develop dementia). I thought they just impaired cognition, which could make you go from 17 to 14 in the Mini Mental State examination or something (with the expected real life consequences), but the effect would disappear as soon as you stopped the drug.
I assume the “dementia” reported in the study was real, and a persisted after the anticholinergic drugs were stopped (which is probably the first thing any doctor does when faced with a new-onset dementia), which would mean they really cause dementia. Is this the right take-home message from this study?
Yes, the study takes that into account. It is about current mental state vs cumulative 10 years of anticholinergics, so the doctors had plenty of time to stop the drugs.
For psychotherapy, I think the N of the therapists is at least as important as the N of the patients. Maybe use sqrt(N_therapists*N_patients)?
I can probably explain your “relapse effective treatment” issue in one word: withdrawal. The human body, brain definitely included, is teeming with opponent processes. Provide meds for a while, then take them away – what happens? Relapse. And what will most easily fix that? Restoring the meds.
The therapist-evaluated effectiveness of therapy is … mind-boggling. I mean, a case study is one thing. But if you’re going to do statistics – use numbers that mean something.
Your hospital has a journal club? Is that something that residents are generally expected to take part in, or is it just you and other doctors who, like you, are inclined towards scholarship?
I took a look at one of the schizophrenia studies they reference (Jauhar et al. 2014). In that study, they use scales measuring overall, positive, and negative symptoms, and hallucinations. These scales have assumably been normed in some relevant populations where the scores are normally distributed.
edit: It seems that the scales used do not in fact have norms. Instead, the effect sizes are calculated based on the pooled standard deviations of the treatment and control groups. This indeed means that the symptom variability that exists in the samples may have a big effect on the results.
You seem to be using “effect size” to mean dimensionless z-score. I don’t think that the normal usage is so specific. I think the normal usage encompasses both raw numbers with units and “standardized” dimensionless measures like the z-score. It’s fine to use it to mean z-score without comment, but when you say
you seem to be contrasting standardized effect sizes to raw effect sizes, which is confusing. I recommend inserting the word “standardized” into that sentence, but all the other uses are fine.
I want to strengthen this comment. People use “effect size” to mean by raw and standardized effect sizes, but when they use it to distinguish the two, they generally mean raw, exactly the opposite of your usage.
What do you mean by “every single”? Does this mean melatonin is unsafe, or just the ones that act more directly on (acetyl)choline?
Also, the article you linked said that the study was only on people 65 and older. If decreasing acetylcholine activity among people whose choline production is already decreased due to old age is bad, how likely do you think it is that it’s also bad among younger people?
Melatonin is fine as far as we know. I mean the common sleeping pills prescribed by doctors, which include Ambien, Lunesta, Desyrel, Vistaril, Benadryl, Restoril, and a few others.
I doubt that using anticholinergics when you’re 30 will do much, but that’s a total wild guess and might depend how long you use them for.
What bad side effects are you thinking of in the case of Desyrel? It seems pretty light on side effects for an antidepressant.
NB, link at the top isn’t working.
I’m pleasantly surprised at how overall effective treatments are actually.
Is there a reason depression is left off of this list? Seems like the one of the more common ones, and I would have liked to see it visually represented next to all the other conditions.
It’s on there as MDD, standing for “major depressive disorder”.
Oh. Derp.
So, jumping on the anticholinergics fear bandwagon, how about Bupropion?
Internet forums (esp Longecity) claim it’s bad for memory, so is this also Bad News?
Okay, so I was concerned about this (because someone close to me takes it, along with an anticholinergic) and nope, I couldn’t find any evidence that it was *anticholinergic*
…this doesn’t promise that it’s perfectly fine for memory, but it’s a little info.
Additionally, depression and anxiety *do* cause problems with overgeneralization of autobiographical memory [DeBeers 2014 is a good study, iirc] so I’d be vaguely suspicious of research on burpropion and memory that wasn’t explicitly controlling for this.
Orange means drugs in the first figure, for anyone who’s confused. Also in the second figure, but that’s labeled.
I wonder if part of the reason that the medications look so good in preventing relapse is that there were comparisons made between people who stopped or were taken off of medication vs people who continued on mediation (and no comparison with people who never took the medication). You might just be getting rebound effects. Going off of lithium can make you manic. Stopping an antipsychotic can make you psychotic. Withdrawal effects are going to inflate the efficacy for prevention of relapse.
Great minds think alike. Or alternately, obvious factor is obvious. Or somewhere in between.
Eeeek, on the sleeping pills news! I take diphenhydramine for insomnia, and I’ve done so for several years!
Hmm I take it from this post you think less of psychology without meds? Where does psychology become useful, or is it useless in almost all cases without medical support? Can psychologists recommend psychiatrists? I have become very interested in psychology as possible professional pathway, but the medical side of it doesn’t appeal/I don’t think would be a strength/would take a long time and a lot of money neither of which are plentiful enough to sustain however long it takes to get a medical degree. This is from an Australian perspective.
there are no placebos if the medication has side-effects. this is true for the patient and the doctor. if a supposedly blinded doctor sees that the patient has a dry mouth or mydriasis or whatever how can it not affect his determination of effectiveness?
and how can PT possibly be faked?
1. Placebo medications can (often) cause placebo side effects.
2. Some trials use active placebos which have minor side effects but aren’t known to have particularly interesting psychiatric effects.
3. Placebo psychotherapy is hard to do. One popular method is to just have unstructured talking about your problems.
I would think a placebo psychotherapy would explicitly avoid doing any of the things that would hypothetically cause psychotherapy’s benefits. “Talking about your problems” obviously has therapeutic effects (for people who don’t have anyone to talk to about their problems), whether or not structured formally or guided with any skill by a professional.
I would think that a placebo psychotherapy, then, would be something that has all the trappings of psychotherapy—going to a professional once a week, sitting in a comfy chair, active listening, etc.—but which doesn’t let you talk about your problems. Maybe you’d be asked to read the therapist a story from a book, instead.
That feels much more likely to mirror the structure of the psychotherapy (subject loses time out of each day; subject visits a trusted authority who defers to their comfort; etc) while being, in benefit, more like the other kinds of conversations you’d expect these people to randomly have throughout the day: rote role-script conversations with cashiers, “on-task” conversations with coworkers, etc. Someone willing to let you talk to them for an hour about whatever is on your mind without the lens of a role-script in the way is not something that can be expected as an ambient part of the environment, so why is it part of the placebo?
“Subjects were randomly referred to either a local psychotherapist, or a reputable hooker.”
Which one is the placebo?
Depends what your problem is.
It’s not exactly uncommon for placebo patients to worry themselves into known side effects like dry mouth or insomnia. And some studies do use “active placebos,” ie drugs that have a similar side effect profile but are not known for the desired effect.
You can fake a specific PT method by doing generic “supportive therapy” instead (and, I guess, hoping the patient doesn’t look up the therapy technique mentioned in the study paperwork and compare what they’re getting to that). That doesn’t help for therapy vs. no therapy at all though.
EDIT: Crossed comments so this wasn’t necessary. Oh well.
I don’t think less of psychology at all! I don’t think great things about psychoanalysis, but cognitive-behavioral, dialectical-behavioral, and a bunch of other therapies are really useful. I mean, just look at the graph up there – that’s not all poor methodology!
The way it works in America (I don’t know about Australia) is that people can go to either a psychologist, a psychiatrist, or often both at once. The psychiatrist will handle the medicine angle and the psychologist the therapy angle. Both sides are usually pretty good about directing someone to the other if they honestly believe the other would be more helpful.
(Usually psychologists know a lot about medications, but are legally blocked from prescribing them for political reasons. In a few US states, psychologists can prescribe some but not all medications. Usually psychiatrists know some therapy, but less well than psychologists, and usually the financial incentive is for them to outsource that to psychologists because meds pay better).
I don’t know about Australia, but I warn you that in the US, becoming a clinical psychologist also takes a lot of time and money, and the competition for jobs at the end of it is really brutal. Make sure you know what you’re getting into. Becoming a social worker, ditto. Becoming a therapist or counselor is pretty easy and I don’t think there are licensing requirements.
I am pretty sure there are licensing requirements for both therapists and counselors in the US. I considered alternate routes; for instance, becoming an MHC (mental health counselor) but found exactly the same cost and approximately the same experience: two years in a master’s program, two internships, debt, with less flexibility in jobs.
Ditto for substance abuse counselors, (in most states), marriage and family therapists, school counselors (except most have additional requirements), etc.
In fact, in my state the only thing I know of that doesn’t require licensure is social workers (many non degree holding people can use the name)! It’s weird and confusing and the national social worker org is trying to fix this.
I agree with all of Scott’s comment otherwise.
If you’re interested mainly in the practice of doing therapy, I would recommend the social work track if it’s similar to the US version of social work. I would flag that it pays less and has less status, but I feel as though I do far more hands-on work and get more training. That being said, I have to push pretty hard to work with research, which continues to make me sad.
Scott, you say your bad at math, but I can’t even read that graph.
In fact, a lot of times when I look at the studies you quote, you’ve come to a dramatic conclusion and I can’t tell if they are trying to say anything at all. Is there a course I can take to help me cut through the bs and weasel words I find in studies, or should I stick letting doctors like you with blogs interpret the studies for me? (And where do I learn to read those kinds of graphs?)
Thanks
If you have no idea where to start, I can recommend a book; “Turning Numbers Into Knowledge” by Jon Koomey.
from the blurb:
The world keeps getting more complex, but becoming a better problem solver can help you make sense of it all. Mastering the art of problem solving takes more than proficiency with basic calculations: it requires (among other things) understanding how people use information, recognizing the importance of ideology, learning the art of story telling, and acknowledging the important distinction between facts and values. Turning Numbers into Knowledge is the first comprehensive guide to these and other essential skills. Full of tools, tricks, and tips for solving problems in the real world, it will prepare you well to make independent judgments about the numerical assertions of others and to generate cogent and compelling analyses of your own.
The blurb is fairly accurate IMO, so might be worth a look
As someone who’s had allergies her entire life, the fact that I’ve been spending 30-odd years taking stuff that might ruin my brain isn’t a heartening one.
As for therapy versus drugs, I personally find that perversely any sort of motivational stuff or otherwise talking encouragingly about my problems invariably makes me feel worse. My mind always twists it all into making me realize just how hopeless I and/or my situation is. It’s exceedingly frustrating. It makes me wonder how effective typical therapy methods are for people in general who tend to overthink or overlogic everything by their inherent natures.
Then there’s me, who seems to get on OK with the sort of unstructured talking-about-problems that seems to fit some descriptions of “placebo psychotherapy” but who has a tendency to get very distressed when, for example, people attempt to apply proper CBT techniques.
I honestly don’t know if I’ve ever had CBT applied to me before. I do know that any time I try myself to retrain my thinking to be more optimistic and confident, it feels fake and like I’m lying to myself about the reality of things.
(I’ve somewhat cynically developed the belief that the goal isn’t actually to be completely sane, but to be just barely insane enough that your mind can successfully lie to yourself in the right ways and gloss over the really painful truths. Kind of related to the Discworld concept of being knurd.)
Have you ever tried training yourself to be less pessimistic rather than more optimistic?
I view myself as a realist, in the sense that I look at all the variables I’m aware of and reach what seems like the most logical and realistic conclusion. So training oneself to be less pessimistic seems like the same difference; it’s potentially looking at things in other than what seems like the most realistic way.
That’s not to say, though, that being realistic can’t result in optimism (or at least less pessimism) if most of the variables in a given situation are instead neutral or positive. Just that it’s hard to train oneself away from thinking whatever the balance actually is.
@Liz
Your observation and reasoning patterns can be biased, making the results less realistic.
Of course, though in my personal case I find people tend to not come up with things I haven’t already thought of.
(Which is, FWIW, a complaint and not a boast, since I never ask for help or advice unless I’ve already exhausted everything I personally can think of and it hasn’t helped.)
I’m talking out my ass here, but I wouldn’t be remotely surprised if CBT had side effects just like everything else does. You could just have an unusually strong adverse reaction.
Oh yes, people who say “therapy not drugs, drugs have side effects” are talking nonsense.
My question is why we seem to see relatively few interventions with large, robust effect sizes even after being studied carefully. My experience with my own life suggests that self-experimentation is quite valuable (I seem to see large effect sizes)… am I fooling myself? Or is it rare to find large effect sizes that generalize across individuals? Or what?
Let’s assume the high-quality researchers know how to calibrate their research and their effect sizes are correct. Three obvious reasons you would get a different-seeming result spring to mind.
First, you may not know how to calibrate your own experiments. I don’t know how you do them, but given that the data here suggests that everything but the most rigorous experiments are prone to inflated positive outcomes, and you cannot rigorously experiment on yourself, you would expect to record better outcomes than real research does.
Second, you are not a random population member. You are a population member who is running an experiment on themselves. This introduces several biases, including your financial and time investments in whatever you’re doing and that you have a belief set that means you expect self-experimentation to be a productive enterprise. In short, large placebo effects and incentives to give yourself the answer you want to hear.
Third, you have whim-level control over when you stop and start your experiments. If you, for example, start all your experiments on things to do to increase your productivity when you’re feeling unproductive, you can expect all of them to look like they worked due to simple regression to the mean.
One wonders whether there’s ever been an extended trial of “patient-led fiddling-with-meds-and/or-therapy”. It’s not easy to define a control group. Anecdotes suggest that it’s an approach that works quite well, OTOH it takes a long time to do, so there’s plenty of time for regression to the mean to kick in.
>giving people “placebo psychotherapy” is more complicated than giving people a sugar pill and a lot of studies don’t bother.
Wait, but why?
Have a “therapist” and a “student” who is just observing. One is a real therapist, one is a randomly chosen volunteer who is instructed to chat with the patient kindheartedly about the problem and be a good listener, they swap…and boom: single blind schema for testing whether various therapeutic methods are better than simple human contact (or just pit two therapies against each other). There’s even a therapist in the room at all times to put a stop to anything potentially bad.
If the patient is aware that they’re in a study and that they might get placebo…it doesn’t seem more complicated or more ethically thorny than placebo pills in principle? From the patient’s perspective, placebo therapy shouldn’t be any more negative than placebo drugs. I suppose it does cost more in terms of therapist-hours than pill experiments cost in doctor-hours, and that’s enough of a disincentive to make a difference.
I know I’ve heard of a similar study–I think from Scott, actually. As I recall, they put out job ads for a secretarial position and a therapy position, and picked half of the therapy pool from each group. Sadly, I don’t remember the results, but I suspect they found personality measures mattered more than which pool the person came from.
My wife is bulimic but she refuses to acknowledge it. She admitted that she was bulimic before we met but she claimed that she hadn’t been symptomatic for years. That was false.
We’re going to have a difficult talk soon, but my question is this: if she agrees to seek treatment, what type of PT would you recommend.
Thanks
Person with eating disorder (admittedly, not bulimic most of the time) here. I have totally done the thing where I said I ‘used to be [anorexic]’ when it wasn’t true. I would add that in my case (and potentially, though not necessarily your wife’s), it was because I thought I was doing food ‘normally’ and my brain was so distorted that it believed it. Then I would remember that people ate more than one meal a day, and try for two, and decide that *now* I was recovered! I’ve had an eating disorder for eight years, and I only broke that cycle in the last three years.
All of that being said, I spend a heck of a lot of time writing about eating disorders and reading eating disorder treatment research. The things I’ve found to be best for working through what treatment works are the Science of Eating Disorders blog (http://www.scienceofeds.org/), and the phone app called Recovery Record (which is demonstrably evidence based, and the only known evidence-based phone app) which is free.
For therapy types beyond that, one particular modality, Cognitive Behavioral Therapy seems to do best (see: http://www.scienceofeds.org/2013/09/22/cognitive-behavioural-therapy-for-bulimia-nervosa-in-the-real-world-whats-the-evidence/), and I tend to prefer therapists who have interacted with eating disorders before (just because it’s easy to misstep).
Finding the right therapist and right therapy seems to me to be so necessary for therapy to work, that I don’t actually understand how double-blind trials could be useful. Even outside of the “placebo therapy” problem. Maybe if you just administer a randomly selected group of people a course of CBT with a randomly selected therapist they do on average get a bit better, but that’s not really capturing the potential of therapy as an intervention. Plus I’d bet that placebo therapy with the right person would be a lot more effective than actual therapy with the wrong person.
It seems a little like trying to do a study of whether relationships improve people’s lives by sending single people on a series of dates with strangers chosen by lottery. (pre-emptive note for quibblers: I fully acknowledge the analogy exaggerates the problem it highlights.)
Of course this opinion is only supported by my own personal experience, which you can probably guess at. But I don’t think the pattern of “tried multiple therapists until I found the right one and then my life got a lot better” is particularly uncommon. I’d be interested in whether there is any evidence base one can use for picking the right therapist. I suspect that it’s a matter of pure luck.
“I’d be interested in whether there is any evidence base one can use for picking the right therapist. I suspect that it’s a matter of pure luck.”
You can try to google scholar ‘therapeutic alliance’ and see where that gets you. It’s not completely the same thing, but I’d expect some of that research to have looked at some of the variables which might be of interest. It seems that a lot of research has been done on this stuff (though it also seems to me that a lot of it is quite poor). Here’s a quote from Handbook of Cognitive-Behavioral Therapies:
“The “therapeutic alliance” refers to the collaborative relationship between the therapist and patient. Research in the early 1980s showed that the therapeutic alliance is positively related to change in various types of psychotherapy (cf. Morgan, Luborsky, Crits-Christoph, Curtis, & Solomon, 1982). A vast body of research has examined the relationship between therapeutic alliance and outcome across a variety of treatment modalities and mental health problems. In general, reviews of the literature indicate that a stronger therapeutic alliance is associated with better treatment outcomes (Horvath & Bedi, 2002; Martin, Garske, & Davis, 2000).
As DeRubeis and his colleagues (DeRubeis & Feeley, 1990; Feeley et al., 1999) noted, however, many of the studies reporting a significant alliance–outcome association do not control statistically for symptom change preceding the assessment of the alliance (e.g., Castonguay, Goldfried, Wiser, Raue, & Hayes, 1996; Gaston, Thompson, Gallagher, Cournoyer, & Gagnon, 1998). Thus, in such studies, a significant alliance–outcome correlation may in part reflect the influence of prior symptom improvement on the therapeutic alliance. Indeed, DeRubeis and Feeley (1990) and Feeley et al. (1999) found that the alliance was not a significant predictor of subsequent therapeutic change. In addition, they found that in the latter half of therapy, the level of therapeutic alliance was predicted by the amount of prior symptom improvement. In other words, these two studies found that good therapeutic alliance early on did not predict good outcome, but that good outcome early on predicted good therapeutic alliance later. […] It is possible that the correlations reported in some studies reflect the impact of good outcome on the alliance rather than any causal effect of the alliance on symptom improvement.”
Some related observations on patient behaviour and therapist responses to patient behaviour:
“Patient resistance typically bodes poorly for treatment effectiveness, unless it is managed skillfully. It is generally assumed that some patients are more likely than others to resist therapeutic procedures. “Resistance” may be characterized as a dispositional trait and a transitory in-therapy state of oppositional (e.g., angry, irritable, and suspicious) behaviors. It involves both intrapsychic (image of self, safety, and psychological integrity) and interpersonal (loss of interpersonal freedom or power imposed by another) factors (Beutler et al., 1996). “Reactance,” an extreme example of resistance, is manifested by oppositional and uncooperative behaviors. […]
Resistance is easily identifiable, and differential treatment plans for patients with high and low resistance are easily crafted. The successful implementation of these plans, however, is often quite a different matter. Overcoming patient resistance to the clinician’s efforts is difficult. It requires that the therapist set aside his or her own resistance to recognize that the patient’s oppositional behavior may actually be iatrogenic. In a study of experienced and highly trained therapists in the Vanderbilt Study of Psychodynamic Psychotherapy, none were able to work effectively with patient resistance (Binder & Strupp, 1997). Rather, therapists often reacted to patient resistance by becoming angry, critical, and rejecting, which are reactions that tend to reduce the willingness of patients to explore problems. […] Research suggests that low-resistance patients may […] do better with authoritative and directive roles compared with nondirective ones (Beutler et al., 2003; Beutler, Engle, et al., 1991).”
More specific guidelines on how to deal differently with patients high and low in resistance levels are also presented in the text. There’s on top of that some research (which I’m not sure if I trust) presented in the text which indicated that some treatment methodologies within the CBT-framework do better than others in specific patient contexts; it may thus be that part of the ‘chemistry’-variable is explained by treatment methodology, rather than things like personality, and to the extent that this is the case one might use the available research to aim for a better fit (don’t use a guy using methodology X if you have problem Z, because it’s expected to work less well than method Y – most therapists presumably combine treatment methodologies, which makes it harder to use methodology as a selection variable, but on the other hand this might also improve average outcomes and make selection less important).
It’s not that there’s no knowledge about this kind of stuff, but I don’t have a big overview over which variables are most important – I’m not sure anyone does. It’s obvious that some factors which are under the control of the therapist affect the likelihood of success of treatment, and some informal guidelines and suggestions for how to optimize outcomes given stuff like this do seem to exist. But that doesn’t help you if your therapist doesn’t follow the advice in the textbook and for example get angry at you if you think his ideas are stupid and unhelpful.
The above comment wasn’t really an answer, but I hope you got something out of the details I added.
By the looks of the graph, I’m not surprised that medicine would outperform therapy for treatment of Schizophrenia. What does surprise me though is that it looks like therapy makes a schizophrenic patient worse off. Or am I misreading that chart?
N=164. The point estimate is negative, but 0 is in the confidence interval.
It may not be that weird. If there was a way more effective antidepressant, antipsychotic etc, it would probably become the most common one — assuming no regulatory inadequacy. Significantly worse drugs will likely not hit the market or be approved. Thus the only ones actually used will be roughly equivalent and somewhat complementary, in that some may work better on some individuals than on others.
Has this kind of thing happened in the past? I know that, e.g., in antidepressants the market has largely moved from MAOIs and tricyclics to SSRIs but my impression was that this had more to do with the side-effect profiles of those drugs than their efficacy. But I know very little about the history of psychopharmacology.
One thing that worries me about basing your evaluation of psychotherapy (Well, mostly CBT, though I can see it applying to other things) on evidence like a list of standard questions is that it seems like it would be easy to fall into the trap of teaching people to beat the test. In CBT, for example, they try to teach people to think more positively, and it seems possible that people don’t actually start feeling better, but just learn to signal it less and give more positive answers on test. This sort of thing seems much harder to fake by medication (especially since you have easier placebos), so this gives another reason why evaluating the effectiveness of psychotherapy is hard.
That worries me too, but I’ve had difficulty explaining my concern, so I wrote a 1400 word essay which attempts to lead up to the concern gently, without triggering peoples psychological defenses.
There’s an economic concept which is similar to this, but I can’t remember its name. When central planners in the Soviet Union decreed that a certain amount of goods be produced, and measured production in tons, then producers started making awkwardly heavy goods…
I also have similar concerns about CBT.
I think you might be looking for Goodhart’s law?
The study says anticholinergic, but everything the study lists is an antimuscarinic. There’s no antinicotinic listed. That seems like it would make a pretty big difference in how it should affect prescribing practices.
What also strikes me about the chart in question is that the confidence intervals of the SMD’s for psychotherapy are greater than the confidence intervals of the drugs, though borderline personality disorder looks difficult to treat no matter which strategy is used. This implies that positive therapeutic results are partially dependent on the therapist in question, which is difficult to control for, as the only indicator of the efficacy of a therapist happens to be the efficacy of the therapist. There are other things one might assess, however — Carl Rogers might argue that a test of implicit association (as a proxy for unconditional positive regard) might do the trick. Or maybe some objective measure of empathic understanding + technical knowledge. Could be worth a shot.
And before anyone objects, yes, I’m aware that Client Centered Therapy is understood to work better on patients with ‘milder’ symptoms. But I believe the real take away is that “client centered therapy” works (or would be beneficial) for people not currently under enough acute mental distress to be actively looking for a therapist.
No, the confidence intervals are large for therapy because the studies are small, not because of any difference in the structure of the studies, let alone a therapist effect.
Thanks so much for writing posts like this. They’re tremendously helpful.
“Also, in psychotherapy, it tends to be the patient’s therapist recording results more often than corresponding pharmacology studies use the prescriber to record results. That eliminates another layer of blinding.”
One important related observation emphasized in a text I recently read (Newman & Kohn’s Evidence-Based Diagnosis – that book is awesome, incidentally, and highly recommended; I wrote a blog post about it here. It’s the sort of book I imagine many readers of a blog like this might like.) is also that proper blinding is in general more important in research using metrics which are to some extent subjective (‘how do you feel?’) than it is in research where metrics are relatively objective (‘blood pressure is 120/80’). As results of psychotherapy are usually quite subjective (compared to a T-cell count) this aggravates the problem.