New study shows that e-cigarette users are no more likely to quit smoking tobacco after a year than non-e-cigarette users. In fact, the trend is in the opposite direction – e-cigarette users are less likely to give up their regular cigarettes. I’m skeptical. r/science is skeptical. The experts are skeptical. Even the authors of the study sound maybe a little skeptical.
The study surveyed tobacco smokers for various information including whether they smoked e-cigarettes in addition to their tobacco. A year later, they went back and surveyed everyone again and asked them if they were still smoking tobacco. And the people smoking the e-cigarettes were no more likely to have quit than the others.
Let’s transition from reality to Hypothetical World. In Hypothetical World, there are only two kinds of smokers, Short Smokers and Long Smokers. The moment someone smokes their first cigarette, God flips a coin and assigns them into one of the two groups based on the result. Short Smokers are predestined to smoke for exactly one year before quitting; Long Smokers are predestined to smoke for exactly fifty years before quitting.
A scientist in Hypothetical World wants to discovery what percent of first-time-smokers become Short Smokers versus Long Smokers (the real proportion is 50-50 since God’s coin is fair, but she doesn’t know that). So she uses the same methodology as this study. She hangs around a tobacco shop and accosts the first thousand people who come in to buy cigarettes, getting their names and phone numbers. Then a year and a day later, she calls them all up to ask if they are still smoking – since anyone who keeps smoking for a year and a day must be a Long Smoker.
So she finds something close to 2% of people are Short Smokers and a whopping 98% are Long Smokers. She incorrectly concludes that God is rolling a d100 and only assigning Short Smoker status to those who come up 99 or 00.
Don’t see why she would make this mistake? Consider a particular generation of Hypothetical people over their lifetimes. The Short Smokers will only smoke a single year out of their lifetime; the Long Smokers will smoke fifty years. When the scientist does her study in a randomly selected year, she only has a 1/average_lifespan chance of catching any given Short Smoker, but a 50/average_lifespan chance of catching a Long Smoker. So, her original sample will contain fifty times more Long Smokers than Short Smokers, and she will mistakenly conclude that their pattern is fifty times more common.
Now transition back to reality. Suppose there are two types of e-cigarette users – successful and unsuccessful. The successful e-cigarette users try e-cigarettes, immediately decide they are better than regular cigarettes, and switch to using e-cigarettes exclusively within one month. The unsuccessful e-cigarette users try e-cigarettes but just don’t get everything they love about tobacco from them. They sort of futz around with e-cigarettes and regular cigarettes and tell themselves that one of these days, they’re really going to stop the regular ones entirely and transition totally to e-cigarettes. These people continue futzing for let’s say ten years before they either finally quit tobacco, give up on e-cigarettes, or die.
In that case, any sample of tobacco smokers taken at a particular time will include a hundred twenty times as many unsuccessful e-cigarette users as successful ones. We expect unsuccessful e-cigarette users to continue their past pattern of futzing around, so it’s not surprising that this sort of sample finds most e-cigarette users not only can’t easily quit tobacco using e-cigarettes, but actually have a harder time quitting tobacco than normal smokers – they’ve already been preselected as The Group That Even E-Cigarettes Can’t Help; as The Group That Tried Something Billed As An Anti-Smoking Aid But Failed At It. It’s a pretty general rule of medicine that people who failed treatment once are more likely to fail treatment a second time.
This is a very speculative explanation and I haven’t heard anyone respectable at a major institution advance it yet, but it seems to me like the most likely reason for these findings. All I have to go for with the study right now is a preliminary “research letter”, but hopefully we’ll know more when the real thing comes out.
Lest this post be entirely pro-drug, here’s a clip of my addiction-medicine teacher and sometime-boss lecturing people about marijuana on Fox News last weekend. He is a great doctor and it’s neat to see him finally getting some of the celebrity he deserves. Even though his politics are terrible.
Typo alert:
So she finds something close to 2% of people are Short Smokers and a whopping 98% are Short Smokers. She incorrectly concludes that God is rolling a d100 and only assigning Short Smoker status to those who come up 99 or 00.
this comment will self destruct if corrected in time..
I’ve been thinking for a while it would be possible to
crowdsourcerecruit study participants on /r/stopsmoking or someplace similar. There’d probably need to be an IRB involved; one of my colleagues at Dartmouth might be able to do something about that.Surely that would introduce sampling bias. People who go on somewhere like /r/stopsmoking are likely to be more motivated to quit smoking than the average person
Isn’t that why you have a control group of r/stopsmoking members who don’t use ecigs?
That was the idea, exactly. Thanks, Ozy.
Except that since they are on r/stopsmoking the control are likely (more than the general population) to be using some other anti-smoking aid.
@Anonymous
Well, okay. But the “general population” isn’t really relevant, is it? Whether e-cigarettes are effective is properly understood with regard to their effect on people motivated enough to try a smoking-cessation aid, not their effects on the general population.
Once a person even thinks to ask “Will e-cigs help me stop smoking?”, we have a reasonable basis for believing that “people who want to stop” is a more representative group than “general population”.
How many e-cig users are doing so with the intent to quit, versus being able to feed their nicotine addiction in places where burning leaves is not allowed, or impolite? Was that controlled for in the study? The page I can see shows a general “intention to quit at time t in the future”, which isn’t the same thing.
The fact that this post linked that other post meant that I got to read this sentence again:
And that makes me really happy.
Hard to tell without access, but non-uniformity in the population trend could also be an issue; e-cig use has been more than doubling per year since 2008. This undermines some standard assumptions.
In this scenario, the majority of e-cig users at any given moment have just started using them, which biases the sample population. The demographics (particularly in age, wealth, psychology and tobacco consumption) of the user base are going to change rapidly as it moves from innovators to early adopters to early majority users.
Your point about short vs. long smokers is entwined with this; the early adopters are probably disproportionately likely to be in the “Tried Something Billed As An Anti-Smoking Aid But Failed At It” class, even compared to hypothetical users in a steady state environment.
Does this apply if the follow-up study is simply looking at the people originally questioned, as in the hypothetical study? Seems to me it only applies if you’re comparing two population samples at different times, not the same sample over time.
The study is comparing e-cig users (a very small and rapidly changing demographic) to non-e-cig users (a large, relatively stable population). There are going to be many uncontrolled variables for which the two populations differ. Some but not all of these differences will be in factors that are rapidly changing within the overall e-cig demographic.
For example, a sample of e-cig users taken in 2011 will probably differ significantly in novelty-seeking[1], age, and motivation to quit from a sample of non-e-cig users taken in 2011. It will also probably differ in those measures from a sample of e-cig users taken in 2021–the population is likely to regress to the mean in a lot of ways by then.
1: There’s some reason to think novelty-seeking itself is correlated to propensity for addiction. That alone would render the whole study fishy.
Ooh, I’ve seen a variant of this problem in action in a completely different context. Kindly permit me to ramble, and maybe some interesting general principle could be derived.
In the setting for Exalted Second Edition, you have, among other parties, the Lunar Exalted, 300 reincarnating wizard shapeshifters who can live for thousands of years and grow more powerful in proportion to their current age this life, and the Terrestrial Exalted, a clan of several thousand true-breeding wizard elementalists who run the Empire at the centre of the world, live for a century or two, and also grow more powerful in proportion to their age, albeit at a slower rate.
(they’re not literal wizards, and all sorts of other simplifications have been made to aspects that would mostly be distracting from the point.)
The books assume that player characters will frequently be young (<5 years) Lunar Exalted setting out to make a name for themselves with their wizardly powers. The problem with this is that there's a bit of a shortage of young Lunars around, because of the lifespan issue. Under a random distribution you'd expect approximately 0.6 Lunars to be under five years old, since their maximum lifespan is in the vicinity of 2500 years. This means one would have to fudge the numbers just to have a party of player character Lunars, let alone other similar-age Lunars for them to interact with in the vicinity.
Enter the war. The Terrestrials and Lunars are at war, and have been so for millennia. Terrestrials try very hard to hunt down and kill any Lunars they find, resetting the power clock on the Lunar in question; Lunars do fight back but aren't trying quite as hard to kill all the Terrestrials, preferring to flee and organize in their own lands at the edge of the world. This should, in theory, make for more young Lunars and allow the players to play as recently reincarnated Lunars.
The problem is that a Lunar's life expectancy goes sharply up with age. At 10 years of age they're already experienced and powerful wizards, and Lunars over 100 or 250 years old have access to whole new tiers of power where they can beat up the majority of gods in the setting, also, they can turn into a swarm of locusts, a mist, a river, or any human they've injured or seduced, making it very hard for the Terrestrials to kill older Lunars or even find those Lunars. So the Terrestrials are disproportionately killing off young, inexperienced Lunars, which compounds the problem by implying that any Lunar who makes it to 100 in the first place is likely to be a professional survivor already and now has access to new godlike powers which greatly raises the chance of living another millennium or two.
So the Lunar life cycle now goes something like this: reincarnate, get killed at age 2, reincarnate, get killed in first month of life, reincarnate, get killed at age 11, reincarnate, get killed at age 4, reincarnate, escape from the Terrestrial Empire and make it to safety, live to 2500, die of old age. Even if the majority of Lunar incarnations die young, the majority of Lunar life-years will still be spent old – which seems to be isomorphic to the issue Scott describes in his post. So you’ve still got a shortage of young Lunars available to the players, and this shortage has only been remedied to the degree that players can expect their characters to get brutally murdered by Terrestrials in short order. You can always say that the player characters are special destined exceptions, but this undermines part of the design intent of creating a setting with 300 Lunar Exalted in the first place, which was to emphasize that the player characters aren’t the only movers and shakers around: there are numerous other people elsewhere in the world who are also chosen by the most high gods and given awesome magical power that they will use in extremely questionable manners.
In my experience, most game masters just ignore this (I expect it doesn’t occur to most of them in the first place), and use sheer handwaving without regard for actuarial expectations to set the relative fraction of PC-tier young Lunars, mentor-tier middle-age Lunars, and plot-device-tier elder Lunars.
Careful, there. If every Lunar lived to the maximum of 2500 years, then their age distribution would be uniform, and the expected age of a randomly sampled Lunar would be 1250. That doesn’t mean the majority of life-years will be spent old, unless by old you mean anything over 100.
But try a simple but non-uniform distribution: say a given Lunar soul gets killed nine times at age 5, then survives to 2500. Then the density of souls in the age range [0,5] is 10 times greater than the density in the range [5,2500], and one expects roughly 6 Lunars between the ages of 0 and 5 to be alive at any given time. Throw in the usual hand-waving about destiny bringing unlikely heroes together, dawn of the third age, Gem blowing up bla bla bla and a party of Lunars isn’t so implausble.
(Apologies if there any glaring errors above, haven’t tried to write out the numbers on paper.)
Yes, I was using “old” in that second sentence you quote to mean “over 100” because that’s well out of normal PC range and into gamebreaking territory, not “over 1250”. Perhaps I should have been more specific about using numbers.
Your math seems to work out.
ohyou.jpg
Anyway, everyone knows 90% of young Lunars form unlikely alliances with two Solars, a renegade Abyssal and a disguised Sidereal.
One party of young Lunars, if the Silver Pact helps bring them together, sure. A whole subculture of young Lunars is harder…
(In a game with two Abyssals, an Infernal, a Dragon-Blood, and a brand new Solaroidish Exaltation.)
I’m glad that things in such an ivory tower academic field like epidemiology can have real-world effects on things like extraterrestrial wizards.
I’m not sure how to take this remark. It looks vaguely like critical sarcasm, but what are you trying to be sarcastic about?
I shared what looks to me to be a similar case where intuitive reasoning about frequencies relative to observations and instances easily goes astray because of the uneven distribution of durations, and that’s a horribly clunky way of trying to describe what I see as a general factor, which is part of why I posted the anecdote: trying to see a less clunky way of describing the common factor, if there is one. I don’t for one moment think that the distribution of extraterrestrial wizards is the non-ivory-tower problem here.
I think the remark is sarcastic but not critical. Scott Alexander is trying to make a light-hearted joke without any ill intent.
What Itai said. I do appreciate the comment and analogy.
I think the general heading for this problem is “selection bias”.
The specific error in the Hypothetical World example is a mismatch between the researcher’s explicit intention to study first-time-smokers, and the methodology of sampling from the customers of a smoke shop. In the Real World study, it’s unlikely that e-cig users who declare an intention to quit are correctly matched to the wider smoking population.
I don’t have access to the full article – does anyone know whether/how they measured cigarette use, or how they determined whether a user had “quit”?
I used to be a pack-a-day smoker, and now I use e-cigs. I’ve purchased two packs of cigarettes in the past three months and haven’t finished them. I’ve only smoked when I ran out of e-cig cartridges (due to shipping delays, mostly) or wanted to make smoking fetish clips (yeah, that’s a thing). Have I “quit” smoking? Depends on your criteria.
I’ve also noticed that when I *do* smoke (without also using e-cigs, e.g. because I’ve run out), I smoke fewer cigarettes: 5-7 per day, as opposed to the ~20 per day I smoked when I wasn’t using e-cigs. They seem more potent and less pleasant to use than they did before – I’m no longer used to the smell, for example, which I don’t really like.
It’s deliberate that the hypothetical researcher said she wanted to study the rates among first-time smokers, then totally failed to screen her cig shop subjects for being first-time smokers?
Seems so to me. If the researcher doesn’t do the obvious thing, Scott’s point about sampling bias is illustrated by the analogy. If she does do the obvious thing, it’s a good protocol but can’t illustrate Scott’s point because the bias is missing. A night-and-day difference in fitness for purpose seems like a good indicator of intent.
It’s a good thing he has such alert readers as us to catch him when he does slip up, yes? 😉