Sleep Support: An Individual Randomized Controlled Trial

I worry my sleep quality isn’t great. On weekends, no matter when I go to bed, I sleep until 11 or 12. When I wake up, I feel like I’ve overslept. But if I try to make myself get up earlier, I feel angry and want to go back to sleep.

A supplement company I trust, Nootropics Depot, recently released a new product called Sleep Support. It advertises that, along with helping you fall asleep faster, it can “improve sleep quality” by “improv[ing] sleep architecture, allowing you to achieve higher quality and more refreshing sleep.” I decided to try it.

The first night I took it, I woke up naturally at 9 the next morning, with no desire to go back to sleep. This has never happened before. It shocked me. And the next morning, the same thing happened. I started recommending the supplement to all my friends, some of whom also reported good results.

I decided the next step was to do a randomized controlled trial. I obtained sugar pills, and put both the sugar pills and the Sleep Support pills inside bigger capsules so I couldn’t tell which was which. The recommended dose was two Sleep Support pills per night, so for my 24 night trial I created 12 groups of two Sleep Support pills and 12 groups of two placebo pills.

Then I asked a friend to flip a coin 24 times, and depending on the result place either a pair of Sleep Support pills or a pair of placebo pills in each slot of a monthly pill planner, and record which slot contained which pills on a secret piece of paper I could see at the end of the experiment. Then every weekend night for three months I took the next pair of pills in the planner and recorded:

– the time I went to bed
– the time I woke up
– my subjective rating of how well-rested I was upon waking
– my subjective rating of how much energy I felt like I’d had that day
– my subjective rating of how vivid my dreams were that night
– my subjective guess about whether I’d taken placebo or experimental that night

The time I went to bed wasn’t intended to be a dependent variable; I generally took the pills just before going to bed, so they couldn’t affect that. And I had no way of measuring what time I went to sleep. It was just so that I could measure my total time in bed that night.

The time I woke up was the hardest to operationalize. Usually I wake up a few times in the morning, groggily check the clock, and decide to go back to bed, then wake up for good once it becomes so late I start feeling guilty about how much of the day I’m wasting. I considered setting wake-up time as the very first time I woke up to check the clock, but sometimes I wake up at 5 AM to go to the bathroom, and I didn’t want that to get recorded as me “waking up” at 5 AM. And if I used a cutoff like “the first time I wake up after 7”, then a night I wake up at 6:59 and go back to bed and wake up for good at 11 would get recorded completely differently from a night I wake up at 7:01 and go back to bed and wake up for good at 11. But if I defined wake-up time as the time I finally woke up for good, then it would be too easy for me to subconsciously bias the experiment. “This feels like a night I took placebo, better stay in bed until at least 11:30”.

I decided to eliminate the whole problem by forbidding myself to check the clock while in bed. I would go to sleep, wake up, either decide to go back to sleep or not, and I wasn’t allowed to check the clock until I had gotten out of bed and gotten dressed.

Here’s the headline results of the experiment – number of hours I slept during experimental vs. control nights, and wake up time during experimental vs. control nights

On average there was no difference between the two groups on either measurement. There was also no difference on any of the subjective measures. My subjective guess about whether I’d taken experimental or placebo capsules that night had no correlation with the reality.

My conclusion isn’t that Sleep Support doesn’t work; I didn’t even try it for its main indication of helping with insomnia. My study was too underpowered to detect even medium-sized effects. And just because it didn’t work for me doesn’t mean it won’t work for somebody else.

My conclusion is that the effect I thought that I observed – a consistent change of two hours in my otherwise stable wake-up time – wasn’t real. This shocked me. What’s going on?

I think my original strategy of “wake up a few times in the morning, check the clock, and finally get out of bed when you really feel like it” is very susceptible to the placebo effect. Usually I might wake up at 9, decide that was too early to face the world, and go back to bed. Maybe I wouldn’t even remember doing this. Part of this was probably inertia – I wasn’t used to getting up at 9, I figured I must not have gotten enough sleep to feel good, and so I didn’t want to do it today. Once I had an exciting new sleep supplement in my system, I woke up at 9, actually checked whether or not I felt ready to wake up, and absent my usual prior that I wasn’t, I found that I was, and woke up.

This hypothesis is supported by the results of the experiment. On about a third of days, I woke up before 10 – again, something I never would have done before starting Sleep Support. I think the active ingredient here was not letting myself look at the clock. Without external cues to tell me how tired I should feel, I was forced to rely on how tired I actually felt, which in many cases was “not tired at all”. This happened regardless of whether I was taking Sleep Support or placebo that day.

Ironically, even though the supplement failed to differentiate itself from placebo, I think this is one of the most successful biohacking experiments I’ve ever done. I’m getting up on average an hour or so earlier than I did before, getting more done, and not feeling any more tired by the evening.

Future research: see if this keeps working even now that I know what’s going on.

You can download my raw data here. If you want to replicate this experiment, you can buy Sleep Support capsules here. There are lots of ways to make a placebo; I found these very large empty capsules helpful.

I’m interested in hearing about anyone else’s experience conducting controlled trials of supplements on themselves; if you do something like this and want to publish it on a blog, let me know.

This entry was posted in Uncategorized and tagged . Bookmark the permalink.

88 Responses to Sleep Support: An Individual Randomized Controlled Trial

  1. daneelssoul says:

    OK. But does Sleep Support have a statistically significant effect on the *variance* of the amount of sleep that you get? The graphs make it appear to, and this might also contribute to your getting up surprisingly early the first two nights.

    • Scott Alexander says:

      That wasn’t a hypothesis of mine before the experiment, and adopting it post hoc feels a little like a fishing expedition, especially since each group was only n = 12. If you eliminate two outliers, I think the experimental condition has lower variance.

      • Roebuck says:

        What is a good reference class for difference in variance hypotheses? How do we stop ourselves searching for the French Pyramids?

        On one hand, a difference in variance sounds like the second natural thing to ask about after the difference in the mean. On the other hand, we would also be very excited if there was a clear bimodality in the experimental distribution (lacking in the control distribution).
        Also, I’m pretty sure we would cheer if the experimental group had an impressive outlier (say, a sleep time of 11:20).

        There must have been maybe millions of people asking themselves this question when looking at RCT results, but I can’t recall seeing any answer.

      • cuke says:

        If you eliminate outliers, doesn’t it look like it increased your sleep time? Did the two sleep time outliers happen early in the experiment? They are presumably the same as the two early waking time outliers? If so, is it your sense that they are due to the same psychological effect you attribute to doing the experiment generally — a kind of treatment excitement?

      • sicromoft says:

        If you eliminate two outliers, I think the experimental condition has lower variance.

        Did you really just suggest, unironically, that eliminating some of the variance results in… lower variance?

        • B_Epstein says:

          Come on. Scott’s claim wasn’t that eliminating outliers lowered variance within the experimental data, it was that after the removal, the variance of the experimental condition was lower than the variance of the control. This is a perfectly valid, non-tautological, statement, if a bit (or a lot) ad hoc.

        • Douglas Knight says:

          Variance is just a metric of dispersion. He obviously means that it’s not true that the one is more dispersed than the other, but that it depends on your choice of measure of dispersion. Variance after eliminating outliers is an underspecified but reasonable metric of dispersion. I prefer interquartile range as a fully specified metric of dispersion insensitive to outliers.

  2. Three Year Lurker says:

    The external light filtering into your room is a low resolution clock that you check without leaving the bed.

    A different solution for people woken by bathroom breaks in the night would be a dim red light for the bathroom to avoid signaling the daytime receptors in your eyes.

    Getting up to use the bathroom then returning to bed indicates that changing the timing and amount you drink also plays a role in length and quality of sleep.

    • Lambert says:

      Can you reliably distinguish between 09:00 and 10:00 at whatever latitude scott’s at?

      • janrandom says:

        Generally, the biological clock is pretty good if you train to listen to it. A lot of people can wake up at a desired time without an alarm clock. A lot of reports of this can be found online e.g. below, but I can’t find studies of this:

        https://www.quora.com/What-is-the-science-behind-people-who-have-a-natural-alarm-clock-and-can-wake-up-at-the-right-time

        I can’t do this exactly, but I can and have changed my sleep rhythm over time a few times, e.g. to wake up early enough to get the kids to school (I have problems going to sleep so I may be still tired at that time but that’s independent).

        Something that you can practice more easily is to estimate the current time without looking at the clock or estimating the duration of a minute. Accuracy of 5 Min for the first and 5 seconds for the second are relatively easy to achieve in my experience.

        • Loriot says:

          I’m not sure exactly what happened, but I seem to have shifted my sleep cycle over the winter holidays, so I now go to bed and wake up shockingly early. I think winter makes it a lot easier to fall asleep early. We’ll see how long I can maintain my new cycle in the face of the ever longer days. Waking up early is very convenient though.

      • keaswaran says:

        Not necessarily between 9 and 10, but at basically any latitude not too close to the Arctic/Antarctic circle, the hour from before to after sunrise is really easy to detect (I can currently very reliably tell the difference between 7 am and 7:30 am in my location in Texas). It’s also easy to tell when one is within an hour or so before sunrise, though it’s harder to be precise about an hour after sunrise.

        I can’t tell if this is related, but I usually have a pretty good sense of direction, though it only works outdoors (not within buildings). And I noticed on a trip to Australia a few years ago, that I was reliably reversing South and North there, which suggests (a hypothesis proposed to me by someone else after I returned to the northern hemisphere) that part of this sense might involve subconscious detection of the sun.

        • Mark Atwood says:

          Once you are outside, know the season, and know how to do the math, it’s surprisingly easy to know the time to within an hour, like so:

          If you are outside a lot during the day, then you know where the high point of the sun is at noon, and you know where the sunrise and sunset points on the horizon. Look at the imaginary line from the sunrise point to the noon point, and mentally divide it into thirds. Each of those is two hours. Cut each of those in half. That’s an hour.

          The width of the solar disc is 1/2 a second arc. If the arc of the path of the sun is at right angles to the horizon, it takes 360 of it’s width from rise to set. So it literally takes one minute of time for the sun to travel it’s own width. Convenient, that!

          That means that if you have comparison point, which could just be your own thumb (which is almost exactly the same angle of arc when held out arms length) and if you have good self body perception, you can easily watch one minute of time pass.

          I strongly suspect that the ease of dividing those arcs in to halfs, thirds, halfs of half, thirds of thirds, halfs of thirds, and thirds of halfs, is fundamentally the origin of base 12 / 60 timekeeping.

          At night, you do literally the same trick, except instead of looking at the sun, you look at the known constellation or a known bright star that was rising opposite the setting sun, and track it on the arc perpendicular to the line from the north star to yourself.

          That most moderns cannot just instantly snap do this would astound the most illiterate peasant of a millennia ago, and make them seriously question whether we were brain damaged or not.

          • Mark Atwood says:

            Self correction. It takes 2 minutes for the sun to cross it’s own width, not 1. Idiot math error on my part.

            And, of course, multiply by the sqrt(2) at 45N or 45S

      • Hackworth says:

        I wake up a minute or two before my alarm clock rings way too often to be coincidence.

  3. Briefling says:

    Isn’t this consistent with Sleep Support having a real effect that lasts more than one day? Which could plausibly happen if e.g. it’s treating a vitamin deficiency.

    Edit: didn’t realize you were only dosing once a week on average. That does make this model less likely.

  4. AM says:

    Is it possible that the sugar might be an active ingredient? Some people (my past self included) find a spoonful of honey before going to sleep helpful for sleep and feeling rested in the morning.

  5. voso says:

    Ingredients from the website, in case anyone else was curious:
    Lemon Balm Extract, Uridine Monophosphate, Bacopa Monnieri Extract, Magnolia Bark Extract, Oleamide, PrimaVie® Purified Shilajit, Sucrosomial® Zinc.

    • Nav says:

      Interesting. I find Zinc makes me more light sensitive in the morning, but I haven’t tested to see whether it’s a placebo.

  6. Matthias says:

    Btw, has anyone ever seen a difference between L-tryptophane and melatonin?

    The former is an essential amino acid that gets metabolised into the latter. Scott had some article a while ago on melatonin and noted that the right dose is very important. I wonder whether involving the bodies metabolism might take care of getting the dosing right all by itself?

    As for sleeping: there are a few stimulants that when taken during the day allow me to fall asleep much easier at night and naturally wake up earlier and well rested. Nicotine (from patches) and methylphenidate are two examples. But I don’t think that’ll generalise to people without ADHD.

    Caffeine is the opposite, even taken early in the day, it prevents me from falling asleep at night.

    • cuke says:

      In the dozen or so people I know who have tried these, I would say more people seem to get help from melatonin than tryptophan but that some portion of people who get benefit from melatonin don’t like to take it because it leaves them feeling groggy in the morning. I also know a number of people who have experienced no benefit from either of them.

  7. duncane says:

    Scott,
    try Valerian supplements. In my personal tests it has a noticeable effect although my tests werent as defined as yours.

  8. drunkfish says:

    How’d you do on your subjective guesses about whether you had gotten a real dose or not? Seems relevant for, with such a small sample size, controlling for other effects like noisy mornings or something. If you consistently guess right, then that’d point toward there being at least some sort of effect going on.

    • Scott Alexander says:

      Totally wrong, no correlation.

      • drunkfish says:

        Huh ok, then yeah that’s a really strong argument for placebo.

        And sorry I didn’t just check myself, I totally missed that you provided the raw data at the end…

  9. AlexOfUrals says:

    Coincidentally, I have just recently observed a somewhat similar effect on myself without any supplements, just after reading this review mentioned in the latest Links post. I had always thought that 8-9 hours of sleep is necessary for me, so if I averaged on less than that during the work days (which is almost always the case), I then stayed longer in bed on the weekend. Then upon reading that 7 hours or maybe even less is optimal, I’ve tried to just don’t stay in bed late (I also usually check the clocks a few times in the morning and over the night, so it was easy), ans all of a sudden I was feeling just fine on ~7.5 hours of sleep on average. I even started to wake up on weekends almost at the same time as on workdays, which is about 7 AM.

    Although that schedule was kind of ruined when I traveled to a 12 hours different time zone for two weeks, and returned just two weeks ago, but I’m getting back to it.

    • Loriot says:

      > had always thought that 8-9 hours of sleep is necessary for me, so if I averaged on less than that during the work days (which is almost always the case), I then stayed longer in bed on the weekend. Then upon reading that 7 hours or maybe even less is optimal, I’ve tried to just don’t stay in bed late (I also usually check the clocks a few times in the morning and over the night, so it was easy), ans all of a sudden I was feeling just fine on ~7.5 hours of sleep on average.

      I think there’s a lot of individual variation (as well as variation due to age). I generally couldn’t sleep for seven hours if I tried (unless I was previously sleep deprived), which used to sometimes cause me stress due to all the articles about how you need to sleep for eight hours and sleep is important and all that.

    • guzey says:

      Comments like this make me so happy!

      It is funny but the exact same thing happened to me in the process of writing that post… Literally just slept 4 hours and am feeling much better than yesterday when I slept 8 – wouldn’t have the courage to do this intentionally before looking deeply into all of this

      (note that I do not advocate always sleeping for 4 hours)

    • Reziac says:

      I require 3 sleep cycles (can scrape by on two, but not well) which totals up just under 7 hours. I expect most people need 3, with some variation in cycle length.

  10. ChrisA says:

    Would be interesting to repeat this study with a smart watch, like a fitbit, that measures your sleep quality and duration automatically. I don’t know how good the quality measurement is, but as far as I can tell the duration is pretty accurate.

  11. bpodgursky says:

    > I’m getting up on average an hour or so earlier than I did before, getting more done, and not feeling any more tired by the evening.

    Your status-quo productivity already leaves me somewhat shocked (eg: job + wildly more productive and regular blogging than I’ve ever managed + impressive amounts of reading). The idea that a supplement will kick you to the next level is a bit unnerving.

  12. Godfree Roberts says:

    See how you sleep after a session of yoga, an early light supper and no booze. Betcha.

  13. Stephen says:

    I ran an analysis using your guess as the independent variable instead of reality. Nothing was significant (SleepRating_AM_: t = 0.2808, p = 0.7815, EnergyRating_endOfDay_: t = -0.5253, p = 0.6046) which somewhat surprised me. I had expected a common source to cause both placebo/supplement guess and the subjective report of sleep quality. I was hoping to be able to back out whether the “1” flag corresponded to placebo or supplement based on the results, but the two subjective measures of sleep quality did not even agree in sign based on your guess.

    Using reality as an IV also resulted in disagreement on the sign of those two outcome measures.

  14. stefanone says:

    Veeeery interesting, I have lotsa sleep issues as well, and now I’m gonna remove any clock from my bedroom.
    Will you repeat your experiment on a 7/7 days basis? and maybe using a sleep tracker like Fitbit or Polar?
    I currently use the Polar Ignite sportwatch, it grants the deepest sleep analisys I’ve ever seen, and even a Nightly Recharge measure which combines data from motion sensors, cardiac sensors and VO2

  15. North49 says:

    I find looking at the clock crucial to going back to sleep. I feel anxiety about if there’s ‘enough time left’ in the night and if I can’t check, I’ll just be awake worrying about it.

    • keaswaran says:

      I’ve been considering putting a large clock in the bedroom and then moving the phone charging station to far from the bed, so I can still have the cushion of knowing what time it is, with no ability to procrastinate before actually getting up.

  16. Confounding anecdata: My sleep quality improved when I put a clock within arm’s reach. I don’t check it much, but it helps me realise that I can go back to bed when it’s light outside but I wake up e.g. two hours before I normally do (e.g. five in the morning). Sadly I am not a very good sleeper, despite doing a ton of things to improve the quality of my sleep – anxieties or my bladder tend to interfere e_e and I wake up ‘properly’ extremely quickly, which, while useful, also means waking up in the middle of the night tends to destroy me. I was failing to get properly back to sleep for years and it gave me a real sleep deficit.

    I’m going to try having a red light in the bathroom, as suggested further up the page. I have a suspicion this will do a lot of difference in my particular case.

    • Lambert says:

      Smart bulbs are cool, if you don’t want a separate red bulb. I’ve not had mine long enough to properly evaluate its effect on my sleep, but control over the amount of blue light and scheduling both feel like features I’m getting the use out of.

    • janrandom says:

      Red lights didn’t help me but your mileage may vary.

      Also, I read about the red light thing being more complicated than expected:

      https://mobile.slashdot.org/story/19/12/17/2152258/night-mode-could-actually-be-worse-for-your-sleep-pattern-study-finds

      • That’s really interesting and valuable, thank you! I do use tools like f.lux – this suggests that maybe I shouldn’t. I’ll experiment a bit. Thanks again – I appreciate the heads-up!

      • Solra Bizna says:

        The study subjects were mice. Humans are diurnal, while mice are nocturnal. If one subscribes to the hypothesis that animals calibrate “day time” using the presence of blue light, then one should expect that it would have the opposite effect in humans as in mice. So when the study showed that mice are more “sleepy” in the presence of blue light than yellow—which is what the “f.lux hypothesis” predicts it would do for mice—this, in my opinion, was evidence for the “f.lux hypothesis”.

        The study does support the important point that brightness has a very large effect on circadian rhythm. In theory, this means holding the brightness constant while shifting color temperature toward red wouldn’t help much. But that’s not what f.lux and Night Mode do. As a side effect of the way they reduce the blue (and green) components of the image, they reduce brightness as well—depending on settings, possibly on the order of 50-80%. So… score two for f.lux?

        Not to mention that mice have poor color vision (two cones instead of three, and not a lot of them) as well as poor vision in general. So… yeah…

        (I permanently wrecked my circadian rhythm by exposing myself to somewhat dim bluish-green light 24/7 for almost two years, so I have had a vested interest in learning about the effects of light on sleep…)

        • Lambert says:

          > two cones instead of three

          It’s not about cones. It’s a different sort of light-sensitive cell that regulates melatonin synthesis.

  17. benf says:

    Melatonin definitely works, but definitely NOT for everyone, for reasons that are mysterious to me. Large RCTs on melatonin find basically no effect at all, but when I take it I feel extremely drowsy 20 minutes later in a way that not even pharmaceutical sleep aids were ever able to deliver. I would either feel nothing or feel drugged, melatonin makes me feel TIRED. But again, this hasn’t been replicated on large scales, and I suspect it must be because there is some biochemical variable not being controlled for because we don’t really understand the system yet.

  18. TJ2001 says:

    Scott, thanks for sharing your data.

    Wake Time and Sleeping Hours are strongly correlated – and it has fairly uniform scatter (+/- 1 hr) for the whole data set. This is going to cause you some difficulty sorting out correlations with other variables.

    Bed time and sleeping hours are much less strongly correlated – but still correlated. In general – the later you go to bed, the less sleep you get.

    So for example – if you woke up at 10 – you slept an average of 8 +/- 1 hrs. If you woke at 11 – you slept an average of 9 +/- 1 hrs.

    If you went to bed at 2:45 am – you got an average of 7.5 hrs sleep vs going to bed at 12:45 with an average of 9hrs sleep.

    This is an example of “Environmental Noise variables” which produce more “Effect” than anything else. You will have to figure out how to control for this somehow before any of the other data becomes meaningful.

    Feynman’s rat running essay seems apropos here… 😉 ;).

    • TJ2001 says:

      The only idea I could come up with that’s not pure torture is to pick a set “Go to Bed” and “Wake time” around whatever you feel your “Best Sleep Time” is – then randomize use of sleep aids. Rank “sleep quality” somehow…. So say you decide on a fixed 9 hrs sleep – From 1AM to 10AM every week-day night. Set an alarm and do it.

      Then maybe give yourself freedom on weekends to do whatever you want due to social life – but don’t log that in the data…

      Rank data is a lot less useful statistically – but this would get rid of the giant correlation in the data which is driving everything else.

  19. AppetSci says:

    This wouldn’t help the falling asleep part, but I’ve often wanted to design some Delayed Action Caffeine pills that have a coating that takes say 8 hours to digest, so you take one when you go to bed and wake up a-buzz with caffeine coursing through your veins. They would be simple and inexpensive and you could easily purchase 6 or 7 hour variants to take dependent on your bed time and required wake-up time.
    Does this exist?

    • Lambert says:

      Keeping caffiene pills under your pillow works but isn’t fun.

    • Scott Alexander says:

      Jornay is a form of Ritalin that works this way. I made fun of the pharma companies for thinking anyone would want it, but I guess I have to eat my words.

    • kupe says:

      When I was more into sleep hacking I used to do “Red Bull Naps” where you quickly drink a coffee or energy drink and then you wake up from your nap 20 -30min later when the caffeine starts to hit.

      • realitychemist says:

        I’ve done this before. It was actually my go-to method for staying up late in college when I needed to. The explanation I heard of why it works was actually quite interesting, although I don’t know how scientifically accurate.

        (Basic idea being that caffeine inhibits binding of adenosine, which helps to prevent you from *becoming* sleepy, but doesn’t do too much if your adenosine receptors already have adenosine bound to them. So, you chug a coffee, and it takes about 20 minutes for the caffeine to have an effect. In that time you take a power nap to help clear adenosine from your receptors, and when you wake up a short time later the caffeine will help prevent the adenosine from rebinding. Again, no idea how factual this is, just what I heard.)

        I did find this to be extremely effective. When I did this I found that I could do productive work – classes, homework, studying, test taking – for nearly 36 hours straight, with just the 20 minute nap to keep me going. Admittedly, though, the world did start to feel somewhat dreamlike after hour 20 or so, and I would *not* have felt comfortable driving or anything like that.

    • Lambert says:

      Huh, you can buy 200 empty enteric-coated capsules for about £20 on Amazon. That gives you 6-7 hours. (though IDK how sleep affects timings.)

      EDIT: are there any pharmacological ‘indicators’ that have some kind of fast, easily noticed side effect that’s not bad in any way? Like how GFP is used in genetics. Caffiene might have too strong a placebo effect to measure the timing properly.

  20. bobbert says:

    >Ironically, even though the supplement failed to differentiate itself from placebo, I think this is one of the most successful biohacking experiments I’ve ever done. I’m getting up on average an hour or so earlier than I did before, getting more done, and not feeling any more tired by the evening.

    It may be the case that Sleep Support remains active in your system for more than 24 hours. If that’s true, the time-delayed effects may be working on subsequent periods of sleep after the first night you take the pill. You would then be getting the beneficial effects you mentioned above regardless of changes in habit, and it would be harder to differentiate between your control nights and your experimental nights.

    You may (I think this is unlikely) be able to tease out these time-delayed effects from your data. Otherwise you can try longer experimental periods similar blind experiment, w/ weeks of treatment randomized instead of single days randomized.

    Or choose not to worry about this.

  21. Radu Floricica says:

    I think Scott mentioned he’s in the minority in which melatonin’s vivid dreaming ends up doing more harm than good. Otherwise yes, 0.3 melatonin a few hours before sleep is the gold standard.

  22. hollobollo says:

    I hope your friend complained about your study design, because only about 16% of “toss a coin 24 times” have 12 occurrences of either side, so you would probably see a pronounced tail end where you’d run out of one of the options. (But I’ll grant you that debating this and doing the maths to back that up plus designing alternatives would be a nice open discussion with a couple of bright Stoch 101 students.)

  23. Rachael says:

    This might be a stupid question because you’re a doctor and I’m not, but is it possible that the extra layer of capsule is reducing or time-delaying the effect of the pill?

  24. janrandom says:

    I am still unhappy with my sleep but there are clearly a few things that can be tweaked without drugs. One thing that worked very well for me is getting up without snoozing. It is relatively easy to tell oneself that ‘I can’t get up quickly, I need to snooze’. I was like that but it felt so illogical: Waking up when I’m fully rested would save me the repeated snoozes and to wake up at a desired time clearly seems something the body can do. So after reading a lot about sleep and trying a lot of things to different degrees of success (e.g. installing red lamps to turn on in the evenings) I came across a trick that now allows me to get up when the alarm rings:

    On a weekend when you don’t need to get up, set the alarm to a time when you are confident that you got enough sleep for sure. Then when it rings get up. Practice a few times.

    Surprisingly, two times was enough for me. Usually, I get up when the alarm rings and I almost never use snooze. This has saved me so many snooze periods. Key is: 1) Setting the alarm to the final time. 2) Get enough sleep in general.

  25. deciusbrutus says:

    How do the data from the blind test compare to the data gathered on the Sleep Support capsules before the test?

  26. Rusty says:

    For me calling this a placebo doesn’t quite capture what I think is going on. I feel this could be best described as a form of CBT. Taking the sleep support as you report it forced you to really interrogate how you were feeling and act on that. I am forever banging on about how effective Allen Carr is in helping with stopping smoking and a large part of his method is clearing away your preconceptions about smoking and actually working out how you really do feel. Once you do that you realise you don’t enjoy it. And here you were able to find out that rising ‘early’ was something you were in fact always happy to do.

  27. Viliam says:

    Scott Alexander — reviewing medical research even when he sleeps.

  28. faith says:

    It’s interesting to me that you connect feeling angry with needing more sleep. As though feeling really good when you awake is the norm and the variation is a sign of trouble. Was there once a time when you would wake up feeling good?

    For me, it’s quite normal to be angry/grumpy/groggy/unhappy when I awake. No matter what time of day. No matter how well or how long I’ve slept. It’s not a sign of anything, that I can tell; it’s just me. There have been times I have had to be up at 5am for a work schedule, and times when I could sleep in as late as I wanted. Always, I wake up in a very irritable mood. I’ve learned to keep my mouth shut for 30-45 minutes, until I am fully awake and in a clear frame of mind to be responsible for my words and attitude.

    My 6 year old son, on the other hand, always awakes in a good mood, no matter how little sleep he’s gotten. He always has, even as an infant. He’s not really a morning person; he just wakes up in a good mood.

  29. Calecute says:

    Seth Roberts used to do that kind of thing a lot, maybe you should check his methodology for sleeping studies with n=1. I remember he also used to have simple reaction time test.

    https://sethroberts.net/

  30. Tatterdemalion says:

    Have you considered doing a second, non-blinded trial? Even if it only makes you feel better via a placebo effect, that’s still a win.

  31. kupe says:

    I’d happily pay a small premium on top of the pills to participate in self-experiments like this. Have a monthly pill planner turn up, loaded with the placebos and instructions.

  32. stevesteve says:

    My study was too underpowered to detect even medium-sized effects.

    Yeah, you can use a power/sample size calculator to determine how many samples you need for a decent chance at detecting a certain size effect.

    Your data set says you have a mean of 8.61 hours of sleep, with a standard deviation of 1.05 hours. To detect a half hour increase in sleep with 80% power with a p value of 0.05, you need 70 samples for each group.

    A sample size of 12 has 80% power for a change in sleep of 50 minutes.

    https://www.stat.ubc.ca/~rollin/stats/ssize/n2.html

    I’m just a software engineer who works with data scientists, not a statistician, so it’s worth double checking what I say.

    I ran a similar experiment on myself with Melatonin after reading Gwerns article. It wasn’t blinded and I instead used data from my Fitbit for the months before and after starting to take the pills.

    I only cared about sleep latency instead of total amount of sleep since total sleep was more dependent on when I got to sleep and my schedule the next day. The results were striking and my sleep latency when using Melatonin was 22 minutes lower.

    You have to be careful with the newer Fitbits though because they auto detect when you fall asleep and can’t calculate sleep latency.

  33. alenaf says:

    I’m surprised nobody mentioned magnesium supplements – I didn’t do a controlled experiment but they seem to help me get better quality sleep. Removing devices an hour before bed, not looking at clocks and using red light is supposed to help too but those didn’t seem to have the same effect on me.

  34. Elephant says:

    I don’t have anything useful to say about sleep, but I just wanted to point out that this is a clear, well done study. Sadly, it’s much better than most of the p-hacked messes, with opaque raw data and overstated conclusions, that the psych/bio/med academic literature is full of.

  35. Edward Scizorhands says:

    Has anyone ever noticed a difference between different brands of melatonin?

    I recently switched, not on purpose, to https://www.naturesbounty.com/our-products/specialty/diet-supplements/melatonin-5-mg-90-tablets/ brand of melatonin, and it doesn’t seem to work as well.

    This is not the only switch I have made: I also have gotten off of traveling for work, which I’m sure had an effect on things.

    But I used to take a pill, lie in bed reading with my legs curled up until I feel tired, turn off the light, and then after a while stretch out, roll over, and relax.

    • Radu Floricica says:

      Link doesn’t work, but if I read well 5mg… that’s a lot. Recommended dosage is 0.3mg. Personally, dosages as high as this end up having the opposite effect somewhat. Not sure why – possibly because melatonin stays in the system all day (shouldn’t, with its short half life…) or more likely something just gets used to it and stops reacting as much to it.

      As for source, Nootropics Depot was recommended by Scott, and I’ve been using it for about a year. The 0.3, and they work perfectly fine.

      • Edward Scizorhands says:

        Dosage doesn’t really seem to matter for me. 0.3mg or 1mg or 5mg, it’s all the same.

        • Lambert says:

          I think studies show that dosages over some value (around 0.3mg) have no extra effect (but can have stronger side effects).

  36. Todd K says:

    Someone posted his deep sleeping time using an Apple watch after taking NR (Nicotinmide Riboside or “Niagen”) for a couple of weeks at maybe 300 mg a day, the amount most take, each morning. He then stopped taking it and slowly watched his deep sleep time go from two hours a night to one hour a night over the next ten days or so, while his 15 minutes of awake time and total sleeping time didn’t change.

    There is an NR/sleep trial in progress but the results probably won’t be published until next spring:

    “This is a double-blind, randomized, crossover study to investigate the effects of 300 mg/d and 1000 mg/d TRU NIAGEN (nicotinamide riboside) compared to a placebo control on cognitive function, mood and sleep in men and women over 55 years of age. The trial [n = 40] is managed by Midwest Center for Metabolic and Cardiovascular Research.

    The study ended April 26, 2019

    https://clinicaltrials.gov/ct2/show/NCT03562468?term=nicotinamide+riboside

  37. Lou says:

    Try this:

    Make an agreement with a friend to wake up at a specific hour the next morning, after waking up you must take a picture of some object located in another room. If you’re reasonably healthy you’ll wake up at that specific hour feeling rested. I’ve tried this with a few friends with positive results.

  38. Neutrino says:

    Have you explored any sleep apnea or similar conditions?
    Do you snore?
    Either one can disrupt sleep, and solutions increase sleep quality while reducing time.

  39. Sammerman says:

    The only thing I’ve found effective for sleep at or before bedtime are travel camping trips out of cell reception. I took one this winter and was bed 30 minutes after sundown each night from day two and on. And woke up real early too. I usually don’t get to bed until midnight or later and have habitual sleep issues. It kind of doesn’t tell me too much because it eliminates noise pollution, light pollution and distractions and also provides me with excellent exercise and cold fresh air and freedom from primary people stressors. FWIW

  40. technillogue says:

    Hallo, I have inexplicable chronic fatigue that gets much worse with mild sleep disruption. At least a few years ago, fitbits correlated best with polysomnography. I’ve done an randomized blinded experiment on whether 50mg modafinil at 7am worsens sleep efficacy (probably not). I also tried to done onef or self-bodywork before that, but had a harder time randomizing and sticking to that. Observational data suggested that caffeine at noon also reduced sleep efficacy.

    I’ve had some strange experiences with zinc, a few times it’s seemed to really make me feel well-rested in the morning, but taking it on a regular basis doesn’t seem to change much.

  41. justin1745 says:

    Scott, just eyeballing they data without any statistical analysis, it looks like there’s a clear benefit if you exclude what appear to be two outliers.

    I wonder if you added substantially to the sample size and got more data points if that would make a difference.

  42. robertskmiles says:

    A question, perhaps for the cryptographers: How do you do this on your own?