SSC Journal Club: Friston On Computational Mood

A few months ago, I wrote Toward A Predictive Theory Of Depression, which used the predictive coding model of brain function to speculate about mood disorders and emotions. Emotions might be a tendency toward unusually high (or low) precision of predictions:

Imagine the world’s most successful entrepreneur. Every company they found becomes a multibillion-dollar success. Every stock they pick shoots up and never stops. Heck, even their personal life is like this. Every vacation they take ends out picture-perfect and creates memories that last a lifetime; every date they go on leads to passionate soul-burning love that never ends badly.

And imagine your job is to advise this entrepreneur. The only advice worth giving would be “do more stuff”. Clearly all the stuff they’re doing works, so aim higher, work harder, run for President. Another way of saying this is “be more self-confident” – if they’re doubting whether or not to start a new project, remind them that 100% of the things they’ve ever done have been successful, odds are pretty good this new one will too, and they should stop wasting their time second-guessing themselves.

Now imagine the world’s least successful entrepreneur. Every company they make flounders and dies. Every stock they pick crashes the next day. Their vacations always get rained-out, their dates always end up with the other person leaving halfway through and sticking them with the bill.

What if your job is advising this guy? If they’re thinking of starting a new company, your advice is “Be really careful – you should know it’ll probably go badly”. If they’re thinking of going on a date, you should warn them against it unless they’re really sure. A good global suggestion might be to aim lower, go for low-risk-low-reward steady payoffs, and wait on anything risky until they’ve figured themselves out a little bit more.

Corlett, Frith and Fletcher linked mania to increased confidence. But mania looks a lot like being happy. And you’re happy when you succeed a lot. And when you succeed a lot, maybe having increased confidence is the way to go. If happiness were a sort of global filter that affected all your thought processes and said “These are good times, you should press really hard to exploit your apparent excellence and not worry too much about risk”, that would be pretty evolutionarily useful. Likewise, if sadness were a way of saying “Things are going pretty badly, maybe be less confident and don’t start any new projects”, that would be useful too.

Depression isn’t normal sadness. But if normal sadness lowers neural confidence a little, maybe depression is the pathological result of biological processes that lower neural confidence a lot. To give a total fake example which I’m not saying is what actually happens, if you run out of whatever neurotransmitter you use to signal high confidence, that would give you permanent pathological low confidence and might look like depression.

This would explain a lot about depression. It would explain why depressed people have such low motivation. It would explain why their movements are less forceful (“psychomotor retardation”). It would even explain why sense data are less distinct (depressed people literally see the world in washed out shades of grey). I thought this was plausible, but said I’d wait for real scientists to say the same thing before believing it too much.

What Is Mood: A Computational Perspective by Clark, Watson, and Friston – is real scientists saying the same thing. Sort of. With a lot more rigor. Let’s look into it and see what they get.

Recent theoretical arguments have converged on the idea that emotional states reflect changes in the uncertainty about the somatic consequences of action (Joffily & Coricelli, 2013; Wager et al. 2015; Seth & Friston, 2016). This uncertainty refers to the precision with which motor and physiological states can be predicted. In this setting, negative emotions contextualise events that induce expectations of unpredictability, while positive emotions refer to events that resolve uncertainty and confer a feeling of control (Barrett & Satpute, 2013; Gu et al. 2013). This ties emotional states to the resolution of uncertainty and, through the biophysical encoding of precision, to neuromodulation and cortical gain control (Brown & Friston, 2012).

In summary, one can associate the valence of emotional stimuli with the precision of prior beliefs about the consequences of action. In this view, positively valenced brain states are necessarily associated with increases in the precision of predictions about the (controllable) future – or, more simply, predictable consequences of motor or autonomic behaviour. Conversely, negative emotions correspond to a loss of prior precision and a sense of helplessness and uncertainty about the consequences of action.

Here they’re saying that emotions – the day-to-day variation in whether we feel happy or sad – is meant to track what kind of environment we’re in. Is it a predictable environment that we should rush out to manipulate so we can harvest a big heap of utility? Or is it an unpredictable environment where we’re probably wrong about everything and should try to limit damage?

It’s not really clear from this quote, but later on they’re going to shift from happiness being “the world is predictable” to “the world is good”, which – sounds a lot more common-sensical. I think this has to do with Friston’s commitment to believing that uncertainty-resolution is the only drive, and every form of goodness is a sort of predictability in a way. See Monday’s post God Help Us, Let’s Try To Understand Friston On Free Energy – or don’t, for all the good it will do you.

Any hierarchical inference relies on hyperpriors. These furnish higher level predictions of the likely value of lower level parameters. From the above, one can see that important parameters are the precisions of prediction errors at high and low levels of the hierarchy (i.e. prior and sensory precision). These precisions reflect the confidence we place in our prior beliefs relative to sensory evidence. If emotional states in the brain reflect the precision of prior beliefs about the consequences of action, then distinct neuronal populations must also encode hyperpriors. In other words, short-term fluctuations in precision (i.e. emotional fluctuations) will themselves be constrained by hyperpriors encoding their long-term average (i.e. mood).

Here, we propose that mood corresponds to hyperpriors about emotional states, or confidence about the consequences of action. In other words, mood states reflect the prior expectation about precision that nuances (emotional) fluctuations in confidence or uncertainty. If emotion reflects interoceptive precision, and is biophysically encoded by neuromodulatory gain control, then this suggests that mood is neurobiologically encoded as the set-point of neuromodulator systems that determine synaptic gain control over principal cells reporting prediction errors at different levels of the interoceptive hierarchy. This set-point is the sensitivity of responses to prediction errors and has a profound and enduring effect on subsequent inference.

The traditional definition says that “mood is like climate, emotions are like weather”. I think they’re saying that mood – long-lasting states like being depressed or being a generally carefree person – are second-level priors about emotions, which themselves are first-level priors about actions.

So suppose you see a vaguely greenish piece of paper on the ground. If you’re happy, you have a prior for the world being good, and so you might be more likely to interpret it as possibly a dollar bill. And you have a prior for the world being exploitable, so you might be more likely to think you can reach down and take it and have an extra dollar. And if you do, and it really is a dollar bill, you might become happier, since you’ve gained a little evidence that your senses are trustworthy (you were right to perceive it as a dollar), the world is exploitable (your cunning plan to pick up the paper and gain $1 worked!), and you’re in the sort of high-reward environment where you should go off and do other exciting things.

On the other hand, if you’re sad, you have a prior for the world being bad, so you might expect it to be litter. You have a prior that you can’t really predict or affect the world, so it might not be worth bending down to pick it up – you might just end up disappointed. But if you did bend down to pick it up, and it did turn out to be a dollar bill, you might brighten up a little, just as the happy person would. You’ve gained a little bit of evidence that you’re in a nice part of the world where good things happen to you, and that you can execute a simple plan like picking up a dollar bill to gain money.

A depressed person would have the same prior that the world is bad and the paper is probably just litter. But if perhaps she did pick up the dollar, and feel tempted to conclude that the world was good and she should feel happy, a higher-level prior would kick in: even when it seems like the world is good, that’s wrong and you should ignore it. The world is never actually good. When good things happen that look like they should convince you that the world is good, those are just lies.

Friston et al bring up learned helplessness. Let’s say you shock a rat a lot. In fact, let’s say you’re even more cruel, and you constantly give the rat apparent escapes, only to close them off at the last second and keep shocking it. You give the rat what look like food pellets, but they turn out to just be rocks painted to look like food. You eventually gaslight the hell out of the rat. Finally, you stop doing this, and you give the rat some actual food and a way out, and the rat just doesn’t care. Yes, food and escape should be good things that make it feel lik the world is reward-filled and exploitable, but it’s been let down so many times before that it assumes anything seemingly-good is a mirage.

Here’s the picture they eventually draw:

Depression is a prediction of bad outcomes with high confidence. Mania is a prediction of good outcomes with high confidence. Anxiety (or “agitated depression”) is a prediction of bad outcomes with low confidence. There’s a blank space where it looks like there ought to be an extra emotion; maybe God will release it later as DLC.

Friston et al speculate that these hyperpriors over emotions can either be genetically encoded, or “learned” over very long periods of consistent stimuli. For example, if your childhood is unbearably terrible, that might be long enough to “burn in” a high-confidence hyperprior that the world is always bad.

(they don’t mention this, but if prediction and action are as linked as everyone always says, I wonder if this would explain why people with terrible childhoods are always mysteriously sabotaging themselves into have adulthoods that are terrible in the exact same way – eg someone with an abusive alcoholic father marrying an abusive alcoholic).

These hyperpriors can reach the level of a mood disorder when they become resistant to feedback. They present a couple of different arguments for how this might happen. In one, a depressed person doesn’t feel any positive emotions, since there’s such a strong prior on everything being terrible that these never reach the level of plausibility. Since positive emotions are a useful tool for figuring out what makes you happy and urging you to do it, depressed people aren’t motivated to make themselves happy, and so never end up contradicting their bias towards believing they’re sad all the time. This fits really well with “behavioral activation”, a common psychotherapy where therapists tell depressed people to just go out and do happy things whether they want to or not, and which often helps the depression resolve.

In another, all the brain’s predictions are so low-precision that it can’t even properly predict interoceptive sensations (the sensations received from organs, eg the heartbeat). Maybe it will think “I guess maybe my heart will beat right now”, but it’s not the sort of clear confident precision that really enters into its mental model. That means these interoceptive sensations are always predicted slightly incorrectly, and this keeps the brain feeling like it’s sick and confused and the world is unpredictable.

They don’t seem to mention this, but it also seems intuitively plausible that the strong prior on negativity could prevent the perception of positive factors directly. You see the piece of paper on the street, you think “the world is always terrible, so no way that’s a dollar bill”, you pass it by, and you miss an opportunity to feel lucky and give yourself a tiny bit of pleasure.

The rest of the paper is just a survey of some findings from biology and neuroscience that seem to support this, though they’re not all very specific. For example, the HPA axis is dysregulated, which fits with predictive processing, but it also fits with everything else. The main part I found interesting was this:

In healthy systems, mood should be affected by the valence of tightly controlled prediction errors. Recent animal work has shown that positive prediction errors (receiving more food than expected), show a strong positive correlation with dopaminergic change in the nucleus accumbens (Hart et al. 2014) with corresponding changes in functional brain activity in humans during a financial reward task (Rutledge et al. 2010). Similarly, it has been shown that signal change in the anterior insula is significantly related to the magnitude of prediction error (Bossaerts, 2010). The pharmacological manipulation of these networks was recently demonstrated where participants were given electric shocks (harms) in exchange for financial reward (gains), and offered the option of increasing the number of shocks in exchange for greater reward. It was shown that citalopram increased harm-aversion, while levodopa made individuals more likely to harm themselves than others (Crockett et al. 2015). This fits nicely with our notion that serotonin levels (and other neuromodulators) encode expectations about likely negative outcomes and encourage the fulfilment of these predictions through action (i.e. low levels promote behaviour with negative outcomes).

Focus on this sentence: “serotonin encodes expectations about likely negative outcomes and encourages the fulfilment of these predictions through action”. Also this one: “Low levels [of serotonin] promote behavior with negative outcomes”.

I don’t think I’m misunderstanding this – the authors cite some evidence that low serotonin causes self-harm, and yes, it certainly does. But what does it mean to have a system for promoting behavior with negative outcomes? Why have a neurotransmitter whose level corresponds to how much you should be trying to do negative-outcome behavior? Surely the answer is just “never do this”.

The only way I can make sense of this is through the paragraph above talking about the shocks-for-money game, where SSRIs decrease people’s willingness to get shocks. It sounds like maybe Friston et al are claiming that we have a “willingness to be harmed” lever so that we can calculate how willing we are to accept some levels of harm in exchange for a greater good. In that case, maybe self-harm is what happens when the “willingness to be harmed” lever is set so high that random noise, the chance of getting other people’s attention, or just passing the time presents some tiny reward, and your harm-for-reward tradeoff rate is so high that even that tiny reward is worth the harm.

More broadly, what should we think of this theory?

In retrospect, if you know Bayesian math, the idea of depression as a prior on bad outcomes seems pretty fricking obvious. I’m not even sure if it’s any different from the sort of stuff Aaron Beck was saying in the seventies. The big advance in this model is uniting “prior on bad outcomes” with “low precision of predictions / low neural confidence”. The low-precision part helps explain anergia, anhedonia, low motivation, psychomotor retardation, sensory washout, and probably (with a little more work) depression with psychotic features. Flipped around, it offers an explanation of psychomotor agitation, grandiosity, psychosis, and pareidolia in mania.

The only problem is that I still haven’t seen “prior on bad outcomes” and “low precision” really get unified. The authors seems to equivocate between “sadness means you’re in an unpredictable environment” and “sadness means you’re in a bad environment where everything sucks”. There is at least a little bit of work to add the hyperprior on top of the prior, so that at least we don’t get suspicious when we remember that depressed people are very confident in their depression. But it still seems like a world of low-precision predictions should be one where people just have no idea whether the paper in front of them is a dollar, not one where they’re really sure it isn’t. A world of high-precision predictions should look more like sitting in a bright room with a metronome, predicting each subsequent beat, rather than a world where everything is great and your life goes well. I’m not even sure this theory can explain why winning the lottery makes you happy rather than sad. It ought to make you think the world is really confusing and unpredictable (really? the thing you thought had a one in ten million chance happened?) – but in fact most lottery winners look pretty happy to me.

If this is confusing, at least it isn’t a new confusion. We know that a big part of the free energy research agenda is to try to unify desire-satisfaction with uncertainty-resolution, and claim that expectation and desire are (somehow, despite how it looks) the same thing. If we just assume that works, for the sake of argument, it allows this paper to be an impressive unification of several lines of research on mood disorder into a coherent and actionable whole.

This entry was posted in Uncategorized and tagged . Bookmark the permalink.

93 Responses to SSC Journal Club: Friston On Computational Mood

  1. poignardazur says:

    (they don’t mention this, but if prediction and action are as linked as everyone always says, I wonder if this would explain why people with terrible childhoods are always mysteriously sabotaging themselves into have adulthoods that are terrible in the exact same way – eg someone with an abusive alcoholic father marrying an abusive alcoholic).

    Quick question: is there any evidence that this is an actual thing that happens, and isn’t confounded by any of the correlation-vs-causation possibilities?

    Like, maybe some girls just emit a smell that’s really attractive to alcoholics, and girls with alcoholic fathers are more likely to have alcoholic husbands simply because they have the same alcoholic-attracting smell their mothers have? If that’s the case, then there doesn’t need to be a “people are mysteriously sabotaging themselves” syndrome at all.

    More mundanely, maybe some people have an hereditary tendency to e.g. be really attracted to impulsive people, in which case they would be sabotaging themselves, with no predictive processing magic needed.

    (same line of questions for the rest of the article; some of this seems really dubious to me)

  2. Toby Bartels says:

    There’s a blank space where it looks like there ought to be an extra emotion; maybe God will release it later as DLC.

    Maybe they don't mention it because it doesn't seem that pathological: hope.

    • poignardazur says:

      That sounds like a writing prompt for Scott to write a predictive-processing-themed reimagining of Pandora’s Box.

    • gorlitsa says:

      I was thinking that quadrant is “contentment”. Things are probably going to be mostly okay.

      • Contentment doesn’t really seem to get at this. In contentment there’s a lack of expectation of positive outcomes, even if contentment is often experienced as positive valence, although it is characterized by a sense of object-level surprise even as it lacks a sense of meta-level surprise.

        Hope makes sense though: it’s a kind of expecting things to happen that you don’t necessarily much think will happen. For this reason, though, I do think it’s pathological even if it’s not typically thought of this way because it encourages putting effort into things that are unlikely to succeed beyond what would be warranted. Yes, hope will not often lead to bad outcomes, but it may leave you perpetually disappointed that things didn’t go your way.

        “Hope” may not be quite the best word to describe this, though, but it is a kind of miscalibration, just not one that most people realize can wreck their life by leading them to spend their time hoping rather than doing.

        • RobJ says:

          It feel like the quadrant could map to something like a “stoner” personality. Low motivation (low confidence of having any predictable affect on the world) combined with a generally positive attitude that things will turn out fine (because somehow they always seem to).

      • Yosarian2 says:

        Contentment seems like the opposite. That square should be for when you think things are maybe going to be really good but have low confidence in it. Contentment seems more like having a high confidence that things are going to to be at least moderately good, combined with a sense that where you are right now is fine and peaceful and you’re not in a position where it’s worth expending a lot of energy to change anything in any big way.

    • Yosarian2 says:

      I was going to call it “cautious optimism” but I guess that’s roughly the same thing.

    • Grek says:

      My guess was the missing emotion was that ‘stomach butterflies’ sensation that I couldn’t think of good English word for. It’s like anxiety, but with positive valence. But I guess Hope is kinda the word for that feeling after all.

      • Nornagest says:

        That’s what I was trying to get at with “excitement”, above. But “hope” is a better word.

        • Upthorn says:

          I think excitement is the right term if you’re looking at predictions of outcomes of a thing you’re going to do, and hope is the right word if you’re looking at predictions of how the world is going without need of your input.

    • tjohnson314 says:

      If Friston believes that humans are uncertainty-minimizers, shouldn’t he say that this quadrant doesn’t exist?

      I think hope is more common if things have worked out for you before, even if you couldn’t predict exactly how it would happen in advance. So maybe it involves uncertainty at one level, but confidence on at meta-level.

    • cuke says:

      I was thinking of that good outcome/low confidence quadrant as equanimity. But I might be translating low confidence in too broad a sense. In Buddhist psychology equanimity is partly the result of seeing the nature of reality as basically good, but not being too attached to any particular outcome because the nature of reality is also unpredictable and changing.

    • Jacob says:

      I feel like I spend most of my life in the top-left corner. I would describe it more as “uncertain optimism” rather than “hope”.

      From the inside, it looks like I’m generally a very confident and optimistic person, but not about anything particular. I’m an opportunist, but without an overarching life mission that I’m dedicated to and confident in. My prior for trying out any new task is that I’ll be good at it: I try new sports all the time, I learn languages, I’m trying to draw etc. On the other hand, I quickly change my mind when disconfirming evidence comes: most of the things I try I suck at, but it doesn’t upset me too much that I turned out to be terrible at basketball (because my hyperprior confidence in the positive prediction was low).

  3. bob0318 says:

    Coming at this from only a passing familiarity with the body of work you’re working in:

    Aren’t the dollar bill and lottery examples confounding neurological prediction with cognitive prediction?

    I don’t think the two reduce to the same thing. Cognitive prediction, motor prediction, sensory prediction could all have a different architectural resilience against this type of global failure in the lower-level prediction mechanics.

    • poignardazur says:

      Oh yeah, I keep wondering about that when I read these articles.

      Especially regarding depression and self-esteem: maybe there is a direct link between low “neurological confidence” and low “cognitive confidence”, but if there is, it’s not at all obvious to me besides in a “these two concepts sound the same so they must be related” way.

  4. VirgilKurkjian says:

    Be careful of waiting for “real scientists” to do something. This is just another form of credentialism — I’m on the inside and I can tell you for sure that there’s nothing special they teach us as part of our training. You’re much more qualified to evaluate research than most “real scientists”.

    • peterispaikens says:

      I’m reading it less as credentialing (i.e. having “a scientist” tell the same thing) but rather about as following the scientific process – instead of Scott or someone else simply speculating about what seems to be true, he’s suggesting waiting until someone actually does the work required to make a proper data-based argument, which requires to actually get proper data – and that’s likely to be a “real scientist” making a publication; one rarely gets to spend the required resources and time otherwise.

      • VirgilKurkjian says:

        I’m sympathetic to this point in general, but I brought up my original concern because “What Is Mood: A Computational Perspective” isn’t an empirical paper; it reviews the empirical work of others. They’re not doing anything someone who is less credentialed but equally qualified couldn’t do.

  5. Shannon Alther says:

    Not strictly related, but many commenters have pointed out that the Wikipedia article on the Free Energy Principle isn’t written for a lay audience. For those of you who think you understand the concept well enough, please take a shot at rewriting parts of the article to make it meet the WP:TECHNICAL standards. There’s a lot of institutional knowledge in here that could be used for the benefit of the public.

  6. warrel says:

    This gets at why the ‘prediction’ model doesn’t seem intuitive:

    Now imagine the world’s least successful entrepreneur. Every company they make flounders and dies. Every stock they pick crashes the next day. Their vacations always get rained-out, their dates always end up with the other person leaving halfway through and sticking them with the bill.

    Shouldn’t this guy be very happy and content, because his predictions are always right?

    • Toby Bartels says:

      If he hired someone to manage the bulk of his money by doing the opposite of whatever he does with the money that he retains in his own control, then maybe he would be happy and content. Of course, that's not the same thing as what you're objecting to at all, but this whole concept seems so vague that it feels as if there ought to be a connection anyway.

      • drunkfish says:

        It’s probably fair to assume that if he tried to hire someone to do that, they would just steal all his money and leave him worse off. Or he’d suddenly start making good predictions that caused him+antihim to net negative. Somehow it’d still go wrong.

    • Doesntliketocomment says:

      Only if he in fact predicted those things. On the face of it, he appears to have predicted the opposite – I would assume he didn’t buy a stock hoping to lose money, for instance.

      If he starts to predict his failures and avoid them then he would be content in some way. For example, say he wants to buy a stock, and then thinks “This will certainly tank”, and doesn’t. If it tanks he will feel good, his prediction was right. I personally know a few satisfied pessimists.

      • Kyp says:

        This is exactly what it is.

        As an example, imagine that our unlucky investor has a bunch of optimist friends. Each time they take an action that fails, they reinforce their own prior about how likely a thing is to work. But their optimist friends keep saying “You’ve gotta keep trying, it’ll work out.” And then they get back on the horse and try again, fail again, and wind up in the same spot. You lose confidence in your own predictive abilities, because it seems like the numbers are in your favor, but then your predictions are wrong, so you rely on other people’s predictions. And then, over time, you lose confidence in their predictions also.

        Having always had depression, I can tell you this is how it usually goes for me. I want to do something, but my evaluation of it is that it won’t end well. But the prior of “you will always fail” isn’t much to go off of, and you know depression affects your perception, so you try to find people that have a more accurate perception of the situation than you do. They tell you that you’re favored, so you take the chance, it doesn’t work out, you adjust your perception of their judgment, and you move on.

        Their judgment could be wrong for any number of reasons, or you could have hit variance, but it’s wrong to view it just as “you know what will happen so you should be happy.” You have a feeling of what will happen and you have your own personal history, but you don’t want to trust those things, both because you know you have depression, and because everyone around you is saying different things. So you’re making predictions about other people’s perceptions, about your own perception, and about what will literally happen. And some of those won’t go the way you predict. If there’s one thing I can say for sure about depression, it’s that no matter how much you expect disappointment, life has a way of always surprising you with it.

        • Kevin C. says:

          If there’s one thing I can say for sure about depression, it’s that no matter how much you expect disappointment, life has a way of always surprising you with it.

          Indeed.

      • nameless1 says:

        So this is how good world and predictable world, desire and expectation are linked: via action? We expect our actions to make our world better and fulfill our desires or else we would have acted differently, and if it does not happen we feel let down and have low confidence in our predictions about what action to take to make things better for us?

        Hm. It is pretty interesting that I went through all the comments and links in both of these posts, and never figured it is action that may be linking expectation and desire, predictable world and good world. And I am actually depressed. Wait, non-depressed people actually expect their actions matter? I just do whatever is expected of me and never expect my actions make me happier, I expect either a miracle will or nothing. I never really planned to win, I planned to fail but in a way that I cannot be blamed because I took all the socially expected actions to avoid it.

        This sounds seriously interesting. How do non-depressed people expect their actions will make them happier? Let’s make an inventory. From 3 to 12 years old my happiest moments were getting exciting toys as presents. That depended on other people giving them to me, not on my actions. From about 12 to 36, it depended on trying to find a pretty girlfriend, which depended on them, not me, as I asked a lot out but most rejected. Finally married now it seems how my evening goes depends on whether my wife and daugther are fighting when I am arriving at home or they are being angels to each other. Nothing seems to depend on me.

        How do non-depressed people do things that make themselves happier that does not depend on other people?

    • benwave says:

      Happy is probably not the right word. From my reading of Scott’s reading of Frisson, Frisson’s actual proposal is that reduction of uncertainty is an the attractor in human-behaviour-space, not that reducing uncertainty makes somebody ‘happy’. ‘happy’ is a possible human-state, a potential outcome rather than a terminal goal of the system.

    • emmajoey says:

      Well, not very happy or content if he actually wants to go on vacation and have it be sunny while he’s there.
      Reminds me of the Rain God

  7. bayeniv says:

    But it still seems like a world of low-precision predictions should be one where people just have no idea whether the paper in front of them is a dollar, not one where they’re really sure it isn’t.

    You can’t assign 50% probability to everything, since there are lots of possible outcomes in the world. So maybe the thinking is something like, “I have no idea what that is on the ground, but it would be hopelessly optimistic to assume it’s a dollar.”

    I’m not even sure this theory can explain why winning the lottery makes you happy rather than sad. It ought to make you think the world is really confusing and unpredictable (really? the thing you thought had a one in ten million chance happened?) – but in fact most lottery winners look pretty happy to me.

    Isn’t it supposed to be about future predictions? Like sure, that lottery prediction in the past was wrong, but now that you’re rich, you can predictably and successfully do lots of good things!

    It seems like there’s a decently plausible hack to align prediction-precision with happiness/goodness/etc: just set priors on baseline human needs to 1. If you have a prior of 1, then no matter how much evidence you observe otherwise, Bayes’ theorem still gives you a posterior of exactly 1. So say you have prior of 100% for things like “I am not hungry”/”I am not in pain”/etc. Then whenever those are not true, you get prediction error, but you can’t fix it by updating your prediction; you have to try to change the situation instead.

    This would explain why the metronome room doesn’t work: if it doesn’t satisfy all your base needs, you aren’t predicting it. There would also probably be a base prediction like “the world is not always the same” or something similar that sets up curiosity/boredom. Then you get a happy state of flow when you have just enough novelty, but can still predict your success with good confidence.

    • benwave says:

      Spitballing here, but what do you think of this narrative: Alice is a person who has hyperpriors (or whatever we’re calling them) for unhappiness, and for low-precision predictions. She sees a green rectangle on the ground. Her brain spits out the prediction: ‘potential dollar’, but her low-precision hyperprior tells her not to have much confidence in that. Since she has a low confidence in that sensory prediction, her brain turns to the optimism/pessimism hyperprior and says ‘hey, what do you think?’. It spits out the prediction ‘everything is always terrible’ answer that it always gives. She doesn’t pick up the green rectangle, and the opportunity to update the hyperpriors is lost, depressive state remains.

      Note that if the unhappiness prior is reversed in this case, it will spit out ‘sweet, it’s a dollar! pick it up.’ Then she will properly test the prediction and be able to update her hyperpriors.

      • benwave says:

        It further occurs to me that it is possible to imagine a situation in which responding to the negative hyperprior allows the opportunity to update, while responding to the positive hyperprior does not – but it is Really Hard: The best I could come up with is Alice sees a shadow pass by her. Her sensory is uncertain what it is but a negative hyperprior tells her it’s a predator, so she stabs it with her spear. It turns out to be a delicious rabbit instead! A positive hyperprior with low accuracy might just spit out ‘it was nothing. Go on about your business’.

        It kind of suggests that for patients presenting with depression or other self-reinforcing mood disorders, a potential therapy could be to present them with situations in which responding to negative priors can persuade them to take a course of action that leads to gathering more data to update those priors, AND in which the data is in the positive direction. Easier abstractly said than done…

  8. Seems like the problem is that Scott is trying to make predictions about the present (so if you predict something good when something bad is there, you will have prediction error, and if you predict something bad when something good is there, you will have prediction error.)

    But we should look at prediction in a more normal way, as about the future. And in that way it makes sense that the more predictable the future, the better it will be, because you expect to make it be that way.

    (longer answer here: https://entirelyuseless.wordpress.com/2018/03/06/how-sex-minimizes-uncertainty/ )

  9. matthewravery says:

    It strikes me that a better way to conceptualize Fig. 1 that Scott copied here would be by showing three bell curves on a single axes, with the left side being “Difficult to Predict World” and the right side being “Easy to Predict World”. The Red bell curve would be narrow and sit towards the left side of the axis. The Green bell curve would be centered at the same place but have a much larger dispersion. The Blue curve would be on the right side and have a narrow variance similar to the Red one.

    Since they’re talking hyperparameters and dispersion, this seems much clearer to me than the display chosen, but maybe I’ve misunderstood the picture.

  10. Doesntliketocomment says:

    This discussion of prediction failure as being the supremely negative stimulus makes me think of an idea a friend of mine had back in college on the nature of Hell. The idea was that Hell was a place where no matter what you did, the outcome was terrible. If you drink from a stream, it turns to bees in your mouth. If you hide in a hole, worms burrow into your flesh. Every choice (even inaction) directly precipitated pain and suffering. As a part of this concept, somewhere in Hell there is a door that leads out. It’s not locked, or guarded, but no one ever escapes – because no one, absolutely no one, ever opens doors in Hell.

  11. snahgle says:

    The “extra emotion” – a prediction of good outcomes with low confidence – sounds a lot like the emotional state of Faith to me.

  12. peterispaikens says:

    “Depression is a prediction of bad outcomes with high confidence. Mania is a prediction of good outcomes with high confidence.” seems like a plausible story for bipolar behavior – i.e. if for whatever reason your confidence is “stuck on high”, being excessively certain about whatever the current direction of mood happens to be, then it would look like a shift between depressive and manic behavior.

  13. Eponymous says:

    It’s not really clear from this quote, but later on they’re going to shift from happiness being “the world is predictable” to “the world is good”, which – sounds a lot more common-sensical. I think this has to do with Friston’s commitment to believing that uncertainty-resolution is the only drive, and every form of goodness is a sort of predictability in a way.

    If my interpretation of Friston is correct (see my comments on the previous post for details), then Friston’s “big idea” is that peoples’ brains encode their preferences as a probability distribution (P) which is *also* their prior about how the world will generally be. But crucially this isn’t the same as their beliefs about the way the world is at this very moment, which is encoded by another probability distribution (Q).

    Thus what people seek to minimize isn’t surprise relative to their current beliefs Q, but surprise relative to their prior/preference probability distribution P (plus a term that captures the inaccuracy of their current beliefs Q).

    I think this interpretation fits well with the argument in this paper, since what I call P seems to correspond with their hyper-prior capturing overall mood / theory of the world. Then loosely speaking, a major mood disorder corresponds to getting your P out of whack.

    In other words, normally we don’t fall prey to the dark room problem because our P beliefs about what (life should generally be like / a good life is like) don’t assign a high number to spending all day in a dark room. But depression occurs when P is altered such that we come to see spending all day in a dark room as a fine/likely thing to do.

    • ADifferentAnonymous says:

      I think P has to be a hyperhyperprior or something and it’s Q that gets out of whack, since depressed people do tend to retain some desire to stop being depressed. That is, P still says that sitting in a dark room is bad, but Q says that doing anything else is even worse.

  14. ADifferentAnonymous says:

    The only problem is that I still haven’t seen “prior on bad outcomes” and “low precision” really get unified.

    I think this is solved by the idea that the brain hacks motivations by putting high priors on good outcomes?

    I’m not even sure this theory can explain why winning the lottery makes you happy rather than sad. It ought to make you think the world is really confusing and unpredictable (really? the thing you thought had a one in ten million chance happened?) – but in fact most lottery winners look pretty happy to me.

    With the above in mind, winning the lottery is a low-level surprise and a high-level antisurprise. Imagine if mid-HMPOR, Harry woke up and learned the Wizarding World was a big virtual reality game after all. He’d be surprised, in that his day-to-day assumptions were all just invalidated, but on a larger scale the world would suddenly have started making a lot more sense.

    So our lottery player is born expecting to have lots of food, safety, love, and other good things. He learns that he doesn’t always have them, but he never forgets the original prior that says he should expect these things*. He’s presumably learned that money leads to those good outcomes, so he believes the lottery win will result in the fulfillment of his prior expectations, even if the event itself is unexpected.

    *The term labelled ‘energy’ here says this directly. It’s the agent’s expectation, under current beliefs, of how unlikely the current world is according to prior beliefs.

    • Radu Floricica says:

      Yeap, it’s not (just) that winning the lottery is per se fulfilling expectations, is that having the money allows one to make all kind of expectations true.

      The problem with this kind of mind theory is that it’s really really easy to apply it to the wrong level. Saying “brain is all about solving uncertainty” is something that applies low level, and _might_, with a lot of indulgence, be used to directly explain the behavior of a worm. Saying that “but people like surprises” is talking about a whole different concept of uncertainty.

  15. MB says:

    The fourth quadrant of forbidden emotion is called hope. While still officially unauthorized, Pandora tried to issue a pirated release not long ago. The gods are currently trying to stamp out these bootlegged versions.

  16. Isaac Carroll says:

    I wonder if the “willingness to be harmed” lever also applies to backtracking in a search process. I’d be interested in the outcome of an experiment which compares manic people, depressed people, people taking stimulants, people taking depressants, and “normal” people in their approach to puzzles or other tasks which require backtracking to solve successfully.

  17. warrel says:

    The authors seems to equivocate between “sadness means you’re in an unpredictable environment” and “sadness means you’re in a bad environment where everything sucks”.

    This.

    I know there are also studies which claim that if you want to incentivize a behavior (in a Pavlovian way) , you make the rewards frequent enough but random, and its more motivating than getting a consistent reward. This seems to contradict the ‘prediction’ model .. (granted the subjects in the studies probably aren’t depressed, but the theory above seems to imply that depression gets created by inability to predict.)

    • Deiseach says:

      This fits really well with “behavioral activation”, a common psychotherapy where therapists tell depressed people to just go out and do happy things whether they want to or not, and which often helps the depression resolve.

      The trouble starts when it’s “Doing this used to make me happy but not any more and I don’t know why”, so recommending “just go out and do more of it!” is not really going to work.

      That being said, the rat example sounded uncomfortably familiar, so this post is getting something right about the theories 🙂

    • Doesntliketocomment says:

      The reason that intermittent reinforcement would work under the prediction model is that the reinforcement implies that there is a predictable, but obscure, pattern. Since the brain is compelled to tease out this pattern, it is more interesting than a “solved” pattern. From the view of the brain, the solution is the true reward, not getting the treat.

      Gambler’s beliefs could be seen as a compelling evidence for this line of thought, as a common theme is that they are close to, or have found, an underlying pattern in events that are designed to be statistically random.

  18. vV_Vv says:

    If this is confusing, at least it isn’t a new confusion. We know that a big part of the free energy research agenda is to try to unify desire-satisfaction with uncertainty-resolution, and claim that expectation and desire are (somehow, despite how it looks) the same thing. If we just assume that works, for the sake of argument, it allows this paper to be an impressive unification of several lines of research on mood disorder into a coherent and actionable whole.

    As I commented on the other post, I think that this “free energy” theory of behavior is unnecessarily convoluted and not particularly informative.

    It’s more promising to unify desire-satisfaction behaviors with uncertainty-reduction behaviors in the standard framework of economic theory as risk aversion: generally speaking, for any good thing you may desire (money, status, sex, affection, entertainment, etc.) you will be willing to trade off some of the expected value for a lower variance of the outcome. In the language of expected utility maximization, this means that your utility function is concave w.r.t. money, sex, etc.

    The interesting thing about risk aversion is that for realistic utility functions (e.g. logarithmic) it creates a baseline effect: if you are offered an investment opportunity that, for $50,000 will make you $1M with probability 10% and $0 otherwise, you will be less willing to take it if your net worth is $50,000 rather than if it is $5M.

    Let’s assume that depression is a pathological underestimation of the expectation of the amount desirable things that you will obtain: since you are risk averse w.r.t. this amount, you will strongly try to reduce the outcome variance: if any option is bad, then you’ll prefer the reliable option that is moderately bad to uncertain option that is potentially catastrophic. If anxiety is a pathological overestimation of the variance of outcomes, then it would cause behavior effects similar to depression (although arguably it would feel subjectively different as your mind is overwhelmed by constantly pondering countless unlikely scenarios, while in depression your mind locks in on the worst-case scenario). Overestimation of the expected amount of desirable things, or underestimation of its variance, would create an apparent risk-seeking behavior: e.g. mania or narcissistic personality disorder.

    • matthewravery says:

      If you want to go the Behavioral Econ route, you’ve got to account for risk-seeking behavior, as well. (So basically Prospect Theory.) People buy lotto tickets, etc. So now you’re back to explaining why some folks are risk-averse and others are risk-seeking and how people’s behavior w.r.t. uncertainty is context-dependent, and I don’t know that this excursion has done anything but relabel some of the questions that “free energy” is attempting to address.

      • vV_Vv says:

        I would write off gambling as irrational behavior caused by the fact that people tend to overestimate the probability of certain rare events that have high emotional salience.

        It’s probably the same mechanism why people worry more about dying in an airplane crash rather than a car crash.

  19. warrel says:

    To add another wrinkle to this, didn’t the one of the main “depressive realism” studies ask depressed patients to asses their ability to predict some outcome, and their (low) assesment was more accurate than the non-depressed people??

    So on one hand they are better predictors than the non-depressed, but maybe only about their non-ability to predict !

  20. Fuge says:

    But the dollar bill thing doesn’t work that way.

    1. I see a flash of green on the ground.
    2. My mind perceives this is unusual for the environment.
    3. I need to make a decision to ignore or investigate.

    But depression or happiness really doesn’t factor into this even as meta-states. Things like “Is it safe to pick it up?” or “Do I have time to?” matter much more than mental state, and those really don’t map to depression or mania unless those conditions are so potent that they would destroy your thesis; that they exist as reactions to your environment.

    Replace the dollar bill with a snake.

    1. I see a thing on the ground hissing at me.
    2. My mind perceives this is unusual for the environment.
    3. I need to decide whether to examine it or leave it alone.

    Would the hyperpriors of depression or mania matter at all in this case? Would manic people be “oh, it’s really a garden hose” and depressed people see every hose as a snake?

    • Doesntliketocomment says:

      I think you are right about the dollar bill example, simply because the action to resolve the situation is too quick and too simple. A better scenario might be something a lottery ticket- it could be worth nothing, or it could be worth a lot, but you would need to take steps to determine which it was. Or in a case from my own experience, you find some old stock certificates. How much effort do you put into determining if they are valuable? A depressed person is willing to invest less effort for the same payoff, as they are using worse odds in their calculations.

      (For the record, they weren’t. The company had gone bankrupt 40 years prior. At least the certificates themselves were attractive.)

  21. andrewnwest says:

    Emotions might be a tendency toward unusually high (or low) precision of predictions

    Potential ‘aha!’ moment for me.

    I spent three years playing poker for a living, and one of my main strengths was avoiding tilt in heads-up games. Every player knows that tilting reduces the quality of their decisions, so tilt control is paramount.

    Tilt is mostly about anger, but I also avoided playing while hungry, tired, sad, happy, etc. My theory was that for most of my learning-time, I wasn’t happy/sad/hungry, so to make my best decisions, I needed to keep my brain-state close to the one I had while learning.

    High level poker is heavily reliant on precalculated solutions, and being familiar with lots of different scenarios. So perhaps a better (or complementary?) theory would be that by avoiding playing in those states, my subconscious predictions of which scenario I was in were better?

    • knockknock says:

      Thanks Andrew — I had to look up “tilt” but it was worth it, found some interesting reading on the concept. I don’t play serious poker but handling your frustrations certainly relates to playing the ponies and more serious life activities

  22. Freddie deBoer says:

    “(they don’t mention this, but if prediction and action are as linked as everyone always says, I wonder if this would explain why people with terrible childhoods are always mysteriously sabotaging themselves into have adulthoods that are terrible in the exact same way”

    guilty

  23. Jared P says:

    So many variables. I can’t quite keep them in my head.

    Prior to some event for a low serotonin individual that is depressed (i.e. makes negative predictions):
    – I don’t even know if I want to make a prediction. Being wrong is painful. Being right is painful. It’s better to be apathetic. I don’t want to take risks.
    After making a prediction:
    – High confidence: Avoid. Avoid. Avoid. Don’t go outside.
    – Low confidence: If I do something, I may be able to prevent this. I must do something. I can’t just sit here.
    After event, negative outcome:
    – High confidence: See, I knew it. Everything is terrible.
    – Low confidence: I didn’t do enough. I should have done more.
    After event, positive outcome:
    – High confidence: Why am I like this? Why am I so terrible? Why can’t I just be happy?
    – Low confidence: Was that actually me? Was that just chance? I cant tell. Keep doing stuff. Never sit still.
    After event, ambiguous outcome:
    – High confidence: My priors say this was going to be bad, and I can justify that. Why are they telling me I should look at the world through rose colored glasses? Don’t they know it hurts worse to be wrong about a happy event than to be right about a sad event?
    – Low confidence: ?

    Prior to some event for a high serotonin individual:
    – It doesn’t matter if I’m right or wrong. What need do I have for low confidence? I should just be as confident as possible and try for the best things! It never hurts to be wrong!

    Makes sense to me.

    • Deiseach says:

      Don’t they know it hurts worse to be wrong about a happy event than to be right about a sad event?

      Yeah. I think this is something that’s not really explored enough. If you’re encouraged to go out and do X because X is fun! X will go well! Negative thoughts lead to negative consequences so think positive thoughts instead! (CBT is really big on this) and you get yourself into a state of “okay, I’m going to do this and I’m going to succeed” and then it goes badly (just like the old negative thoughts that you ignored would have said) – that is a lot more painful and discouraging than “I’m not expecting much out of this; oh look, it’s as miserable as I thought it would be” ever is.

      Morrissey expressed it:

      There’s a club if you’d like to go
      You could meet somebody who really loves you
      So you go and you stand on your own
      And you leave on your own
      And you go home and you cry
      And you want to die

      When you say it’s gonna happen “now”
      Well when exactly do you mean?
      See I’ve already waited too long
      And all my hope is gone

      • engleberg says:

        According to Hock’s The Four Temperaments (nihil obstat), melancholics cannot forget insults, brood over them, to think of them excites us anew- well, with that kind of attitude we aren’t going to forgive and forget the times we were encouraged to do X! X will be Fun! They said! I cannot forget them saying it would be fun; I brood over this, to think of being told this falsehood excites me anew, and I am tempted into the sin of wrath.

  24. Hitfoav says:

    Has anyone else noticed how closely this parallels Jordan Peterson’s ideas about learning and motivation in his book Maps Of Meaning?

  25. micje says:

    > A world of high-precision predictions should look more like sitting in a bright room with a metronome, predicting each subsequent beat, rather than a world where everything is great and your life goes well.

    No, if you’re able to make high-precision predictions, and you’re able to choose among different actions, you can predict which action has the best outcome, and choose that action. Doing that repeatably will cause you to be in a world where everything is great and your life goes well.

    • Right.

      “You can predict which action has the best outcome, and choose that action.”

      Not only can you, you predictably *will* do this in a predictable situation rather than sitting in a room with a metronome. So a highly predictable world will predictably be one where your life goes well in fact.

  26. shrikesnest says:

    I think “good outcome, low confidence” is what we’d casually refer to as “normal.” It’s interesting to think of that as another emotional state with actual attributes, instead of a blank slate onto which emotions are written.

  27. Nornagest says:

    There’s a blank space where it looks like there ought to be an extra emotion; maybe God will release it later as DLC.

    Predicting good outcomes with low confidence? That sounds like excitement to me.

  28. asapverg says:

    Does anyone else feel like the description of less depressed people being less willing to accept shocks is counterintuitive? Making the choice to take the shock and the money doesn’t match my mental picture of depression.

    It’s like bending down to pick up the dollar, only bending down is going to hurt. Do we know all of a sudden think that depressed people are MORE likely to bend down to pick up the dollar than their happy-go-lucky counterpart?

    • Grek says:

      I dunno, I’m depressed and I would accept being shocked with electricity for small amounts of money.

      It’s painful, certainly, but it seems low effort and hard to screw up. You just sit there and feel hurt. I can sit there and feel hurt all day. I’m doing it right now. Finding out that someone is willing to give me money for doing that would be really nice. If I had to exert an effort to do it, like if you wanted to pay me minimum wage to jog on a treadmill all day, it would be different. I wouldn’t agree to the treadmill thing because it sounds incredibly exhausting and I’d probably give up half way through and not get any money. But if there’s a machine that will give me a small electric shock and a quarter every single time I put my hand into it, that seems imminently doable.

  29. jrgarrett says:

    Near the end of his life, Jung was asked how, after he died, he would know that he was in hell. He replied, “there would be answers.” Perhaps this is a partial differentiator of hopeful vs. hopeless people. Hopeful people look for questions, not answers.

  30. Ezra says:

    Supposing we put “hope” into that vacant corner, “the opposite of depression is hope” seems very much in line with a conventional understanding.

    “The opposite of anxiety is mania” is a new one on me, but it doesn’t seem obviously wrong.

  31. mtl1882 says:

    This explanation of depression really resonated with me. In my experience, depression doesn’t feel much like sadness, but instead it’s like every thought I generate causes my brain to respond with, “Yeah, right, don’t even bother.” There is a total inability to build momentum, and while anxiety in the past may have caused low confidence, I at least could push through the anxiety and gain confidence by completing the action. With depression, it’s like no matter how many times I push through and do it, I never get into a rhythm, or “on a roll.” It has no cumulative effect, and confidence does not build. My brain still tells me, “don’t even bother going to Panera, that’s too much to even attempt,” even if I managed to drag myself there for dinner the past few days. I go to work every day convinced I won’t be able to get my tasks done, and surprise myself every day by doing it. But no sense of comfort/routine developed. It also seems like a set of converging events that lowered my confidence level on several fronts predisposed me to depression and then I bottomed out, as suggested by this post. I can’t trust myself or take my thoughts seriously. It is a horrible feeling. Everything seems pointless, because I went through a period of time where it did seem that every effort I made backfired, and now any time I go to put in effort my brain predicts it won’t be worth it. I had never seen psychomotor retardation listed as a symptom before — I thought I was just going crazy. One of the things I notice most about my depression is that when I go to do something like open a car door, there’s no force behind it, and I can’t seem to summon up enough strength to push it open.

  32. Grek says:

    The authors seems to equivocate between “sadness means you’re in an unpredictable environment” and “sadness means you’re in a bad environment where everything sucks”.

    I think this makes more sense if you internalize it instead. Then the equivocation is between “sadness means you keep failing to predict the environment” and “sadness means you suck and are bad at everything”, which seems a lot more forgivable.

    This seems to match with Depression (“I am an awful piece of shit who can’t do anything right, so whatever I think looks correct is almost certainly a mistake somehow.”); Mania (“I’m the greatest human being to ever walk the face of this planet, so all of my thoughts and actions are probably just as perfect and insightful as they seem.”) and Anxiety (“I have this nagging feeling that everything is going to go wrong and it’ll all be my fault somehow, but I don’t understand how or why that could be.”)

  33. Jeremiah says:

    Podcast version delayed…

    Some of you may be wondering where the audio version is, unfortunately I’m not in a position to record for the next few days, so I’m releasing I Can Tolerate Anything Except the Outgroup in lieu of the current post, but I’ll catch up on Monday at the latest. You can find it on the feed or the link will take you to the MP3.

    Also a reminder that earlier I did Meditations on Moloch as well.

  34. jordanrastrick says:

    In this model, why does mania transition easily into paranoia?

    • Nootropic cormorant says:

      Because if you’re absolutely confident about everything you can’t tolerate ambiguity.

    • Grek says:

      If manic people are very confidently but incorrectly predicting that they are great, the environment is rich and that everything is going to go well (as manic people do) and then they suffer a series of shocking setbacks due to the situation not being as good as their manic perceptions say it is, the brain is going to start generating hypothesises about what keeps going wrong. The mania keeps insisting that it’s not a personal or environmental problem, so the only possibility left is malicious sabotage by jealous enemies. Except that there aren’t any real enemies, just completely foreseeable environmental/personal obstacles that keep getting overlooked due to mania.

  35. Nootropic cormorant says:

    I think that introducing “good outcome” and “positive outcome” into the explaination is getting us further from understanding this stuff, there isn’t supposed to be anything good except “reducing free energy” after all.

    By this I mean that basic needs would also need to be somehow encoded as predictions, perhaps an association of hunger – eating – satiation as a cycle, with ‘eating’ initiially compromising only of digestion, but later expanded by learning of mastication, taste and so on – and thus forming a robust set of predictions that figures in other predictions, thus constituting a motivation.

    The key thing to keep in mind is that these predictions also concern the subject himself, if I could know everything with 100% confidence, I wouldn’t bet that I’d be playing around with metronomes, rather I’d predict myself a very rich and happy man.

    (The above is written with confidence appropriate to my hyperpriors, but I really know nothing about this stuff other than what Scott just posted, I’m not even sure what a Bayesian blanket is)

    Also, some comments seem to confuse precision(s) (of the priors), expected precision (first hyperprior) and precision of precision (second hyperprior), the first is how confident you are about a particular prediction, the second is how confident you’ll be on average, and the third how much can various confidences differ from each other.

    • >I think that introducing “good outcome” and “positive outcome” into the explaination is getting us further from understanding this stuff

      Right. I think this is the main thing confusing Scott. Instead of talking about “good outcome”, we should talk about “outcomes that people predictably stumble towards when they are available,” and instead of talking about “being happy”, we should talk about “conditions that people predictably stay in when they can,” and instead of talking about “being unhappy,” we should talk about “conditions that predictably result in people stumbling around until they find their way out of them,” or something like this.

      Then if you remember, as you said, that the predictions also concern the subject himself, it is easy to see why predicting good outcomes minimize uncertainty. It is just predicting that you will do the kind of stuff that people normally do, which is an unsurprising prediction.

      • Nootropic cormorant says:

        Yeah, that’s exactly how I understood it, although reading solweintraub’s comment below with Friston quotes makes me doubt my interpretation.

  36. Fossegrimen says:

    So; Predicting good outcomes and being mostly wrong and enjoying the surprise of finding out exactly how badly it goes fits in the forth quarter? If so, it exists an it’s a good place to be really.

    All we need is a label.

  37. solweintraub says:

    The only problem is that I still haven’t seen “prior on bad outcomes” and “low precision” really get unified.

    I think that simply results from goals being encoded in a probability distribution P over states. Everything in the world being considered bad is equivalent to no state having a particularly high probability under P, i.e. the distribution is rather flat which corresponds to low precision in the predictions based on P. In that case, no state is very attractive, so hard-wired states for self-preservation become attractive such as lying in bed all day and doing the absolute minimum to survive.

    A world of high-precision predictions should look more like sitting in a bright room with a metronome, predicting each subsequent beat, rather than a world where everything is great and your life goes well.

    IIRC, in some video lecture, Friston proposed that the dark room/bright room examples of absolute predictability are avoided by some innate hyperpriors that prefer/expect change (manifesting e.g. in bi-stable optical illusions where the interpretation automatically shifts from one to the other), so the while extrinsic value makes the agent seek experiences that are highly predictable, epistemic/intrinsic value makes it seek out experiences that yield high information gain, i.e. which change your beliefs about Q a lot. This seems to be the basis for boredom and this falls right out of the equations (see equation 3 in this paper).
    The metronome would ultimately become boring. The prior for change is basically the result of selective pressures trading off the advantages of intrinsic reward for world exploration vs avoiding getting stuck with highly predictable patterns (autism).

    I’m not even sure this theory can explain why winning the lottery makes you happy rather than sad.

    This was actually covered here: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2660582/

    Although, wining a lottery may be improbable it is not surprising, in the sense you expected to win on entering; imagine you won a lottery that you had not entered: you would immediately think there had been a mistake (which would be unexpected and of little value).

    This still seems quite unconvincing and it’s actually testable (is causing massively false expectations unethical?).

    • Toby Bartels says:

      Although, wining a lottery may be improbable it is not surprising, in the sense you expected to win on entering; imagine you won a lottery that you had not entered: you would immediately think there had been a mistake (which would be unexpected and of little value).

      Maybe this is addressed in the paper, but people win lotteries that they haven't entered all the time, or at least they are led to believe so, and they get so excited about it that they give their banking information to complete strangers. But it's probably hard to get the statistics showing what portion of people fall for that scam.

  38. Swami says:

    This topic intersects with something I have been noticing lately; the vitriol aimed at those suggesting that the world is (broadly) getting better.

    The most recent example is Steven Pinker with his last two books (first on the decline of violence and now on Enlightenment Progress.) Before that we had Matt Ridley and Julian Simon, among others.

    The responses to their observations often tend to lack any factual arguments. Just an anecdote or two and anger that anyone would have the nerve to even suggest that progress is in any way real or even possible. In other words, the mere suggestion or possibility of progress seems to offend them, and deeply.

    As a side note, yesterday I was chatting with a person with a difficult childhood who was always denigrated by his father. He had just learned his cancer was gone after two rounds of chemo. Yet he told me that receiving the good news was harder for him to handle than the original diagnosis. Good news was harder than bad.

    I asked him why and he told me that he “knows” that he doesn’t deserve good news and good outcomes. He was brought up believing he was worthless, he was told so constantly.

    Connecting back to my original point, I wonder if there is a sub segment of the population whose world view or framing absolutely depends upon the world going to Hell. My life sucks, but so does everyone else, so it must be OK. If you suggest it isn’t going to Hell, indeed that things are broadly getting better, you have just attacked their defense mechanism.

    Just wondering.

  39. loki-zen says:

    The only way I can make sense of this is through the paragraph above talking about the shocks-for-money game, where SSRIs decrease people’s willingness to get shocks. It sounds like maybe Friston et al are claiming that we have a “willingness to be harmed” lever so that we can calculate how willing we are to accept some levels of harm in exchange for a greater good. In that case, maybe self-harm is what happens when the “willingness to be harmed” lever is set so high that random noise, the chance of getting other people’s attention, or just passing the time presents some tiny reward, and your harm-for-reward tradeoff rate is so high that even that tiny reward is worth the harm.

    I think it’s actually simpler than that? I don’t know how much you’ve talked to your patients about why they self harm but for past-me, people I know and some people who’ve written about their experiences, it’s not random noise or attention or boredom. It actually made me feel a bit better. Idk why, endorphins maybe. But that’s a more straightforward case of ‘willingness to be harmed for reward’ combined with ‘lack of other sources of reward/relief that you predict will work’.

  40. Upthorn says:

    I feel like this theory is missing a layer: predictions about outcomes of agentive action are not necessarily tightly bound to predictions about what sort of world you’re in.

    I’ve definitely experienced simultaneously coexisting feelings that “my life is going in a very positive direction” and that “I can’t do anything without fucking up.”

  41. ohwhatisthis? says:

    >(depressed people literally see the world in washed out shades of grey)

    Bullshit.

    https://www.frontiersin.org/articles/10.3389/fpsyg.2015.00153/full

    What a giant mess of words put together incorrectly,that article.

  42. WATTA says:

    This would explain how gratitude journaling makes you happier. You think of things that are evidence for the goodness of your environment and so the prior about your environment shifts toward being more confident about it being good. The surprising thing about this is perhaps the feeling that mere thinking and writing shouldn’t change your expectations. But of course thoughts you haven’t had before are new experiences and any new experience can be evidence that changes expectations in your mind.

  43. cuke says:

    “…are second-level priors about emotions, which themselves are first-level priors about actions.”

    I would have said “which themselves are first-level priors about thoughts” rather than actions.

    This model seems overly dependent on looking at “outcomes” objectively as either good or bad when so much of the terrain of anxiety and depression seems to be about distortion fields around thoughts, or how we interpret what we’re calling outcomes.

    Anxious or depressed people may say, “my teacher said my performance was fine — which makes me feel like a loser because I don’t stand out as exceptionally good” while someone not in the midst of a mood disorder may say to the same feedback, “right, I’m doing good enough.”

    A depressed person may say, “My friend asked me if I joined a gym this year in order to lose weight — that’s because my friend thinks I’m fat” while another person hearing the same question may have the thought “my friend is interested in what I’m up to.”

    A depressed person drops and breaks a cup and thinks, “God, I’m such a stupid shit” while the non-depressed person thinks, “I’ll go get the broom.”

    One of the things I’m struck by in working with people who have anxiety or depression is how much perfectionism and black-and-white thinking tend to go along with those mood states. So there’s a very narrow band of “good outcome” and everything outside of that is considered a “bad outcome.” Everything turning out just right is “success” while everything short of perfect is “failure.”

    But when not in the grips of depression, those same people may allow for more flexible interpretations. Which makes it seem in part that people are able to update their priors better when they’re not depressed, even if they are people who experience major depression. How would this model deal with dynamic systems in which people’s way of interpreting outcomes changes based on mood?

    • nameless1 says:

      We rarely work directly on terminal goals, there are long goal-chains so after every outcome there is a new prediction, the interpretation of the outcome cannot be separated from the prediction of how closer I got to the terminal goal (which may not even be a conscious ine). This could explain the narrow band and perfectionism: unless things are going extremely well, the depressed person expects things will just go wrong next step in the chain. If I had 95% on a freshman test, gosh, I had 5% wrong, and this was an easy test, things will get harder and harder and when the tests are hard enough I will fail them and never get a job etc. etc. but if it is 100% then there can be still doubt about future performance but at least no current evidence that could be inflated into a mountain of worry. There is no EXCUSE for expecting things to go wrong in a later step.

      When breaking a cup, AFAIK “I can never do anything right” is more common than “I am so stupid”. So it sounds like a prediction of future performance.

      Do you know depressed humor? Yes, it exists. It’s the Murphy’s Law kind of things. Things that can go wrong, will. Things that cannot go wrong, still will. Ha ha. Situation Normal: All Fscked Up.

  44. jollybard says:

    I have bipolar type II. In my experience, depression does not necessarily feel like sadness, nor does hypomania feel like happiness, believe it or not. One can be depressed and still have loads of fun hiding in a dark room binge-watching anime and making ironic memes about depression, and one can be manic and be in complete despair because you are so utterly convinced that all your life will be going to shit in the next week.

    In fact, what does seem to work best to describe what my mood episodes feel like is the prediction thing. When I’m depressed, everything feels uncertain, I have no idea what the hell I’ll be doing in the future, so nothing makes sense and I don’t care about anything; I’m also both physically and cognitively sluggish, as you mentioned. But that doesn’t mean I can’t have fun in the moment (in fact, the turning point in realising that was indeed depressed was when I became completely anhedonic).

    When I’m hypomanic, I am hugely confident about everything, good or bad. So one day I might be ecstatic about a new project I have, and the other I might crippled by panic attacks because I’m convinced I have heart problems (which is a priori extremely unlikely given my age and family history). Everything is more salient, tiny things just in the corner in my eyes feel like they’re screaming right in my ears, and I’m holding about four thoughts at once all the time. I also move faster and speak more.

    Given all this, I can’t agree with the leap from mood to emotion that they are trying to make. First of all depression feels like low confidence in everything, not high confidence in bad things. I’d much rather we kept those separate.

    (Caveat: this is my personal experience and bipolar symptoms manifest themselves in myriad ways.)

  45. Radu Floricica says:

    Read this paper by Baumeister some years back. Can’t find the free pdf anymore, but seems relevant to the subject.

  46. nameless1 says:

    Scott,

    I don’t remember if I raised it before – there are two kinds of depression, A) where your ability to feel pleasure is numbed B) you can feel pleasure perfectly well but you are not motivated to seek it out.

    That friend you had away from his computer to a party, and he enjoyed it, but next time you have to drag him again. He is just not making that sort of “I like it, therefore I am gonna do it” connection.

    That is me. I don’t see the world in grayscale. This seems to be common in my family. SSRI’s don’t really help, because maybe it is more dopamine-related.

    Perhaps a more striking example, something you can identify easier with your patients. People like me feel the need to pee but not actually stand up and go. Just put up with it. I don’t know why. Just because it is unpleasant it does not really mean I have to do something about it.

    Psychologically it could be some kind of a wrong thought like “I don’t deserve pleasure because I am guilty and bad” or “You have to pay for pleasure with future pain, better avoid it, and current pain is okay as you are paying some debt off and you will have less future pain” or something of that sort but frankly I cannot even identify such thoughts in me. I don’t really think I believe stuff like this consciously.

    Another, perhaps related thing is always being somewhere else with thoughts and not being in the here and now. Meditation used to be very useful for this but currently meditation feels very hard to do.

    In this model it woulds like now neural confidence but not big priors on bad outcomes. Except it is not actually anxiety, nor agitatedness.

    This is the first time I hear about behavioral activation but sounds like similar to what I prescribed to myself: try random things, random foods, music etc.

    • jollybard says:

      This is what I experience during my depressive phases, as I explained in a comment above. I believe this is called atypical depression? But yes, it feels like low neural confidence in everything.

      In fact, I have only rarely experienced what is called melancholic depression, which is what one usually thinks of when thinking of depression, which would explain why it took such a long time for me to connect the dots.

  47. Ambi says:

    In that case, maybe self-harm is what happens when the “willingness to be harmed” lever is set so high that random noise, the chance of getting other people’s attention, or just passing the time presents some tiny reward, and your harm-for-reward tradeoff rate is so high that even that tiny reward is worth the harm.

    This came off as typical minding to me. I self-harm every once in a while and when I’m in that state it’s like the entire concept of “good” has changed to the point where I actively don’t want to be happy. I’m sure the incentive of getting others’ attention is the cause of some self-harm but in my experience, thinking about it as if it’s a normal functioning mind misses something key about the experience.

    • minorin says:

      Indeed. This strikes me as a rather extreme misunderstanding of the motivations behind self-harm. The pain incurred isn’t on the “harm” side of the tradeoff at all; that’s the *reward*. Why? I think that varies from person to person; perhaps it’s a mechanism of processing intense emotions, or an avenue for control over a bodily experience. Regardless of why, self-harm is often for its own sake, not on account of “random noise” or “for attention” (the latter, in particular, is a common and nasty way of dismissing self-harm as unworthy of help).

  48. hollyluja says:

    serious question: given this framework, can one be an anxious optimist?