G.K. Chesterton On AI Risk

[An SSC reader working at an Oxford library stumbled across a previously undiscovered manuscript of G.K. Chesterton’s, expressing his thoughts on AI, x-risk, and superintelligence. She was kind enough to send me a copy, which I have faithfully transcribed]

The most outlandish thing about the modern scientific adventure stories is that they believe themselves outlandish. Mr. H. G. Wells is considered shocking for writing of inventors who travel thousands of years into the future, but the meanest church building in England has done the same. When Jules Verne set out to ‘journey to the center of the earth’ and ‘from the earth to the moon’, he seemed but a pale reflection of Dante, who took both voyages in succession before piercing the Empyrean itself. Ezekiel saw wheels of spinning flame and reported them quite soberly; our modern writers collapse in rapture before the wheels of a motorcar.

Yet if the authors disappoint, it is the reviewers who dumbfound. For no sooner does a writer fancy himself a Poe or a Dunsany for dreaming of a better sewing machine, but there comes a critic to call him overly fanciful, to accuse him of venturing outside science into madness. It is not enough to lower one’s sights from Paradise to a motorcar; one must avoid making the motorcar too bright or fast, lest it retain a hint of Paradise.

The followers of Mr. Samuel Butler speak of thinking-machines that grow grander and grander until – quite against the wishes of their engineers – they become as tyrannical angels, firmly supplanting the poor human race. This theory is neither exciting nor original; there have been tyrannical angels since the days of Noah, and our tools have been rebelling against us since the first peasant stepped on a rake. Nor have I any doubt that what Butler says will come to pass. If every generation needs its tyrant-angels, then ours has been so inoculated against the original that if Lucifer and all his hosts were to descend upon Smithfield Market to demand that the English people bend the knee, we should politely ignore them, being far too modern to have time for such things. Butler’s thinking-machines are the only tyrant-angels we will accept; fate, ever accommodating, will surely give them to us.

Yet no sooner does Mr. Butler publish his speculations then a veritable army of hard-headed critics step forth to say he has gone too far. Mr. Maciej Ceglowski, the Polish bookmark magnate, calls Butler’s theory “the idea that eats smart people” (though he does not tell us whether he considers himself digested or merely has a dim view of his own intellect). He says that “there is something unpleasant about AI alarmism as a cultural phenomenon that should make us hesitate to take it seriously.”

When Jeremiah prophecied Jerusalem’s fall, his fellow Hebrews no doubt considered his alarmism an unpleasant cultural phenomenon. And St. Paul was not driven from shore to shore because his message was pleasant to the bookmark magnates of his day. Fortified by such examples, we may wonder if this is a reason to take people more seriously rather than less. So let us look more closely at the contents of Mr. Ceglowski’s dismissal.

He writes that there are two perspectives to be taken on any great matter, the inside or the outside view. The inside view is when we think about it directly, taking it on its own terms. And the outside view is when we treat it as part of a phenomenon, asking what it resembles and whether things like it have been true in the past. And, he states, Butler’s all-powerful thinking machines resemble nothing so much as “a genie from folklore”.

I have no objection to this logic, besides that it is not carried it to its conclusion. The idea of thinking machines resembles nothing so much as a fairy tale from the Arabian Nights, and such fairy tales inevitably come true. Sinbad’s voyages have been outstripped by Magellan’s, Abdullah’s underwater breathing is matched by Mr. Fleuss’ SCUBA, and the Wright brothers’ Flyer goes higher than any Indian carpet. That there are as yet no genies seems to me less an inevitable law than a discredit to the industry of our inventors.

There is a certain strain of thinker who insists on being more naturalist than Nature. They will say with great certainty that since Thor does not exist, Mr. Tesla must not exist either, and that the stories of Asclepius disprove Pasteur. This is quite backwards: it is reasonable to argue that the Wright Brothers will never fly because Da Vinci couldn’t; it is madness to say they will never fly because Daedalus could. As well demand that we must deny Queen Victoria lest we accept Queen Mab, or doubt Jack London lest we admit Jack Frost. Nature has never been especially interested in looking naturalistic, and it ignores these people entirely and does exactly what it wants.

Now, scarce has one posited the possibility of a genie, before the question must be asked whether it is good or evil, a pious genie or an unrighteous djinn. Our interlocutor says that it shall be good – or at least not monomaniacal in its wickedness. For, he tells us, “complex minds are likely to have complex motivations; that may be part of what it even means to be intelligent”. A dullard may limit his focus to paper clips, but the mind of a genius should have to plumb the width and breadth of Heaven before satiating itself.

But I myself am a dullard, and I find paper clips strangely uninteresting. And the dullest man in a country town can milk a cow, pray a rosary, sing a tune, and court a girl all in the same morning. Ask him what is good in life, and he will talk your ear off: sporting, going for a walk in the woods, having a prosperous harvest, playing with a newborn kitten. It is only the genius who limits himself to a single mania. Alexander spent his life conquering, and if he had lived to a hundred twenty, he would have been conquering still. Samuel Johnson would not stop composing verse even on his deathbed. Even a village idiot can fall in love; Newton never did. That greatest of scientists was married only to his work, first the calculus and later the Mint. And if one prodigy can spend his span smithing guineas, who is to say that another might not smith paper clips with equal fervor?

Perhaps sensing that his arguments are weak, Ceglowski moves from the difficult task of critiquing Butler’s tyrant-angels to the much more amenable one of critiquing those who believe in them. He says that they are megalomanical sociopaths who use their belief in thinking machines as an excuse to avoid the real work of improving the world.

He says (presumably as a parable, whose point I have entirely missed) that he lives in a valley of silicon, which I picture as being surrounded by great peaks of glass. And in that valley, there are many fantastically wealthy lords. Each lord, upon looking through the glass peaks and seeing the world outside with all its misery, decides humans are less interesting than machines, and fritters his fortune upon spreading Butlerist doctrine. He is somewhat unclear on why the lords in the parable do this, save that they are a “predominantly male gang of kids, mostly white, who are…more comfortable talking to computers than to human beings”, who inevitably decide Butlerism is “more important than…malaria” and so leave the poor to die of disease.

Yet Lord Gates, an avowed Butlerite, has donated two billion pounds to fighting malaria and developed a rather effective vaccine. Mr. Karnofsky, another Butlerite, founded a philanthropic organization that moved sixty million pounds to the same cause. Even the lowly among the Butlerites have been inspired to at least small acts of generosity. A certain Butlerite doctor of my acquaintance (whom I recently had to rebuke for his habit of forging pamphlets in my name) donated seventy-five hundred pounds to a charity fighting malaria just last year. If the hardest-headed critic has done the same, I shall eat my hat1. The proverb says that people in glass houses should not throw stones; perhaps the same is true of glass valleys.

I have met an inordinate number of atheists who criticize the Church for devoting itself to the invisible and the eternal, instead of to the practical and hard-headed work of helping the poor on Earth. They list all of the great signs of Church wealth – the grand cathedrals, the priestly vestments – and ask whether all of that might not better be spent on poorhouses, or dormitories for the homeless. In vain do I remind them that the only place in London where a poor man may be assured of a meal is the church kitchens, and that if he needs a bed the first person he will ask is the parish priest. In vain do I mention the saintly men who organize Christian hospitals in East Africa. The atheist accepts all of it, and says it is not enough. Then I ask him if he himself has ever given the poor a shilling, and he tells me that is beside the point.

Why are those most fixated on something vast and far away so often the only ones to spare a thought for the poor right beside them? Why did St. Francis minister to the lepers, while the princes of his day, seemingly undistracted by the burdens of faith, nevertheless found themselves otherwise engaged? It is simply this – that charity is the fruit of humility, and humility requires something before which to humble one’s self. The thing itself matters little; the Hindoo who prostrates himself before elephants is no less humble than the Gnostic who prostrates himself before ultimate truth; perhaps he is more so. It is contact with the great and solemn that has salutary effects on the mind, and if to a jungle-dweller an elephant is greatest of all, it is not surprising that factory-dwellers should turn to thinking-machines for their contact with the transcendent.

And it is that contact which Mr. Ceglowski most fears. For he thinks that “if everybody contemplates the infinite instead of fixing the drains, many of us will die of cholera.” I wonder if he has ever treated a cholera patient. This is not a rhetorical question; the same pamphlet-forging doctor of my acquaintance went on a medical mission to Haiti during the cholera epidemic there. It seems rather odd that someone who has never fought cholera, should be warning someone who has, that his philosophy prevents him from fighting cholera.

And indeed, this formulation is exactly backward. If everyone fixes drains instead of contemplating the infinite, we shall all die of cholera, if we do not die of boredom first. The heathens sacrificed to Apollo to avert plague; if we know now that we must fix drains instead, it is only through contemplating the infinite. Aristotle contemplated the infinite and founded Natural Philosophy; St. Benedict contemplated the infinite and preserved it. Descartes contemplated the infinite and derived the equations of optics; Hooke contemplated infinity and turned them into the microscope. And when all of these infinities had been completed – the Forms of Plato giving way to the orisons of monks, the cold hard lines of the natural philosophers terminating in the green hills of England to raise smokestacks out of empty fields – then and only then did the heavens open, a choir of angels break into song, and a plumber fix a drain.

But he is not trapped in finitude, oh no, not he! What is a plumber but one who plumbs infinite depths? When one stoops to wade among the waste and filth to ensure the health of his fellow men, does he not take on a aspect beyond the finite, a hint of another One who descended into the dirt and grime of the world so that mankind might live? When one says that there shall certainly never be thinking-machines, because they remind him too much of God, let that man open his eyes until he is reminded of God by a plumber, or a symphony, or a dreary Sunday afternoon. Let him see God everywhere he looks, and then ask himself whether the world is truly built so that grand things can never come to pass. Mr. Butler’s thinking-machines will come to pass not because they are extraordinary, but precisely because they are ordinary, in a world where extraordinary things are the only constant of everyday life.

[1: EDIT 4/2: Mr. Ceglowski wants to clarify that he does in fact give to charity]

This entry was posted in Uncategorized and tagged , , . Bookmark the permalink.

140 Responses to G.K. Chesterton On AI Risk

  1. R Flaum says:

    The use of the term “science fiction” seems anachronistic. While “scientifiction” was coined during Chesterton’s lifetime, I don’t think “science fiction” came into wide use until later.

    • Protagoras says:

      Are you suggesting that this might be a forgery?

    • Scott Alexander says:

      What do you think Chesterton would have called it?

      • R Flaum says:

        I’m not certain that someone back then would have thought of it as a genre, as something recognizable when you point to it, at all. When Gernsback suggested the term “scientifiction”, he felt it necessary to define the qualities that these stories have in common: “a charming romance intermingled with scientific fact and prophetic vision”. Someone in the 1910s might well have just thought that some stories feature fictional scientific advances and others don’t in the same way that some stories are set in England and others in France, without needing to have a term that means stories-set-in-England. That’s just my guess, though.

        Okay, ignore all that. As semiel points out below, “Scientific romance” would be the correct term.

      • semiel says:

        The term “Scientific Romance” was apparently invented in the 1850s, and seems less anachronistic to me.

        That said, I read this as “GK Chesterton is magically in the present day”, so it didn’t personally bother me when he used the term.

        • Itai Bar-Natan says:

          Why magically? I think this post was generated by a neural network that was given a corpus of Chesteron’s writings as input.

      • axiomsofdominion says:

        Well, stories such as those involving Tom Swift, invented and published by the Stratemeyer Syndicate, were classed as adventure stories or even science adventure. You might alternatively call them “travelers’ tales.” Erewhon, Gulliver’s Travels and other such stories are examples. You might also see such stories referred to as “pulps”, or even planetary romance, a term applied to stories similar to those of Edgar Rice Burroughs. Indeed he invented both John Carter and Tarzan, which shows the common threads of adventure/travelers’ tales and the various early sci fi stories. Lovecraft, whose stories thread the needle of sci fi and fantasy was only published in the pulps during his own lifetime. Pulps came out a bit after GK’s time. The late 1800s.

        • Pulps came out a bit after GK’s time. The late 1800s.

          I assume “GK” is “Gilbert Keith” (Chesterton). He was born in 1874. His death is usually dated to 1936, although the OP appears to be evidence for a somewhat later date.

        • tumteetum says:

          lovecraft (1890-1937) in his letters to robert e. howard, uses the term “weird fiction” for his own stuff.

          and whilst i’m here, hey cassander! many thanks for recommending The First Law trilogy, its deeply twisted but i very much enjoyed it.

    • Thecommexokid says:

      Spelling “paperclip” as one word also reads anachronistic to me. Google ngram viewer.

    • 27chaos says:

      Typo and nitpicking thread:

      There is a certain strain of thinker who insists on being more naturalist than Nature. They will say with great certainty that since Thor does not exist, Mr. Tesla must not exist either, and the the stories of Asclepius disprove Pasteur.

      Double “the” before stories, delete one, replace with “that”.

      • domain322 says:

        > and he will talk you ear off

        “your”

      • Aaron Brown says:

        “Jeremiah prophecied Jerusalem’s fall” — the verb should instead be “prophesied”.

        (Or “prophesised”/”prophesized” but I feel like GKC would’ve used “prophesied”.)

      • Karl Narveson says:

        Nature has never been especially interested in looking naturalistic, and it ignores these people entirely and does exactly what it wants.

        Personified Nature should be “she”, and “want” for wish/whim/will is an anachronism. Compare (from The Napoleon of Notting Hill, first chapter, closing paragraph)

        Then the people went and did what they liked.

        So

        Nature … does exactly what she likes.

      • Winter Shaker says:

        Our host is well known for trolling his readers with ‘did you notice the double ‘the’?’ setups. Maybe the habit is hard to break.

  2. Jody Lanard says:

    Thank you, Scott, for that beautiful and oddly moving post. You are a spectacular writer in any voice you choose.

  3. coreyyanofsky says:

    I gave up on that Ceglowski essay about a third of the way in, when I encountered:

    Doing this is the ethics version of the early 20th century attempt to formalize mathematics and put it on a strict logical foundation. That this program ended in disaster for mathematical logic is never mentioned.

    …really? Really?

    …Really?


    ARE YOU FUCKING KIDDING ME?
    I guess I just imagined all that hullabaloo about Löb’s theorem then.

    • nelshoy says:

      My grandparents still remember the fallout from when Gödel made all our current math stop working, and the undecidable number of people killed as a result.

    • Protagoras says:

      I think you could be clearer about what point you are making here. It appears that you think “disaster for mathematical logic” is an odd description of the early to mid 20th century state of affairs, with which I would certainly agree. But you seem to be making a more specific version of the point that I don’t follow. How is Löb’s theorem specifically relevant?

      • coreyyanofsky says:

        Never mind that describing Gödel’s incompleteness theorems as a disaster for mathematical logic is ridiculous — he means something like the theorems were a disaster for Hilbert’s program of grounding mathematics in axiomatic systems whose consistency was to be proven by finitary methods. This is true insofar as it goes, and in a counterfactual world where AI safety researchers failed to understand this and its implications for their own program, the criticism would be fair.

        The point is that incompleteness theorems and other theorems of self-reference such as Löb’s theorems bear directly on the question of how a formal system can prove to itself that its actions won’t change its terminal goals, and this is fully understood by researchers in AI theory. Anyone who’s scratched the surface of AI safety research is going to come face-to-face with the pitfalls of self-reference in formal systems. So when Ceglowski writes, “That this program ended in disaster for mathematical logic is never mentioned” he betrays such thoroughgoing Dunning-Kruger effect and/or intellectual dishonesty that I can safely write his ideas off as not worthy of my time. (I don’t think he even realizes that the comparison he was making between formalized mathematics and formalized ethics is deeper than just a surface analogy.)

  4. nelshoy says:

    I love this. I didn’t know it was possible to craft an amazing parody and a serious rebuttal at the same time! Chesterton would be proud.

  5. John Richards says:

    This is a fantastic attempt to replicate Chesterton’s style, especially his rhetorical jiu-jitsu.

  6. Forge the Sky says:

    Spot-on style. Well done.

    As a dude who’s fairly decent at emulating other people’s thinking and writing styles, I was pretty blown away lol

    • LHN says:

      Agreed. The one point at which I was jarred was the word “piss” in the last paragraph, which I can’t remotely imagine Chesterton using in print. Aside from that it felt very right.

      • Scott Alexander says:

        Thanks, I’ve changed that word.

        • Rachael says:

          If we’re being very nitpicky about style, “go fix a drain” sounds both American and modern – I’d expect a Brit born any time before about 1980 to say “go and fix a drain”.

          Seemed flawless apart from that though.

  7. SEE says:

    Simply wonderful, Scott.

  8. summerstay says:

    You got his voice down perfectly. If you are interested in reading C.S. Lewis’s thoughts on transhumanism, you could read The Abolition of Man.

    • Scott Alexander says:

      Yeah, I feel kind of bad because I think real Chesterton would have been totally against all of this. But that’s also what made it a fun challenge.

      • hlynkacg says:

        I’m not so sure.

        Several of the arguments sound like things he would and in some cases did say. The bits about fairy-tales, humility, and “our tools rebelling against us” in particular all strike me as positions he actually held.

      • Cecil Harvey says:

        I think you’re right that he’d be against it. But he wouldn’t have argued it were impossible. Nor do I think he’d believe that superintelligent AI would be an affront to God. His principal objection would be to the relentless utilitarian pursuit of efficiency being aesthetically displeasing.

        • hlynkacg says:

          I think his response would be a bit more pointed than that. Not an affront to god, but to humanity.

          Having come to doubt whether humanity can be combined with progress, most people, easily pleased, would have elected to abandon progress and remain with humanity. Mr. Shaw, not being easily pleased, decides to throw over humanity with all its limitations and go in for progress for its own sake.

  9. northernreaction says:

    That’s very interesting, especially in light of the fact that Lewis’s The Abolition of Man was also about AI Risk.
    https://northernreaction.wordpress.com/2016/10/01/the-abolition-of-man-by-machine/

  10. J says:

    > stumbled across a previously undiscovered manuscript

    Not entirely unknown, as it turns out. In Jorge-Luis Borges’ 1949 “The Aleph”, he writes:

    Chesterton tells us of a valley of great glass spires, a panopticon whose prison is the whole world, with a great mechanical man at the center obsessing over the smallest detail.

    “I picture him,” he says, “on his throne, as though in the watchtower of a great city, surrounded by telephones, telegraphs, phonographs, the latest in radio-telephone and motion-picture and magic-lantern equipment, and glossaries and calendars and timetables and bulletins…”

    Oddly, this collation of the intimacies of millions of souls comes not from any base voyeuristic urge. Indeed, its work proceeds with the utmost impersonality and deliberation, as vegetables do, or planets.

    • Scott Alexander says:

      Whoa, that’s really interesting. But when I try to search I just get the Borges quote. Anyone know which Chesterton this is from?

      • Le Maistre Chat says:

        Knowing Borges, probably one he made up. Ever encountered a peryton in a fantasy RPG?

        • Scott Alexander says:

          Hm, okay. It’s still really creepy. Especially given the title of the story. Nothing is ever a coincidence.

      • theodidactus says:

        Borges habitually made stuff up, in order to send people on wild citation hunts. See also Pertyon and Uqbar. Not saying that’s what happened here, but it’s a possibility.

        Also: I’ll owe up to this one, you totally got me. Didn’t even get suspicious until halfway through. This reads SO MUCH like Chesterton

    • summerstay says:

      J’s playing a prank, I think. It’s a combination of a real quote from The Aleph with some stuff J just made up.

      • Scott Alexander says:

        Well, darn.

        • J says:

          I will tell my grandchildren of the day I fooled Scott Alexander! 🙂

          In reconciliation, let me offer my story about the Celestial Emporium. In grad school I discovered Foucault’s quote about the Borges quote, and being new to Borges didn’t realize the list was a phantom reference in his essay on Wilkins (who I did not know until just now was an *actual* philosopher.)

          But anyway I loved the Foucault quote and the list, and it seemed so relevant to the process of research that I taped it outside my office door. But my advisor didn’t get it, and it disturbed him enough that he made me take it down. This confirmed my low opinion of him as a man of little creativity, and shortly thereafter I gave up on grad school and took a job at a Big, Hip Company.

          To endear myself to such an enlightened, brilliant group of intellectuals, I promptly posted the list on my new office door, and decided I would randomly use entries from the list to set my status in our newfangled chat client: one day it would be “Those that belong to the emperor”, and the next it would be “Those that have just broken a vase”.

          But any old shell script could do that, so I started mixing them in with programming puns, so after “Suckling Pigs”, I punned “Enumerable Ones”, and with a flourish on the Friday afternoon of my first week, set my status to “The Weakened”.

          This earned me a visit from my very kindly yet concerned boss, who delicately enquired whether I was making subtle threats to murder all of my coworkers.

          • Anon. says:

            (who I did not know until just now was an *actual* philosopher.)

            He does seem a bit like a fictional character doesn’t he?

          • summerstay says:

            The part of Wilkins program that was actually carried out we now call “the metric system.”

  11. summerstay says:

    I quote Tolkien’s thoughts on Butler’s War Against the Machines here: http://machinamenta.blogspot.com/2013/06/tolkien-and-darwin-among-machines.html
    Here’s a short excerpt:
    “So we come inevitably from Daedalus and Icarus to the Giant Bomber. It is not an advance in wisdom! This terrible truth, glimpsed long ago by Sam Butler, sticks out so plainly and is so horrifyingly exhibited in our time, with its even worse menace for the future, that it seems almost a world wide mental disease that only a tiny minority perceive it. Even if people have heard the legends (which is getting rarer) they have no inkling of their portent…”

    • hnau says:

      +1

      In particular, the quotation at the end of the post does a great job of explaining why pseudo-Chesterton (and the real Chesterton!) puts so much weight on fairy-tale references.

    • TheEternallyPerplexed says:

      He’d not liked reading the developments in“The Last Ringbearer”, I guess 🙂 (great parts of which, btw, sound more like John le Carré).

  12. Le Maistre Chat says:

    I’d suggest deleting ” A certain Butlerite doctor of my acquaintance (whom I recently had to rebuke for his habit of forging pamphlets in my name) donated seventy-five hundred pounds to a charity fighting malaria just last year.” I felt pulled out of the parody and my intelligence insulted by such a large neon WINK WINK.

    • Scott Alexander says:

      Hm, I don’t think I can keep the part about cholera in without that, though. And I don’t want to leave open the counterargument that I just cherry-picked a few rich people.

      • J says:

        I was mainly caught up in enjoying the Chestertonesque writing, and I haven’t followed Butler/Ceglowski, so I wasn’t paying a lot of attention to the underlying EA argument. Perhaps Chat felt similarly.

      • Jai says:

        I enjoyed this bit, possibly because I’m a sucker for fourth-wall breaking recursion.

  13. Le Maistre Chat says:

    On a positive note, Scott, my favorite line was
    “there have been tyrannical angels since the days of Noah, and our tools have been rebelling against us since the first peasant stepped on a rake.”
    That’s so uncannily him.

  14. meltedcheesefondue says:

    That is sublime.

    Somehow it feels more Chesterton than Chesterton himself – after reading this, I have a better grasp of his tone and style than after merely reading his own work.

    Maybe because it’s a distillation of the crucial elements of his style?

  15. leoboiko says:

    Slow applause. Well done.

  16. Aevylmar says:

    Given that you (observably) *can* write like Chesterton, why do you ever not?

    Seriously, this is amazing.

  17. ronh says:

    I am a huge GKC fan. I created an account just so I could comment on this.

    I am in awe, sir. This was brilliantly and wonderfully crafted. I think The Man Himself would have been delighted. Well done.

  18. Le Maistre Chat says:

    Wait, wait, so was the Butlerian Jihad named in honor of Samuel Butler? All this time, I thought it was named to produce a mental image of actual butlers becoming guerrillas because robots took their jobs.

  19. Rachael says:

    That is fantastic. The style is very convincing.

    One might almost be fooled into thinking it wasn’t by the author of Unsong if it weren’t for “What is a plumber but one who plumbs infinite depths?” (That’s not meant negatively or critically. It’s a delightful line, and a pleasing in-joke for fans of your fiction.)

  20. The Pachyderminator says:

    This is a crazy good Chesterton impersonation. Not sure if that makes it a good April Fool’s joke or not (is convincingness one of the axes along which April Fool’s jokes are rated?), but an excellent post. (Oddly, just yesterday I saw someone make the (quite true) observation that C.S. Lewis fans trying to imitate the style of The Screwtape Letters never works out.)

    • MugaSofer says:

      In true Chestertonian fashion, the trouble with writing Screwtape knockoffs isn’t that it’s hard, but that it’s too easy. You just switch “good” and “bad” around and add in some demonic cackling. People get lazy.

    • DavidS says:

      I found ‘screwtape proposes a toast’ a particular fan fiction failure. Replaces self-aware self-criticism and human universals with rants against his personal political pet peeves

      • Ilya Shpitser says:

        But I mean, The Screwtape Letters isn’t particularly universal either — it’s just C.S. Lewis’ moral pet peeves. So in that sense it’s fan fiction faithful to the intent of the author.

    • Evan Þ says:

      I think Randy Alcorn did a not-too-bad job in Lord Foulgrin’s Letters. It isn’t quite the same, either in focus or style, but IMO it’s pretty good.

  21. dyfed says:

    This really is marvelous, Scott.

    I agree with you earlier when you said you felt bad because you thought Chesterton would have disapproved—he certainly would have—but not of your parody, just with the position. And I think he would have been terrifically delighted with the opportunity to rebut himself.

    I think your master-stroke here is that one can very easily imagine Chesterton agreeing because he felt that mysticism was necessary to man; AI figures God like a shadow-play. The response, on consideration, is probably that Chesterton would disavow mysticism that is not both ordinary and orthodox in the most positive sense—there is a mysticism we must have because it is true and a mysticism we must not because it is malarkey.

    This is the Chesterton of fierce and fancy rhetorical flights, without the Chesterton of hard theological reason and evidence. But it is a very good Chesterton and I find myself edified by reading both it and the essay it is a response to.

    Well done.

  22. rkn69 says:

    Really nicely done. I was so enthralled I forgot it was April 1st.

    Newton may not have found Love, but I agree with Richard Feynman, it’s more important even than physics.

  23. hnau says:

    Doesn’t quite pass the Literary Turing Test for me… but still delightful. Well done.

  24. Mark says:

    I loved the first paragraph.

    And, I have to say, if the comment section of SSC were slightly less obsequious, or slightly less pedantic, I wouldn’t be making this comment.

    So here it is – I don’t think this really captures the clarity, or vivacity, of G.K. Chesterton.

    Obviously, that’s not really any great criticism.

    [I think Chesterton tends to tell a story with his essays that people can instantly identify with: “I was talking to my friend about a sign i saw”… “my nurse used to tell me these stories”… “this guy said this, what rot”… “in starbucks last week I saw this etc. etc.”… “there is a place blah blah blah”… “A question is current in our looser English journalism touching what should be done with the German Emperor after a victory of the Allies. Our more feminine advisers incline to the view that he should be shot. This is to make a mistake about the very nature of hereditary monarchy. Assuredly the Emperor William at his worst would be entitled to say to his amiable Crown Prince what Charles II. said when his brother warned him of the plots of assassins: “They will never kill me to make you king.” Others, of greater monstrosity of mind, have suggested that he should be sent to St. Helena. So far as an estimate of his historical importance goes, he might as well be sent to Mount Calvary. What we have to deal with is an elderly, nervous, not unintelligent person who happens to be a Hohenzollern; and who, to do him justice, does think more of the Hohenzollerns as a sacred caste than of his own particular place in it. In such families the old boast and motto of hereditary kingship has a horrible and degenerate truth. The king never dies; he only decays for ever.”]

  25. Jeremiah says:

    That really was fantastic. I think you nailed Chesterton.

    Oh, also Jerusalem really is going to fall

  26. av says:

    One wonders if you don’t already have one of these terrible angels already, writing for you as you get on with your life. I don’t know how else you find the time to be so prolific.

  27. Rand says:

    Not quite a fairy tale, but there’s is a delightful exchange between Usbek and Rhedi in Montesquieu’s 1721 “Persian Letters”:

    [From Letter 105, Rhedi to Usbek]

    You are aware that since the invention of gunpowder, no fort is impregnable: which means, Usbek, that there is no asylum on earth against injustice and violence.
    I am always afraid that they will eventually succeed in discovering some secret which will provide a quicker way of making men die, and exterminate whole countries and nations.

    [From Letter 106, Usbek to Rhedi]

    You say that you are afraid of the discovery of some method of destruction that is crueller than those which are used now. No; if such a fateful invention came to be discovered, it would soon be banned by international law; by the unanimous consent of every country the discovery would be buried. It is not in the interest of rulers to make conquests by such means: they ought to look for subjects, not territory.

    It’s worth reading the complete letters (different translation, only one I could find on Google Books). There’s a broader discussion of the ethics of nuclear and chemical warfare within.

  28. NIP says:

    I really don’t mean to insult our host, but I honestly can’t believe Scott wrote this. To prove I’m not trying to slag him off, let me just say that if he did write it himself, he has been practicing diligently and congratulations are in order, because I’ve never seen a more convincing imitation of Chesterton written by someone else. Then again, it was a very short piece (by the standards of either author) on a very specific topic, in imitation of someone with a very un-subtle style, so perhaps any Chesterton fan with enough prep time could have done the same. Still, I would never have expected someone who thinks Chesterton’s theological opinions make him a fuzzy humanist heretic to be able to (even partially) pass a Chestertonian Turing-test.

    I am also in agreement with other posters who pointed out that while Chesterton certainly might have expressed his disagreement with Ceglowski’s AI-alarmist-skepticism in similar terms (though we can’t know for sure) he definitely wouldn’t have been enthusiastic about AI itself. But then, Scott didn’t exactly make faux-Chesterton take that position, so well-played.

    • Scott Alexander says:

      Thanks!

      “I’ve never seen a more convincing imitation of Chesterton written by someone else”

      Are there other people who have tried this? I’d be really interested in seeing their work.

      • NIP says:

        Thanks!

        You’re welcome.

        Are there other people who have tried this? I’d be really interested in seeing their work.

        To clarify, I’ve never actually seen anyone else attempt it (probably because if you don’t do it nearly perfectly, you end up sounding really silly). Which is still technically consistent with what I said, though I know it looks like I’m implying I’ve seen other examples. But even if I had, I’d never have expected you to do it so convicingly. I know you’re a big fan of Chesterton, of course, but after reading this post of yours back in the day, I assumed you had completely missed the point of his entire life’s work and thus would be incapable of replicating his style. I stand corrected on the latter point. Good work.

      • Nornagest says:

        There’s Chesterton pastiche in Neil Gaiman’s Sandman comics, but that’s the only other place I’ve seen it.

      • Alejandro says:

        Case for Three Detectives is a delightful pastiche mystery novel in which a locked-room murder is given three different and equally ingenious explanations by detectives which parody Sayers’ Peter Wimsey, Christie’s Hercule Poirot, and Chesterton’s Father Brown. (The true solution is eventually found by the unimaginative local police sergeant.) The voice of the Brown character is not bad at capturing Chesterton’s style.

  29. Luke85 says:

    For years, I have been trying to get my well-read friends to read Chesterton. Not being able to share the witty banter with anyone has been a uniquely frustrating experience approached only by my inability to get my blog reading friends to try SSC for a few long posts.

    The fact that you have combined these two frustrations into a blindingly brilliant unattainable post that no current friend of mine will ever come close to appreciating is both wonderful and soul-crushing. Thanks for that.

  30. HaltingProblem says:

    Great piece, very enjoyable read.

    Maciej may or may not have donated >$7500 to anti-malaria charity, but he has donated $20,000 to a homeless shelter in San Francisco. It wasn’t direct, but he won money for his bookmarking startup in a Hacker News poll and asked that the money be donated to the charity directly instead of going to him. See https://twitter.com/Pinboard/status/788199455572963328

    It’s unclear if the nature of this donation obligates the author to eat their hat.

  31. Parth says:

    I read up on G.K. Chesterton after this, and came across this quote:

    The whole modern world has divided itself into Conservatives and Progressives. The business of Progressives is to go on making mistakes. The business of the Conservatives is to prevent the mistakes from being corrected.

    Scott, you should just add this to the SSC masthead.

  32. I very much enjoyed it. Any chance your source could dig up a lost Orwell essay for next year?

  33. R Flaum says:

    I feel like, if you told Chesterton “Our computers keep getting more powerful and we’re worried that they’ll get too hard to control,” his response would be “Then stop making more powerful computers.”

    • Nancy Lebovitz says:

      I’m reminded of Chesterton disagreeing with people who said “you can’t set the clock back”– of course you can set a clock back.

  34. 75th says:

    Mommy, Daddy, don’t fight

    (I’m a big fan of both Scott and Maciej for separate non-AI-related reasons and the conflict is very decomforting)

    • marvy says:

      Conflict? I don’t think they see it that way (can’t speak for either one of course).
      Tis a perfectly friendly discussion/debate/conversation, if perhaps (slightly) more heated than ideal.

      • Paul Crowley says:

        “This scenario is a caricature of Bostrom’s argument, because I am not trying to convince you of it, but vaccinate you against it.

        • marvy says:

          That is a legitimate thing to do, so long as you admit that’s what you’re doing. It’s basically saying, “I am attacking a strawman”.

          • IrishDude says:

            I don’t think it’s productive to attack a strawman of an argument if the goal is to vaccinate against the actual argument.

          • rlms says:

            @IrishDude
            The idea is that if people see the strawman being knocked down enough, they will pattern match all similar arguments to it (even if they are valid), so they are vaccinated against even considering the actual argument seriously. It’s like how a Society For Promotion Of Wolfish Interests might employ a Boy to cry wolf, so when an actual wolf eats someone everyone ignores it.

          • IrishDude says:

            @rlms
            It could be an effective tactic, at least to less sophisticated audiences (is that who it’s aimed at?). I don’t think it’s productive for getting at truth or vaccinating sophisticated audiences that are less prone to bad pattern-matching.

  35. Eli says:

    Is this all a vast, elaborate excuse to reference Frank Herbert’s Butlerian Jihad?

  36. MereComments says:

    As a huge Chesterton fan, this is just brilliant. Thank you!

  37. daniel says:

    I’m a bit sad that “Chesterton” couldn’t resist a good quip and felt obliged to start off with:

    (though he does not tell us whether he considers himself digested or merely has a dim view of his own intellect)

    As witty as it is, the comment strikes me as quite unkind and facetious(?) at best. It’s as if a claim of a single Antelope spotting is proof that a Lion is not an animal that eats antelopes and that if you heard of an unspecified animal seeing a lion and surviving undigested it you would surmise it couldn’t possibly be an Antelope.

    Since the whole manuscript revolves around tearing apart arguments of Mr. Ceglowski which exhibit similar failures of logic it left with me a sour taste throughout.

    • Jtp says:

      That was my reaction to the unwarranted jump from “if everyone contemplates the infinite” to “if everyone fixes drains.”

    • carvenvisage says:

      First off, it’s a joke, not a refutation. It’s in the style of chesterton. His writing is full of good humoured jokes of this kind. Personally this essay totally failed the ‘turing test’, but this aspect at least is 100% in the parameters. This is not Scott’s serious, logical voice.

      Beyond that, I think you’re just wrong about those things being at all analogous:

      Speaking loosely to illustrate neutral information is obviously different than speaking loosely because it makes people who you think are misguided look stupid.

      The former is motivated by (mild) good will, and the latter by (mild) bad. If you’re a habitual generaliser you might find it confusing when you get extra leeway in the former but not the latter case, but there’s nothing mysterious about it. Generalise to illustrate information: thanks. Generalise out of laziness: some people mind most people don’t. Generalise ‘framing’ someone in a bad way: most people at least mildly dislike.

  38. Very good. Of course Chesterton in fact would have disagreed with the position. There is also something here which is actually right, but gives a hint of one of the reasons why the AI risk idea is wrong. That is, fanatically seeking a single goal actually is a failure mode of genius, not of ordinary people. But this most definitely does not mean that no matter how much you increase intelligence, the more likely something will fall into that failure mode; rather, just as people seek to avoid failure modes in general, so they seek to avoid that particular one, and anything worthy of the name “superintelligent” will find some way to avoid falling into it.

    • MugaSofer says:

      That is, fanatically seeking a single goal actually is a failure mode of genius, not of ordinary people. But this most definitely does not mean that no matter how much you increase intelligence, the more likely something will fall into that failure mode…

      Much though I enjoyed AU!Chesterton making it, this is not actually a mainstream AI-risk argument. I believe Scott invented it for the occasion as a Chesterton-y thing to say.

      • semiel says:

        I don’t think this works as an affirmative argument for AI risk, but I think it does pretty effectively blow up the counterargument that a smart AI would figure out it was misaligned. If that were true, we ought not see examples of smart people behaving in misaligned ways, but in fact we see that all the time.

  39. Walter says:

    Mr. Chesterton classes up SSC. Fantastic idea to invite him.

  40. n8chz says:

    Wait a minute, you say this is a faithful transcription of a G. K. Chesterton manuscript?

    Are you saying “AI” as an abbreviation for “artificial intelligence” was a thing in Chesterton’s time? Let alone the phrase “AI risk?” And that personal fortunes on the order of £2 billion existed? I assume you wouldn’t fake something like that, but the article is dated April 1.

    The part about Arabian Nights reminds me of something in Schroedinger’s Cat Trilogy:

    They gradually comprehended that all their myths
    had been memories of the future, available to them through the nonlocal
    activity of the quantum waves making up their brains. Age-old religious visions
    of Immortality, for instance, they recognized as precapitulations of the
    inevitable end product of their current longevity research. The “magic
    carpets” and “seven-league boots” they already had; the New
    Heaven and New Earth they were rapidly building. The superhuman heroes and
    heroines of romantic fiction were the humans they were themselves becoming as
    the HEAD Revolution accelerated them toward greater intellectual efficiency,
    more flexible emotional equilibrium, neurosomatic rapture, and metaprogramming
    wisdom.

    • Paul Crowley says:

      I have bad news for you – C S Lewis didn’t really transcribe a cache of diabolical letters either, and he didn’t even wait for April 1 to publish.

  41. Witbrock says:

    There is a “then” that should be a “than” or vice versa. I’ve forgotten where.

    “Fervour” is spelled like that.

    And there is no such word as “critiquing”. It’s “criticizing” or “offering a critique”.

    Lovely job though.

    • Witbrock says:

      Also “two thousand million pounds”. At the time, a billion was a million million, outside America, which presumably used it to translate the milliard de francs used to find its revolution. (I’m making that up, but I’m right about the word).

  42. Jayson Virissimo says:

    PSA: My city, Phoenix, has a local Chesterton Society, which is a (pipe smoking friendly) men’s club for the discussion of G. K. Chesterton’s writings and their application to our modern context. Also, many other cities have similar such things.

  43. John Nerst says:

    I haven’t even read Chesterton but I enjoyed this anyway, wonderful.

  44. wintermute92 says:

    This was delightful. That you can produce a near-flawless pastiche of Chesterton is truly impressive – he’s not an easy one to emulate. That you do so as a casual April Fool’s joke makes it even better.

  45. Marie says:

    I’d be really curious to see how this would score on your AI persuasion experiment from a while back. (If this is one of the results of you trying to write persuasive essays on AI risk, I heartily endorse the project, if only for the delightfully different tacks you’re experimenting with).

    The time travelling church building, the narrow-focused genius, and the factory-dweller’s contact with the transcendent strike me as particularly Chestertonian conceits. (I went on a google hunt convinced Chesterton had used the church example himself somewhere, especially given how much he ragged on Wells, but l think I’d just confused/conflated it with occasions when he treated the Church itself similarly)

    • hlynkacg says:

      Observation: G K Chesterton was very persuasive.

      Conclusion: Write like G K Chesterton.

      😉

  46. Matthew says:

    “The human race, to which so many of my readers belong, has been playing at children’s games from the beginning, and will probably do it till the end, which is a nuissance for the few people who grow up. And one of the games to which it is most attached is called, “Keep to-morrow dark,” and which is also named (by the rustics in Shropshire, I have no doubt) “Cheat the Prophet.” The players listen very carefully and respectfully to all that the clever men have to say about what is to happen in the next generation. The players then wait until all the clever men are dead, and bury them nicely. They then go and do something else. That is all. For a race of simple tastes, however, it is great fun.

    . . .

    “But in the beginning of the twentieth century the game of Cheat the Prophet was made far more difficult than it had ever been before. The reason was, that there were so many prophets and so many prophecies, that it was difficult to elude all their ingenuities. When a man did something free and frantic and entirely his own, a horrible thought struck him afterwards; it might have been predicted. Whenever a duke climbed a lamp-post, when a dean got drunk, he could not be really happy, he could not be certain that he was not fulfilling some prophecy. In the beginning of the twentieth century you could not see the ground for clever men. They were so common that a stupid man was quite exceptional, and when they found him, they followed him in crowds down the street and treasured him up and gave him some high post in the State. And all these clever men were at work giving accounts of what would happen in the next age, all quite clear, all quite keen-sighted and ruthless, and all quite different. And it seemed that the good old game of hoodwinking your ancestors could not really be managed this time, because the ancestors neglected meat and sleep and practical politics, so that they might meditate day and night on what their descendants would be likely to do.

    “But the way the prophets of the twentieth century went to work was this. They took something or other that was certainly going on in their time, and then said that it would go on more and more until something extraordinary happened. And very often they added that in some odd place that extraordinary thing had happened, and that it showed the signs of the times.

    “Thus, for instance, there were Mr. H. G. Wells and others, who thought that science would take charge of the future; and just as the motor-car was quicker than the coach, so some lovely thing would be quicker than the motor-car; and so on for ever. And there arose from their ashes Dr. Quilp, who said that a man could be sent on his machine so fast round the world that he could keep up a long chatty conversation in some old-world village by saying a word of a sentence each time he came round. And it was said that the experiment had been tried on an apoplectic old major, who was sent round the world so fast that there seemed to be (to the inhabitants of some other star) a continuous band round the earth of white whiskers, red complexion and tweeds—a thing like the ring of Saturn.

    . . .

    “All these clever men were prophesying with every variety of ingenuity what would happen soon, and they all did it in the same way, by taking something they saw ‘going strong,’ as the saying is, and carrying it as far as ever their imagination could stretch. This, they said, was the true and simple way of anticipating the future. ‘Just as,’ said Dr. Pellkins, in a fine passage,—’just as when we see a pig in a litter larger than the other pigs, we know that by an unalterable law of the Inscrutable it will some day be larger than an elephant, just as we know, when we see weeds and dandelions growing more and more thickly in a garden, that they must, in spite of all our efforts, grow taller than the chimney-pots and swallow the house from sight, so we know and reverently acknowledge, that when any power in human politics has shown for any period of time any considerable activity, it will go on until it reaches to the sky.'”

    — GK Chesterton, The Napoleon of Notting Hill, “Introductory Remarks on the Nature of Prophecy” link text

  47. P. George Stewart says:

    Great stuff. It actually reads a bit like Aleister Crowley too (who I think was one of the greatest English prose stylists of all time), but then they were of similar vintage, similary-educated. For anyone reading who’s into this beautiful Victorian/Edwardian style of prose, there’s an amusing literary interchange between the two, AC responding to a Chesterton review of one of AC’s poems:-

    http://www.100thmonkeypress.com/biblio/acrowley/books/mr_crowley_and_the_creeds_1904/c_and_creeds_text.pdf

  48. mondsemmel says:

    This was great! Well done.

    FYI, Scott, the link to “On the Impossibility of Supersized Machines” on Philpapers is down. Maybe switch it to a link on arxiv or something?

  49. Leon says:

    Magnificent. Also a good argument I think, though maybe this just shows how taken I am with Chesterton’s style and thought-world. Relevant tweetstorm here.

  50. Leon says:

    Also worth checking out are Chesterton’s parodies of other poets:
    http://www.unz.org/Pub/LivingAge-1921jan15-00165

  51. JenniferRM says:

    Bravo.