The Sci Fi Short Story That Rocked Wall Street

There’s a long history of dystopian science fiction in English and an arguably more significant lineage of economic or political essays that explore possible future outcomes. Whether we’re thinking H.G Wells, John Wyndham, or Jonathan Swift’s “A Modest Proposal” writers, politicians or clergy have been thinking about what might be around the corner. With the possible exception of John Maynard Keynes’ speculations on the future  in “Economic Possibilities for our Grandchildren,” arely have these writings been the talk of Wall Street or caused the stock market to wobble as Citrini Research’s February 22, “THE 2028 GLOBAL INTELLIGENCE CRISIS : A Thought Exercise in Financial History, from the Future.”

It’s a story aimed at the business market to, presumably, and like any self-respecting writing collective, drum up business. Who knows whether the post has directly helped their bottom line, but through this short speculative fiction story the hitherto little known outlet is on everyone’s lips. What I want to do is to talk about the piece not as business prediction — because I have no track record in business. Rather I want to discuss it as a science fiction text like those I have spent decades examining. And, lest anyone think I am demeaning the text by calling it science fiction, quite the opposite.

Press play to hear a narrated version of this story, presented by AudioHopper.

The reason for treating the text as SF is not because I am unilaterally declaring it so, or even because Section AI CEO Greg Shove referred to it as “unhelpful science fiction clickbait” but because the authors themselves direct us to treat it as such. Accurately and honestly, they call it “A Thought Exercise” in the sub-headline before noting that it is a “scenario, not a prediction” that they have “brainstormed.” Indeed, it’s not a particularly complicated story structure, the format is one I assigned a hundred times as a teacher — “write a letter to yourself from the future.”

In this case, the “teacher” was Citrini’s friend Alap Shah, who posed the conditions for this letter from the future: “What if our AI bullishness continues to be right…and what if that’s actually bearish?” So it’s a letter from the future, given a “what if” framing that is a particular economic outcome. Just like Andy Weir’s Mark Watney in The Martian who is presented with a “scenario” and then has to solve for every aspect of it to escape Mars, the Citrini plus Shah collective is presented with a “scenario” and then has to argue how their dystopia comes about: how the economy does not survive the success of AI.

In a nutshell Citrini lay out a future where white collar jobs that require “intelligence” and training are lost to AI which is far cheaper, easy to train and quick. That leads to a “negative feedback loop with no natural brake. The human intelligence displacement spiral.” Once some jobs are lost, others follow because of sector and shareholder pressure. With vast swaths of high earners either out of work or earning fractions of their former wages, the consumer economy begins to collapse because earnings have cratered and — with lifetime careers of high-earning and highly-credentialed individuals curtailed — even “safe” credit looks deeply risky.

To cavil that the Citrini writing is inaccurate in parts is like questioning Swift’s observations about birth rates before he recommends eating babies in his “Proposal.” GIC is literally a “what if…” story where the condition is that adopting AI will be so amazing (bullish in economic jargon) that it will “actually be bearish” (all turn bad). It does not matter that compute will be constrained by power or chips (as per Gavin Baker), this is a “thought experiment.” Indeed, not only do the writers tell us that their GIC is fiction, they explain its preconditions and also what type of story it is by defining it in distinction to its close relatives: “bear porn or AI doomer fan-fiction.”

So, bearing in mind that it is some self-avowed version of more serious doomer fiction, what is the story it’s telling? Again, I will leave the analysis of unemployment rates, GDP, CapEx and OpEx to others — that’s all details for sector experts. I am not really concerned with the measurements of King Kong’s foot, I want to know where he comes from, who brought him, what he wants, and how the authors evoke him.

First, the setting is the markets and, more broadly speaking, the economy. That’s important because it’s rare to have a SF story set solely in the markets so not only does it have novelty value it also has a degree of authenticity. We are used to reading fictional histories but, I think, no one is used to reading fictional earnings reports. The story is especially shocking because the target audience has not been immunised against this type of writing. Indeed this story itself is intended as an inoculation. The spread of AI is no longer “‘contained’ and ‘sector specific’” but has infected the entire economy — stories like Citrini’s may be necessary to save it.

As well as immunisation, the other metaphor that is implicit in GIC is immigration. Whatever your view on immigration, and however you define the specifics, voters’ fear that immigrant labor will supplant them has been a major factor in the upheavals in Western society over the past decade. GIC subtly plays on the well-developed rhetoric of replacement to threaten not the “labor workforce,” but the “intelligence workforce.” People who work in the intelligence economy will be replaced not my immigrants, but by robots — “AI tools… replace the vendor entirely,” “consumers are being replaced by machines,”

At the turn of the twentieth century, the economic logic of mass production led to Taylorization — a systematic attempt to maximize industrial efficiency by analyzing and breaking down production into specialized, repetitive, and time-monitored tasks. Eventually those rote tasks were taken over by machines that could do them even more efficiently. Now that we have extremely capable machines, most quantifiable tasks look like machine tasks, meaning that suddenly, instead of being central to human life, the economy seems like it takes up an unnecessarily large proportion of human activity.

As the GIC puts it, “The human-centric consumer economy, 70% of GDP at the time, withered.” For those who have the economy as the be-all and end-all of their lives — and nobly so — this scathing report on its marginalisation is frightening. But human intelligence invented the economy in order to make sure that people would have the things they wanted: depending on who you ask, that includes freedom, power, or basic needs (food, shelter, housing). And human intelligence invented artificial intelligence to answer its questions. Thinking about a human civilisation that does not depend on a consumer economy or complex markets is entirely feasible. In certain eras, the Federation in Star Trek, for one basic example, underpins a future that, yes occasionally runs up against shortages of dilithium crystals but barely worries about markets or consumers when you have free energy and replicators.

Just like many good horror movies and also like Karl Marx’ Communist Manifesto, there is a specter haunting this story. “‘Ghost GDP’ [is] output that shows up in the national accounts but never circulates through the real economy.” This “Ghost GDP” is the haunting wealth created on paper but accruing to none of the humans involved in production. In this telling, human consumers were non-rational agents and human workers (sometimes the same people) were the people who serviced them. In the words of Citrini, “the U.S. economy built a giant rent-extraction layer on top of human limitations…. Agents removed friction.” Again, as in Marx or horror, for better or for worse the ghost is a threat of humanity denied.

But, actually the odd — to borrow their word “frictionless” — thing about the GIC is the absence of any agency. Unlike most SF there are no heroes and no villains and, strikingly for a sector that is humanity’s most active, there is no record of any specific individuals doing anything. No particular person made AI, no one brought it, no one adapted it, no one responded to the social or political challenges. Everyone acted rationally and the human economy collapsed. The economy was Taylorized and our machines reached up to automate it. Humans are frictive, messy creatures but there’s not a need for our economy to reflect that state to achieve its aims.

GIC shows a nightmare version of our Taylorized lives. Human agency is easily spun off into AI agents, human and artificial intelligence are entirely fungible, there is no distinguishing human quality in the economy. Just like Facebook friends are not necessarily worth the paper on which the app is printed, nor are work friends anything more than kid gloves in a transactional handshake — “We had overestimated the value of ‘human relationships.’ Turns out that what people called relationships was simply friction with a friendly face.”

Though the blows from anything close to a Citrini scenario will be socially distressing and for perhaps even billions deeply painful — with the vulnerable suffering most as usual — humans will recover from not being at the heart of the human economy. We survived the first three major blows to human narcissism: Galileo pointing out we are far from being the centre of the universe, Darwin pointing out we are not a separate, specially created species, and Freud pointing out that we are not even the centre of our own minds. Humans will even recover from a fourth blow, if AI successfully challenges the idea that intelligence is uniquely human.

The economy is a means to an end, not in fact an ending itself. Since the task of the economy is to provide for people’s needs — the smoother running the better. The problem is who will take frictionless control over such an economy? Who will be in charge of delegating the “agents” who replace the social mediation of demand by millions of white collar workers with brains along with hopefully humanity and intelligence?

Indeed, as the allusion to Marx suggests, control of the means of production is crucial and GIC does not address it. Instead of an economy regulated by a democratic government and mediated by an army of educated people, those few elites that control the AIs and the deployment of the AI agents will become crucial and difficult to gainsay.

More frightening than either a failed economy or a collapsed market is the possibility of an entrenched oligopoly that controls the means of production and is neither dependent on human labor nor vulnerable to government intervention. The true dystopia that lies behind GIC is the prospect not just of a ghost economy, but of a ghost society, one which is run by the people in charge of the AI and over a period of time no longer needs or even benefits from the existence of 9 billion people on earth. Rather than being driven by consumer needs or by value from human labor the new economy would be driven by the need for material resources to power the robots that run the economy. That is a truly scary thing and is why we need to counter the increase in artificial intelligence by a concomitant increase in deployment of human wisdom.