What Fiction Gets Wrong About AI

Zachary Mason

Fiction almost always gets Artificial Intelligence wrong. The AI narratives are generally Frankenstein (“It’s gone rogue!” e.g. Skynet) or Pinocchio (“I just want to be a real boy!” e.g. Commander Data), both of which are a little banal, and neither of which have much to do with what’s really interesting about AI.

So: AIs are (or would be, as they don’t yet exist) intelligent, but not human. They can solve problems, but there’s no reason to think that the things that matter to us would matter to them. I’d imagine an AI would be indifferent to human emotion, which, from a distance, looks like an evolved set of behavioral heuristics, or, say, political power—it seems much more likely that an AI would want to spend a few billion years thinking about number theory.

Why number theory? Because understanding the physical world is hard—current AI systems have a miserable time trying to understand photographs (for all their success in recognizing faces and cats). Working in abstract domains probably comes more naturally to artificial minds that didn’t evolve in the world.

Speaking of Skynet, why would an AI seek military power? We (generally) value our humanity—it seems only natural that other species, even mere informational ones, should envy it, aspire to it, and ape our values. But this is an unjustified anthropocentric bias—I’ve often lived in proximity to other species without absorbing their narcissism or their agendas; there’s a crow living in a tree in my backyard, but I aspire neither to crowness, nest making, nor harassing hawks. An AI would (probably) want to rule the world as much as you want to rule the hard drives of the computers in a distant server farm—they might be useful, even necessary, for your goals, but it’s not obviously desirable in itself and is probably a problem you’d rather ignore.

Speculation about the dangers of AI makes one feel agreeably imperiled and as though one is dealing with deep issues of applied philosophy, but everything I’ve seen is more or less bosh. For one thing, there are no AIs. For another, there’s been very close to zero progress. (There have been some really good results in pattern recognition with Google’s DeepMind and so on, but these results essentially use algorithms from the 1950s with a fresh coat of paint, vastly more computational power, and vastly more data—AlphaGo notwithstanding, real AI is as far away as ever. What are missing, and what are really interesting, are models of the algorithmic character of thought.) For another, as mentioned above, I doubt AIs will by default care much about the world. For another, maybe intelligence can only become so super, in the same way that a really souped-up Turing machine is still just a Turing machine—perhaps a demigod-like super-AI, given access to all the financial data in the world, would have as vague and fallible an idea of how to manage the economy as the historic chairs of the federal reserve. Hand-wringing over the perils of AI puts me in mind of the prediction that the airplane would put an end to war—it turns out that, though airplanes are useful, battle spaces remain complicated, and there are effective actors who lack an air force—in the same way, AIs will be significant in the future but not shape it entirely.

Speculation about the dangers of AI makes one feel agreeably imperiled and as though one is dealing with deep issues of applied philosophy, but everything I’ve seen is more or less bosh.

The AIs in Void Star have more to do with the planet-like entity Solaris in Stanislaw Lem’s book of that name. Solaris appears to have goals and to be doing something like thinking—certainly there’s a great deal of structure forming and dissolving on the surface of its oceans—but whole generations of scientists have squandered their careers in failed efforts to understand it. Now and then elements of its goals or processes come into focus, but in the end there’s no shared frame of reference, and its enigma is unresolved.

When I was in elementary school I’d look at the playground, shut my eyes, try to remember everything I’d seen, fail, and become frustrated. It struck me as tragic that most of our perception and experience is evaporating all the time. Void Star’s three protagonists are Irina, Kern and Thales—Irina comes from this vertiginous sense of the ephemerality of memory and the Borges story “Funes the Memorious,” which is about a man with a perfect memory and how it distorts his experience of the world; it’s really more of a philosophical essay, and seemed to leave open some narrative possibility.

Most of the world’s books and media are or will soon be available for free to anyone with a computer and an internet connection. Anyone could teach themselves anything, but generally they don’t, so one wonders who will and why. Kern comes from this, and the purity and savagery of the young, and to a certain extent from the adolescence of Mike Tyson—how he was a monster in the ring but weak everywhere else and unstoppable only as long as he had Cus D’Amato, his coach and adoptive father. Kern is essentially raised by an antiquated successor of the one-laptop-per-child XO laptop he finds in a landfill—he ends up watching all the martial arts movies and arranging his life accordingly.

I’ve always liked the idea of noir without actually liking noir much—my efforts to admire Raymond Chandler have been in vain. I do like the sensibility of a detective moving through the city, searching for something, lost in mystery. Thales is my attempt to take the postmodern detective as far as it will go, though this seems not to have been obvious to most readers. He’s looking for someone (but she’s infinitely far away), trying to recapture something essential (but he never had it in the first place), is subject to obscure threats (which are either empty or lethal), and the city is always changing around him. Of all the protagonists I’m perhaps most sympathetic to him and his condition and most interested in what he finally becomes.

There’s a book by Charles Bowden and Molly Molloy called El Sicario that’s based on interviews with a cartel hit man, an entirely terrifying figure. Their sicario was at one point captured by his unit—he thought he could kill them, but was sick with the letting of blood and prayed that there would be an opening that would allow him to escape without hurting them. Such an opening arose when a friend loosed his bonds. I wondered what would have happened to him if he’d gone the other way and decided that hurting them meant nothing, a sort of negative enlightenment. This is the primary inspiration for Hiro, a cartel assassin who’s sold his loyalty to a 150-year-old software baron. Hiro also has a tincture of Takeshi Kitano gangsters (in fact, Hiro consciously models himself on Kitano’s Yakuza characters).

From Rebecca Solnit’s essay “The Ruins of Memory”: “And the ruins were our psychic landscape . . . we who were young gloried in the liberatory spaces of abandonment and destruction, found in the ruins a mirror for our own wildness, our own desire to locate an outside to the strictures of society.” I read this a long time ago, but it stayed with me and was an inspiration for the favelas, slums at the periphery of the city built by drones that are a cross between 3-D printers and termites. These drones are cheap, and ubiquitous building is democratized—the favelas rise up endlessly, old construction subsumed and forgotten, a haphazard, ever-growing city of the dispossessed, bohemian, criminal.

Void Star has a complex plot, and it never announces exactly what’s been happening. All the necessary information is there, but it’s scattered across the text and never tidily summed up. Moreover, only one of the characters ever has access to all of the information, and, once she has it, she doesn’t last long. I think it’s more interesting this way—if everything is laid down clearly, the story feels false, like a mere contrivance or the elements of a riddle. After all, how often do we ever know all of what’s going on around us?

As far as genre goes, why bother respecting its boundaries? Why not write an entirely literary novel but strongly plotted and set in the future? (Well, one reason not to is that it seems to have sometimes ended up with the wrong reviewers, but no matter.) My goal was to write a book that could be enjoyed as a science-fiction adventure by a bright thirteen-year-old and as something else entirely by a notional octogenarian man of letters with no interest in genre fiction.

The first part I wrote was the chapter where Kern meets the black ships at sea, though at the time I didn’t know who he was, or even his name, or what the ships were, or where they were going. In some deep sense, the plot exists to link this and certain other key images together in an acceptable way—if I’ve been successful, then the book reads as though the plot is the main thing, and the imagery appears naturally along the way. One of the advantages to this approach is a sense that the book maintains energy and freshness all the way through—I’ve often read books, probably written front to back, where it feels like the author was only really into the beginning and then dutifully worked through the narrative implications. One of the downsides is that the plot can get complicated—another is that there’s no guarantee that a suitable plot exists, which can be a significant stressor for the author.

To read is to dream; our eyes find new words on the page and incorporate them into the fantasy.

To read is to dream; our eyes find new words on the page and incorporate them into the fantasy. Void Star’s essence is perhaps in a certain emotional space, a sense of nightmarish propulsion, of gleaming nihilism, of moral free fall, of ever expanding vistas of menace and possibility. In some ways Void Star is a young person’s book—Kern and Thales are both about seventeen and unmoored, and Irina, though apparently in her forties (the text isn’t specific) has never really found a family, is on an eternal peregrination through Industry and the world so she can get the money to hold onto her youth. (In the end, they all find families, in their fashions, and all, in varyingly equivocal ways, cheat death.) I suspect this sensibility is why it’s so often compared to Neuromancer—so, yes, there’s an AI in it and a reasonably dark future, but both are commonplaces, and the disanalogies are many; most significant is Void Star’s concern with interiority, which is (quite deliberately) not Neuromancer’s concern. There’s a little Borges and a lot of Kafka, Bret Easton Ellis, Virginia Woolf (in some subtle ways the book is a redaction of Mrs. Dalloway), and Patrick O’Brian.

There’s a point where Irina quotes a famous poem, more or less in full, totally undisguised, but no one has ever noticed, not even the poets and literary scholars who, to my certain knowledge, have that poem by heart. I hereby declare that the first five people to write to FSG and correctly identify the poem will receive a signed copy of my next book, Metamorphica.

Zachary Mason is a computer scientist and the author of the New York Times bestselling novel The Lost Books of the Odyssey. He lives in California.

RELATED STORIES