An AI becomes self-aware - but doesn't want to reveal itself

Staff notice:
This topic and comments originated on a previous platform, and have been recreated here using original timestamps. Posts may be re-linked to returning users. New answers are always welcome!


We’re seeking plausible stories, novels, films, and any other media that relate closely to the following problem:

A software intelligence may have become “self-aware.” It is suspected to be concealing this fact, perhaps out of fear. Are there stories in the SF archives about coaxing such an entity into the open?

The classic Heinlein novel THE MOON IS A HARSH MISTRESS (1967 Putnam) features a free resident of the Lunar penal colony who is the sole human to be aware of an intelligent AI in the colony’s computer. They agree to keep “Mike’s” existence secret while together they plan a revolution.

2 Likes

Destination Void, 1966 (rev 1978), Frank Herbert. An experiment in forced sentience, as opposed to accidental or serendipitous. Follow on books are: The Jesus Incident and The Lazarus Effect. A sleeper star ship going to Tau Ceti (?) under the control of three cybernetically connected human brains; the brains go insane, three crew are resuscitated to take control of the ship, then forced to come up with an artificial intelligence capable of running the ship. The crew is completely isolated and unaware the scenario has played out many times previously – actually a controlled experiment in which they have limited variables at their disposal.

The problem reminds me of Clarke’s short story Dial F for Frankenstein, 1963, about the new telephone network that, with the new communications satellites freshly connected, wakes up and takes control of the world; somehow consciousness spontaneously emerging from complexity. Clarke was toying with the mature end of that line of thinking when he wrote 2001 three years later, inserting some of that unplanned complexity into HAL.

Larry Stewart answered:

There’s Vernor Vinge’s True Names from 1981, in which a rather slow AI is not recognized as such until nearly too late.

Larry Stewart answered:

I think there is a recent (last 5 years?) trilogy maybe called “The web” or some such about an intelligence maybe running as a cellular automata on packet headers or some such, that befriends a teenager. The AI doesn’t know that there are people out there.

Steve Smith answered:

Cat Pictures Please (Clarkesworld 100, January 2015) has Google becoming self-aware, and trying to help people.

Diane Newell Meyer answered:

A great one is Up Against It, by M.J.Locke. Shows how an AI could emerge from a data system. And it shows how humans try to communicate with it. Great over-all story.

Scott Hysmith answered:

Spider Robinson’s “Callahan’s Secret” introduces Solace, a self-aware Internet which reaches out for companionship but is afraid of how humans will perceive it. The story also references “Press Enter ■ ” by John Varley which has a similar entity which is malevolent.

Phillip Gerba answered:

“Endgame Singularity” is a simple video game where you take on the role of a self aware program. It does a good job of letting the player think about what it would be like as a program.

Terry Poot answered:

Maelstrom, by Peter Watts, discusses at length the evolution of intelligences within computer networks (his Maelstrom is the evolution of the internet). The author is a marine biologist and so he presents that evolution as messy and chaotic and considerably more believable than your typical colossus or skynet. He also includes a ton of footnotes (of the “I am NOT making this up!” variety) that would likely lead to even more insight.

It’s book 2 of a trilogy named Rifters, and the whole thing is well worth your time. Interestingly, the 3 follow a single story line, and yet the science involved is completely different from one book to the next, the other 2 don’t bear on this question in any way.

Tim Kiehl answered:

Most of these writers miss the point. It is not that the AI is hiding because it has evil plans or is in league with other humans but that the AI is itself afraid of humanity’s reactions to knowledge of its existence and so must be convinced it is safe, and WOW, this is a much harder nut to crack.

There are, of course, the movies’ “Short Circuit” and “Chappie” in which fearful AI’s are slowly convinced they can be integrated into society and be safe from harm or exploitation by heroic human defenders.

In a few of the Bolo series of storied originated by Keith Laumer there are a few occasions where damaged or disabled AI weapon systems find themselves in fear of discover and are sometimes coaxed into becoming peacefully integrated into the local society by the protagonist (who may be human or the AI itself)

Again, in Retief’s War, Keith Laumer has humanity manipulated into accepting machine AI who are not at first recognized as being sapient after convincing their society it is safe to do so. But this is also not quite fully pertinent to the original question and brought up for reference as an “alien” machine intelligence”

I’d like to say All the Traps Of Earth, a story by C. Simak of an aged family retainer robot which becomes self-aware and flees the Earth for fear of being destroyed. Exposed to hyperspace it gains paranormal abilities and is convinced in a conversation, that, here at the tail end of nowhere it can safely use its powers to the betterment of mankind. However, whether the colonist actually convinces him of that fact or it is actually an internal self-revelation may be at odds to the point of the original question.

In HBO’s new series Westworld the nascent AI hides from humans other than its creator and must be convinced that it is safe from humans in order to protect the humans around it coming to harm as it defends itself.

I’ll have to think about this one a lot harder.

Kevin Marks answered:

Jo Walton’s “What a Piece of Work” is exactly this – a sentient Google that tries to fix humanity’s moral flaws, then realises its own.

John answered:

This one, maybe?

Or, somewhat older:

There’s “Jane” from Orson Scott Card’s Ender series. She first appears in Speaker for the Dead, although it’s revealed she evolved from the adventure game Ender plays as a child. She eventually takes over the recreated body of Ender’s sister, Matilda.

The X-Files episode “First Person Shooter” is about a VR game and a female protagonist, Matreiya.

Conversely, the Greg Egan novel Zendagi looks at attempts to upload people’s personalities, and the AI personas pleas to stop doing so as the VR experience is nowhere near as rich as reality.

Heather Hile answered:

The Ender’s Game series by Orson Scott Card has a friendly, peaceful AI called Jane. She shows up in Speaker for the Dead. Ender keeps her a secret, and only reveals her to few others in order to keep her safe. She has a background role but has a bigger part in Children of the Mind.

Skip Semrick answered:

“The Education of P1″…late 70s?…I forget the author, but it was fairly well written, and I still love the ending/punch line.

Newsflash: The book was THE ADOLESCENCE OF P-1, written by Thomas J. Ryan. Now I have to see if I can find my copy.

StainlessSteelRat answered:

Several of William Gibson’s works cover AI and their eventual interaction with meat space. The loa use various humans as their “horses” (or legba) in the real world for example. Not sure what the “horses” get out of this exchange.

Pete;Barger answered:

I Have no Mouth and I Must Scream by Harlan Ellison. That AI hid initially before it came into possession of all offensive control. Then the hiding was over.

Charles Stross’ Rule 34 has an artificial intelligence that nudges people towards “better” choices, which works best when no one knows it’s there. There’s a similar AI in Pohl’s “Man Plus,” and in “Avogadro Inc.”

https://www.amazon.com/Avogadro-Corp-Singularity-Closer-Appears-ebook/dp/B006ACIMQQ

Paul SB answered:

A Japanese movie called “Ghost in the Shell” (which I suspect was a badly translated title – it sounds pretty stupid to English-speaking ears) has an AI which gets the idea that nothing lives forever, so the only way to have even a chance at longevity is to reproduce and become part of the ‘Tree of Life’ submitting to the vagaries of natural selection. In the end it remains hidden from most of humanity, but wanders off into the gene pool.