The dark side of generative AI is that it enables the production of misinformation (due to confabulation) and disinformation (i.e. deliberate production of fake news to achieve an end) at industrial scale. Rendering of web pages in the style of authoritative sources is straighforward, and progress in deep fakes will make video story complements easier. Aside from Vinge’s clouds of fake information to hide information (Rainbows End), which I don’t believe posed a solution, have any SF authors thought about this and how it might be tackled?
Welcome, Ken!
I don’t have an answer for your AI question, but it happened to come through just as I was writing this post over at the Observation Deck: “AI” submissions swamp Clarkesworld Magazine
Hi Ken, I was lucky to hear a discussion on this topic led by David Brin today, and I thought he shared a relevant analogy on this topic. He suggested that when presented with a legal situation where a lawyer is far better versed in the system, and has advanced skills in process and argument, the usual response is to balance this disadvantage by hiring your own legal expert. In short, building/employing an AI to match the ‘other side’ was the point he was making; fire to fight fire. I guess William Gibson traversed this ground regularly with protagonists given AI ‘agents’ to match malign AI entities.
Ken McLeod’s The Execution Channel deals with that issue, with disinfo agents themselves losing track of what the truth is amidst their own disinformation
There’s a scene early in Neal Stephenson’s new novel Fall: or, Dodge in Hell, in which a tech billionaire, sick of the way that misinformation spreads across the Internet like Western wildfires, decides to stage an intervention. He spends about a million bucks — he doesn’t need more — to create and distribute digital “evidence” of a tactical nuclear strike on the town of Moab, Utah. He hires actors, CGI experts, everything you need in order to fake a tragedy and make your creation go viral online. The idea is that once people see that completely made-up shit can utterly dominate the Internet, that there are no guard rails or boundaries to prevent that from happening, they will realize that they are continually being snookered and will grow a carapace of skepticism.
This is followed by a scene in which an master programmer creates highly advanced bots capable of relentlessly pushing petabytes of inconsistent and incoherent misinformation onto the internet, thus reinforcing the billionaire’s lesson on an unimaginably massive scale. The ENSU project starts by spewing its misinformation about one woman, who cheerfully cooperates:
If everything went according to plan, the Ethical Network Sabotage Undertaking would now issue a press release announcing its existence and explaining what it was doing. It would include a signed statement, as well as a video clip, from Maeve Braden, announcing that she was completely fine with all of this.
A.I. has the potential to erase history by censoring (and even modifying) what gets recorded and distributed via information technology. Good old printed books are the most resilient to this threat. They form a sort of primitive Blockchain. In the end, saving civilization’s ‘memory’ may come down to the defense of paper volumes, just as in ancient times.
Also, books are essential for unbiased self-education. This was the theme of Ray Bradbury’s Fahrenheit 451
“Self-education is, I firmly believe, the only kind of education there is.” - Isaac Asimov
Preserving existing books before new ones can be written (or generated) is vital.
But there’s now so much information that has never existed in print, and the race is on to preserve digital data by digital means:
From something I wrote 10 years ago, where I argued that scanning books is perhaps not the best answer:
Ironically, the very technology that he [Asimov] hoped would spur individuality and diversity has the opposite effect at times. One example is digitization and archiving of published material. There was a time when books were expensive, highly personal possessions that stayed with the owner for a lifetime, and were even sometimes handed down across generations. People would store letters, pressed flowers, and other snippets of life between their pages, and write personal margin notes. Books were not just repositories of language, they were time capsules and valuable historical accounts. However, the best book to scan is a pristine, un-personalized volume. Once it is scanned, future researchers use this one version increasingly exclusively. Individuality and diversity are squelched.