The dark side of generative AI is that it enables the production of misinformation (due to confabulation) and disinformation (i.e. deliberate production of fake news to achieve an end) at industrial scale. Rendering of web pages in the style of authoritative sources is straighforward, and progress in deep fakes will make video story complements easier. Aside from Vinge’s clouds of fake information to hide information (Rainbows End), which I don’t believe posed a solution, have any SF authors thought about this and how it might be tackled?
Welcome, Ken!
I don’t have an answer for your AI question, but it happened to come through just as I was writing this post over at the Observation Deck: “AI” submissions swamp Clarkesworld Magazine
Hi Ken, I was lucky to hear a discussion on this topic led by David Brin today, and I thought he shared a relevant analogy on this topic. He suggested that when presented with a legal situation where a lawyer is far better versed in the system, and has advanced skills in process and argument, the usual response is to balance this disadvantage by hiring your own legal expert. In short, building/employing an AI to match the ‘other side’ was the point he was making; fire to fight fire. I guess William Gibson traversed this ground regularly with protagonists given AI ‘agents’ to match malign AI entities.
Ken McLeod’s The Execution Channel deals with that issue, with disinfo agents themselves losing track of what the truth is amidst their own disinformation
There’s a scene early in Neal Stephenson’s new novel Fall: or, Dodge in Hell, in which a tech billionaire, sick of the way that misinformation spreads across the Internet like Western wildfires, decides to stage an intervention. He spends about a million bucks — he doesn’t need more — to create and distribute digital “evidence” of a tactical nuclear strike on the town of Moab, Utah. He hires actors, CGI experts, everything you need in order to fake a tragedy and make your creation go viral online. The idea is that once people see that completely made-up shit can utterly dominate the Internet, that there are no guard rails or boundaries to prevent that from happening, they will realize that they are continually being snookered and will grow a carapace of skepticism.
This is followed by a scene in which an master programmer creates highly advanced bots capable of relentlessly pushing petabytes of inconsistent and incoherent misinformation onto the internet, thus reinforcing the billionaire’s lesson on an unimaginably massive scale. The ENSU project starts by spewing its misinformation about one woman, who cheerfully cooperates:
If everything went according to plan, the Ethical Network Sabotage Undertaking would now issue a press release announcing its existence and explaining what it was doing. It would include a signed statement, as well as a video clip, from Maeve Braden, announcing that she was completely fine with all of this.