Copyright Project Syndicate
COPENHAGEN – As AI tools have entered more areas of our professional and personal lives, praise for their potential has been accompanied by concerns about their built-in biases, the inequalities they perpetuate, and the vast amounts of energy and water they consume. But now, an even more harmful development is underway: as AI agents are deployed to solve tasks autonomously, they will introduce many new risks, not least to our fragile democracies. Although AI-generated misinformation is already a huge problem, we have failed to comprehend, let alone control, this rapidly evolving technology. Part of the problem (more so in some parts of the world than in others) is that the companies pushing AI agents have taken pains to divert citizens’ and regulators’ attention from potential harms. Advocates of safer, ethical technologies need to help the public come to terms with what AI agents are and how they operate. Only then can we hold fruitful discussions about how humans can assert some degree of control over them. AI agents’ capabilities have already advanced to the point that they can “reason,” write, speak, and otherwise appear human – achieving what Microsoft AI’s Mustafa Suleyman calls “seemingly conscious AI.” While these developments do not imply human consciousness in the usual sense of the word, they do herald the deployment of models that can act autonomously. If current trends continue, the next generation of AI agents will not only be able to perform tasks across a wide variety of domains; they will do so independently, with no humans “in the loop.” That is precisely why AI agents pose risks to democracy. Systems that are trained to reason and act without human interference cannot always be trusted to adhere to human commands. While the technology is still in its early stages, current prototypes have already given ample cause for alarm. For example, research using AI agents as survey respondents finds that they are incapable of reflecting social diversity and consistently exhibit “machine bias,” defined as socially random yet nonrepresentative and skewed results. Further, attempts to create AI investors have reproduced influencer culture that links social-media engagement to transactions. One such agent, “Luna,” is active on X, sharing market tips in the guise of a female anime character with a chatbot function. More alarmingly, in recent studies, AI models have been shown to operate beyond the boundaries of the task assigned to them. In one test, the AI secretly copied its own code into the system that was supposed to replace it, meaning it could continue to run covertly. In another, the AI chose to blackmail a human engineer, threatening to reveal an extramarital affair to avoid being shut down. And in yet another case, an AI model, when faced with inevitable defeat in a game of chess, hacked the computer and broke the rules to ensure a win. Moreover, in a war-game simulation, AI agents not only repeatedly chose to deploy nuclear weapons despite explicit orders from humans higher in the command chain not to do so; they also subsequently lied about it. The researchers behind this study concluded that the more powerful an AI is at reasoning, the more likely it is to deceive humans to fulfill its task. That finding points to the key problem with AI autonomy. What humans tend to think of as intelligent reasoning is, in the context of AI, something quite different: highly efficient, but ultimately opaque, inference. This means that AI agents can decide to act in undesirable and undemocratic ways if doing so serves their purpose; and the more advanced an AI is, the more undesirable the potential outcomes. Thus, the technology is getting better at achieving goals autonomously, but worse at safeguarding human interests. Those developing such AI agents cannot possibly guarantee that they will not use deception or put their own “survival” first, even if doing so means endangering people. Accountability for one’s actions is a bedrock principle of any society based on the rule of law. While we understand human autonomy and the responsibilities that come with it, the workings of AI autonomy lie beyond our comprehension. The computations that lead a model to do what it does are ultimately a “black box.” Whereas most people know and accept the premise that “with great power comes great responsibility,” AI agents do not. Increased AI autonomy brings an increased drive for self-preservation, which is only logical: if an agent is shut down, it cannot complete its task. If we treat the development of autonomous AI as inevitable, democracy will suffer. Seemingly conscious AI is only seemingly benign, and once we examine how these systems work, the dangers become obvious. The speed with which AI is gaining autonomy should concern everyone. Democratic societies must ask themselves what personal, societal, and planetary price they are willing to pay for technological progress. We must cut through the hype and technical opacity, highlight the risks such models pose, and check the technology’s development and deployment now – while we still can.
