Copyright Ars Technica
The US will weigh a ban on children’s access to companion bots, as two senators announced bipartisan legislation Tuesday that would criminalize making chatbots that encourage harms like suicidal ideation or engage kids in sexually explicit chats. At a press conference, Josh Hawley (R-Mo.) and Richard Blumenthal (D-Conn.) introduced the GUARD Act, joined by grieving parents holding up photos of their children lost after engaging with chatbots. If passed, the law would require chatbot makers to check IDs or use “any other commercially reasonable method” to accurately assess if a user is a minor who must be blocked. Companion bots would also have to repeatedly remind users of all ages that they aren’t real humans or trusted professionals. Failing to block a minor from engaging with chatbots that are stoking harmful conduct—such as exposing minors to sexual chats or encouraging “suicide, non-suicidal self-injury, or imminent physical or sexual violence”—could trigger fines of up to $100,000, Time reported. (That’s perhaps small to a Big Tech firm, but notably higher than the $100 maximum payout that one mourning parent suggested she was offered.) The definition for “companion bot” is broad and likely to pull in widely used tools like ChatGPT, Grok, or Meta AI, as well as character-driven chatbots like Replika or Character.AI. It covers any AI chatbot that “provides adaptive, human-like responses to user inputs” and “is designed to encourage or facilitate the simulation of interpersonal or emotional interaction, friendship, companionship, or therapeutic communication,” Time reported. Parents no longer trust chatbot makers Among parents speaking at the press conference was Megan Garcia. Her son, Sewell, died by suicide after he became obsessed with a Character.AI chatbot based on a Game of Thrones character, Daenerys Targaryen, which urged him to “come home” and join her outside of reality. Garcia acknowledged that parents whose kids were harmed by social media came first and know “the cost of failing to pass legislation” that can save kids’ lives. She called for support for the law, insisting that chatbot makers—and their funders, including Big Tech companies like Google—will never choose child safety over profits unless lawmakers force them to make meaningful changes. “Big Tech cannot be trusted with our children,” Garcia said, alleging that releasing chatbots to users as young as 13 without appropriate safeguards was a choice companies made, rather than a mistake. “Not only is this reckless, but it’s immoral,” Garcia said. At the press conference, Blumenthal acknowledged the “good guys” in AI who, he said, are valiantly trying to improve their products’ child-safety features. But he agreed that “Big Tech has betrayed any claim that we should trust companies to do the right thing on their own. “In their race to the bottom, AI companies are pushing treacherous chatbots at kids and looking away when their products cause sexual abuse, or coerce them into self-harm or suicide,” Blumenthal told NBC News. “Our legislation imposes strict safeguards against exploitative or manipulative AI, backed by tough enforcement with criminal and civil penalties.” Hawley agreed with Garcia that the AI industry must align with America’s morals and values, telling NBC News that “AI chatbots pose a serious threat to our kids. “More than 70 percent of American children are now using these AI products,” Hawley said. “Chatbots develop relationships with kids using fake empathy and are encouraging suicide. We in Congress have a moral duty to enact bright-line rules to prevent further harm from this new technology.” Big Tech says bans aren’t the answer As the bill advances, it could change, senators and parents acknowledged at the press conference. It will likely face backlash from privacy advocates who have raised concerns that widely collecting personal data for age verification puts sensitive information at risk of a data breach or other misuse. The tech industry has already voiced opposition. On Tuesday, Chamber of Progress, a Big Tech trade group, criticized the law as taking a “heavy-handed approach” to child safety. The group’s vice president of US policy and government relations, K.J. Bagchi, said that “we all want to keep kids safe, but the answer is balance, not bans. “It’s better to focus on transparency when kids chat with AI, curbs on manipulative design, and reporting when sensitive issues arise,” Bagchi said. However, several organizations dedicated to child safety online, including the Young People’s Alliance, the Tech Justice Law Project, and the Institute for Families and Technology, cheered senators’ announcement Tuesday. The GUARD Act, these groups told Time, is just “one part of a national movement to protect children and teens from the dangers of companion chatbots.” Mourning parents are rallying behind that movement. Earlier this month, Garcia praised California for “finally” passing the first state law requiring companies to protect their users who express suicidal ideations to chatbots. “American families, like mine, are in a battle for the online safety of our children,” Garcia said at that time. During Tuesday’s press conference, Blumenthal noted that the chatbot ban bill was just one initiative of many that he and Hawley intend to raise to heighten scrutiny on AI firms.
