by We have entered the brave new world of AI chatbots. This means everything from rethinking how students learn in school to protecting us from mass-produced disinformation. It also means heeding growing calls to regulate artificial intelligence to help us navigate an age where computers write as fluently as humans. Or even better.
So far, there is more agreement on the need to regulate AI than on what it would entail. Mira Murati, head of the team that created chatbot app ChatGPT — the fastest-growing consumer Internet app in history — said governments and regulators should get involved, but did not suggest how. At a company event in March, Elon Musk similarly spoke with less than exacting precision: “We need some kind of, like, regulator or something to oversee the development of artificial intelligence.” Meanwhile, ChatGPT’s wide range of uses has bolstered European efforts to regulate single-purpose AI applications.
To break the deadlock, I propose transparency and traceability requirements tailored specifically to chatbots, which are computer programs that rely on artificial intelligence to converse with users and produce fluent text in response to typed requests. Chatbot apps like ChatGPT are an extremely important corner of artificial intelligence that is poised to reshape many everyday activities — from how we write to how we learn. Relaxing chatbots creates enough problems without getting bogged down in broader AI legislation created for autonomous weapons, facial recognition, self-driving cars, discriminatory algorithms, the economic impact of widespread automation, and the small-but-nil chance of catastrophic disaster some fear the AI may eventually fire. The tech industry is relentlessly on the chatbot gold rush. we need immediate, focused legislation to go along with it.
The new rules should track the two stages that AI companies use to create chatbots. First, an algorithm is trained on a huge amount of text to predict the missing words. If you see enough sentences that start “It’s cloudy today, it might…”, you’ll know that the most likely conclusion is “rain”—and the algorithm learns that, too. The trained algorithm can then generate words one by one, just like the autocomplete feature on your phone. Human reviewers then painstakingly rate the algorithm’s output on a handful of metrics, such as accuracy and relevance to the user’s query.
The first regulatory requirement I propose is for all consumer-facing applications that include chatbot technology to make public the text on which the AI was first trained. This text is hugely influential: train yourself on Reddit posts and the chatbot will learn to speak like a Redditor. Train them on the Flintstones and they’ll talk like Barney Rubble. A person concerned about web toxicity may want to avoid chatbots trained on text from inappropriate websites. Public pressure could even prevent companies from training chatbots on things like conspiracy-theory “news” sites—but only if the public knows what text the companies are being trained on. In Mary Shelley’s 1818 novel Frankenstein, gave a glimpse into the monster’s mind by listing the books read by this literary progenitor in artificial intelligence. It’s time for tech companies to do the same for their own eerie chatbot creations.
Human evaluators also greatly shape a chatbot’s behavior, which points to a second transparency requirement. One of ChatGPT’s engineers recently described the principles the team used to guide this second stage of training: “You want it to be useful, you want it to be true, you want it to be—you know—nontoxic.… It should also be made clear that it is an AI system. It shouldn’t assume an identity it doesn’t have, it shouldn’t claim abilities it doesn’t have, and when a user asks it to do tasks it shouldn’t, it should write a decline message.” I suspect that the guidelines given to evaluators, which included low-wage contract workers in Kenya, were more detailed, but there is currently no legal pressure to reveal anything about the training process.
As Google, Meta and others scramble to integrate chatbots into their products to keep up with Microsoft’s embrace of ChatGPT, people deserve to know the guiding principles that shape them. Elon Musk is reportedly recruiting a team to build a chatbot to compete with what he sees as ChatGPT’s over-the-top “smartness.” Without more transparency into the training process, we wonder what this means and what previously off-limits (and potentially dangerous) ideologies his chatbot will espouse.
Therefore, the second requirement is that the guidelines used in the second stage of chatbot development are carefully formulated and publicly available. This will prevent companies from training chatbots in a slapdash fashion and will reveal what political leanings a chatbot might have, what topics it won’t touch, and what toxicity developers haven’t avoided.
Just as consumers have the right to know the ingredients in their food, they should know the ingredients in their chatbots. The two transparency requirements suggested here give users the chatbot ingredient lists they deserve. This will help people make healthy choices about their informed diet.
Detection drives the third necessary requirement. Many teachers and organizations are considering banning content produced by chatbots (some have already done so, such as Wired and a popular coding Q&A site), but a ban isn’t worth much if there’s no way to detect chatbot text. OpenAI, the company behind ChatGPT, released an experimental tool to detect ChatGPT output, but it was terribly unreliable. Fortunately, there’s a better way—one that OpenAI may implement soon: watermarking. this is one technical method for changing chatbot word frequencies which is not perceptible to users, but provides a hidden stamp that identifies the text with the chatbot’s author.
Instead of just hoping that OpenAI and other chatbot producers will implement watermarking, we should enforce it. And we should require chatbot developers to register their chatbots and their unique watermark signatures with a federal agency like the Federal Trade Commission or the AI watchdog proposed by Rep. Ted Lieu. The federal agency could provide a public interface that would allow anyone to plug in a snippet of text and see which, if any, chatbots likely produced it.
The transparency and traceability measures proposed here will not slow the progress of artificial intelligence or diminish the ability of chatbots to serve society in positive ways. It would simply make it easier for consumers to make informed decisions and for humans to recognize AI-generated content. While some aspects of AI regulation are quite nuanced and difficult, these chatbot regulations are clear and urgently needed steps in the right direction.
This is an opinion and analysis article and the views expressed by the author or authors are not necessarily those of Scientific American.
