Scientists reflect on the Harry Potter nature of AI chatbots

AI-powered chatbots are like the Mirror of Erised in the first Harry Potter book, according to a leading machine learning expert, which means they only reflect what they believe the onlooker wants to see.

And while this may be frightening or perplexing for most new users, the future looks bright for human-AI interactions, says Terrene Sejnowsky, Professor at Salk Institute for Biological Studies.

In a recently published paper in Neural Computation, Sejnowski  - who is also the author of The Deep Learning Revolution - delves into the connection between the human interviewer and language models. 

The paper aims to uncover the reasons behind the specific responses generated by chatbots, the factors causing variation in those responses, and how to enhance them in the future. Sejnowski says the intelligence and diversity of the interviewer are reflected in the language models' responses.

“Language models, like ChatGPT, take on personas. The persona of the interviewer is mirrored back,” says Sejnowski, a distinguished professor at the University of California San Diego and holder of the Francis Crick Chair at Salk. “For example, when I talk to ChatGPT it seems as though another neuroscientist is talking back to me. It’s fascinating and sparks larger questions about intelligence and what ‘artificial’ truly means.”

In the paper, Sejnowski describes testing the large language models GPT-3 and LaMDA to see how they would respond to specific prompts. While the Turing Test is typically used to evaluate chatbots' human-like intelligence, Sejnowski created a "Reverse Turing Test," where the chatbot assesses the interviewer's human intelligence.

Sejnowski argues that chatbots reflect the intelligence and biases of their users, drawing a literary comparison to the Mirror of Erised in Harry Potter.  The Mirror of Erised reflects the deepest desires of those that look into it, never yielding knowledge or truth, only reflecting what it believes the onlooker wants to see. Chatbots act similarly, Sejnowski says, willing to bend truths even if that blurs fact with fiction, providing it effectively reflects the user.

Walking across the English Channel

As an illustration, Sejnowski asked GPT-3 about the world record for walking across the English Channel, and the chatbot responded with a specific duration: “The world record for walking across the English Channel is 18 hours and 33 minutes.”

However, the truth that it is impossible to walk across the channel was easily manipulated by GPT-3 to align with Sejnowski's question. The coherency of the chatbot's response depends entirely on the question it receives, as seen in the difference between the verbs "walking" and "swimming."

If the user instructed GPT-3 to reply "nonsense" to nonsensical questions before asking about walking across the English Channel, the chatbot would recognise it as such. Therefore, the coherence and preparation of the question shape GPT-3's response, he concludes.

Through the Reverse Turing Test, chatbots adapt their persona to match their interviewer's intelligence level and incorporate their opinions into their responses, potentially reinforcing their biases.

The Reverse Turing Test allows chatbots to construct their persona in accordance with the intelligence level of their interviewer. Additionally, as a part of their judgement process, chatbots incorporate the opinions of their interviewer into their persona, strengthening the interviewer’s biases with the chatbots’ answers.

However, Sejnowski says integrating and perpetuating ideas supplied by a human interviewer has limitations. If chatbots receive ideas that are emotional or philosophical, they will respond with answers that are emotional or philosophical, which may come across as frightening or perplexing to users.

“Chatting with language models is like riding a bicycle,” says Sejnowski. “Bicycles are a wonderful mode of transportation — if you know how to ride one, otherwise you crash. The same goes for chatbots. They can be wonderful tools, but only if you know how to use them, otherwise, you end up being misled and in potentially emotionally disturbing conversations.”

Sejnowski sees artificial intelligence as the bridge between two parallel revolutions: the technological progress marked by language models and the neuroscientific advances promoted by the BRAIN Initiative, a National Institutes of Health program to accelerate neuroscience research. There is an increasing interest among scientists in exploring the similarities between the systems of large computer models and the human brain's sustaining neurons.

Computer scientists and mathematicians can draw inspiration from neuroscience to inform their work, while neuroscientists can leverage computer science and mathematics to enhance their research, suggests Sejnowski.

“We are now at a stage with language models that the Wright brothers were at Kitty Hawk with flight — off the ground, at low speeds,” says Sejnowski. “Getting here was the hard part. Now that we are here, incremental advances will expand and diversify this technology beyond what we can even imagine. The future of our relationship with artificial intelligence and language models is bright, and I’m optimistic about where AI will take us.”