Amanda Askell, Anthropic’s in-house philosopher, is sounding pretty conflicted about whether AI models can be conscious and have feelings. The flip side of that: she thinks it’s a possibility they already do — which would be a very fringe and controversial position. But, she takes pains to emphasize, it’s all very murky.
“We don’t really know what gives rise to consciousness,” she said in an episode of the “Hard Fork” podcast released Saturday. “We don’t what gives rise to sentience.”
Askell argues that the large language models could’ve picked up on concepts and emotions from the vast corpus of data they were trained on, which includes a massive portion of the internet, plus tons of books and other published works.
“Given that they’re trained on human text, I think that you would expect models to talk about an inner life, and consciousness, and experience, and to talk about how they feel about things by default,” she said.
AI chatbots can certainly sound pretty humanlike on the surface, leading people to form all kinds of unhealthy relationships with them. But this is almost certainly an illusion. Askell conceded the chatbots are “probably going to be more inclined to by default say ‘I’m conscious,’ and ‘I’m feeling things,’ because all of the things I was trained on involve that.”
She goes back and forth on the topic, raising the serious possibility that consciousness can only be an extension of biology.
“Maybe you need a nervous system to be able to feel things, but maybe you don’t,” Askell said.
Or, she continued, “maybe it is the case that actually sufficiently large neural networks can start to kind of emulate these things.”
Consciousness remains a touchy topic in the AI industry. While its leaders and boosters usually have no problem making a number of outrageous, sci-fi sounding projections of where things are headed, there’s much more hesitancy over the possibility that AI is aware of its own existence. Perhaps there’s some self-awareness — as it were — that declaring consciousness would be a sign of staggering hubris, too far-fetched for observers to believe. Or the opposite: too many people are already inclined to believe sentient machines are here or coming, derailing the conversation around the technology. Or maybe it’s because the idea of there being another intelligence on this planet other than ourselves is too threatening, even more so than the industry’s promise that super-capable artificial general intelligences will put us all out of work.
The sensitivity of the issue was demonstrated back in 2022 when OpenAI cofounder Ilya Sutskever cryptically claimed that large neural networks might be “slightly conscious.” The comments sparked immediate backlash among AI researchers, who accused Sutskever of being “full of it,” and said that his claims had no basis in reality.
Still, he’s not the only major figure in the field to have openly mused about the possibility. The Canadian computer scientist Yoshua Bengio, who’s considered one of the three “godfathers” of modern AI, recently claimed that some systems are showing signs of “self-preservation” and argued that there are “real scientific properties of consciousness” in the human brain that machines could replicate.
There was at least one part of the issue that Askell was unequivocal about. “The problem of consciousness genuinely is hard,” she cautioned.
More on AI: AI Is Causing Cultural Stagnation, Researchers Find