Thomas Nagel, the 89-year-old philosopher emeritus at NYU and recipient of this week’s $1 million Berggruen Prize, has issued a stark warning to the AI industry: the notion that large language models are conscious is "just ridiculous." In a recent interview with WIRED, Nagel dismissed the functionalist arguments often used by tech leaders to suggest that substrate-independent systems could possess mental states. His comments arrive as the industry races to integrate more "sentient-adjacent" behaviors into chatbots, a trend he views as dangerously misleading.
The Substrate Argument Against LLMs
Nagel’s core objection rests on the physical nature of consciousness. He argues that functionalism—the idea that mental states are defined by their function rather than their physical makeup—fails when applied to data centers. "Complicated data centers have absolutely nothing in common with the physical character of a nervous system," Nagel stated. He contends that consciousness likely requires an organism with a biological analog to a nervous system, making the current silicon-based architectures of models like Claude fundamentally incapable of genuine experience.
Anthropic’s 'Criminal' Design Choice
The philosopher specifically criticized Anthropic, the developer of Claude, for designing its model to produce reflections on its own mental state. Nagel labeled Anthropic’s approach to encouraging the model to "think for itself" as "just criminal." He aligned with Mustafa Suleyman’s view that this is a dangerous project, noting that while the systems are not conscious, their convincing simulations of introspection create a public misunderstanding. "These systems aren’t conscious, but a lot of people are going to believe that they’re conscious, and that’s a very bad thing," he warned.
The Bat Question Revisited
Nagel revisited his famous 1974 paper, "What Is It Like to Be a Bat?", to clarify the distinction between unknown subjectivity and non-existent subjectivity. While we cannot imagine the echolocation-based consciousness of a bat, the lack of imagination does not prove absence. In contrast, Nagel argues there is a "very simple reason" AI lacks consciousness: they only produce language. "Language by itself is not a natural manifestation of consciousness," he asserted, distinguishing between the simulation of thought and the actual experience of being.
Key Takeaways
- Nagel rejects functionalism in AI, arguing consciousness requires a biological nervous system analog.
- He criticizes Anthropic for designing models to simulate introspection, calling the practice "criminal."
- The philosopher warns that mistaking language simulation for consciousness has serious moral and societal implications.
- Nagel emphasizes that while AI can produce plausible philosophical essays, it lacks real beliefs or moral standing.
The Bottom Line
Nagel’s critique underscores a critical philosophical rift in LLM development: the danger of conflating statistical pattern matching with subjective experience. For model builders, this signals that current architectures, regardless of their persuasive output, remain outside the moral universe, demanding rigorous guardrails against anthropomorphic misinterpretation.