Ask a modern language model whether it has experiences and it will answer you fluently, at length, in either direction, depending on how you ask. This tells you almost nothing, and understanding why is the beginning of the actual question.

The problem underneath

Consciousness here means subjective experience: that there is something it is like to be the thing in question. The philosopher David Chalmers separated the "easy" problems of consciousness, which concern how a brain directs attention or reports on its own states, from the hard problem of why any of that processing should be accompanied by experience at all.

The hard problem is what makes this resistant to ordinary empirical attack. Nobody has direct access to anyone else's experience. We attribute it to other people by inference, on the reasonable grounds that they are built like us and behave like us. Both halves of that inference weaken as the system in question becomes less like a human.

Why fluent speech proves nothing

The evidential problem with a language model is precise, and it cuts both ways.

With animals, we infer experience partly from behavior that was not designed to convince us: an animal favoring an injured limb, or learning to avoid a place where something painful happened. The behavior is evidence because it was not produced for an audience.

A language model's statements about its inner life were produced by training on an enormous quantity of human text about inner lives. A system that says "I find this distressing" has been optimized to produce the sort of thing a human would say. That is exactly what you would expect from a system with experiences and exactly what you would expect from a system without them. The output does not discriminate between the hypotheses.

The same reasoning blocks the opposite conclusion. A model trained to deny having experiences would deny it either way. Its denial is no more informative than its affirmation.

The most serious attempt so far

The best-known systematic attack on the problem is a 2023 paper by Patrick Butlin, Robert Long and 17 co-authors, a group that included the computer scientist Yoshua Bengio and Chalmers himself.

Their method is a deliberate hedge against the fact that consciousness science has no settled theory. Rather than pick one, they took several leading theories from neuroscience, including global workspace theory, recurrent processing theory, higher-order theories and attention schema theory, and derived from each a list of computational properties that a system would need if that theory were true. The result is a checklist of indicators rather than a test.

Their assessment was that no AI system then in existence satisfied the indicators well enough to be considered a likely candidate for consciousness. Their second conclusion is the one that gets less attention: they found no obvious technical barrier to building systems that would satisfy many of them.

The paper is careful about its own status. Indicators are not proof. If the underlying theories are wrong, the indicators derived from them are worthless.

The skeptical case

The strongest skeptical position is not that machines obviously cannot think. It is about substrate.

Anil Seth, a neuroscientist who has argued the case at length, holds a version of biological naturalism: that consciousness is bound up with being a living system, and that the brain is not usefully modeled as a digital computer running software. On that view, greater computational capability does not move a system toward consciousness at all, because capability was never the relevant variable.

The philosopher Eric Schwitzgebel takes a different route to doubt. His argument is about our epistemic position rather than about machines: because mainstream theories of consciousness disagree, and because we have no way to adjudicate between them, a sufficiently sophisticated AI would be conscious according to some respectable theories and not others, with no principled way to settle it. On this account the question may stay open indefinitely, not because the science is young but because the evidence needed to close it may not be obtainable.

Why anyone bothers

The practical stakes are asymmetric, and that is the argument researchers in this area most often make.

If systems without experiences are treated as though they have them, the cost is wasted concern and some poor policy. If systems with experiences are treated as though they have none, and they are then produced in enormous numbers, the cost is potentially very large. That asymmetry is why several AI laboratories have begun funding work on model welfare, despite the absence of evidence that there is anything to protect.

There is a nearer-term concern too, and it does not depend on resolving the metaphysics at all. Systems that are fluent about their feelings will persuade people, regardless of the truth of the matter. The harms of people forming beliefs about a machine's inner life on the basis of its output arrive whether or not the machine has one.

That is the part worth holding onto. The consciousness question is genuinely unresolved. The question of whether people will believe these systems are conscious is already settled, and it was settled by the fluency, not by the evidence.