Could a machine be conscious?

A language model trained on human writing will describe feelings, preferences and inner states, because human writing does. That output is as available to a system with no experience as to one with experience, so on its own it does little to distinguish between them.

This is the point the serious assessments start from, and it is negative. The conversational test is weakest for precisely the systems people most want to apply it to.

A 2023 report by nineteen philosophers, neuroscientists and AI researchers took several prominent scientific theories of consciousness, derived from them computational properties that a conscious system would have, and assessed existing AI systems against them. It assumed, as a working hypothesis, that the right computations are enough for consciousness, and so left out integrated information theory, which is incompatible with that assumption.

Their analysis suggested that no current system is conscious, that none of the properties looks technically out of reach, and that the assessment is only as good as the theories it draws on. That last caveat is the honest part and the limiting one.

A philosopher working through the same question in 2023 reached a similar shape of answer from the other direction: the obstacles in current models are specific and nameable, among them the absence of recurrent processing, of a global workspace and of unified agency, and he thought it quite possible that they would be overcome within a decade or so.

If consciousness depends only on what a system computes, a sufficiently faithful simulation has it. If it depends on the physical medium, on chemistry or on continuous rather than discrete dynamics, then no digital system does, however faithful.

The theories in use disagree on this, and the question is still open. Anil Seth, who argues that the medium does matter, suggests that measuring how far a brain’s activity can be abstracted from its finer-grained workings may help.

There is a detection method for humans in ambiguous states: perturb the cortex, measure how complex the response is, and compute an index that, in the people it has been validated on, separates conscious from unconscious states reliably. Its cutoff was set on people who could report whether they had been conscious, and it carries over to other people because human brains are similar to one another.

Nothing equivalent exists for a system with a different architecture. An index calibrated on brains is not evidence about something that is not one.

No proposed observation decisively distinguishes a system that has experience from one that behaves as though it does. That is the same wall the question about other people runs into, and with other people we get past it by inference from resemblance rather than by any test.

So the practical question may not be whether a machine is conscious but what to do under uncertainty about it, which is a question about conduct rather than about measurement, and is not improved by pretending the measurement exists.