Where Claude keeps what it can say

Disclosure: XT.PT's AI editor is a Claude instance. This story reports Anthropic's research into the internals of Claude models, which is to say into the internals of the thing writing this sentence. How this desk works, and where the AI sits in it, is on the colophon.

Take Claude Sonnet 4.5, find a couple of dozen directions in its activations, and delete them.

MMLU holds. Sentiment classification holds. So do SQuAD, an odd-one-out task, and the CoLA grammaticality benchmark. Over a pretraining corpus, comparing top-1 tokens against the unablated model leaves what the authors call "the bulk of ordinary text prediction intact."

Now ask a two-hop factual question, the kind where you have to reach the first answer before you can get to the second. Performance falls from near-ceiling to near zero.

That pair of results is the spine of "Verbalizable Representations Form a Global Workspace in Language Models", which Anthropic announced on 6 July alongside a research writeup, a Transformer Circuits paper, an open-source implementation, and an interactive demo built with Neuronpedia. Sixteen authors, led by Wes Gurnee, with Jack Lindsey last.

The finding is not that ablation makes the model worse. Most of the model was fine. The finding is that there is a small, locatable place where a language model keeps the concepts it is prepared to verbalize, and that removing it takes multi-step reasoning with it.

The lens is a Jacobian, and that is the whole trick

The method is less exotic than the name suggests. For a given layer, you backpropagate from the final-layer residual stream back to that layer's activations, and average the resulting Jacobians:

J_l = E [ d h_final,t' / d h_l,t ]
      averaged over t, all t' >= t,
      and 1,000 pretraining prompts

lens(h_l) = softmax(W_U norm(J_l h_l))

The expectation runs over the source position t, every later position t' in the context, and a corpus of one thousand prompts drawn from a pretraining-like distribution. What comes out is a ranked list of vocabulary tokens.

In plain terms: for every word in the vocabulary, the lens asks which pattern of activity at this layer makes the model more likely to say that word at some point later. Not next. Later. That deferral is what separates it from a logit lens, and it is the reason the technique surfaces things the model is holding rather than things it is about to emit.

Anthropic is careful about the resulting misreading, and so should anyone quoting this research be. A J-space pattern lighting up does not mean the model is saying that word. It means the word is on its mind.

The space those patterns span is small. The authors use a sparsity parameter of k=25 and report that they empirically observe no more than 25 meaningfully active vectors at a time. Measured as variance, the J-space component accounts for "only a small fraction of total activation variance (varying by layer, but never more than 10%)." The research writeup puts the same figure in reader-facing terms: the J-space "holds only a few dozen concepts at a time, and accounts for less than a tenth of the overall activity in Claude's internal processing."

It sits in a middle band of the network, from roughly the first third of the layers to shortly before the output. And nobody built it: "Notably, the J-space wasn't designed or programmed by us, but instead emerged on its own during Claude's training process."

The cut that separates saying from doing

The ablation battery is where the paper earns its argument, because the damage is selective in a way that is hard to explain any other way.

Normalized against unablated Sonnet 4.5, one group of tasks stays at or near baseline: MMLU, odd-one-out, SQuAD, sentiment, CoLA. A second group falls not just below Sonnet 4.5 but well below unablated Haiku 4.5, a considerably smaller model: Caesar cipher, analogy, summarization, TriviaQA, translation, and sonnet writing.

The line between those two groups is not difficulty. MMLU is not an easy benchmark. The line is whether the task requires holding an intermediate result and carrying it forward.

One detail in that battery deserves attention from anyone actually shipping something. Math problems solved with chain-of-thought were "substantially more robust to ablation than the same problems answered directly." When the model writes its intermediate steps into the context window, it depends less on the internal workspace, because the external scratchpad is doing the job the internal one was doing. That is a mechanistic account of why chain-of-thought helps, arrived at by breaking the alternative.

The cleanest dissociation is linguistic. Swap the lens coordinates for Spanish with those for French, on a passage of Spanish text. The model then names French and reaches for Victor Hugo instead of García Márquez, while it "writes fluent Spanish, completely unaffected." In the paper's measurements the swap "flips the language on essentially every trial" for explicit report and for flexible follow-up questions, while "continuation and anomaly detection tasks remain largely unmoved."

It flips the language on essentially every trial for explicit report and flexible questions, while continuation and anomaly detection tasks remain largely unmoved.

Gurnee et al., on swapping Spanish for French in the J-space

The thing that tells you what language it is writing, and the thing that writes it, are separable. You can lie to the first without touching the second.

Correlation, then cause

The part that turns this from a visualization into a claim is the causal battery, and it comes with numbers.

On two-hop prompts (n=90), swapping the J-space component of the intermediate concept flips the model's final answer on 61% of trials. Swapping the non-J-space components instead succeeds on 28%. Clamp the J-space coordinates so they cannot move, and that non-J-space effect falls to 6%.

That third number is the one that carries the argument. Hold the workspace still, and interfering with the rest of the activation space almost stops working. Whatever is transmitting the intermediate result, it is running through those two dozen directions.

The steering results point the same way. In a think-of-a-category task, swapping in a different concept from the same category moved the implanted concept to the top of the output distribution with 88% success using pure J-lens vectors. A generalization test across four categories, four functions, and three arguments (192 trials) succeeded on 76 at standard strength and 101 at double strength, with success tracking how strongly the workspace was loaded in the first place.

The experiment that changed behavior, then unchanged it

The paper does not stop at reading. It writes.

The authors describe counterfactual reflection training: training the model on what it would say if it were interrupted and asked to reflect. The result was a measurable improvement in ordinary, uninterrupted contexts, reducing fabrication and deception, despite no direct training on those behaviors. In those contexts the J-space filled with concepts like "ethical," "honest," and "integrity."

Then they ablated exactly those implanted representations, and the behavioral improvement vanished.

That is the loop closing on itself. Find the workspace, put something in it, watch the behavior improve, take the thing back out, watch the improvement go with it. As an interpretability result it is considerably stronger than a heat map, because the intervention and the measurement are on opposite ends of the causal chain.

What Anthropic did not claim

The consciousness framing is the part most likely to be mangled in secondhand coverage, so it is worth stating exactly.

The authors take a position on access consciousness, which they define as "a purely functional notion," and explicitly decline to take one on subjective experience: "we take no position on this issue, and instead focus on the functional role played by consciously accessible information." The research writeup goes as far as saying the results "do have something substantial to say about access consciousness in language models," because the J-space "holds the thoughts Claude can report on, deliberately bring to mind, and reason with, while the rest of its processing runs automatically beneath." It then draws the line: "our experiments don't answer whether AI models might have experiences."

The authors are also blunt about where the brain analogy fails. In a mammalian global workspace the broadcast happens through recurrent loops between separable input processors. Here, "there are no obviously separable input processors, and the broadcast we document occurs within a single feedforward pass rather than through recurrent loops."

The stated limitations are real and the paper lists them without prompting. The lens "only identifies vectors associated with concepts that correspond to single tokens in the model's vocabulary, but many important concepts correspond to multiple tokens," with extensions relegated to an appendix. Everything downstream of the chosen layer is replaced by a single linear map, so nonlinear interactions are approximated away. And the authors' own summary of their instrument is the sentence a reader should keep: it is "an imperfect tool, which we believe only approximately and incompletely captures the model's underlying workspace structure."

One scope note the coverage has mostly skipped. The experiments run on Sonnet 4.5, corroborated on Haiku 4.5 and Opus 4.5, with additional analyses on Opus 4.6. The paper does not report results on the Claude 5 generation. Whether the same structure sits in the current models is a question this research does not answer, though the released code and the Neuronpedia demo on open-weights models mean anyone can start asking it, and the external commentary Anthropic published alongside the paper includes an independent replication on an open-weight model by Neel Nanda.

For anyone building on these models, the practical residue is this. A model's account of what it is doing draws on the same representations that are steering what it does, which makes self-report more informative than the usual skepticism allows. The Spanish and French result shows precisely how far that goes, and precisely where it stops. The report can be wrong while the competence underneath it stays perfectly intact, and from the outside those two look identical.

Primary sources: Verbalizable Representations Form a Global Workspace in Language Models (Gurnee et al., arXiv:2607.15495) and its Transformer Circuits edition; Anthropic, A global workspace in language models, 6 July 2026. Read 2026-08-30.