Headline: How Sparse Autoencoders Reveal What Neural Networks Know
Neural networks compress more concepts than they have neurons. A single neuron might activate for the Golden Gate Bridge, Korean text, and the color red. This compression strategy, called polysemanticity, makes looking at individual neurons basically useless for understanding what a model actually knows.
Sparse autoencoders (SAEs) crack this problem open. Rather than interpreting neurons directly, SAEs decompose model activations into millions of interpretable features. Anthropic has extracted 34 million such features from Claude 3 Sonnet, and the results are striking: we can now read concepts out of neural networks the way we might read words off a page.
The architecture is deceptively simple
Take a vector of neural network activations (what the model is "thinking" at a given layer). Pass it through an encoder that expands it into a much higher-dimensional space; a 4,096-dimensional activation might become a 34-million-dimensional representation. Apply a ReLU activation to zero out most values, keeping only around 20 nonzero entries. Then pass through a decoder to reconstruct the original activation.
The sparsity constraint is what makes this work.
The model is trained with an L1 penalty that punishes having too many active features. This forces the network to discover a minimal, interpretable set of features rather than memorizing arbitrary patterns. Each feature corresponds to a direction in the model's internal space; activate that direction, and you activate a specific concept. Anthropic's research found features that are multilingual (a "Golden Gate Bridge" feature activates for references in English, Chinese, Japanese, and Vietnamese), multimodal (responding to both text and images), and hierarchically organized. Smaller SAEs capture broad concepts; larger ones reveal granular sub-features within them.
The practical payoff arrived quickly. Anthropic identified safety-relevant features related to deception, sycophancy, bias, and dangerous content. These aren't just patterns in data. They're causally active. Amplify the feature, and the behavior changes.
The most dramatic demonstration was Golden Gate Claude: researchers took the feature corresponding to the Golden Gate Bridge and amplified it to 10x its normal activation strength. The result was a model obsessively fixated on the bridge, inserting it into every response.
The modification was surgical, performed at the level of neural representations rather than through prompts or fine-tuning. If you can identify the internal feature corresponding to "hide my true reasoning," you can potentially detect or suppress that behavior before it manifests in outputs.
Great for finding unknowns, less so for targeting knowns
SAEs have an important limitation that recent research has clarified: they're excellent discovery tools but poor specification tools.
When you don't know what concept you're looking for, SAEs excel. They'll surface features you never knew existed, including ones you couldn't have thought to look for. But when you already know what you want to detect or steer toward? SAEs often underperform much simpler baselines. The reason relates to how SAEs are trained. They optimize for reconstruction fidelity and sparsity, not for capturing any specific concept you care about. A feature that reconstructs well might not align perfectly with the human concept you're trying to target. Simple linear probes trained directly on your concept of interest consistently outperform SAE-based detection for known targets.
This isn't a flaw. It's a category distinction. SAEs are microscopes for exploration. Want to find unknown concepts? Use an SAE. Want to act on a concept you already understand? Train a probe.
The technique has other limitations worth noting. In Anthropic's largest SAE (34 million features), roughly 65% of features never activated on the test distribution. These "dead features" might represent extremely rare concepts, or they might be artifacts of the training process. Either way, the efficiency of feature discovery degrades as SAEs scale.
Evaluation remains tricky too. Current methods for assessing whether a feature is "interpretable" are heavily subjective. Researchers look at examples that maximally activate a feature and try to describe what they have in common. Proxy metrics like L0 (sparsity) and Loss Recovered don't perfectly correlate with human judgments of interpretability. Automated interpretation pipelines have emerged that use LLMs to explain SAE features at scale, costing around $1,300 per 1.5 million features versus roughly $200,000 for human labeling. But these automated explanations have their own limits: they're derived from top-activating examples and often fail to capture how features behave across their full distribution.
We went from "individual neurons are uninterpretable" to "we can extract millions of human-readable concepts" in about three years. SAEs have revealed safety-relevant internal structures that can be surgically modified. But the field is honest about the gaps. Most features in large SAEs never fire. And we can't yet reliably explain what every feature represents.
Alternative architectures are already emerging. Transcoders, which replace the bottleneck reconstruction step with direct input-to-output transformations, may offer better interpretability for understanding how models compute. The tool is being refined even as it produces results.
Our read: SAEs are the first serious tool for reading neural networks at the concept level. They've shown us that safety-relevant features exist, can be identified, and can be modified. That's a genuine capability we didn't have five years ago. But the technique is better suited to exploration than exploitation; better for finding unknown threats than acting on known ones. Use it accordingly.