The Concept Dictionary of a Robust ResNet-50

A sparse autoencoder finds 6,132 concepts inside an image classifier. Here you can see which of them the network finds in a photo, which ones decide its answer, and what happens when you paint one in.

Author

Eli Chen

Affiliation

AI Fund

Published

October 2026

Every image classifier carries around its own vocabulary, the things it has learned to look for on the way to deciding that a photo shows a golden retriever and not a German shepherd, and I've always wanted to be able to read that vocabulary directly. The explorer below is my attempt. It takes the last block of an adversarially robust ResNet-50, splits what the network sees into 6,132 concepts with a sparse autoencoder, and lets you watch which concepts it finds in a photo, which ones push it toward its answer, and what happens when you paint one in yourself.

Loading the network and the dictionary (about 75 MB)…

Hover to see the concepts at a spot · click to pick one · drag to paint it in

The network is an adversarially robust ResNet-50; every concept is a direction in its last block's 2048 channels, found by a sparse autoencoder. Concepts are listed by how much they add to the network's score for its answer; click another class to ask why not that one.

Neurons, concepts and a dictionary

Why not just look at the neurons? The network's last block has 2,048 channels at each of the 7×7 spots it divides a photo into, and you can certainly look at what makes each channel fire, but many of them respond to several unrelated things at once, because a network that has to tell a thousand classes apart packs more ideas into those 2,048 numbers than it has numbers to spare. A sparse autoencoder undoes some of that packing. It learns a dictionary of directions in those channels and rewrites the activation at every spot as a sum of just 32 of them, so that each direction gets to mean one thing, and it's trained only to make that rewrite faithful, which means nobody ever tells it what the concepts should be.

I trained one with 16,384 directions on about 250,000 ImageNet training photos, using the TopK recipe from Gao et al.4 About half of those directions ended up firing almost never, so the dictionary here keeps the 6,132 that fire on at least one patch in 10,000, and dropping the rest changes nothing I could measure. The rewrite isn't perfect: it captures about 75% of the variance in the network's activations, and if you swap it in for the real activations, the network still gets 61.4% of held-out ImageNet photos right against 64.1% on its own, and gives the same answer as before 86% of the time.

Why a robust network

The network is a ResNet-50 that Salman et al.7 trained to resist adversarial examples, and that choice matters more than anything else here. An ordinary classifier can be pushed to any answer by changes too small to see, which means the directions it's sensitive to don't have to look like anything, and if you ask it to draw a concept you mostly get colored noise. Robust training closes that loophole, so the only way to excite a concept is to make the image actually look more like it, as Engstrom et al.8 showed for single neurons.

That's why the render next to each concept shows recognizable pineapples and dog faces, and it's also what makes painting work. When you drag across the photo, the explorer runs gradient steps that raise the selected concept under your brush, drawn through the same smooth Fourier parameterization that Lucid uses for its renders, and on a robust network the cheapest way to raise a pineapple concept is to paint something that looks like pineapple. Santurkar et al.9 built the first version of this paint tool on single neurons of a robust network, and here you get to paint with the dictionary's concepts instead.

Reading the answer

So how do we know which concepts decide the answer? Here the architecture does us a big favor, because the block the dictionary reads from is the network's last one: its output is averaged over the 49 spots and fed straight into a linear classifier. That means a concept's contribution to a class score is simply its average activation times the dot product of its direction with that class's classifier weights, with no gradients or approximations involved, and the explorer computes it exactly for every concept in the photo. Across 60 random validation photos, the concepts account for a median 94% of the top class's score above its baseline, the rest being whatever the autoencoder fails to reconstruct, and the ten strongest concepts carry about two thirds of that. Clicking another class in the top five reruns the same sum for that class, which turns out to be a lovely way to ask the network why it didn't say German shepherd.

Naming 6,132 concepts

A concept is only useful if you can tell what it is, so I gave Claude each concept's render next to the six validation photos where it fires hardest, darkened everywhere it's quiet, and asked for a short name and an honest grade: clear when five of the six photos share the named thing, mixed when about half do, and unclear when there's no common thread at all. That came out to 64% clear, 25% mixed and 12% unclear, and the unclear ones mostly respond to lighting or backgrounds, like dim scenes and blown-out patches, more than to any object. Some names repeat, such as four separate soccer ball concepts, because the autoencoder sometimes splits one idea across several directions, and you can usually see the siblings side by side under nearest concepts. Treat the names as one reading of the examples, and when in doubt trust the examples themselves, since Borowski et al.10 found that people predict what a unit responds to better from natural photos than from renders.

Limits

There's a lot this doesn't show. The dictionary only covers the last block, so it tells you what the network has concluded about each part of the photo while the steps that got it there stay hidden. The renders are what a concept would draw if nothing held it back, and they can exaggerate or mislead. Concepts that fire rarely tend to be muddled, which is why they were left out, and even among the ones kept, some are clearly two things sharing one name. And the robust network is a weaker classifier than an ordinary ResNet-50, about 64% top-1 against 76%, which is the price it pays for being readable.

Related work

This explorer leans on a lot of earlier work. Its layout comes from The Building Blocks of Interpretability1, which combined feature visualization, attribution and per-image groups of neurons for GoogLeNet, and the dictionary here plays the role of those neuron groups with one vocabulary shared by every photo. Sparse autoencoders took off for language models2,3,4, Gorton5 applied them to InceptionV1 and found curve detectors that its neurons were missing, and closest to this project, Gorton and Lewis6 trained sparse autoencoders on robust ResNet-18s to study how adversarial training changes the way features share neurons.

What I love about this is how little of it is hidden. Every number in the explorer is computed live from the network in your browser, so when it says a concept added 0.7 to the score for golden retriever, that's exactly what happened, and you can go paint that concept onto something else and watch the answer move. It feels like a real step toward networks we can simply read, and I think that's a future worth being excited about.

References

  1. Olah, Satyanarayan, Johnson, Carter, Schubert, Ye and Mordvintsev. The Building Blocks of Interpretability. Distill, 2018.
  2. Bricken et al. Towards Monosemanticity: Decomposing Language Models With Dictionary Learning. Transformer Circuits, 2023.
  3. Cunningham, Ewart, Riggs, Huben and Sharkey. Sparse Autoencoders Find Highly Interpretable Features in Language Models. ICLR 2024.
  4. Gao et al. Scaling and Evaluating Sparse Autoencoders. 2024.
  5. Gorton. The Missing Curve Detectors of InceptionV1: Applying Sparse Autoencoders to InceptionV1 Early Vision. ICML 2024 Mechanistic Interpretability Workshop.
  6. Gorton and Lewis. Adversarial Examples Are Not Bugs, They Are Superposition. 2025.
  7. Salman, Ilyas, Engstrom, Kapoor and Madry. Do Adversarially Robust ImageNet Models Transfer Better? NeurIPS 2020. The network here is their ResNet-50 trained with an L2 budget of 3.
  8. Engstrom, Ilyas, Santurkar, Tsipras, Tran and Madry. Adversarial Robustness as a Prior for Learned Representations. 2019.
  9. Santurkar, Tsipras, Tran, Ilyas, Engstrom and Madry. Image Synthesis with a Single (Robust) Classifier. NeurIPS 2019.
  10. Borowski, Zimmermann, Schepers, Geirhos, Wallis, Bethge and Brendel. Exemplary Natural Images Explain CNN Activations Better than State-of-the-Art Feature Visualization. ICLR 2021.

Photos are from the ImageNet validation set. The renders use Lucid's image parameterization and its direction objective with cosine similarity cubed.