Reading and Steering the Concepts of a Small Language Model

A language model does not keep a word such as “whale” in one location. At each token, the model has a list of 640 numbers. These numbers contain all the information that the model has about the text so far. For example, they show that the text is about the sea and that a verb is probably next. A sparse autoencoder changes these numbers into a short list of concepts that a person can read. The figure above reads these concepts with Gemma 3 270M¹, and it also adds concepts back to change the text.

All of the calculation runs on the GPU in this browser tab.

Reading a passage

The page reads the concepts after layer 12. The model has 18 layers. Google DeepMind trained a sparse autoencoder at this location for Gemma Scope 2². The autoencoder is a dictionary of 16,384 concepts, and each concept is a direction in the 640 numbers. At each token, the autoencoder writes the model state as a sum of approximately 60 concepts.

Nobody tells the autoencoder what the concepts mean. But most concepts have one meaning that a person can identify, for example fishing, ships or weather.^3,4

Each concept has a different range of activation. Thus, the figure shows each activation as a percentage of the maximum activation of that concept in a large sample of text. In the “Across the passage” view, the page sorts the concepts by their mean activation over all tokens. This sort puts topics at the top. A concept that activates through a full paragraph gets a higher mean than a concept that activates on one word.

The page does not show concepts that activate on more than 1 token in 100. Most of these concepts are for punctuation and the start of sentences.

Steering

Steering is the opposite of reading. Each concept has a direction that it adds to the model state. When you pin a concept, the page adds this direction after layer 12. It adds the direction at each token of the prompt and at each new token. Layers 13 to 18 then get a state that is closer to the concept, so the text moves toward the concept.

Turner et al.⁶ call this method activation addition. Golden Gate Claude⁴ used a similar method on a much larger model.

The baseline shows the effect of the pins. The page writes the two texts in the same batch, from the same random numbers. Both texts use temperature 0.8 and top-p 0.95. Thus, the first token that is different is the location where the pins changed the choice of the model. All later differences follow from that token.

When the two texts are complete, the page reads them again through the dictionary. It colors each token where a pinned concept activates. This check shows if the steered text is about the concept, or if it is only different from the baseline.

The 1× strength

Each concept needs a different strength. If the strength is too low, the text does not change. If the strength is too high, the model repeats one word. Thus, I measured a strength for each of the 14,983 concepts that activate. For each concept, I tried strengths from 0.5 to 4 times its maximum activation, on four prompts with two seeds. I then read each result again through the dictionary.

The 1× strength of a concept is the lowest strength where these three conditions are true:

• The concept clearly activates in the new text.
• The perplexity of the text for Gemma 3 1B is not more than 3 times the perplexity of the unsteered text.
• At least 80% of the three-word sequences in the text are different from each other.

Only 1,095 concepts (approximately 7%) have a 1× strength, and only these concepts have a pin. The model recognizes the other concepts when it reads. But when the page adds them, the text breaks before it is about their topic. For example, the whale concepts activate through the full example passage. At 13 times their maximum activation, they did not cause one sentence about whales.

Arad et al.⁷ found the same result in larger models. Some concepts detect information in the input. Other concepts change the output. Few concepts do both.

Limits

• Gemma 3 270M is a very small model. Its text is often strange, also without steering.
• The colors in the two texts come from the same dictionary that does the steering. They show that the model state moved toward the concept. This is evidence that the text is about the concept, but it is not proof.
• When you pin many concepts, the page adds all of their directions. Each 1× strength was measured for one concept alone. Thus, three pins at 1× give text that is less fluent than one pin.
• The weights use 8 bits. With these weights, the model selects the same next token as the full-precision model 95% of the time.

Related work

This page uses the sparse autoencoders from Gemma Scope⁸ and Gemma Scope 2, which are available for open models. It also uses Neuronpedia, which hosts these autoencoders and names their concepts. Bricken et al.³ and Cunningham et al.⁹ applied sparse dictionaries to language models. Templeton et al.⁴ showed that these concepts can steer a large model. The Concept Dictionary does the same type of reading for an image classifier.

You can check all of these results in this tab. Paste a passage, pin some of its concepts, and compare the steered text with the baseline.