GPT-2 ATTENTION

Real GPT-2 small, running in your browser. Every one of the 144 attention heads, computed live from the actual OpenAI weights.

Query token: last token click a token to jump to the head that focuses on it most · or pick any cell in the all-heads grid below

Layer 0 · Head 0 attention

Most likely next token

Each row is a token attending back to earlier tokens; brighter cells take more of that row's attention, and every row sums to 1. The first column is the attention sink and the diagonal is self-attention.

Attention from ·

All 144 heads · layer (rows, 0–11) × head (cols, 0–11) hover to read · click any cell to inspect