Interactive Lab · Transformers
Read this: "The animal didn't cross the street because it was too tired." You instantly know it = the animal. A transformer figures that out with self-attention: every word looks at every other word and decides how much to pay attention to each. Click any word below to see exactly which words it looks at, and watch the model resolve the ambiguity when you change one word.
🎬 Watch the 45-second explainer, then turn the knobs yourself below ↓
▶ The attention probe, click any word
Click a query word. Every other word shades by how much the query attends to it. Thicker arc = stronger attention.
Click it in the sentence above, watch which word lights up brightest. That's the model deciding what "it" refers to.
Attention is the mechanism reading this very sentence inside every LLM you use, it's how “it” finds its noun, how a verb finds its subject, and how a model juggles context across thousands of tokens. Search ranking, translation, protein folding (AlphaFold), and image generation all route information the same way.
The code that's actually running
In Sentence A ("…too tired"), click itpredict first: does it attend more to animal or street? Now switch to Sentence B ("…too wide") and click it again. Only one word changedyet where "it" looks flips. Why? A street can be wide; an animal gets tired. The model routes attention using the whole sentence. Also click a verb like crossdoes it look back at its subject animal? That's how the model tracks who-did-what.
The takeaway
softmax(QKᵀ/√d)·V.