Room 2 of 5
Attention
Read this sentence: “The bank by the river was steep.”
What kind of bank? You knew instantly. A riverbank.
But a machine doesn’t “know” anything. It can’t see a river. It’s never been outside. So how does it figure out that “bank” means dirt and rocks, not money and vaults?
It draws invisible lines between words, connecting them with weights that tell it which words to pay attention to. “Bank” is strongly connected to “river.” So “bank” means the geographic kind.
You’re about to prove this by breaking it. See those lines connecting the words? Grab the one between “bank” and “river.” Drag it down.
This visualization shows the principle of attention. Real models use dozens of attention heads simultaneously, each drawing a different map.
Watch the predictions change. “Eroded” and “muddy” vanish. “Fees” and “interest” appear. You just turned a riverbank into a financial institution, by severing a single connection.
The model doesn’t understand the word “bank.” It understands which other words “bank” is connected to, right now, in this sentence. Change the connections, change the meaning.
This is attention. This is how large language models process language.
They don’t understand. They draw a map. A brilliant map gives you brilliant answers. A bad map gives you confidently wrong ones. And the model has no idea which kind of map it drew.
What this means when you use AI
When you write a long prompt, the model draws an attention map across everything you wrote. If your key instruction is buried in the middle of five paragraphs, it may get weak connections — the model pays less attention to it.
Put your most important constraints at the beginning and end. Use explicit structure: headings, numbered lists, clear separation between background and instructions. You’re not formatting for aesthetics — you’re shaping the attention map.