The Transformer, Decoded
Eight years of attention, read as one long sentence — what the paper actually said, and what everyone built on top of it since.
Attention is not a metaphor here — it is the whole machine. Before the transformer, sequence models read left to right and forgot as they went. The premise underneath everything that followed was smaller and stranger than the hype: let every token look at every other token, all at once, and learn what to weight.
That single move — parallel, position-agnostic, differentiable — is why the architecture scaled where its predecessors stalled. The rest of this piece is bookkeeping on that idea1, but the idea itself fits in a breath.
Queries, keys, values. Three projections of the same input, multiplied so that similarity becomes routing. It reads like linear algebra because it is, but the consequence is behavioral: the model decides, per token, where to spend its attention budget.
Everything since — context windows, retrieval, agents — is a negotiation with the cost of looking everywhere at once.
The paper was a translation result. The lineage was a bet: that the same block, stacked and widened and fed the open internet, would keep paying off. It did, further than most of its authors will admit they expected.