What is a transformer in AI?
The idea behind the AI that can chat, translate, and write code—and why one little word can change the whole meaning.
Picture a group reading together
Imagine everyone in a reading group has one word from a sentence. To understand “it,” you might look back at the other words for clues. In “The dog chased the ball because it was excited,” “it” probably connects to “dog.”
A transformer does something a little like that with numbers. Each word-piece can gather useful clues from other pieces around it. The technical name for this clue-gathering is attention.
See attention in action
Tap a word below, then choose which word it should connect to for a useful clue. Try changing the sentence.
Make a guess. Does “it” mean the dog or the ball here?
This playful guessing game is not showing a real AI’s attention scores. Real models use many layers of learned numbers, and their internal patterns are more complicated than one word choosing another.
From words to a next guess
A model first splits text into tokens—pieces that might be whole words, word parts, or punctuation. It turns those pieces into lists of numbers called vectors, adds information about their order, and processes the numbers through layers.
Attention helps each piece gather relevant context. Other parts of the network refine the information. In a common chat model, the system then scores possible next pieces, chooses one, and repeats. That’s how a short prompt can grow into a full answer.
Why is it such a big deal?
Transformers are good at finding relationships across lots of pieces, and much of their training can happen in parallel. The same basic idea can be adapted for text, images, audio, and code.
A transformer turns pieces into numbers, uses attention to mix in helpful context, and can use that context to make a next-token guess.
Curious about the next piece?
These questions pick up right where this one leaves off:
Inspired by Vaswani et al., Attention Is All You Need (2017). This is a simplified introduction; transformer designs vary.