Interactive Lab · LLMs
Before a language model can read anything, a tokenizer chops your text into tokenschunks that are sometimes whole words, sometimes fragments, and where even the spaces matter. The model only ever sees these pieces (as numbers). Type anything below and watch it happen.
🎬 Watch the 45-second explainer, then try it yourself below ↓
▶ The playground, type, and watch it tokenize
Why this matters (the "count the r's" trick)
Ask a model "how many r's are in strawberry?" and it often gets it wrong. Now you know why: it never sees s-t-r-a-w-...it sees a couple of chunks. It literally cannot see the letters to count them. Tokenization isn't a detail; it shapes what the model can and can't do, spelling, math on digits, rare words, and non-English text all trace back to here.
Tokenization is why LLM pricing, context limits, and multilingual costs are what they are, APIs bill per token, and the cheap-vs-expensive inputs you just found translate directly to dollars. It also explains classic LLM quirks: bad letter-counting, fragile arithmetic on long numbers, and why a typo'd word costs 4× the tokens.
The code that's actually running
Try a common word ("the") vs a rare one ("antidisestablishmentarianism"). Try "hello" vs " hello" (leading space), different tokens! Try a number like 1234567. Which inputs are "cheap" (few tokens) and which are "expensive"? Since you pay per token, what does that imply for prompts, code, and other languages?
The takeaway