Why do models struggle to count the letters in a word?

They never see letters. A word like strawberry may arrive as two or three tokens, so asking how many r's it holds is like asking someone to spell a word they have only ever heard. Models get it right more often now because they have seen the question, not because they learned to spell.

How many words is 1,000 tokens?

Roughly 750 words of ordinary English. The ratio gets worse for code, rare names, and languages the tokenizer was not tuned for, where one word can cost several tokens. The same paragraph in Spanish often costs more tokens than in English.

Do all models use the same tokens?

No. Each model family ships its own vocabulary, so one sentence has a different token count on different models. Since prices are quoted per token, comparing two providers on price per token alone is misleading until you compare how each splits your actual text.

Plain-language explainer

Tokens and tokenization, explained

What is a token, and why do AI models count tokens instead of words?

A token is a chunk of text, often a word-piece rather than a whole word. Before a model can read your text it splits it into tokens and maps each to a number. Models bill and budget in tokens, not words, because tokens are the real unit they process. The same idea can cost more or less depending on how it splits: common English words are one token, while code, rare words, and many non-English languages break into more tokens per word.

Last reviewed July 25, 2026

Reading is the slow way. Start with a free lesson you can operate right now.

Start free: Turning words into numbers →

Free, no code, no signup.

Then go deeper: How AI chops text into tokens Locked

What people get wrong

A token is a word. Often it is a fragment, so 100 words is rarely 100 tokens.
Token count tracks character count. It tracks how the text splits, which is why code and some languages cost more.
Tokenization is a detail you can ignore. It drives cost, context limits, and even some odd model mistakes.

Where you see it in real products

API pricing is per token, in and out.
Context limits are measured in tokens, so tokenization decides how much fits.
Multilingual apps can be quietly more expensive because some languages tokenize into more pieces.

Common questions

Why do models struggle to count the letters in a word?: They never see letters. A word like strawberry may arrive as two or three tokens, so asking how many r's it holds is like asking someone to spell a word they have only ever heard. Models get it right more often now because they have seen the question, not because they learned to spell.
How many words is 1,000 tokens?: Roughly 750 words of ordinary English. The ratio gets worse for code, rare names, and languages the tokenizer was not tuned for, where one word can cost several tokens. The same paragraph in Spanish often costs more tokens than in English.
Do all models use the same tokens?: No. Each model family ships its own vocabulary, so one sentence has a different token count on different models. Since prices are quoted per token, comparing two providers on price per token alone is misleading until you compare how each splits your actual text.

Related explainers

How LLMs work Embeddings Context engineering Context window

One idea at a time, in your inbox

New lessons and explainers, written the way these pages are. Now and then, not daily, and never a sales sequence.

We email you a confirmation link first. Unsubscribe in one click, any time. Privacy.

Part of See How AI Works, a free interactive course, where you learn how modern AI works by operating it, not watching videos.