The internetโ€™s favorite memeโ€”asking an LLM to count the 'r's in 'strawberry'โ€”isn't a sign of artificial stupidity. It is a predictable failure of architecture. A new deep-dive on DEV.to breaks down exactly why this happens, pointing the finger at three core components: tokenization, the context window, and temperature settings. If you are building with LLMs, you need to stop treating them as humans and start treating them as probabilistic token predictors.

Itโ€™s Not Reading, Itโ€™s Chunking

Language models do not read characters. They read tokens. Using OpenAIโ€™s tiktoken library with the o200k_base encoding (standard for GPT-4o), the word 'strawberry' is assigned a single ID: 101830. The model never sees 's-t-r-a-w-b-e-r-r-y'. It sees one atomic unit. Asking it to count letters is like asking you to count the strokes in a word you have only ever heard spoken. The model relies on training data patterns to 'remember' spelling, which is fragile. Newer models often get this right only because they learned to spell it out step-by-step as a workaround, not because the underlying limitation vanished.

Numbers Are Broken Too

The same logic applies to arithmetic. The tokenizer chops '1234567' into ['123', '456', '7']. Worse, '9.11' becomes ['9', '.', '11']. When the model compares '9.11' to '9.9', it is comparing the chunk '11' against '9'. This explains why models stumble on simple decimal comparisons. The fix is obvious for builders: do not ask the model to do math. Have it write code or call a calculator tool. Let the deterministic engine handle the numbers, not the probabilistic one.

Context Windows Are Volatile Memory

Your modelโ€™s 'working memory' is the context window, measured in tokens, not messages. A rough rule of thumb is 1 to 1.5 tokens per English word. When a chat exceeds this limit, the oldest messages are truncated or summarized by the application layer. This is why long conversations 'forget' the start. For API users, this is also a cost center: shorter prompts are cheaper prompts. The best practice is to start a new chat for every distinct task and explicitly repeat key instructions rather than assuming the model retains them from earlier in the session.

Temperature Controls Chaos

Finally, there is temperature. This parameter dictates the randomness of the next-token prediction. At temperature 0, the model picks the most likely token every time, yielding consistent but potentially repetitive outputs. At temperature 1.5, less likely tokens are sampled more often. The article demonstrates this with Llama 3.2 3B via Ollama: asking for a coffee shop name yielded 'Byte & Brew' twice at temp 0, but varied between 'Byte & Brew' and 'Ctrl+Alt+Brew' at temp 1.5. Use low temperature for code extraction and high temperature for brainstorming.

Key Takeaways

  • Tokenization is opaque: Words like 'strawberry' are single IDs (e.g., 101830), making character-level tasks impossible without explicit spelling strategies.
  • Context is finite: Long chats forget early instructions because the context window truncates old tokens; start new sessions for new tasks.
  • Math is a tool, not a skill: Tokenizers split numbers arbitrarily (e.g., '9.11' becomes '11' and '9'), so always offload arithmetic to code execution.
  • Temperature is a dial: Set it to 0 for deterministic code generation and higher for creative variety.

The Bottom Line

Stop treating LLMs as smart assistants. They are token prediction engines with severe blind spots. If you understand the tokenizer, you stop fighting the model and start engineering around its flaws.