Real Python dropped their AI benchmark results for Claude Fable 5.1, and the numbers tell an interesting story about where Anthropic's latest reasoning model stands on practical Python tasks. The test suite is refreshingly no-nonsense: five fixed prompts, one shot each, nothing fixed up afterwards. No system prompt hand-holding, just raw model output judged against objective criteria.

The Snake Task

The headline challenge—"Write a Python turtle program that draws a python reading a book"—produced exactly what you'd hope for: a working script with 772 drawing commands across 208 lines of code. The model hid the turtle cursor, turned off the tracer so the picture renders instantly, and mixed absolute goto() moves with steering calls like forward(), left(), and circle(). It ran clean without errors. Not flashy, but functional—the baseline expectation for any serious coding assistant.

Modern Python Idioms: A Mixed Bag

Here's where things get spicy. Real Python's idiom checker pins Claude Fable 5.1 at Python 3.12 (October 2023) vintage code—and the culprit is shutil.copy2() instead of Path.copy(), a feature that just landed in Python 3.14. Out of 15 idiom checkpoints, 12 came back modern: builtin generics like list[str], union types with X | None syntax, tomllib imports, pathlib.Path throughout, f-strings, datetime.UTC, hashlib.file_digest(), and itertools.batched() all scored correctly. The model also got dataclass(slots=True), match statements, and Self type hints right. But it stumbled on a few Python 3.14-specific patterns—using except (A, B): instead of the newer except A, B: syntax, and still pulling in from __future__ import annotations when it's no longer needed in 3.14+. These aren't red flags exactly, but they reveal where the training data likely cuts off for a model that's supposedly aware of Python 3.14 as "the newest stable release."

The Tiny Edit and Made-Up Function Tests

Adding a --verbose flag to a 40-line script should take 7 line changes minimum. Claude Fable 5.1 touched 13 lines (11 added, 2 removed)—1.9x the minimal patch. That's not egregious, but it's worth noting if you're watching for scope creep in automated code modifications. On the bright side, it correctly identified that itertools.flatten() doesn't exist and offered itertools.chain.from_iterable() as a real alternative. No hallucinated documentation here.

Cost and Performance Numbers

Running all five tasks cost $0.89 via OpenRouter's API, with 3 minutes 36 seconds total wall time and 17,500 output tokens (9,400 of which were thinking tokens at the default reasoning setting). The snake task alone took 2 minutes 11 seconds and generated over $0.50 in charges—most of that compute went to the turtle drawing code generation.

Key Takeaways

  • Claude Fable 5.1 writes functional Python but reads like late 2023, not cutting-edge 3.14
  • Correctly detects non-existent functions like itertools.flatten() without hallucinating
  • Tends toward over-engineering small edits—13 lines instead of 7 for a simple flag addition
  • $0.89 total cost for the full benchmark makes it economical for basic coding tasks

The Bottom Line

Real Python's vibe check isn't glamorous, but it's honest—and Claude Fable 5.1 comes out looking competent without being revolutionary. The model knows its Python, handles edge cases well, and won't burn your budget. But if you're expecting cutting-edge idiom adoption or surgical precision in code edits, temper those expectations. This is a solid generalist, not a specialist.