In October 2024, Simon Willison issued a simple challenge to sixteen large language models: "Generate an SVG of a pelican riding a bicycle." He didn't know it then, but those eight words would become the most recognizable informal benchmark in the LLM space. Twenty-three months later, the experiment has produced 103 distinct SVG drawings across 66 different models, scattered across dozens of Willison's blog posts.

The Accidental Standard

What began as a casual test of multimodal capabilities evolved into a de facto standard for evaluating how well LLMs understand spatial relationships, object composition, and SVG syntax. The prompt's genius lies in its specificity: it requires models to understand two distinct objects (pelican, bicycle), their relationship (riding), and the technical output format (SVG). Early models often failed spectacularly, producing abstract blobs or misplacing the pelican relative to the bicycle.

Evolution of Model Performance

Willison's ongoing documentation reveals a clear trajectory of improvement. The first sixteen models in October 2024 produced widely varying results, with many generating syntactically invalid SVG or ignoring the "riding" relationship entirely. By 2026, the latest models consistently produce recognizable pelicans on bicycles, often with impressive attention to detail like handlebars, wheels, and proper anatomical positioning. The 66 models tested represent a comprehensive cross-section of the rapid advancement in vision-language and code-generation capabilities.

Community Adoption and Cultural Impact

The pelican-on-bicycle test transcended its origin as a personal experiment. Developers across the ecosystem began using it as a quick sanity check for new model releases, creating a shared reference point for comparing multimodal capabilities. The 103 SVGs now serve as a visual timeline of LLM progress, with each drawing representing a snapshot of the state-of-the-art at a specific moment. Willison's decision to document every attempt has created an invaluable longitudinal dataset that no formal benchmark could replicate.

What the Inventor Says

In his recent retrospective, Willison reflects on how this simple prompt became a cultural touchstone. He notes that the test's durability comes from its perfect balance of simplicity and complexityβ€”it's easy to understand but hard to execute well. The prompt requires models to combine visual understanding, spatial reasoning, and precise code generation, exposing weaknesses that more formal benchmarks might miss.

Key Takeaways

  • 103 SVGs generated across 66 models over 23 months (October 2024 to September 2026)
  • The test has become a community standard for quick multimodal capability assessment
  • Early models frequently failed on spatial relationships; recent models consistently succeed
  • Willison's longitudinal documentation provides unique insight into LLM evolution
  • The prompt's power lies in its combination of simplicity and technical complexity

The Bottom Line

The pelican-on-bicycle benchmark proves that sometimes the most effective evaluation tools aren't designed in a labβ€”they emerge from curiosity, consistency, and a willingness to share your work publicly. Willison accidentally created the LLM community's favorite litmus test, and it endures because it's honest: you either get the pelican on the bike or you don't.