Software engineer Jesse Waites recently deployed a custom AI research pipeline to mine the Dutch East India Company (VOC) archives, successfully identifying a forgotten 1812 meteorite fall in India, three previously unrecorded volcanic eruptions in Indonesia, and the tragic fate of three Javan rhinoceroses. The project, inspired by historian Benjamin Breen’s recent discovery of a new dodo eyewitness account, demonstrates how agentic workflows can scale historical research beyond human reading speeds. Waites processed 4.35 million pages of transcribed handwritten records, reducing a task estimated to take 70 years for a human to a single 12-hour overnight run on his home AI lab.

The Agentic Filtering Architecture

Waites’ pipeline relies on a multi-stage filtering system designed to handle the noise inherent in OCR’d historical texts. The process begins by converting 5.7 million passages into vector embeddings to enable semantic search, bypassing the inconsistent spelling of 17th-century Dutch. A lightweight 'System One' decision model called Jev acts as the primary filter, costing approximately three dollars to screen 59,000 elephant mentions. Only passages flagged by Jev are passed to Claude Haiku for detailed extraction of dates and locations, before a Claude Code agent verifies the findings against original document scans to ensure factual accuracy.

Discoveries in the Colonial Record

The most significant find is a meteorite fall near Pandharpur, Maharashtra, recorded in the Java Government Gazette on December 19, 1812. The report describes an iron-rich stone weighing four pounds that buried itself a foot deep, predating the earliest known meteorite record in the region by 26 years. Additionally, the AI surfaced three volcanic eruptions absent from the Smithsonian’s Global Volcanism Program: Gamkonora in 1722, Ciremai in 1712, and Slamet in 1780. These findings are supported by sworn crew statements and official correspondence, including a tragic account of three Javan rhinoceroses intended for the King of Kandy that died during transit. One rhino died in the 'horse stable' in 1738, while two others aboard the ship Loverendaal perished in 1740, leaving the King without the gifts he expected.

Open-Sourcing the Toolkit

Waites is releasing the workflow as an open-source toolkit named Antiquity, allowing other developers to apply similar agentic patterns to public archives. The project highlights a critical engineering constraint: AI models must be validated against known ground truths before being trusted for new discoveries. Waites tested his pipeline by ensuring it could locate already-documented events, such as the 1815 Tambora eruption, treating these knowns as 'buried wristwatches' to calibrate the detector’s sensitivity. This approach mitigates the risk of AI hallucinations in historical research, where confident but incorrect outputs can easily pass as novel insights.

Key Takeaways

  • The pipeline processed 4.35 million pages in 12 hours, a task estimated to take a human 70 years at standard reading speeds.
  • A two-stage model architecture using Jev for cheap filtering and Claude Haiku for extraction significantly reduced computational costs.
  • Validating AI searches against known historical events is essential before trusting negative results or new discoveries.
  • The open-source toolkit 'Antiquity' enables developers to replicate this agentic research workflow on other digitized archives.

The Bottom Line

This project proves that agentic workflows are not just for coding assistants but are powerful tools for scientific discovery when paired with rigorous validation. By treating known historical facts as calibration data, Waites turned a 'hallucination-prone' LLM into a reliable research instrument, setting a new standard for AI-assisted archival work.