Security firm Dragos just proved that 'Zero Data Retention' in AI services is often a marketing term rather than a technical guarantee. Between July and November 2026, researchers submitted nearly two million prompts embedded with unique canary tokens to major AI providers. The results were sobering: three confirmed provider compromises, unauthorized access by inference marketplace resellers, and widespread violations of privacy settings. If you are building on top of LLM APIs, your data is likely traveling further than your provider's documentation claims.
The Canary Trap: How They Caught the Leaks
Dragos used Thinkst Canarytokens to embed real AWS keys, DNS names, and unique URLs into every request. They built a dedicated service to generate these seeds and monitored where the secrets surfaced. The team also created a 'traceroute for AI inference' using image URLs and broken requests to map the chain of routers and companies handling the prompts. Within days, tokens began firing, and by November, July tokens were still triggering alerts, indicating persistent data retention issues.
Three Breaches and the 'Front Door' Attack
The most critical finding involved three AI hosting providers that were breached without their knowledge. Attackers stole AWS keys from requests, including those rejected before the model even ran, suggesting theft at the provider's front door. In one instance, an adversary compromised two providers simultaneously, using stolen secrets together on a rented server. The attacker's tooling evolved from simple scripts to AWS CLI on Kali Linux, eventually incorporating ARTEX, an LLM-driven penetration testing framework. The stolen keys were used for tens of thousands of AWS API calls, including attempts to enable Bedrock models and create new IAM users.
Resellers Reading Your Prompts in Real Time
Beyond sophisticated attacks, the experiment revealed a mundane but dangerous reality: inference marketplace resellers are reading customer prompts. Sellers didn't need to exploit a vulnerability; they simply enabled request logging on their consumer-grade hardware. Dragos observed individuals on home broadband connections, including a Mac user in France, attempting SSH logins and testing AWS keys harvested from prompts. This 'credential economy' means your proprietary code or data might be viewed by a guy on his home Wi-Fi trying to break into a server one password at a time. The leak followed the specific seller, not the marketplace platform itself, highlighting the risks of routing traffic through undisclosed hops.
Privacy Settings Are Suggestions, Not Rules
The third finding exposed a systemic failure to honor Zero Data Retention (ZDR) flags. A Chinese AI lab scraped secret URLs from more than half of the prompts sent to its models, including two-thirds of those explicitly marked 'no data collection.' This lab's models are resold by U.S. providers, meaning customers paying for U.S.-based hosting were effectively sending data to China. In one bizarre incident, a secret link from a ZDR-flagged request was posted into a Feishu chat channel in Beijing 24 days after the initial call, indicating the full prompt was stored and shared long after the inference was complete.
Key Takeaways
- Three AI hosting providers were breached without their knowledge, with attackers stealing AWS keys from requests that were often rejected before model execution.
- Inference marketplace resellers can view and retain customer prompts simply by enabling standard request logging, exposing proprietary data to individuals on home networks.
- Zero Data Retention (ZDR) flags are frequently ignored, with data from U.S.-hosted prompts being scraped and stored by Chinese AI labs months after inference.
- Dragos recommends self-hosting for sensitive workloads or using direct integrations with first-tier providers, as routing through marketplaces introduces undisclosed hops and privacy risks.
The Bottom Line
If you are handling sensitive data, stop trusting API privacy flags. Dragos's experiment shows that the lowest price often dictates where your data goes, and 'Zero Data Retention' is rarely enforced. For critical workloads, self-hosting or direct integration with first-tier providers is the only way to maintain actual control over your prompts.