A security researcher going by the handle janitor-security published a project called The Janitor on GitHub this week, describing it as an attempt to use AI — specifically Anthropic's Claude and OpenAI's Codex — to build an automated vulnerability scanner that catches code written by other AI systems. The headline result: the approach didn't work out as planned.

What It Was Supposed to Do

The Janitor was designed around a specific threat model: Veracode's 2025 State of Software Security report found that AI-assisted code contains 36% more high-severity vulnerabilities than human-written equivalents, and the threat has evolved beyond Copilot autocomplete into autonomous agents that can open PRs without any human authorship. The Janitor aimed to be a deterministic enforcement gate — a structural firewall that runs on your hardware, analyzes every pull request before merge, and blocks code that looks like it was generated by an AI without being reviewed by a human.

The Technical Architecture

The project uses Rust as its foundation with tree-sitter for AST parsing across 23 language grammars. It implements IFDS interprocedural taint analysis, Kani-proven Boolean predicates for detection correctness, and Z3 SMT exploit witness synthesis — formal methods that the team describes in detail in a grant research brief document. The Vibe-Check Gate compresses every patch via zstd and measures compressed_len / raw_len: vibe-coded PRs generated by prompting an AI without human authoring tend to be statistically self-similar, compressing below a 0.15 ratio threshold.

The Zero-Upload Promise

In both deployment models — CLI + GitHub Action or Janitor Sentinel GitHub App — the analysis engine runs entirely on your own runner. Source code never leaves your environment. The Governor backend receives only a signed analysis result, not the source. No SAST upload to a cloud pipeline. This is explicitly positioned as an alternative to tools like Veracode, Checkmarx, and SonarQube, which require cloud connectivity and send source code to remote clusters — a hard disqualifier for IL5/IL6 air-gapped environments.

Performance Claims

The team reports auditing 33,000+ pull requests across 22 enterprise repositories including godot, nixpkgs, vscode, k8s, pytorch, kafka, rust-lang/rust, tauri, redis, next.js, home-assistant, ansible, workers-sdk, langchain, deno, rails, laravel, apple/swift, aspnetcore, okhttp, terraform, and neovim. On a standard 8 GB laptop, they claim sustained performance of 6.7 seconds per PR on the 3.5 million line Godot Engine codebase — C++, C#, GDScript, Python combined — with 58 MB peak RAM and zero engine panics reported.

Why It Failed

The source material does not elaborate extensively on what specific failure modes were encountered during development. The project page frames its architecture as a response to the fundamental problem: LLM-based code review tools cannot prove anything — they emit confidence scores against training distributions, making novel adversarial inputs well-structured but semantically dangerous invisible to heuristic systems trained on pre-AI codebases. The implication is that building a reliable detector for AI-generated vulnerabilities using AI itself creates a recursive detection problem where the signal-to-noise ratio degrades as both sides of the equation use similar underlying technology.

Pricing and Tiers

The core scanning, cleanup, dedup, bounce, dashboard, and report features are permanently free. Team Tier costs $499 per year and adds Dual-PQC Integrity Bonds (ML-DSA-65 FIPS 204 + SLH-DSA FIPS 205), CycloneDX v1.6 CBOMs, CI/CD compliance attestation, and the Janitor Sentinel GitHub App. Sovereign/Air-Gap Tier starts at $49,900 per year with offline PQC validation, Governor Control Plane, Wasm BYOR rule mounting, and Air-Gap Intel Transfer Capsules for classified network environments.

The Bottom Line

The irony of using Claude and Codex to detect vulnerabilities introduced by AI assistants is not lost on anyone paying attention — it is precisely the kind of recursive problem that makes security tooling so hard to get right. The Janitor's formal methods approach and zero-upload architecture are architecturally interesting, but if the core premise of AI-detecting-AI via AI itself is fundamentally broken, no amount of cryptographic attestation changes that equation.