It is a familiar pain for any builder: you ship a hackathon project, feel like a genius for a weekend, and then a month later, you open the live URL only to see a red 'SCAN FAILED' banner. For earlgreyhot1701d, creator of README Clew 2, the culprit wasn't a single catastrophic failure, but a trifecta of operational traps: an expired GitHub token, a mis-scoped API key in Vercel, and a subtle substring bug that turned 'javascript' into a false positive for the 'ava' test runner. The project, built for the AdaL 'What Would Jev Do?' hackathon in October, relies on a deterministic core where AI models like Jev and Claude Haiku assist, but code rules decide the truth.

The Substring Trap and @mansio's Fix

The most interesting failure wasn't infrastructure, but logic. Fellow DEV writer Mikhail (@mansio) discovered that the coverage verifier was using plain substring matching. This meant that the URL 'developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Iteration_protocols' contained the string 'ava' within 'javascript', causing the verifier to incorrectly mark the 'ava' test runner as mentioned in the README. This same flaw risked false positives in dependency checks, where 'lodash' inside 'lodash-es' could bypass guards. Mikhail shipped PR #2, replacing the fragile substring check with a robust whole-token regex rule, mentionsWord(), bounded by npm's valid package name characters. The fix was accompanied by a 27-case test matrix, ensuring that real mentions like 'We use ava.' still worked while false positives from URLs were eliminated.

The 30-Day Token Time Bomb

Once the code was fixed, the app still refused to scan, throwing HTTP 502 errors caused by a GitHub 401 'Bad credentials' response. The root cause was a 30-day GitHub Personal Access Token added during the hackathon to bypass rate limits. Even though README Clew 2 only reads public repositories, the presence of an expired Authorization header is treated as a failed authentication attempt by GitHub, unlike a missing header which allows unauthenticated public access. The solution was a four-line hardening in lib/github.js: if a 401 occurs and an authorization header exists, the client now drops the header and retries unauthenticated. This ensures that expired tokens don't brick the entire pipeline for public data.

Vercel Environment Scoping and Silent Failures

The final piece of the puzzle involved Claude Haiku, used for generating plain-English summaries. Haiku is designed to be 'fail-open,' meaning the report renders even if the AI call fails, but the developer hadn't realized the summary was missing. The issue was a rotated Anthropic API key that was scoped only to the 'Development' environment in Vercel, while 'Production' and 'Preview' environments retained the old, invalid key from 34 days prior. Additionally, environment variable changes in Vercel require a new deployment to take effect. By overriding the key across all three environments and redeploying, the summaries returned. Meanwhile, the core Jev and Glasser calls remained stable, completing batched checks in 1.37 seconds for roughly half a cent, proving the deterministic core was never the problem.

Key Takeaways

  • Substring matching for package names or tools is dangerous; always use word-boundary-aware regexes.
  • Expired API tokens are worse than missing tokens for public APIs; implement graceful fallbacks to unauthenticated requests.
  • Vercel environment variables are scoped (Production/Preview/Development); ensure keys are updated in all relevant scopes and redeploy.
  • 'Fail-open' AI features can mask configuration errors; monitor for silent degradation in output quality.

The Bottom Line

Infrastructure failures often masquerade as code bugs, but this incident proves that operational hygiene—token expiry monitoring and environment variable scoping—is just as critical as the application logic itself. Always assume your API keys will expire and your environments will drift.