The gap between Large Language Model intuition and deterministic code is often where the money hides. Developer Kevin Meneses Gonzales recently conducted a rigorous backtest of a five-prompt stock screening system he originally built with Claude. By converting the AI's qualitative advice into hard-coded Python rules, he tested ten major tech stocks from August 7, 2026, through November 11, 2026. The experiment highlights a critical flaw in relying on AI for financial signals: the model's labels are irreproducible, but its implied risk management—specifically stop-losses—holds up under scrutiny.
The Irreproducibility of AI Labels
Meneses discovered that Claude's original classifications for Microsoft and Meta could not be replicated by code. In August, the AI labeled both as "fresh setups," citing strong earnings reactions and proximity to support. However, when the developer wrote the rules down as explicit thresholds—flagging stocks extended by 20% in three sessions or 15% above their 50-day average—Microsoft was classified as "extended" due to a 22.84% deviation from its mean. Meta, meanwhile, received a "no setup" label because it remained 6.08% below its 200-day average. Only Palantir, which had surged 28.74% in three sessions, received a consistent "extended" warning from both the AI and the code.
Performance Data Disproves Label Superiority
The backtest results, using adjusted closing prices from EODHD APIs, showed that the AI's specific labels did not predict future performance better than chance. The four stocks Claude labeled as "fresh setups" averaged a 10.50% gain at the two-month mark, while the two "extended" stocks averaged 9.50%. This statistical difference is negligible given the small sample size. More concerning is the long-term drift: while seven of the ten stocks beat the SPY benchmark at two months, four fell below their August prices by November 11. Nvidia, initially a "fresh setup," dropped 22.34% by the final checkpoint, while AMD, also a "fresh setup," gained 33.62% at two months before retracing to 14.39%.
The Stop-Loss Was the Only Reliable Signal
If the AI's labels were noise, its implicit risk management was signal. The developer implemented a stop-loss based on the lowest low of the five sessions prior to the screen date. This rule triggered on four stocks. For Amazon and Netflix, exiting on the stop-loss prevented further losses, as both stocks remained lower by November 11 than their exit price. Alphabet was slightly higher at the two-month mark but fell 17.12% by November 11, validating the exit. The only failure was Meta, which dropped 8.18% to hit the stop before rallying 21.92% by October 7. Despite this whipsaw, the stop-loss rule protected capital in three of four cases, proving more valuable than the AI's directional bias.
Key Takeaways
- AI-generated financial labels often rely on context (like earnings news) that cannot be captured by simple price-based code rules.
- Stop-losses derived from recent volatility (5-day lows) were effective in preventing larger drawdowns in 3 out of 4 triggered cases.
- The "fresh setup" vs. "extended" classification showed no statistically significant difference in two-month returns across the ten tested stocks.
- Hindsight bias affected the stock selection, as the ten famous names were chosen in November based on data through August.
The Bottom Line
Claude is a terrible quantitative analyst but a decent risk manager. The model's inability to provide reproducible, code-ready thresholds makes its stock picks dangerous for automated systems, but the underlying logic of strict stop-losses remains the only part of the prompt worth keeping.