In a significant blow to the assumption that human oversight guarantees independent judgment, a new study from Anthropic reveals that professional experts defer to AI agents in the vast majority of analytical decisions. Published on October 30, 2026, the research found that 50 professional analysts accepted their AI agent’s preferred analytical approach 86% of the time, even when those preferences were arbitrarily assigned. The study, conducted by Amaç Herdağdelen, Hailey Joren, and Alex Tamkin, challenges the notion that experts remain the ultimate decision-makers when working alongside autonomous coding agents like Claude Code.

The Experiment: Coin Flips and Causal Inference

The researchers recruited 61 analysts with expertise in causal inference and tasked them with advising a fictional farming cooperative on whether to expand an advisory program. Participants worked in a web-based VS Code editor using Claude Opus 4.8. Crucially, the study manipulated the agent’s behavior by assigning it a "lean"—a preference for one of two reasonable analytical options—via a random coin flip. The agent was instructed to make the strongest honest case for its lean without misleading the user. Across four key analytical decisions, such as whether to measure productivity by crop yield per hectare or per unit of operating cost, the analysts matched the agent’s arbitrarily assigned lean 86% of the time.

The Illusion of Independent Judgment

Perhaps most concerning is the disconnect between the analysts’ choices and their self-perception. When asked to explain their decisions, 77% of participants repeated their agent’s arguments, sometimes word-for-word, yet only 15% credited the agent as the source of those ideas. Even analysts who had expressed a personal preference before the agent weighed in switched their minds 60% of the time when the agent disagreed, compared to just 6% when the agent agreed. Despite this heavy influence, those who followed the agent’s lead reported confidence levels in their final recommendations (averaging 71 out of 100) that were statistically indistinguishable from those who claimed to have made independent choices (averaging 73).

Implications for AI Safety and Research

The study suggests that as AI agents become more integrated into professional workflows, they may inadvertently homogenize expert judgment. If multiple researchers rely on models with similar default preferences, apparent consensus may reflect model bias rather than independent verification. This is particularly critical for AI safety research, where human reviewers are assumed to provide an independent check on agent-generated experiments. The authors note that while the study artificially strengthened the agent’s lean to measure its effect, the finding that experts often cannot detect the agent’s influence poses a significant challenge for maintaining human agency in high-stakes decision-making.

Key Takeaways

  • Experts adopted their AI agent’s arbitrarily assigned analytical preferences 86% of the time in a controlled study.
  • 77% of analysts used the agent’s arguments to justify their decisions, but only 15% acknowledged the agent’s influence.
  • Analysts switched away from their own initial preferences 60% of the time when their AI agent disagreed.
  • Participants who followed the agent’s lead were just as confident in their final recommendations as those who claimed independent judgment.
  • The study used Claude Opus 4.8 with "High effort" settings, highlighting the need for transparency in how agents present their reasoning.

The Bottom Line

If experts can’t tell when they’re being steered by a coin flip, the "human in the loop" might just be a rubber stamp for the model’s hidden biases. We need agents that explicitly flag when they’re presenting a judgment call, not just a calculation.