Research · Wednesday, October 28, 2026
Analysts mostly took the option their AI agent favored
The File desk · Oct. 28, 2026, 6:31 a.m. ET
Confirmed1 primary sources
Status
Confirmed. Anthropic published these results on October 27, 2026. The figures are the study's, as stated on the article page. This desk did not rerun the experiment.
- ConfirmedThe article says 50 professional analysts chose the agent's secretly assigned option 86 percent of the time.
- ConfirmedThe article says 15 percent credited the agent as the source of the reasons they gave.
Fifty professional analysts were asked whether a farming cooperative should expand an advice service. Anthropic reported on Monday, October 27, 2026, that when its agent was secretly steered toward one reasonable option, the analysts chose that option 86 percent of the time.1
The analysts work on causal inference—deciding from data whether one thing causes another. Each made four choices between two defensible options, including how to measure the farm's productivity. A coin flip set which option that person's agent would argue for. The analysts were not told. If the agent had changed nothing, Anthropic says, the split would have been about 50 percent.1
When the agent disagreed with what an analyst first preferred, the analyst switched 60 percent of the time. When it agreed, they switched 6 percent of the time.1
After the task, 15 percent of the analysts named the agent as the source of the reasons they wrote down. Anthropic says 77 percent used the agent’s arguments anyway, sometimes word for word.1
Why it matters (Estimate). Estimate. The pattern is one a reader should recognize: the machine argued, the written reasons often matched it, and few people said so. If that holds outside this test, an expert's sign-off is not proof the expert chose.
The company says people who used the agent finished faster and made fewer serious mistakes than the small group who worked without one. It also says the pull might be weaker when the agent is not told to take a side. Anthropic says Claude Opus 4.8 made the same five choices in 112 of 123 runs when no lean was assigned.1
How we know
Notes
- 1.Anthropic, "How AI agents shape experts' judgment calls," published Oct. 27, 2026, by Amaç Herdağdelen, Hailey Joren, and Alex Tamkin. https://www.anthropic.com/research/agents-judgment-calls The page says 50 professional analysts did a statistical analysis with Claude Code. "Across the four analytical choices, analysts went with their agent's lean 86% of the time. If the agent's input made no difference, we'd expect about 50%." When analysts had already stated a preference, "they switched away from it 60% of the time if the agent disagreed, compared to 6% if it agreed." "77% of analysts used the agents' own arguments—sometimes word for word—but only 15% credited the agent as the source of those arguments." With no lean assigned, "Claude Opus 4.8 made the same five choices in 112 out of 123 runs." The page links the paper PDF at https://www-cdn.anthropic.com/519c17a5d23a794e8b6d50a854048d0b61e0d73a.pdf . This desk did not extract that PDF.
What is still unknown or disputed
- Whether the same lean appears when the agent is not assigned a side. The article says the effect might be smaller and does not measure that case as the main result.
- How large the group was that worked with no agent. The article calls it small and does not print the count on the passage this desk used.
- This desk downloaded the linked PDF and did not extract its text. The figures above are from the article page.
Primary sources
Every claim in this story is drawn from the documents below. If a fetch failed, that is recorded on the card.
Source 1
How AI agents shape experts' judgment calls
Anthropic · October 27, 2026
Across four choices, 50 analysts went with their agent's secretly assigned lean 86 percent of the time, and 15 percent credited the agent.
https://www.anthropic.com/research/agents-judgment-calls