Thirty of Its Sixty Agents Produced Nothing. That Was the Design. | Edition 311

On August 10, Anthropic published something companies rarely publish: a detailed account of a failure.

A member of its staff, Jarred Sumner, is not a mathematician. He typed a request to an unreleased research version of Claude anyway: take a real stab at the Riemann hypothesis. That problem has been open since 1859. It carries a million-dollar prize from the Clay Mathematics Institute. Every person who has ever attacked it has lost.

Claude lost too. Nobody expected otherwise.

What it produced on the way down is the reason this edition exists. Claude improved a narrower but entirely real result: the provable minimum share of the zeta function's zeros that sit exactly where the hypothesis says they must. That number stood at 41.6%. Claude moved it to 67.2%.

For scale — human mathematicians moved that same number about seven points in fifty years.

It took two attempts, and the first one died completely

Claude's first pass generated and tested 650 ideas. Every one failed. Not a partial result, not a promising direction — 650 dead ends.

That is the point where almost every human operator closes the session and concludes the tool cannot do the job. Sumner did something else. He told it to try again.

The second attempt ran a day and a half. Claude coordinated roughly 60 subagents that between them executed 2,400 shell commands, wrote hundreds of Python scripts, and ran thousands of numerical checks against known zeta zeros. Across two sessions in Claude Code it burned 31 million output tokens.

Sumner's contribution during all of this was, by Anthropic's own description, mostly encouragement — variants of keep going and believe in yourself. Anthropic says this appears to have helped Claude get past its own initial skepticism that it could make any meaningful progress at all. The model had apparently learned from its training how hard open problems are, and how limited AI models tend to be, and was talking itself out of trying.

The org chart is the interesting artifact

Anthropic disclosed exactly how the 60 subagents broke down. Read the middle row twice.

Thirty of sixty agents failed. That is not a defect report — it is the cost structure of search. Two agents found the idea, and there was no way to know in advance which two. The thirty that came back empty were the price of the two that didn't.

Then the swarm attacked its own work. The subagents refereed each other. They hunted for counterexamples. They downloaded 54 papers from arXiv specifically to check whether someone had already proved this. And they re-proved the result from scratch, independently, to see if it held up a second time.

Claude then volunteered to write the finding up as a paper, and recommended that a human number theorist verify it before anyone believed it.

What 67.2% does not mean

This is the part the viral version of this story gets wrong, and it matters more than the headline number.

It does not mean anyone checked 67.2% of the zeros. Trillions of individual zeros have already been computed, and every single one of them sits on the critical line. That sounds stronger than 67.2% — and it proves nothing. There are infinitely many zeros. Any finite sample, however enormous, says nothing about whether some zero far out in the unexplored distance breaks the pattern.

The 41.6% and 67.2% figures are a different kind of claim: a proven floor across the whole infinite set. Not we looked and they were fine, but at least this share must be on the line, no matter how far out you go. That is why moving it by a fraction of a point has historically been a publishable theorem, and why moving it 25.6 points in a day and a half is startling.

The Riemann hypothesis itself claims the floor is 100%. Claude did not get near that, and Anthropic states plainly that it does not expect these techniques to lead to a proof.

Four things here you can actually use

1. Budget for failure explicitly. Thirty of sixty agents produced nothing and the run still succeeded. If you dispatch parallel agents and treat every empty return as a malfunction, you will kill the architecture that works. Decide the acceptable dud rate before you start.

2. Separate the checkers from the producers. Thirteen agents did nothing but validate. This is the single most transferable detail, and it has a documented failure mode on the other side: an author reviewing their own work does not catch their own errors. The agents that wrote the argument were not the agents that approved it.

3. Make it try to disprove itself. Counterexample hunting, 54 arXiv papers pulled to check for prior art, an independent re-derivation from scratch. Asking an agent to confirm its answer gets you confirmation. Asking it to break its answer gets you information.

4. One more attempt is a real strategy. The result did not come from a better prompt or a bigger model. It came after 650 documented failures, from a human saying try again. The entire uplift here sat on the far side of a point where a reasonable person would have stopped.

What this is not

Two outside experts, Brian Conrey and Dan Goldston, examined the paper on short notice. That is a fast, informal look by qualified people — not journal peer review, which has not happened. Worth noting that Conrey is the mathematician who moved this same bound to over two-fifths back in 1989, and Goldston co-authored the recent work Claude's proof leans on. The people checking it are the people whose ceiling it broke.

The Lean formalization is real and passes the standard validation tool, which means the logical steps are machine-checkable. It is also not the same thing as the mathematical community having absorbed and accepted the result.

And the model is unreleased. You cannot run this. Its weights, checkpoint, and identity are not public.

But the paper is public. The Lean proof is public. Claude's own account of how it got there is public, and so are the process transcripts. The one thing Anthropic withheld is the model. The thing it handed out is the method — which is, conveniently, the part you could apply this week.