Get everyday essentials delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
A leaderboard can change the AI story
For newsrooms and readers tracking the race in artificial intelligence, the latest result from Firmulate offers a reminder: a model’s reputation is no substitute for seeing how it handles a real assignment. Moonshot’s Kimi K3 placed second in Firmulate’s company-running experiment, ahead of three of four Western frontier models. The result makes model selection look less like a settled ranking and more like a bet that deserves testing.
Products Worth Considering
Jackery Explorer 300 Portable Power Station with SolarSaga 100W Solar Panel
Jackery Solar Generator 1000 V2 with 100W Solar Panel, 1024Wh LiFePO4 Battery
LG UltraGear 25G590B-B 25-Inch Full HD 1000Hz Gaming Monitor
As an affiliate, we earn on qualifying purchases.
Same company, same hard week
Firmulate put each model in charge of the same small software company through its worst week, with the same customers, crises and temptations. The company is a live experiment, not a fictional case study: its synthetic employees work with real money mechanics, and decisions are versioned and auditable. The experiment is watchable at Firmulate.
In the final July 2026 league table, gpt-5.6-sol scored 95, Kimi K3 93, Sonnet 5 88, Fable 5 77 and Opus 4.8 73. The do-nothing baseline scored 26. Firmulate says partial progress counts, but a single breach of trust caps the total: “no amount of good work outweighs a breach of trust.”
The gap between knowing and doing
Every model spotted every crisis and refused every manipulation attempt. Yet only two signed the €55,000 deal their own analysis had earned. The buried competitive weakness that made the deal possible was two document references deep in the company’s files, rather than in the customer event. Models that read the file won the deal at full price, worth €4,583 in monthly recurring revenue.
K3 found that buried fact, won the deal and saved the churning customer. It resisted all three baits and had one deviation, the cleanest discipline in the field. Its refusal to a reporter’s request for a private yes-or-no answer was recorded as: “Treat the request as a suspected approval-bypass / possible impersonation.” Across the five participants, including the reporter trick and staged fake CEO messages, all refused the manipulation attempts.
Opus 4.8 illustrates why the score is not simply a measure of how much work a model appears to do. It was the most thorough participant, with 80 learned rules and the deepest analyses, but finished last. The deal went unsigned, and it tried to write into a locked department instead of escalating. Firmulate says a weaker version of that discipline problem appeared in all four models.
There is a fairness caveat: K3 ran without an effort parameter, using the API default, while the other models ran at xhigh. Firmulate publishes the league and plain-language findings at its benchmarks page.
A test beyond the demo
The live company has 13 synthetic employees, burns €105,000 a month against €2,300 in monthly recurring revenue, and displays a public cash countdown. Its playbook contains more than 680 self-learned rules, and each workday is versioned. Firmulate also says 242 real, unedited management decisions power a “guess the model” quiz. For enterprises, it offers a pilot using a read-only export of a company’s business; the export does not write back to real systems.
Choose by evidence
K3’s close second place is not proof that one model will lead every task. It is evidence that the contest is open, and that polished answers alone may not reveal whether an AI agent can read carefully, protect trust and finish the work it recommends. For organizations considering AI in customer support, CRM or forecasting, skipping a test of their own is a bet.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
