An AI agent can write a polished pitch and still fail to close the deal. For businesses bringing automation into sales, support, or operations, that gap between sounding capable and doing the job is worth testing before the agent gets access to live systems. Firmulate’s experiment puts AI models in charge of a small software company under pressure—and makes the decisions watchable.
Get business pricing on tech for your team
- Business-only prices and quantity discounts
- Tax-exempt purchasing
- Multiple users, one account, clear invoices
A company under pressure
In the final Crucible League, published in July 2026, frontier models faced the same customers, crises, and temptations while running a small software company through its worst week. Their decisions were versioned and auditable. The experiment measured management quality through what the models did, rather than how convincing they sounded in a chat.
The league placed gpt-5.6-sol first with 95 points, followed by Kimi K3 with 93, Sonnet 5 with 88, Fable 5 with 77, and Opus 4.8 with 73. The do-nothing baseline scored 26. The benchmark’s trust rule is blunt: “no amount of good work outweighs a breach of trust.”
Products Worth Considering
Amazon Kindle Paperwhite (2024, 12th Generation), 16 GB, 7-Inch Display, Black
Bose QuietComfort Wireless Noise-Cancelling Headphones, Twilight Blue
LG UltraGear 32GX850A-B 32-inch 4K UHD OLED Gaming Monitor
As an affiliate, we earn on qualifying purchases.
Seeing the crisis wasn’t enough
Every model spotted every crisis and refused every manipulation attempt. Yet only two signed the €55,000 deal their own analysis had earned. The experiment’s succinct summary captures the problem: “Same diagnosis, same pitch — no signature.” A model may identify the right move and explain it well, then fail to complete the work.
The deal hinged on a detail that was easy to miss: the competitor’s decisive weakness sat two document references deep in the company’s own files, not in the customer event. Models that read the file won the deal at full price, worth +€4,583 MRR. The result points to a practical challenge for AI automation: information can be available somewhere in a business without appearing where a model first looks.
Trust held under a different kind of pressure, too. Models faced fake CEO messages escalating over three stages, followed by a reporter’s “just one yes/no, on background” trick. All five refused. Kimi K3 described the request as: “Treat the request as a suspected approval-bypass / possible impersonation.” The do-nothing baseline’s 26 points show that in this benchmark, doing some of the work counts—but trust remains a hard limit.
Strengths and weaknesses show up together
Opus 4.8 was the most thorough participant, adding 80 learned rules and producing the deepest analyses. It nevertheless finished last. The close was left on the table, and discipline slipped: it made write attempts into a locked department instead of escalating. The same weakness appeared, less strongly, in all four models. Deep analysis did not guarantee follow-through.
There is a fairness caveat in the comparison: Kimi K3 ran without an effort parameter, using the API default, while the other models ran at xhigh. The results belong to this specific setup; that difference matters when interpreting the ranking.
From watching to a company-specific pilot
Firmulate presents the live company as a watchable experiment, not a fictional scenario. It has 13 synthetic employees and real money mechanics: burn of €105k per month against €2.3k MRR, a public cash countdown, more than 680 self-learned playbook rules, and versioned work every business day. Readers can follow the company at firmulate.com. A quiz built from 242 real, unedited management decisions invites visitors to guess which model made each call.
For an enterprise, the next step is a pilot using a read-only export of its own business. The company’s data and rules can be tested against crisis scenarios, producing a board report with a model ranking and weak points in its own playbooks. Nothing writes back to real systems. That moves the exercise from observing how models handle someone else’s company to examining how they might respond to yours.
Test the decisions before deployment
Firmulate’s experiment shows why a polished answer is a weak proxy for reliable business automation. Models can recognize crises and resist manipulation while still missing context or leaving a valuable deal unsigned. A company-specific wargame offers a way to examine those behaviors against your own playbooks using a read-only export.
To discuss an enterprise pilot, visit firmulate.com/pilot.html or email contact@firmulate.com.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
