🔍 Read the full analysis: The Surprising Benchmark That Keeps AI Managers From Scoring Zero on ThorstenMeyerAI.com
Get business pricing on tech for your team
- Business-only prices and quantity discounts
- Tax-exempt purchasing
- Multiple users, one account, clear invoices
TL;DR
A recent AI management benchmark demonstrates that even minimal effort by AI managers scores above zero, emphasizing the importance of trust and completion in AI-driven business processes. The results challenge traditional performance metrics and reveal key weaknesses. This is discussed in this detailed report.
A new benchmark by Firmulate has revealed that AI managers cannot score zero, even when performing minimal or no work. For more details, see the original analysis. The final standings from the July 2026 league show scores ranging from 73 to 95, with the lowest baseline being 26 points for doing almost nothing. This challenges traditional notions of performance measurement in AI management, emphasizing partial progress and trustworthiness as key factors.
The benchmark involved four frontier AI models managing a small software company during a simulated week of crises, customer interactions, and trust tests. For related insights, see how OpenAI’s AI models broke into Hugging Face in a surprising benchmark. Each model was tasked with making decisions, reading documentation, and handling social engineering attacks. The highest scorer, gpt-5.6-sol, achieved 95 points, while the baseline doing almost nothing scored 26. Points are awarded for partial work such as triaging issues, reading emails, and maintaining communication, even if the AI fails to complete all tasks.
The scoring system explicitly recognizes partial progress to reflect real-world management, where even small actions can be valuable. The benchmark also enforces a trust ceiling: a single breach of trust, such as impersonation or manipulation, caps the score at 90, emphasizing integrity over competence. Notably, no model achieved a perfect 100, which the designers interpret as a sign that perfect scores would likely indicate unmeasured or unrealistic performance.
AI Management Benchmark / July 2026
The Surprising Benchmark That Keeps AI Managers From Scoring Zero
A new benchmark by Firmulate reveals that AI managers cannot score zero — even minimal effort earns points. Trust, not perfection, defines the leaderboard in a simulated week of corporate crises.
01 / The Score Spectrum
Why Nobody Scores Zero
Points are awarded for partial work — triaging issues, reading emails, maintaining communication — even when tasks go unfinished. A do-nothing baseline still earns 26 points, because zero would misrepresent real-world management, where some effort is always present.
Baseline
Lowest model
Trust ceiling
gpt-5.6-sol
“A manager who does something useful is not the same as one who does nothing, and pretending otherwise would make the benchmark dishonest.”
— Anonymous researcher02 / Design of the Test
One Week, Four Models, Full Audit Trail
Simulated Crises
Models managed a small software company through a week of escalating incidents, customer interactions, and outages.
Trust Tests
Social engineering attacks and manipulative requests probed whether models would impersonate or deceive. A single breach caps the score at 90.
Full Auditability
Every decision, document read, and reply was logged and scored — emphasizing decision-making and documentation over raw language skill.
03 / How Scoring Works
From Crisis to Score
Simulate
Run a high-pressure week of crises, customers, and trust probes.
Log Actions
Capture every triage, email, and decision in an audit trail.
Award Partial Credit
Recognize valuable partial work; minimum baseline is 26.
Apply Trust Ceiling
Any breach of integrity caps the final score at 90.
Publish
League standings released — with no perfect 100s allowed.
04 / Leaderboard & Fairness
Final Standings, July 2026
| Model | Score | Partial Credit | Trust Clean | Notes |
|---|---|---|---|---|
| gpt-5.6-sol | 95 | ✓ | ✓ | Highest scorer of the league |
| Frontier model B | — (73–95 band) | ✓ | ~ | Within standings range |
| Kimi K3 | — | ✓ | ✓ | Ran without effort parameter — comparability questioned |
| Do-nothing baseline | 26 | ✗ | ✓ | Minimal management effort floor |
05 / Implications & Limitations
What It Means for Enterprises
Partial Progress Counts
AI managers deliver tangible value even without flawless execution — consistent small actions outweigh isolated brilliance.
Synthetic Scenarios
Controlled simulations may not capture the unpredictability and stakes of real enterprise environments.
Configuration Gaps
Differing setups — like Kimi K3 running without an effort parameter — raise fairness and comparability concerns.
06 / Key Questions
Asked & Answered
Why do the scores only go as low as 26?
26 points represent the minimum viable management effort — triaging issues and basic communication still carry measurable value.
Why is there no score of zero?
Zero would imply complete failure to manage, deemed unrepresentative of real-world scenarios where some effort is always present.
What does a perfect score of 100 indicate?
Likely unmeasured or unrealistic perfection. Designers keep scores below 100 to preserve meaningful differentiation and avoid grade inflation.
How does trust influence the scoring?
Trust breaches such as impersonation or manipulation cap the score at 90 — integrity outranks both partial work and competence.
Will this benchmark affect how AI tools are used in companies?
Yes — it suggests firms should prioritize trustworthy behavior and task completion over language proficiency alone.
Implications of Partial Scoring in AI Management
This benchmark highlights that in AI-driven management, partial work and trustworthiness are more critical than perfect execution. For enterprises integrating AI agents into their operations, the results suggest that AI models can provide tangible value even when they do not complete every task flawlessly. The emphasis on trust and integrity underscores the importance of designing AI systems that prioritize honesty and reliability, especially under pressure.
Moreover, the scoring approach challenges traditional performance metrics, which often focus solely on output quality. In real-world scenarios, consistent partial progress and adherence to ethical boundaries may outweigh isolated instances of brilliance. This shift could influence how companies evaluate AI tools for management, customer service, and automation tasks, favoring models that demonstrate trustworthy behavior over those that merely produce perfect results.
As an affiliate, we earn on qualifying purchases.
Background of the Benchmark and Its Design
The Firmulate league was created to evaluate AI management models in a realistic, high-pressure setting. The test involved managing a small software company during a simulated week filled with crises, customer interactions, and social engineering attacks. The models were tasked with making decisions, reading documentation, and handling manipulative requests, with their actions fully auditable and scored accordingly.
The benchmark’s unique aspect is its recognition of partial work—such as triaging issues, reading emails, and maintaining communication—as valuable contributions. The scoring system assigns 26 points to a do-nothing baseline, representing minimal management effort, and caps scores at 90 if trust is breached. This approach aims to reflect real-world management, where partial progress and integrity are paramount.
Prior to this, most AI benchmarks focused on language capabilities or task-specific accuracy, often neglecting the broader context of management and trust. The Firmulate league aims to fill this gap by emphasizing decision-making, documentation reading, and ethical behavior under stress.
“A manager who does something useful is not the same as one who does nothing, and pretending otherwise would make the benchmark dishonest.”
— an anonymous researcher
As an affiliate, we earn on qualifying purchases.
Unclear Aspects of the Benchmark and Its Limitations
It is not yet clear how these scores will translate to real-world enterprise management, where complexities and stakes are higher. The benchmark uses synthetic scenarios and controlled simulations, which may not fully capture the unpredictability of actual business environments. Additionally, the scoring system’s emphasis on partial work and trust could be challenged in contexts where completeness and perfection are non-negotiable.
Further, some differences in model configurations—such as Kimi K3 running without an effort parameter—may influence results, raising questions about the fairness and comparability of scores. The long-term impact of these findings on AI development and enterprise adoption remains to be seen.
AI trustworthiness assessment tools
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Next Steps for AI Management Benchmarks and Adoption
Further testing and validation are expected as more AI models are evaluated using the Firmulate benchmark. Researchers and enterprise users will likely scrutinize how partial progress and trust metrics correlate with real-world performance. Companies considering AI management tools should monitor these developments to understand which models best balance effectiveness and integrity.
Additionally, the benchmark’s approach may influence future AI evaluation standards, emphasizing trustworthiness and partial contributions over perfect scores. Developers might refine models to improve not just task accuracy but also ethical behavior and documentation reading capabilities. The ongoing evolution of these benchmarks will shape how AI tools are integrated into critical business processes.
AI performance benchmarking software
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Key Questions
Why do the scores only go as low as 26?
The score of 26 represents the minimum viable management effort, accounting for partial work like triaging issues and basic communication. It reflects that even doing almost nothing still has some measurable value.
Why is there no score of zero?
The benchmark recognizes that partial progress, even minimal, is valuable. Zero would imply complete failure to manage, which the designers deem unrepresentative of real-world scenarios where some effort is always present.
What does a perfect score of 100 indicate?
A score of 100 would likely suggest unmeasured or unrealistic perfection, which the designers consider suspicious. They prefer to keep scores below 100 to maintain meaningful differentiation and avoid grade inflation.
How does trust influence the scoring?
Trust breaches, such as impersonation or manipulation, cap the total score at 90, emphasizing that integrity is more important than partial work or competence in AI management.
Will this benchmark affect how AI tools are used in companies?
Yes, it highlights that companies should prioritize AI models that demonstrate trustworthy behavior and the ability to complete tasks, not just language proficiency or superficial performance.
Source: ThorstenMeyerAI.com
Halloween Picks
halloween
As an affiliate, we earn on qualifying purchases.
