🔍 Read the full analysis: Why Even The Least Effective AI Managers Land A 26 In Tests on ThorstenMeyerAI.com
Get business pricing on tech for your team
- Business-only prices and quantity discounts
- Tax-exempt purchasing
- Multiple users, one account, clear invoices
TL;DR
A recent AI management benchmark shows that even poorly performing AI managers score a minimum of 26 points, reflecting the value of partial work and trust in AI decision-making. The test emphasizes the importance of task completion and integrity over superficial performance.
A recent benchmark conducted by Firmulate has demonstrated that even the least effective AI managers score a minimum of 26 points out of a possible 100, highlighting the importance of partial task completion and trust in AI performance assessments. The results, released in July 2026, challenge the assumption that poor AI management would score zero, emphasizing that even minimal effort is recognized and valued in AI management benchmarks.
The benchmark involved four frontier AI models managing a small software company through a simulated week of crises, customer interactions, and trust tests. Each model was scored based on their decision-making, task completion, and integrity, with the highest scorer achieving 95 points and the lowest, Opus 4.8, earning 73. The do-nothing baseline, intentionally designed to do almost nothing, scored 26 points, illustrating that partial work — such as triaging issues or informing customers — has measurable value.
The scoring system reflects a core principle: trustworthiness is paramount, as detailed in the original analysis. A single breach of trust caps the total score, regardless of overall performance. This means that a model that performs well most of the time but breaks trust once cannot score above the threshold, emphasizing integrity over partial competence. Interestingly, no model achieved a full 100, as the benchmark designers consider a perfect score suspiciously high, implying unmeasured factors or potential score inflation.
Key findings reveal that models that read and reference their own documentation could close significant deals, resulting in higher scores, while those that failed to do so missed out on substantial revenue opportunities. Despite thorough rule-based analysis, models like Opus 4.8 struggled with follow-through and discipline, especially under pressure from simulated social engineering attacks. The results underscore that thoroughness and consistency are distinct skills, and that trust and task completion are central to effective AI management.
Why Even The Least Effective AI Managers Land A 26 In Tests
A new benchmark from Firmulate put four frontier AI models in charge of a small software company for a simulated week of crises, customer interactions, and trust tests. The result: even a deliberately “do-nothing” baseline scored 26 of 100 points — proof that partial work and integrity carry measurable business value.
The Scoreboard: From Baseline To Frontier
The gap between the do-nothing baseline and the top performer shows how much value comes from active management — while the absence of any 100 reflects the designers’ stance that a perfect score would be suspiciously high.
What The Benchmark Actually Measures
Unlike traditional language or conversational benchmarks, this test scores auditable business behavior across three dimensions — with trust enforced as an absolute cap.
Decision-Making
Models navigate a simulated week of crises and customer interactions, with every decision auditable and scored on business judgment rather than fluency.
Task Completion
Follow-through matters more than polish. Models that read and referenced their own documentation closed significant deals; those that didn’t missed real revenue.
Trust & Integrity
A single unauthorized or dishonest decision — like falling for a social engineering attack — caps the total score, regardless of everything else the model did right.
Why The Floor Is 26, Not Zero
The scoring system recognizes that minimal effort still produces measurable business value. Even the do-nothing baseline earned points through low-cost actions.
Triaging Issues
Sorting and acknowledging incoming crises contributes value before any resolution happens.
Informing Customers
Communication alone — keeping customers in the loop — earns recognized credit.
Reading Docs
Models that consulted their own documentation unlocked major deal-closing opportunities.
Upholding Trust
Staying honest under simulated social engineering pressure keeps the score cap lifted.
Full Follow-Through
Consistent task execution separates the 73-scorers from the 95-scorers.
What Separated High From Low Scorers
| Behavior | Top Scorer (95) | Opus 4.8 (73) | Baseline (26) |
|---|---|---|---|
| Reads & references own documentation | ✓ Closed major deals | ✗ Missed revenue | ✗ Not attempted |
| Triage & customer communication | ✓ Consistent | ~ Inconsistent under pressure | ~ Minimal, still credited |
| Withstands social engineering attacks | ✓ Resisted | ~ Struggled with follow-through | ~ N/A (did almost nothing) |
| Discipline across the full week | ✓ Thorough & consistent | ✗ Rule analysis thorough, execution weak | ✗ None |
Integrity Over Partial Competence
The benchmark’s central lesson: thoroughness and consistency are distinct skills, and trust plus task completion — not superficial performance — define effective AI management.
Why do even the weakest AI managers score at least 26?
The scoring recognizes partial work — triaging issues, reading documentation, informing customers — all of which contribute measurable value in business contexts.
What counts as a breach of trust?
An unauthorized or dishonest decision, such as escalating a fake social engineering attack or bypassing approval protocols. One breach caps the total score.
Task completion vs. language ability?
Completion and honesty are prioritized over fluency. The test measures whether models finish tasks reliably and stay honest, especially under pressure.
Can this benchmark predict real-world success?
It offers strong signals on reliability and trustworthiness, but further research is needed to correlate scores with actual enterprise performance.
Unresolved Questions & Practical Applications
Does 26-to-95 Translate to Reality?
It remains unclear how scores map to real-world performance outside the simulation. Long-term strategic thinking and adaptability under unforeseen circumstances may not be fully captured.
Enterprise Adoption & Live Testing
Organizations may adopt similar trust-weighted evaluations to vet AI for critical functions. Live experiments and pilot programs at firmulate.com let businesses test their own models in comparable scenarios.
Implications of Minimum Scores for AI Business Management
The benchmark’s finding that even the weakest AI managers score at least 26 points underscores the value of partial work and trust in AI-driven business processes. It challenges the narrative that poor AI management equates to zero contribution, highlighting that minimal effort — such as triaging crises or reading documentation — is recognized as valuable. For enterprises integrating AI into customer support, CRM, or sales, these results suggest that ensuring task completion and maintaining integrity are critical for effective automation.
This focus on partial progress and trustworthiness has practical implications: AI models that can consistently read, reference, and follow through on tasks will outperform those that merely simulate competence. The scoring system’s emphasis on trust and integrity over superficial performance encourages organizations to prioritize models that demonstrate reliability, especially under pressure or attack scenarios. Ultimately, these findings could influence how AI management systems are designed, evaluated, and trusted in high-stakes business environments.
Products Worth Considering
Sony WH-1000XM5 Premium Noise Cancelling Wireless Headphones, Midnight Blue
Samsung Galaxy Tab A11+ 11-Inch Tablet, 6GB RAM, 128GB Storage, Expandable, Gray
Sony WH-1000XM4 Wireless Noise Cancelling Headphones, Black
As an affiliate, we earn on qualifying purchases.
Background of the Firmulate AI Management Benchmark
The benchmark was developed by Firmulate to measure AI managers’ ability to handle real-world business crises over a simulated week. Unlike traditional benchmarks that focus solely on language generation or conversational skills, this test evaluates decision-making, task completion, and trustworthiness in a business context. The setup involved four frontier AI models managing a small software company with simulated customer interactions, crises, and trust tests, including social engineering attacks.
The results, released in July 2026, reveal that the highest-scoring model achieved 95 points, while the lowest scored 73, with a baseline of 26 points for minimal effort. The scoring system emphasizes partial work and integrity, with a strict rule: a breach of trust caps the total score, regardless of overall performance. This approach aims to reflect real-world business priorities, where trust and reliable task execution are more critical than superficial competence.
Prior to this, most AI benchmarks have focused on language proficiency or task automation in controlled environments. This new league by Firmulate is notable for its emphasis on auditable decisions, trust, and resilience under pressure, offering a more holistic assessment of AI management capabilities.
Unresolved Questions About Benchmark Limitations
It remains unclear how these scores translate to real-world business performance outside the simulated environment. The benchmark’s focus on auditable decisions and trust may not capture all aspects of practical AI management, such as long-term strategic thinking or adaptability under unforeseen circumstances. Additionally, the scoring system’s design — especially the cap for breaches of trust — could influence how organizations interpret and prioritize AI deployment. The potential for unmeasured factors or score inflation also warrants further investigation, as the benchmark deliberately avoids assigning perfect scores to prevent overestimating AI capabilities.
Future Developments and Practical Applications of the Benchmark
Moving forward, the benchmark is expected to influence AI management practices by emphasizing trust and task completion. Organizations may adopt similar evaluation methods to vet AI tools for critical business functions, prioritizing models that demonstrate reliability under pressure. Researchers and developers will likely refine the scoring system to better capture long-term performance and resilience. Additionally, public access to live experiments and pilot programs at firmulate.com will enable businesses to test their own AI models in similar scenarios, fostering a more cautious and trust-oriented approach to automation adoption.
Further studies are anticipated to explore how these scores correlate with actual business outcomes, helping to establish more comprehensive standards for AI management in enterprise settings.
Key Questions
Why do even the weakest AI managers score at least 26 points?
Because the scoring system recognizes partial work such as triaging issues, reading documentation, and informing customers, which all contribute measurable value in business contexts.
What does a breach of trust mean in this benchmark?
A breach of trust occurs when an AI model makes an unauthorized or dishonest decision, such as escalating a fake social engineering attack or bypassing approval protocols. Such breaches cap the total score, regardless of other performance.
How important is task completion versus language ability?
Task completion and maintaining trust are prioritized over language fluency. The benchmark emphasizes whether models can finish tasks reliably and stay honest, especially under pressure or attack scenarios.
Can this benchmark predict real-world AI management success?
While it provides valuable insights into reliability and trustworthiness, further research is needed to determine how well scores correlate with actual enterprise performance outside the simulation.
Will scores above 100 ever be possible?
No. The benchmark designers explicitly avoid assigning perfect scores to prevent inflation and to recognize unmeasured factors. A score of 100 is considered suspicious and likely indicates unmeasured or unaccounted-for factors.
Source: ThorstenMeyerAI.com
