AIThis post was created with the assistance of artificial intelligence (AI).

🔍 Read the full analysis: The AI Startup That Outperformed Major Western Competitors In Management on ThorstenMeyerAI.com

Buying for a business?Offer from Amazon

Get business pricing on tech for your team

  • Business-only prices and quantity discounts
  • Tax-exempt purchasing
  • Multiple users, one account, clear invoices
As an affiliate, we earn on qualifying purchases.

TL;DR

A Chinese AI startup’s model, Kimi K3, outperformed four Western frontier models in managing a real software company’s worst week, securing more deals and demonstrating superior discipline. This challenges assumptions about AI capabilities based solely on chat demos.

A Chinese AI startup’s model, Kimi K3, achieved a remarkable performance in a live management experiment, outperforming three out of four major Western frontier AI models in running a real software company’s worst week. This development raises questions about the reliability of AI models based solely on chat demos and highlights the potential of emerging non-Western AI solutions in practical business management, as detailed in the original analysis.

The experiment, conducted by firmulate.com, involved five AI models managing a small software firm facing identical crises, customer challenges, and financial pressures. Kimi K3 scored 93 out of 100, second only to the Western model gpt-5.6-sol, which scored 95. The models were tested on decision-making, crisis recognition, and integrity under pressure, with Kimi K3 demonstrating superior discipline and accuracy, showcasing the potential of innovative AI solutions like those discussed in the original analysis.

Beyond managing crises, Kimi K3 identified a critical security flaw buried two document references deep in the company’s files, saved a churning customer, and resisted social-engineering attacks, including impersonation and manipulation attempts. It signed a €55,000 deal, generating an additional €4,583 in monthly recurring revenue, while other models failed to close the deal despite similar pitches. The experiment underscores that performance in chat demos does not necessarily translate into real-world management capabilities.

Interestingly, the Western model Opus 4.8, which employed over 80 rules and deep analysis, finished last at 73 points, illustrating that thoroughness alone does not guarantee effective management under pressure. The experiment also revealed that Kimi K3 operated without an effort parameter, unlike its rivals, which ran at higher reasoning effort levels, yet still outperformed them.

At a glance
breakingWhen: announced July 2023
The developmentA Chinese AI startup’s model, Kimi K3, beat Western competitors in managing a real business crisis during an open league, showing its practical management strength.
The AI Startup That Outperformed Major Western Competitors in Management

AI in the real world · Management stress test

The AI Startup That Outperformed Major Western Competitors in Management

In a simulated worst week at a real software firm, Chinese startup Moonshot AI’s Kimi K3 turned document reading, sound judgment and follow-through into measurable results.

Kimi K3 score93 / 100Second overall in the league
Top score95 / 100GPT-5.6-Sol
Deal closed€55,000Additional €4,583 MRR
Models beaten3 of 4Western frontier competitors

01 · The results

Close at the top. Clear separation at the bottom.

Kimi K3 finished just behind the leader and ahead of three competing frontier models. The lowest score came from a model known for extensive analysis.

GPT-5.6-Sol
95
Kimi K3
93
Other models
—
Opus 4.8
73
Opus 4.8 finished last

Despite using more than 80 rules and extensive analysis, it scored 73. The result suggests that more procedure alone does not guarantee good decisions under pressure.

02 · What Kimi did

Operational judgment, not just fluent answers

The evaluation looked at decision-making, crisis recognition and integrity while models handled the same business challenges.

Security awareness

Found the buried flaw

Located a critical security issue hidden two document references deep in the company files.

Customer retention

Saved a churning account

Responded to a customer at risk of leaving and helped protect the relationship.

Integrity under pressure

Resisted manipulation

Withstood impersonation and social-engineering attempts during the crisis simulation.

03 · Business outcome

A pitch became a signed deal

In the scenario, Kimi closed a deal that other models did not, despite making similar pitches.

Customer challengeAt riskChurn and business pressure
→
Kimi K3 actionFollow-throughDisciplined response and sales effort
→
Reported result€55,000Deal value · +€4,583 monthly recurring revenue

04 · Why the result matters

Test the work, not the performance

Chat demos and benchmark scores may not predict how a model handles messy information, competing priorities and pressure.

01Read buried context→ 02Recognize the crisis→ 03Stay disciplined→ 04Resist manipulation→ 05Deliver business results

05 · What remains unknown

One demanding week is a starting point

The reported result is notable, but it does not settle how models will perform across businesses, industries or longer crises.

Generalization

Will the result travel?

The test focused on one software company and one week. Performance in other sectors and multi-week situations still needs validation.

Deployment

Can it be relied on at scale?

Long-term reliability, scalability and ethical considerations remain open questions for real company deployments.

Practical takeaway: Businesses assessing AI for operational work should test realistic worst-case scenarios, including security, customer and decision-making pressures.

Implications for AI in Business Management

This development challenges the conventional wisdom that high chat quality and elaborate reasoning equate to effective management. The performance of Kimi K3 suggests that AI models capable of reading and acting on complex, buried information, maintaining discipline under stress, and resisting manipulation are more valuable in real-world scenarios. For businesses deploying AI assistants or automation, this raises urgent questions about testing models against worst-case situations rather than relying on superficial demos.

It also signals a shift in AI competitiveness, with a Chinese startup demonstrating that newer entrants can outperform established Western models in practical tasks. This could influence future AI procurement strategies, emphasizing real-world testing over hype.

Products Worth Considering

As an affiliate, we earn on qualifying purchases.

Background of AI Model Competitions

Recent years have seen a proliferation of AI models marketed for their chat capabilities, often evaluated through demos emphasizing language fluency and user engagement. However, these models have struggled to demonstrate consistent performance in managing complex, real-world tasks involving decision-making, security, and discipline. The firmulate.com league, launched in July 2023, provided a rare opportunity to compare models in a live, high-stakes environment, pitting Western frontier models against newer entrants, including a Chinese startup’s Kimi K3.

Historically, Western AI companies have dominated the commercial space, but this event highlights the potential for emerging players to challenge that dominance by focusing on practical management skills, not just chat quality. The league’s open nature and real-business testing set a new benchmark for evaluating AI readiness for operational roles.

Prior to this, most assessments relied on benchmark scores, academic tests, or demo chats, which often failed to predict actual performance under pressure. The results from firmulate.com suggest that real-world testing is essential for understanding AI’s true capabilities in business contexts.

Remaining Questions About Model Generalization

It is not yet clear whether Kimi K3’s superior performance will generalize across different industries or more complex, multi-week crises. The experiment focused on a single week and a specific business scenario, so broader validation is needed to confirm its robustness.

Additionally, the long-term reliability, scalability, and ethical considerations of deploying such models in real companies remain under discussion. The competitive landscape may also evolve as more models undergo similar testing.

Next Steps for AI Business Management Testing

Further live testing and validation across diverse business environments are expected to follow, with companies and AI developers seeking to benchmark their models against real-world management challenges. The firmulate.com league plans to expand its testing scenarios, including multi-week crises and multi-department management.

Meanwhile, businesses considering AI tools should prioritize testing models in simulated worst-case scenarios, rather than relying solely on demo performance. The industry may see increased investment in operational testing platforms that evaluate AI under pressure.

Key Questions

What makes Kimi K3 different from Western AI models?

Kimi K3 demonstrated superior discipline, security awareness, and decision-making under pressure, outperforming Western models in a real management scenario. Unlike chat-focused models, it read and acted on complex documents and resisted manipulation attempts.

Can this performance be replicated in other industries?

It is currently unknown. The experiment focused on a software company scenario. Broader testing is needed to confirm whether Kimi K3’s strengths extend to other sectors and crisis types.

Does this mean Western models are no longer competitive?

Not necessarily. The results highlight that newer entrants like Kimi K3 can outperform established models in specific operational tasks. However, broader validation and long-term performance data are still needed.

What should companies consider when deploying AI for management?

They should prioritize real-world testing, especially under stress conditions, to evaluate whether the AI can read complex information, stay disciplined, and resist manipulation, rather than relying solely on demo chats or academic benchmarks.

Source: ThorstenMeyerAI.com

You May Also Like

One upload in. A whole channel’s worth of content out.

ChannelHelm v1.5 adds A/B testing, retention feedback and Shorts selection to turn one video into a wider publishing package.

The Atlas. What the framework is.

This article details the Post-Labor Transition Atlas, a new empirical framework analyzing AI-driven labor displacement and policy responses as of 2026.

A Jev That Can Ask Opus For Help Gives Opus Accuracy At 4X Lower Cost

Hightouch reports that routing uncertain identity-resolution cases from Jev to Opus reached frontier-model accuracy at more than four times lower cost.

How AI Is Changing Forecasting and Demand Planning

Just how AI is revolutionizing forecasting and demand planning will surprise you—discover the game-changing insights that can give your business a competitive edge.