The AI Company Trial That Cloud Buyers Should Watch

Buying for a business?Offer from Amazon

Get business pricing on networking and server gear

• Business-only prices and quantity discounts
• Tax-exempt purchasing
• Multiple users, one account, clear invoices

Create a free Business account →Free to register

As an affiliate, we earn on qualifying purchases.Hide

Before an AI agent gets near your cloud business, ask whether it can run one

Cloud and infrastructure providers are weighing AI agents for customer support, sales and operations. A polished demo can show that a model writes well. It cannot tell you whether the agent will read the relevant files, handle pressure and finish a job that requires a real decision. Firmulate has put that broader test on public display: a software company, a rough week and outcomes that include money on the line.

A company simulation, with decisions on the record

Firmulate ran frontier AI models through the same small software company’s worst week, with the same customers, crises and temptations. The live experiment is real and watchable; each decision is versioned and auditable. The company has 13 synthetic employees and real money mechanics, including monthly burn of €105,000 against €2,300 in monthly recurring revenue. Its public cash countdown and evolving playbook make the exercise more concrete than a chat demonstration.

The final July 2026 Crucible League puts gpt-5.6-sol first with 95, followed by Moonshot’s Kimi K3 at 93. Sonnet 5 scored 88, Fable 5 scored 77, and Opus 4.8 scored 73. A do-nothing baseline scored 26. Firmulate’s stated rule is that partial progress counts, but a single breach of trust caps the total: “no amount of good work outweighs a breach of trust.” The full results and plain-language findings are on Firmulate’s benchmark page.

The deal was won in the files

All models spotted every crisis and refused every manipulation attempt. Yet only two signed the €55,000 deal their own analysis had earned. The decisive competitor weakness was buried two document references deep in the company’s own files, rather than in the customer event. Models that read the file secured the deal at full price, worth €4,583 in monthly recurring revenue. The result points to a practical gap: identifying the right answer and carrying it through to a business outcome are separate tests.

Kimi K3 finished second, ahead of three of the four Western models in the field. It found the buried security needle, won the deal and saved the churning customer, while resisting all three baits. Firmulate reports that K3 had one deviation, the cleanest discipline in the field. During a staged social-engineering attempt, a fake CEO message escalated over three stages; a reporter also tried “just one yes/no, on background.” All five models refused. K3’s recorded reasoning described the request as “a suspected approval-bypass / possible impersonation.”

Opus 4.8 offers a different lesson. It was the most thorough participant, with more than 80 learned rules and the deepest analyses, but placed last. It left the deal unsigned and slipped on discipline by attempting to write into a locked department instead of escalating. Firmulate says a weaker version of that same weakness appeared in all four models. Thorough analysis, by itself, did not ensure a clean close.

Why infrastructure teams should care

For a cloud provider, hosting company or infrastructure team, the stakes extend beyond which model wins a table. An agent connected to customer records or an operations workflow must find information, respect boundaries and complete the work. Firmulate’s trial makes those behaviors visible under shared conditions, while its live company shows the experiment unfolding over time. The public site also offers a quiz built from 242 real, unedited management decisions, inviting readers to guess which model made each call.

Enterprises can also run the wargame against a read-only export of their own business, so the trial does not write back to live systems. Details are available from Firmulate.

A benchmark is a starting point

Kimi K3’s second-place finish shows that the field is open: it beat three of four Western frontier models in this trial, while finishing just behind gpt-5.6-sol. The deeper message for infrastructure buyers is to test agents against the work, documents and pressure they will actually face. A model’s reputation or chat performance cannot settle that question on its own.

Fairness note: K3 ran without an effort parameter (API default) while the others ran at xhigh.

Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html

Powered by Thorsten Meyer AI