How AI Agents Win or Lose Big Deals by Reading Your Files Two Layers Deep

Imagine an AI that doesn’t just respond on the surface but digs into your company’s files to make decisions. In a recent live experiment, AIs that read deeper into documents secured lucrative deals, while those that didn’t missed out—even when every other factor was identical.

The Power of Reading Deeper: A Live AI Wargame

At the forefront of AI evaluation, a live experiment by Firmulate tested four top models—gpt-5.6-sol, Kimi K3, Sonnet 5, and Opus 4.8—by running them through a simulated week of a small software company’s worst crises. Every model faced the same challenges: customer issues, ethical tests, and manipulation attempts. The goal? See which AI truly understood the company’s files and how that affected their ability to close a $55,000 deal.

Key Outcomes of the Experiment

• All four models successfully identified every crisis and refused manipulative tactics, such as fake CEO messages and reporter tricks.
• Only two models, GPT-5.6 and Kimi K3, managed to sign the deal based solely on their analysis—each earning full credit for their diagnosis.
• Despite identical diagnoses and pitches, the other two models failed to close the deal, leaving the opportunity on the table.

The Hidden Factor: Reading Your Files Two Layers Deep

The decisive difference was not in the superficial responses or chat behavior but in how deeply the models explored the company’s internal files. The winning models uncovered a critical, buried fact—located two document references deep—that proved essential in sealing the deal. Those that didn’t read beyond the surface missed this key insight, losing the opportunity entirely.

Implications for Business AI Adoption

This experiment underscores a critical property for enterprise AI: the ability to read and understand your files thoroughly before acting. As firms integrate AI into CRM, support, or decision-making processes, the question isn’t just about how well an AI writes but whether it can follow through on what it reads and stay honest under pressure.

Beyond Surface-Level Responses

Models that only skim the surface or fail to delve into context risk missing vital clues buried within internal documents. For example, the Opus 4.8, while the most thorough in rules learned and analysis depth, still left a deal on the table due to process slips—a sign that even the best deep-dive model can falter without disciplined execution.

Real-World Application: The Firmulate Live Wargame

Firmulate’s live platform hosts ongoing simulations where AI models run entire companies through crises, with real money mechanics and self-learned rules. The experiment’s transparency allows users to see how each model performs in realistic business scenarios, making the case that AI’s true enterprise value hinges on thorough understanding and integrity—not just language prowess.

Currently, 14 benchmark runs are queued, and the league table shows GPT-5.6 scoring 95, with Kimi K3 close behind at 93. Sonnet 5 and Opus 4.8 trail slightly, indicating that even the most comprehensive models can leave opportunities behind if their discipline slips.

What This Means for Your Business

As AI models continue to evolve, their ability to read your internal documentation—finding crucial insights buried deep—is becoming a defining advantage. Whether it’s closing deals, avoiding manipulation, or maintaining ethical standards, the difference lies in how deeply and accurately the AI can interpret your data.

To explore and test your AI workforce before deploying it live, firms can run similar wargames using Firmulate’s platform, which keeps every decision auditable and ensures no real systems are ever affected.

Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html

Powered by Thorsten Meyer AI