Research · November 2026

What decides an AI agent's result: the model, or the decision layer?

We tested eight language models running a company for a simulated year, alone and with WhAI. On final cash, the decision layer explains 72% of the variance. The choice of model explains 5%.

Schema: cinque modelli collegati al decision layer di WhAI. Varianza della cassa finale: decision layer 72%, scelta del modello 5%.Diagram: five models connected to the WhAI decision layer. Variance in final cash: decision layer 72%, model choice 5%.
The test

One company, one year, eight models

1

The benchmark

YC-Bench: an agent runs a start-up for one simulated year and is scored on its final cash.

2

Two set-ups

Each of the eight language models alone, and connected to WhAI.

3

What WhAI does

From the history of the run, it estimates the quantities the agent can't observe, and ranks candidate plans by simulating cash with Monte Carlo.

4

The check

A paired test on four new worlds with no published runs: five of the models, each alone and with WhAI, on the same world and the same seed.

The results

Every pair, better with WhAI

48 / 48

pairs better with WhAI, for the five models in the paired test on four new worlds. Sign test p = 3.6×10⁻¹⁵.

$1.72M

mean gain per pair, 95% confidence interval from 1.52 to 1.92.

72% vs 5%

of the variance in final cash explained by the decision layer, and by the choice of model.

$3.45M to $3.82M

for seven mid-size models with WhAI on the published configuration, against $2.87M for the best published model alone, at up to one twentieth of its inference cost.

What it means for you

What it means, depending on who you are

If you build agents

Look at the step that decides

In this test, the decision layer weighed more on the result than the language model. That's where to connect WhAIQ.

Integrate WhAIQ

If you run a platform

Add a calculation to call

Your agents keep running the process and call WhAIQ when a step needs a number.

For agent platforms

If you run a company

Start from one decision

Pick a decision your company makes often. We calculate the annual return on your numbers before you sign.

How to start
Questions about the paper

The benchmark and the results

Who built the benchmark?

Collinear AI, not Vedrai. YC-Bench is public and has its own leaderboard.

Which models did better with WhAI?

All five models in the paired test, in every pair: 48 out of 48. On the published configuration, seven mid-size models with WhAI ended above the best model running alone.

Where can I read it?

Grazioli, Fusai, Magliano, Murro, Cugliari, Decision Layers Matter More than Model Choice for Language-Model Agents: A Paired Evaluation in a Long-Horizon Business Simulation, Preprints.org, 13 November 2026. Open the preprint
Read next

Decision layer compared

With decision models, planning AI, world models

Read

Integrate WhAIQ

For integrators

Read

WhAIQ

The calculation engine

Read

Building agents? Test the decision layer on a real case

Partner test access connects WhAIQ to your agents through MCP.