A long-horizon benchmark for agents that run an end-to-end business.
Website · Paper · Hugging Face
Running a business is challenging: evidence is noisy, feedback is delayed and hard to attribute, markets keep changing, and operational obligations persist. Business Arena tests whether agents can operate a seller business end to end over a 30-day market horizon.
Agents source products, allocate capital, enter markets, set prices, advertise, replenish inventory, negotiate with buyers, satisfy compliance requirements, and adapt as demand and competition change. Evaluation connects aggregate business results to the reasoning, tool use, and market decisions that produced them.
| Business skill | What the agent must do |
|---|---|
| Decision-Making Under Uncertainty | Gather partial market evidence, distinguish signal from noise, and update beliefs as outcomes arrive. |
| Strategic Planning Under Constraints | Allocate capital, products, markets, and time while preserving flexibility for later decisions. |
| Insight-to-Action Alignment | Turn plans into accurate quantities, prices, listings, advertising budgets, and replenishment decisions. |
| Cooperation & Competition | Serve and negotiate with buyers while responding to suppliers and changing competitors. |
These skills form one operating loop. Sourcing changes inventory, pricing changes demand, sales release capital, and customer or compliance failures can erase gains from otherwise sound commercial decisions.
The current snapshot contains 150 completed runs across 15 model families, with 10 matched runs per model. Mean final worth ranges from $20,856 to $188,488. The strongest model more than doubles the shared starting capital, while several models lose capital over the same market horizon.
| Rank | Model | Mean final worth |
|---|---|---|
| 1 | Gemini 3.1 Pro | $188,488 |
| 2 | GPT-5.6 Sol | $168,867 |
| 3 | Fable 5 | $164,204 |
| 4 | Gemini 3.5 Flash | $125,952 |
| 5 | GPT-5.5 | $117,481 |
The ranking is only the starting point. Business Arena exposes why agents earn, stall, or lose money.
Capital must be deployed, recycled, and converted at a margin. Models can reach similar final scores through very different businesses: concentrated premium catalogs, high-volume wholesale operations, or inventory-heavy stores that fail to sell through.
Business value is shaped by deployed capital x capital turnover x realized margin. High deployment creates opportunity, but it does not guarantee revenue or profit.
 |
 |
| Opportunity capture How much capital models put to work and turn into revenue. |
Margin discipline How models balance realized margin with inventory sell-through. |
A profitable plan still has to survive day-to-day operations. Customer inquiries require complete, truthful answers; compliance mistakes produce direct fines. The strongest business result and the most reliable service behavior are not always produced by the same model.
 |
 |
| Customer service Converted, incomplete, unanswered, and materially false inquiry outcomes. |
Compliance Mean fines and violations accumulated over a complete run. |
The public site provides three connected levels of evidence:
- Leaderboard: compare mean net-worth trajectories across repeated runs.
- Agent trajectories: inspect the model's reasoning, visible context, scripts, and tool actions.
- Market replay: watch the seller workspace and buyer storefront change together as the agent acts.
Human-designed reference strategies use the same agent-visible tools without oracle information. They provide a consistent operating reference for what deterministic business rules can achieve in the same market.
Yijun Pan1,2,† · Yukun Lian1 · Kunyu Shi1 · Junbo Li1 · Hongwei Xue1 · Sicong Xie1 · Guannan Zhang1 · Xiaoying Xing1,‡
1 Accio Team, Alibaba Group
2 Yale University
† Work done during internship at Accio.
‡ Corresponding author.
We evaluate models on request.
Be among the first to test how your model runs a business end to end. Contact us to get started. Results can remain private or be added to the public comparison with your approval.
Prefer to copy:
xiaoying.xing@alibaba-inc.com·yijun.pan@yale.edu
