Releases: DrobyshevDev/stadion
Release list
stadion 0.1.0
First release. Six operational decision tasks where an agent is scored against
two references it cannot argue with: the classical operations-research method
for the problem, tuned on seeds held out of the evaluation, and the exact
optimum computed by backward induction.
pip install stadion-rlWhat a run reports. A normalised score — 0 at the classical method, 1 at the
optimum — with a bootstrap interval over paired instances, and what one point of
score is worth in return. "Indistinguishable from the classical method" is a
verdict, not a rounding error, and where the classical method is already optimal
the score is reported as undefined rather than divided by noise.
How much room each task has, over 40 instances × 20 episodes:
| Task | Classical | Optimum | Headroom |
|---|---|---|---|
inventory |
204.141 | 204.890 | +0.4% |
joint-pricing |
110.099 | 114.812 | +4.3% |
pricing |
24.490 | 26.006 | +6.2% |
queueing |
21.911 | 25.611 | +16.9% |
supply-chain |
−37.532 | −31.027 | +17.3% |
energy |
16.778 | 21.234 | +26.6% |
The spread is deliberate. A price threshold with no forecast leaves a quarter of
the battery's value unclaimed; the newsvendor formula is within half a percent of
the exact optimum, so an agent reporting a large improvement on inventory has
found a bug rather than a policy.
Instances are generated, not stored. Demand levels, cost structures and
horizons are drawn at evaluation time from a documented distribution, so the
numbers an agent is asked about did not exist before the run.
One agent interface covers numeric policies and language models alike: a
view carries the written brief and the numbered menu beside the observation
vector and the live environment. Continuous action spaces reach every player as
the same menu, and the classical rule is snapped to it and tuned through the
snap.
The optima are checked against themselves. stadion verify computes each
dynamic program's analytic value and, separately, simulates the policy that same
program emits — two routes to one number, which have to agree within Monte Carlo
error. It runs in CI, because every other number here is measured against the
optimum and nothing else would catch a wrong recurrence.
Environments and classical policies come from
decisionrl unmodified.
Python 3.10+, tested on Linux, macOS and Windows. MIT.