Skip to content

AirOpsEnv v0.1.0

Choose a tag to compare

View all tags
@earakely-scale earakely-scale released this 14 Oct 04:22
· 3 commits to main since this release
v0.1.0
0ea8919

AirOpsEnv's first release: an agent runs JetBlue's operations desk at JFK or Boston through a real storm day, watch by watch on AgentEnv's virtual clock, and is graded against the plan a MIP proves optimal in hindsight.

  • Env airops-day: per-flight cancel, restore and hold decisions, executed on the day as it really happened (BTS on-time data, the FAA Command Center's ground stops and ground delay programs as hourly slot caps, TAFs and METARs as issued). Tools: get_situation, check_plan, confirm_changes, advance.
  • Pack b6-v1: 57 days from 13 months of 2022–2025. 20 eval days (12 JFK, 8 BOS), 3 control days, 34 train days. Every eval day qualifies: doing nothing costs 26–66% over the hindsight optimum and the re-planning follower lands within 15% of it. The references are deterministic, and the pack rebuilds byte-identically from pinned inputs.
  • Viewer: the network map with NEXRAD radar, the hub in 3D from OpenStreetMap, the tail chart with the slot caps, and the ops panel with each watch's news and tool calls. agent-env airops view and agent-env airops record.

GPT-6.1 Sol on 5 eval days (sol-v1, one run each, $2.99 in all):

Day GPT-6.1 Sol Follower Blind Doing nothing JetBlue
b6-bos-20250216-1500 0.672 0.937 0.937 0.366 0.224
b6-jfk-20230714-1400 0.441 0.782 0.229 0.212 0.200
b6-jfk-20240804-0700 0.382 0.901 0.627 0.213 0.202
b6-jfk-20250709-1500 0.324 0.614 0.743 0.224 0.200
b6-bos-20240107-0600 0.276 0.417 0.436 0.220 0.202
Mean 0.419 0.730 0.594 0.247 0.206

The attached film is Sol's run of the Boston storm of 16 February 2025 (60 s, 1080p).

Basemap © OpenMapTiles © OpenStreetMap contributors via OpenFreeMap; hub twin © OpenStreetMap contributors (ODbL); radar NEXRAD n0q via the Iowa Environmental Mesonet.

Assets 3
Loading