v0.7.2

v0.7.2

Patch release over v0.7.1. Replay ephemeral lifecycle + oversub metric depth + aenv CCL pinning + cubesandbox hang bounds + vm_monitor correctness. No CLI breaking changes.

Replay mode & ephemeral lifecycle

• #239 — exec_ephemeral mode: bare-exec ephemeral sandbox (no pause/resume, no admission) — the real "spin up once → run → kill; to reuse, spin up again" workload. Oversub sweep default.
• #241 — exec_ephemeral CCL bind + per-sandbox multiplier (M trajectories per slot on a fresh sandbox) + stop_when_done.
• #243 — gate the ephemeral M-loop on persistent create-failure, not is_alive (kill_one flips is_alive per trajectory, so the multiplier ran 1 not M).
• #244 — per-pass run identity in fixed mode (repeated passes stay separate run rows) + per-run elapsed = create + kill + stress decomposition.

Replay correctness

• #240 — aggregate manifest-skip warnings into one INFO summary (no per-file WARNING spam).
• #248 — count trajectory_attempts in run_summary.json so failed is never negative (slot-count vs trajectory-completion dimensional cross).
• #256 — keep failed steps out of per-action latency stats (was dragging obs Per-step-timings avg_s toward 0).

Oversub driver & observability

• #237 — multi-template linearity (cap, clamp, fleet, rank) so the oversub ratio holds across templates.
• #236 — surface the executed command sequence in replay outputs.
• #258 — per-command exec time columns in ratio-summary.csv.
• #260 — per-stat degradation + same-trajectory cross-ratio compare + auto-compare in oversub-compare.
• #257 — p90 percentile in calc_percentiles and replay report surfaces.

aenv CCL pinning

• #245 — sysfs-detected CCD/CL domains + one-shot all-thread firecracker pinning (no polling); zen5 + Kunpeng verified.
• #254 — drop dead numa_bind from aenv (AENV firecracker ignores the e2b FC_BIND env var).

vm_monitor

• #242 — compute vm_cores as a single aggregate jiffies delta (bounded by construction; the per-VM normalized-rate sum overshot physical cores on NaN-sparse pivots).
• #247 — AMDuProfPcm (uprof) as a 4th log-capture rotation slot + selective tools: / --rotation-tools membership.
• #249 — normalize YAML boolean enabled/capture so vm_monitor is not silently disabled (str(True)="True" ≠ lowercase check).

cubesandbox

• #250 — bound cube SDK control-plane calls (session-pin adapter + _call_bounded daemon-thread wrapper + SDK 0.7.0 guard) + shared-deadline Phase-6 wait_all + concurrent cleanup — a wedged runner can no longer hang the bench.
• #253 — skip torn-down sandboxes in cleanup_all so ephemeral runs stop re-killing dead handles ("Killed 175 of 990" misread).

Config, deps, docs

• #261 — default task_batch.interval 5 → 1 across replay-*.yaml (tighter per-batch dispatch).
• #255 — pin e2b to 2.46.0 (2.47+ breaks AENV).
• #238 — pin vuejs/core base_commit in all ts Dockerfiles.
• #246 — CLAUDE.md catch-up.

Full changelog: v0.7.1...v0.7.2

v0.7.1

Patch release over v0.7.0 (final, 2026-10-13). v0.7.0 stays published; this is a clean semver patch.

Changes since v0.7.0

• feat(aenv): pin firecracker to CCL vCPU set in exec_only replay mode (#232) — in exec_only replay, firecracker is pinned to the CCL vCPU set.
• fix(vm_monitor): source resource_report CPU sheets from raw /proc deltas (#233) — CPU sheets now derived from the always-on raw /proc snapshots.
• fix(vm_monitor): correct raw_report memory units — 1024x underreport (#234) — host_resources.xlsx underreported mem_used_gb / mem_cache_gb / buffers_mb / dirty_mb by 1024x: host_mem_raw.csv holds MB (_read_meminfo converts kB->MB before dump), but raw_report divided as if kB. Also fixes the stale "kB fields" _ProcRawWriter docstring and the /proc formulas in docs/dev/metrics-reference.md. Existing reports can be regenerated without re-running the benchmark: vm-monitor raw-report <log_dir>.

Full changelog: v0.7.0...v0.7.1

v0.7.0

Final stable release of 0.7.0. Major release spanning rc1 → rc2 → rc3 → final, consolidating all changes since v0.6.0 (~140 commits across 8 feature areas).

1. Replay workflow (trajectory / lifecycle / exec replay)

Deterministic replay of recorded SWE-bench trajectories against sandbox backends. Three modes: exec_only (long-lived continuous exec, baseline), lifecycle (create_all then per-step resume/pause via aenv LifecycleCapable; oversubscription via snapshot memory reuse), trajectory (ephemeral create_one/kill_one via aenv EphemeralCapable; oversubscription via queue limiting + launch_interval_sec pacing).

Replay stack #114-#134 mega-merge: trajectory loading + cached pool, ReplayStep/Trajectory dataclasses, ReplayTaskRunner/ReplayRoundRunner, RoundRobinTaskManager replay wiring, LifecycleSeriesWriter (thread-safe JSONL series), 6-timestamp _run_slice, admission controllers, ReplayMetrics segment lists + retry accumulators, QPS-gate exec bucket, report formatting + e2e smoke.

Follow-ons: lifecycle hot-path contention fix #161 (writer drain thread - root cause of ~4x wall inflation), exec_only lifecycle-series writer fix #190, manifest as authoritative run set #196, two-tier per-run aggregation #197, e2b SDK pin <2.47 for AENV #203, replay.yaml action_timeout=300s #207/#209, per-step action text, replay.py split into config+formatter #192.

2. Aenv provider (lifecycle + ephemeral capabilities)

New --provider aenv: subclasses E2BProvider, reuses E2B SDK pointed at an AENV server via E2B_API_URL. Adds pause/resume (E2B beta_pause/Sandbox.connect), create_one/kill_one (trajectory mode), snapshot_sizes. default_replay_mode = lifecycle. Satisfies LifecycleCapable + EphemeralCapable + SnapshotSizeCapable. Documented in CLAUDE.md #169.

3. Oversub benchmark driver + cross-arch comparison

• oversub-bench (#143-#150): standalone driver over the kernel; emits run_summary.json + sweep-config YAML. Kernel emits raw facts, driver computes valid. Flushes partial artifacts on interrupt #150, round_count pass-through #149, terminal phase logs #164, traj denominator fix #163.
• oversub-compare #191: manifest-driven join of N sweep dirs (ARM baseline vs x86 freq/L3 configs); emits tidy + ratio-summary + 4-sheet xlsx. Per-run row collapse #200.
• Ctrl+C teardown for sweep #225 and replay stress #224.

4. RFC 0002 - workflow plugin architecture

Internal refactor, no CLI breaking changes. TaskRunner ABC + WorkflowSpec registry + RunContext #177, migrate runners + collapse dispatch #178, typed WorkflowConfigBase views #179/#180, report layer onto ReportFormatters strategies #181, drop hardcoded workflow set #182. Data-driven capability error hint #175. RFC accepted/Active #170/#174.

5. vm_monitor enhancements

• Always-on raw snapshots #213/#219: /proc/meminfo + /proc/vmstat + /proc/stat + per-cpu + per-VM + per-device /sys/block + ublk-daemon jiffies, written to raw_data/ on a background daemon thread.
• PSS field #210 (smaps_rollup, <2% perturbation); ublk-daemon cores + fleet-total PSS timeline #211.
• iostat-style disk capture #185 (IOPS + avg request size); disk util_pct 100x fix #221.
• Per-collector --no-X switches + devkit mem/topdown split #186; guaranteed resource_report.xlsx + opt-out chart phase #189; ksys rotation fix #223.
• Log capture rotation #220; host CPU utilization curve #202; --host-proc #229; reclaim-stall rates #230; SVG into svg/ subdir #231; host_resources.xlsx raw-CSV consumer #222; cubesandbox VMM type #218.
• CI: vm_monitor test job #226; fixture relocation #227; standalone usage guide EN/ZH #206.

6. cubesandbox provider (Cloud Hypervisor microVM, full lifecycle)

New full-lifecycle provider #136 RFC then #214 Phase1 exec then #215 Phase2 lifecycle: exec + pause/resume + create_one/kill_one + snapshot_sizes. RFC relocated to docs/dev/rfcs/0003 #216. vm_monitor integration + real snapshot_sizes deferred.

7. Observability workbook (_obs.xlsx)

8-sheet workbook: Overview, Per-step timings, Lifecycle overhead, Trajectory summary, Step detail, Concurrency states, Gantt, Snapshot sizes. Per-step timing + lifecycle overhead % in Overview #183/#184/#201/#204. Atomic xlsx write + legend fix; per-trajectory replay_result.json export. Name shortened #162. matplotlib core dep for Gantt PNG #165.

8. Document bench + misc

pdf/xlsx document bench configs #188; optimized Dockerfiles PGO+LTO CPython 3.12.3; docs reorganized into architecture/guide/dev #166; env_provider tests grouped by provider #199; TaskManager into task_manager package #194/#195; issue templates to RFC guide #217; httpx/httpcore logging capped at WARNING.

Version indexes

• pyproject.toml: 0.7.0rc3 to 0.7.0
• version.py: 0.5.0/alpha to 0.7.0/stable (synced single-source-of-truth header).

No CLI breaking changes.

v0.7.0rc3 — Release candidate 3

Release candidate 3 of v0.7.0

Over v0.7.0rc2 (28 squash-merged PRs). No CLI breaking changes; the TaskRunner refactor (RFC 0002) is internal. src/ suite: 844 tests collected, CI green on Python 3.10/3.13.

RFC 0002 — workflow plugin architecture

• #177 RFC 0002 accepted (TaskRunner ABC + registry + RunContext)
• #178 migrate runners to TaskRunner + collapse workflow dispatch (P1)
• #179 / #180 typed WorkflowConfigBase views + wire from_raw (P2-1 / P2-2)
• #181 collapse report layer onto ReportFormatters strategies (P3)
• #182 drop validate() hardcoded workflow set (P2-2 follow-up)
• #174 / #175 RFC 0002 marked Active; data-driven capability hint + registration touch-point docs

Replay observability

• #183 per-step timing + lifecycle overhead % in obs Overview
• #190 write lifecycle series for exec_only to populate Step detail / Trajectory summary / Concurrency / Gantt sheets (were header-only)
• #161 lifecycle replay hot-path contention — background writer drain thread for the global lock+flush (root cause of ~4x wall-time inflation: 49 vs 28 min)
• #184 document obs Overview per-step timing + overhead %, enrich result tables

vm_monitor

• #189 guarantee resource_report.xlsx + opt-out chart phase (root-cause: detach + --no-charts; no SIGKILL reaper) for oversub-scale runs
• #186 per-collector --no-X switches + devkit mem/topdown split
• #185 iostat-style IOPS + avg request size to disk capture
• #187 add commented monitor.skip block to replay yamls

Oversubscription tooling

• #191 oversub-compare — manifest-driven cross-arch/config comparison of oversub-bench sweep results (delta CSVs + 4-sheet comparison.xlsx; Overview / Per-ratio with lifecycle_overhead_pct chart / Component heatmaps exec-resume-pause-wait / Per-trajectory delta). Runs no sandboxes — compares already-produced results.
• #188 document bench pdf/xlsx configs for the bench-core kernel

Other

• #192 split replay.py into config + formatter modules
• #166 reorganize docs/ into architecture / guide / dev structure
• #169 document aenv provider in CLAUDE.md
• #167 e2b: persist template binding across create → detect boundary
• #168 drop zeroed failed-step timestamp sentinels in trajectory elapsed
• #163 replay traj snapshot/report denominator was pool×fleet (root-cause fix)
• #162 shorten replay observability xlsx name to <prefix>_obs.xlsx
• #165 make matplotlib a core dependency (replay Gantt PNG)
• #164 emit oversub-sweep phase logs to terminal

Full changelog: v0.7.0rc2...v0.7.0rc3

v0.7.0rc2

Release candidate 2 of v0.7.0

Stabilization over v0.7.0rc1 (19 squash-merged PRs). No breaking API changes. src/ suite: 757 passed, CI green on Python 3.10/3.13.

Oversub benchmark driver (new feature)

• #143 oversub-bench driver + run_summary.json contract (kernel emits raw facts, driver computes validity)
• #147 default output_root under results/
• #148 per-trajectory timing breakdown (12 components) in trajectory-detail.csv
• #149 round_count pass-through + run-to-completion validity (fixes the multi-day never-finishing run)
• #150 flush partial oversub artifacts on interrupt

Replay observability hardening

• #151 / #153 vm_monitor Excel row-overflow guards (Raw_VM_Data + all per-sample timeline sheets)
• #152 guard happy-path trajectory export so a step failure cannot crash the run
• #154 render-OOM root-cause fix: _segment rejects a<=0 (a failed steps zeroed pause_end no longer produces a ~1.8e9-second segment → OOM/hang); plus Gantt/per-step downsampling and a 6h trial_timeout_sec backstop; wait-decomposition refactor + docs
• #157 reap vm_monitor on process exit, not xlsx appearance
• #159 wait-decomposition all-zero bug root-cause fix: attribute the slept think-delay to natural_delay; drop the redundant +delay term in interaction_total (value-preserving: interaction_total stays 568)

CI / tooling / docs

• #139 English bench-core usage guide
• #140 version-independent snapshot_size stub + Python 3.10/3.13 CI matrix
• #142 bump ruff/mypy target to py310 + pyupgrade auto-fix
• #144 remove obsolete fix_scripts
• #145 rename vm_monitor/replay excel outputs
• #146 remove orphaned add_l3_hit_rate_to_excel
• #141 document squash-merge multi-commit PR rule in CLAUDE.md

Full changelog: v0.7.0rc1...v0.7.0rc2

v0.7.0rc1 — Replay feature release candidate

Overview

Release candidate for the trajectory replay feature: deterministic replay of recorded SWE-bench trajectories against sandbox backends (--provider aenv / e2b / docker / fake), with a host-agnostic stress kernel (bench_core) + provider contract (env_provider) under src/. Ships as a 21-PR stack (#114–#134), now merged to main.

Highlights

Replay workflow (workflow_type: replay, config/common/replay.yaml)

Three modes keyed by sandbox lifecycle, not "whether a rate limiter is attached":

• exec_only — long-lived sandboxes, continuous step exec, no pause/resume. Baseline for pure exec-replay cost (e2b/docker/fake).
• lifecycle — long-lived sandboxes: create_all → pause once → per-step resume()/pause(). Oversubscription = snapshot memory reuse (pause frees RAM, so k×N sandboxes fit in N running slots). Emits initial_pause + per-slice pause/resume segments + snapshot_size events.
• trajectory — ephemeral create_one/kill_one per trajectory; per-step resume/pause still runs. Oversubscription = queue limiting (M slots gate concurrent trajectories).

Observability workbook (<prefix>_obs.xlsx, 8 sheets)

Overview (Run / Throughput & overcommit / Admission & QPS / Retry impact as grouped color-coded sections) · Per-step timings · Lifecycle overhead · Trajectory summary (per-trajectory cost sums + stacked bar chart + n_failed/n_timeout/success_rate + DataBar/ColorScale conditional formatting) · Step detail (per-trajectory per-step, 6 lifecycle sub-segments) · Concurrency states · Gantt · Snapshot sizes. Plus a text report + _lifecycle_series.jsonl (time-time()-stamped for join with vm_monitor host samples).

Per-trajectory export (PR #134)

One replay_result.json per trajectory under <output_dir>/trajectories/<sanitized_tid>/ — a superset of the reference replay-aenv-main schema (steps[] + aggregates + pause-resume overhead + bench-core sub-segments + create/kill error string). A trajectories/index.json catalog browses the whole fleet at a glance.

vm_monitor integration

Auto-enables during the stress phase only for providers with a VMM (vmm_type: e2b/aenv → firecracker; docker/fake → skipped). Runs as a stress-file-sync subprocess bracketing active stress; outputs CSV + analysis_report.xlsx + SVG to <output_dir>/vm_monitor/. Atomic xlsx write (#132) survives the monitor-reap race; SVG written before xlsx so reap-on-xlsx drops nothing (#126 fix).

Multi-template routing

Trajectories declare per-file concrete templates via a side template_manifest; the fleet is round-robin-allocated over the pool's (template, trajectory) pairs and runners route by template affinity.

Entry points

bench-core --provider {fake,e2b,docker,aenv} --config config/common/replay.yaml · vm-monitor -t <vmm>

Status

Release candidate — feature-complete replay stack, locally + CI verified (full pytest matrix green: bench_core kernel, env_provider providers, legacy e2b_bench/docker_bench/vm_bench/llm_replay, vm_monitor). Feedback welcome before the 0.7.0 final.

v0.6.0

Features

• EnvironmentProvider abstraction + host-agnostic bench_core kernel (RFC 0001)
• bench_looper: in-image runner for browser / coding-go / coding-ts (ARM)
• bench_looper: --interval-min/--interval-max inter-round sleep + --quiet
• aenv_latency: AgentENV cold-boot + snapshot/lazy-load latency benchmark (#102)
• vm_monitor: Disk I/O / Host-mem-detail / ublk metrics + SVG time-curve export
• vm_monitor: generic host pressure + disk latency + adaptive SVG units (#101)

Fixes

• bench_looper: default browser URLs to 127.0.0.1:8080 (host networking)
• bench_looper: no-op coding warmup + per-round progress logging
• bench_looper: scrub public IPs + fix stale Dockerfile build-context comments
• log: downgrade non-fatal warmup exceptions to WARNING

Refactor

• vm_monitor: consolidate host per-NUMA sheets into NUMA_Overview (#97)
• vm_monitor: extract export_to_excel into per-sheet helpers
• vm_monitor: split _add_charts into per-chart helpers

Docs

• bench_looper: ARM build context + usage guide; per-scenario run commands; resource limits
• vm_monitor: expand README with CLI usage + new metrics/outputs

Tests

• vm_monitor: structural snapshot for export_to_excel

v0.5.0

Features

• E2B: add Python coding benchmark variant (#75)
• E2B: classify sandbox-unreachable errors and log round_id (#74)
• Dockerfile: add openEuler browser/TS/Go coding images (arm64 + x86_64)
• Add RFC process, issue/PR templates, community health files (#87)
• Codify RFC 0001 (EnvironmentProvider) and engineering-issue template (#88)

Fixes

• E2B: anchor coding verify import at coding_project_dir (#92)
• Dockerfile: set dnf sslverify=0 to skip corp-proxy self-signed cert
• Dockerfile: rename openEuler temp container to avoid name collision
• Scrub leaked E2B credentials, replace hardcoded public IPs (#94)

Refactor

• E2B: collapse three Config construction paths into one field table (#90)

Packaging

• Drop deprecated/broken console scripts; fix requires-python floor to >=3.10 (#93)

Docs

• Document PDF and XLSX E2B execution flows (#89)
• Add openEuler browser image build guide

v0.4.0 - Coding & document benchmarks, NUMA binding, cleanup

🚀 v0.4.0 - Coding & document benchmarks, NUMA binding, cleanup

This release adds coding (TS/Go) and document (PDF/XLSX) benchmarks, per-sandbox NUMA node binding, and arch-aware (x86_64) Dockerfiles, then removes the deprecated qemu_monitor / session-replay packages and translates remaining Chinese comments/docstrings/CLI text to English.

Features

💻 Coding benchmarks (E2B)

• TypeScript benchmark against vuejs/core — multi-process verify via npx tsx (default 3 concurrent processes), compiler-core baseParse trace-faithful verification
• Go benchmark against gohugoio/hugo — go clean -cache before every verify for cold-compile CPU measurement
• verify_repeat config (yaml + --coding-verify-repeat CLI) — repeat verify N times with independent processes
• Per-pair edit semantics; shared DEFAULT_VERIFY_TEMPLATES pool; language-aware root discovery

📄 Document benchmarks

• PDF and XLSX document benchmarks via document-bench-validate
• Group document operations into phases; retry transient ready-check failures

🧬 NUMA node binding (E2B)

• Bind each sandbox to a NUMA node via round-robin distribution across a numa_bind list
• Normalize numa_bind to a list-of-nodes at config load; multi-node numa_bind form documented
• numa_node_for_index contract tightened (type hints, explicit bool guard)

🏗️ Arch-aware builds

• x86_64 Dockerfile variants for coding-ts and coding-go
• push_to_harbor.sh made arch-aware via ARCH env var (browser / coding-ts / coding-go)

Refactor

• Config & dockerfile reorganization: config/ and dockerfile_build/ reorganized by scenario (browser/coding/document)
• Logging migration: replace print() with the logging module across e2b_bench and docker_bench; benchmark report emitted via logger.info, routed to stdout to preserve pre-refactor output destination
• Round-robin cleanup guaranteed on errors

Removed

• qemu_monitor/ package and qemu_monitor.py entry script (deprecated in v0.1.0 → v0.2.0) — use vm_monitor
• session-replay/ — use llm_replay

Notes

• Remaining Chinese comments, docstrings, and CLI text translated to English
• Version bumped to 0.4.0

Full changelog: v0.3.0...v0.4.0

v0.3.0 - Round-Robin Benchmark Mode

🚀 v0.3.0 - Round-Robin Benchmark Mode

This release adds round-robin benchmark mode for E2B sandbox testing, enabling memory migration stress testing through controlled sandbox group rotation.

Key Features

🔄 Round-Robin Sandbox Rotation

• Group-based execution: Splits sandboxes into equal groups (configurable via round_count or round_size)
• Cycling support: Automatically cycles through groups when rounds exceed group count, enabling continuous testing within test_duration
• Per-round metrics: Independent statistics tracking for each round with accurate latency extraction

🌊 Wave-Based Warmup for Large Scale Tests

• Automatic batching: When total_count > 100, warmup executes in waves of 100 sandboxes each
• Cross-process state: Supports --detect mode by splitting detected sandboxes into waves
• Memory preheating: Opens multiple tabs per sandbox, executes operations to allocate memory before benchmark

📊 Tab Operation Benchmark Mode

Each benchmark round executes:

• New tab → Opens a new browser tab with URL from browser_urls (round-robin selection)
• Snapshot → Captures DOM state
• Click → Interacts with first available element
• Screenshot → Captures visual state

📈 Multi-Level Metrics Collection

Level	Metrics	Purpose
Per-Sandbox	Task count, success rate, latencies, step timing	Individual sandbox health
Per-Round	Task delta, success delta, latency distribution	Round comparison
Per-Step	open_tab, page_load, snapshot, click, screenshot timing	Performance breakdown
Aggregate	Total tasks, overall success rate, P99 latency	Summary statistics

🔍 Enhanced Error Diagnostics

• Detailed error recording with exit code, stderr, stdout, and context
• Automatic error type classification (tab operation errors, legacy browser errors, timeout, other)
• Thread-safe metrics collection with threading.Lock()

💾 Swap Monitoring Enhancement

• Swap cache and activity metrics in vm_monitor
• Swap_Timeline sheet with chart in Excel summary
• Improved swap display formatting for real-time and summary report

Configuration

test:
  benchmark_mode: "round_robin"   # Enable round-robin mode

  # Choose ONE:
  round_count: 5                  # Number of groups (auto-calculated size)
  # round_size: 20                # Sandboxes per group (auto-calculated count)

  round_interval: 30              # Gap between rounds (seconds)

Bug Fixes

• Round comparison latency statistics correction
• Round baseline timing for accurate task counts (sentinel entry)
• Step timing data type fix for click/screenshot errors
• Config priority enforcement: CLI > YAML > defaults
• Wave mode duplicate sandbox creation fix
• Division by zero when tab_ids is empty
• Warmup delay config override fix

Testing

• 162 tests total (39 new for round-robin features)
• All tests pass ✅

Full Changelog

v0.2.0...v0.3.0