AgentKernelArena 0.3.0

AgentKernelArena 0.3.0 expands kernel-agent integrations, adds task quality auditing and optional GPU analysis tools, and strengthens benchmark integrity across the task suite.

This release tags the existing main commit 0acf65b3a967ef1025dbfc5fd4b415b259e3bd43.

Highlights

Agents and task quality

• Added GEAK v4 integration, including lifecycle handling for its background optimization workflow. Arena independently checks and scores the resulting kernel. See #73.
• Added Forge integration and compatibility with KernelForge as shipped in Hyperloom, including kernel identity metadata, editable-source boundaries, and task-owned correctness and benchmark drivers. See #69, #79, and #92.
• Added the separate quality_loop workflow: task validation and bounded repair, one optimization candidate, centralized evaluation, independent Codex review, and gated baseline/case improvements. Accepted changes can be bundled into one draft PR. See #76.
• Strengthened task-validator report finalization, completion handling, and benchmark-fairness checks. Quality-loop proposals now exclude generated reports and build artifacts. See #105.

Benchmark integrity and task coverage

• Expanded canonical graph-replay timing across kernel benchmarks, with device-event fallback and state restoration support. Added shared Python and native HIP helpers, stricter case matching, and benchmark-method consistency checks. See #77.
• Added harness integrity checks around agent execution and strengthened protection of test/configuration files, performance entrypoints, and colocated harness code.
• Added the image_kernel task type for kernels shipped in the selected runtime image, plus MI355X attention, MoE, quantization, and related tasks. Correctness and performance drivers were aligned to exercise the scored work. See #69 and #82.
• Added FlyDSL optimization examples grouped by compute-pattern complexity. See #43.
• The tagged source contains 426 task packages across nine task types, including 21 image_kernel packages. Availability depends on each task's runtime and architecture contract.

Runtime and optional analysis

• Added a Slurm/Spur launcher that allocates MI355X GPUs and runs the existing Docker workflow on the compute node. See #95.
• Added a pinned, non-root RDNA4 gfx1201 Docker runtime, automatic first-use image builds, an RDNA4 quickstart, and architecture-specific agent guidance. See #97.
• Added opt-in isolated evaluation-tool sidecars for Triton FpSan, ROCm GPU ASan, rocJITsu race detection, Waitcheck, ConSan, and HIP-FpSan, with capability checks, structured evidence, and advisory/required policies. See #89.

Upgrade notes

• Removed geak_v3, geak_v3_triton, and mini_swe_triton. These values and their normalized aliases are no longer accepted as agent.template. Select a supported integration; use an earlier revision to resume an old run with a retired integration. Their num_parallel and run_mode configuration guidance was also removed.
• quality_loop is a separate repository maintenance workflow. Normal task runs still use centralized evaluation; they do not automatically start its independent reviewer.
• Run comparison is available at src/tools/compare_runs.py.
• Docker remains the supported experiment path. Consult the tagged compatibility matrix for runtime selection and dependencies.

Validation and limitations

The CI run for the exact release commit passed on the Ubuntu CPU runner with Python 3.11:

• make check-perf-helpers
• python -m pytest -q tests
• python -m compileall -q src tasks tests

No fresh GPU benchmark or task-validator campaign was run as part of publishing this release. These CPU checks do not establish GPU correctness or performance for every task.

Evaluation tools remain experimental and candidate-dependent. Their current hardware qualification is scoped to MI355X (gfx950); task support on MI300/MI325 or RDNA4 does not imply sanitizer support. A passing sidecar startup control does not establish that a candidate was instrumented, and the current agent/evaluator arrangement is not a tamper-resistant reward boundary. See the evaluation-tool guide.

RDNA4 runtime checks were performed on a 16 GB gfx1201 device during development; task memory, LDS, language, and architecture restrictions still apply. See the runtime guide.

Full changelog: v0.2.0...v0.3.0