Hyperloom v1.0.0a1

Highlights

Unified macro-cycle orchestration and budget accounting

Short and long sessions now share the same cyclic optimization model. Phase budgets use consistent charge-back accounting, short runs stop dispatching after their phase budget is exhausted, and new macro-cycles only start when at least three hours remain. This removes legacy cyclic-mode branches and makes phase progression more predictable across bounded and long-horizon runs.

Opt-in Ray serving and GPU execution

The opt-in Ray path now places all serving and GPU-specialist operations under the whole-machine serving_slot mutex, including framework-agent benchmarks, integrate_patch, and concurrency sweeps. GPU specialists can queue without blocking the Coordinator, serving receives scheduling priority, and stale AITER JIT locks are cleaned before server launch. Local subprocess execution remains the default; multi-node behavior is unchanged.

vLLM and serving-environment reliability

Hyperloom now isolates co-located Ray heads, uses per-session free ports for persistent serving processes, and reaps orphaned vLLM/SGLang process groups safely. Package-root MAGPIE_PATH entries are kept out of PYTHONPATH, preventing main-environment Torch packages from shadowing isolated vLLM environments. TraceLens patching and dependency discovery also support isolated vLLM/AITER installations more reliably.

GEAK-first kernel optimization

GEAK is now the default owner of the complete KERNEL phase. Forge remains an explicit opt-in and runs only when KERNEL_OPT_BACKEND_ORDER=forge is set. GEAK candidates remain provisional until Hyperloom revalidates them with its benchmark harness; successful revalidation can now complete before SWEEP begins, while failed or inconclusive validation exits cleanly without blocking the session.

Baseline, evaluation, and reporting integrity

Relative evaluation-result paths are resolved against the benchmark output directory, preventing false baseline-accuracy failures. Persistent servers use unique ports across retries, framework-phase gains are attributed correctly in session breakdowns, and concurrency-sweep internal tasks are no longer rejected by their own singleton policy.

Security and policy hardening

Multi-node restart arguments are validated and shell-quoted before execution. Explicit framework source roots must remain inside the configured allowlist, and queued tasks are revalidated by PolicyGate before dispatch to prevent forged task rows from bypassing normal authorization and Critic gates.

CI and release engineering

Every PR targeting main now runs a single-GPU Qwen3 smoke test on the dedicated Hyperloom E2E runner. The sharded Python 3.10/3.11 test suite has stronger failure visibility and completeness checks. Release version reporting now comes from installed package metadata, keeping hyperloom.inference_optimizer.__version__, wheel metadata, and the release version aligned.

What's included

Orchestration and phase behavior

• Unify short/long macro-cycle behavior and charge-back phase budgeting — #954
• Expand Ray serving_slot coverage and improve eval/JIT reliability — #955
• Invoke the Framework Agent through python -m instead of PATH lookup — #964
• Preserve queued GEAK win revalidation before SWEEP — #999

vLLM, serving, and installation

• Isolate vLLM Python paths, recover stale Ray state, and harden serving lifecycle behavior — #968
• Use per-session free ports for persistent server reuse — #971
• Refresh compatibility guidance and architecture documentation — #966, #969, #970, #1001

Security and policy

• Validate multi-node restart arguments and replay policy checks before task dispatch — #959

Reporting

• Attribute FRAMEWORK_AGENT and integrate_patch gains correctly — #975

CI and dependencies

• Add mandatory single-GPU PR smoke testing — #991
• Update GitHub Actions dependencies — #960, #961, #962, #963, #995
• Reduce internal complexity and remove dead paths — #965

Release

• Set the packaged version to 1.0.0a1 — #996
• Read the runtime version from package metadata — #1002

Requirements

ROCm platform with a supported AMD Instinct GPU; Docker with AMD ROCm support, Kubernetes with AMD GPU nodes, or the supported bare-metal setup; Cursor IDE with Remote SSH; an Anthropic API key or compatible Anthropic gateway; tmux, jq, Node.js >= 18, and Claude CLI.

Alpha notice

v1.0.0a1 is an alpha release. Configuration surfaces, orchestration policies, and installation workflows may continue to evolve before the final v1.0.0 release.

Hyperloom v0.9

Highlights

Ray-managed single-node execution

Every GPU/serving unit (baseline, profile/roofline, explore, sweep, conc_sweep, and needs_gpu specialists) now runs on a Ray-managed GPU lease, with Ray owning GPU queueing, device isolation, and a whole-machine serving mutex. The path fails fast on an infeasible cluster, times out stuck specialist scheduling, detects dead actors, and keeps the raylet stable across per-round server reboots. Multi-node is unchanged (gated off).

Accuracy-gate & eval-result integrity

lm-eval output is wired to a session-scoped EVAL_RESULT_DIR; a baseline that produces no accuracy verdict now hard-stops instead of optimizing an unvalidated baseline; leaked eval results are salvaged back from the InferenceX checkout / local-disk mirror; and session-breakdown attribution no longer fabricates credit from a seeded stack.

Provider-direct LLM configuration

Hyperloom runs against Anthropic + DeepSeek directly (provider-only paths), with env-driven, consistent gateway auth/endpoint resolution and case-insensitive Anthropic-endpoint handling; the Critic can run over the native provider endpoint.

Model-path & workload-default consistency

A single --model value (local path or HF repo id) resolves identically across baseline, roofline, and the kernel agent; prompt-stated ISL/OSL/CONC/TP are honored as flags instead of silently defaulting; and the model_arch freshness guard is org-aware across HF-cache snapshot paths.

Kernel / Forge / GEAK

forge-fusion is adopted end-to-end with hardened subprocess timeouts, GEAK v4 installs via pip/one-click, advertised kernel backends are aligned with the runtime, and a TraceLens-free bypass benchmark harness ships as a Magpie drop-in for text-gen + xDiT.

Framework agent

The FRAMEWORK phase gains cross-framework rating + PR-KB discovery, a flag-gated config-exploration subphase, and a candidate-free local-exploration arm so a dry PR feed no longer wastes the phase.

What's included

Accuracy gate & eval integrity

• Wire EVAL_RESULT_DIR for lm-eval output + guard stale eval on RUN_EVAL-off - #885 (addresses #909)
• Stop the run on baseline accuracy failure; revert after baseline - #923
• Fix lm-eval cleanup deleting Magpie traces - #907 (closes #904)
• Salvage leaked eval results / eval-dest redirect on the baseline path - #925, #942 (closes #927; addresses #934)
• Session-breakdown attribution de-fabrication (A1/A2/A3) - #921

Model path & workload defaults

• Resolve HF repo-id --model to a local dir across all in-process readers - #886
• Config-driven vLLM --block-size + resolve repo-id in compat preflight - #900
• Honor prompt-stated workload arg defaults (ISL/OSL/CONC/TP) - #908 (addresses #903; issue open)
• Case-insensitive Anthropic endpoint + org-aware model_arch guard - #936 (closes #929, #930)

Provider-direct LLM

• Single-gateway fallback + env-driven gateway auth/endpoint - #841, #867
• Provider-only execution paths + native-provider Critic - #860, #864

Ray-managed execution

• Unify single-node GPU/serving on Ray + raylet-stability fix - #910
• Ray robustness: infeasible fail-fast, sched-timeout, dead-actor detect - #937
• Enable macro loops for short runs - #940

Kernel / Forge / GEAK / TraceLens

• forge-fusion E2E adoption + subprocess-timeout hardening - #838, #878
• GEAK v4 one-click / pip git+ install - #839, #888
• Align advertised kernel backends with runtime - #897
• TraceLens-free bypass benchmark harness (text-gen + xDiT) - #801, #872
• Canonicalize TraceLens graph-capture attribution before GEAK dispatch - #806 (closes #805)

Framework agent

• Cross-framework rating + PR-KB discovery - #803
• Flag-gated config-exploration subphase - #811
• Candidate-free local-exploration arm - #937

Install & portability

• Bare-metal installer + configurable ROCm installs - #802, #827
• Quick-start Docker image + local demo/examples - #813, #847, #932
• Portable per-revision dependency cache - #925 (advances #767)
• Fix vLLM container start / docker exit - #866 (closes #837)

Reliability, cleanup & hardening

• Phase-budget charge-back + conc-sweep boost + token efficiency - #919, #899, #853
• Repo-wide de-dup / dead-code / comment / test consolidation - #863, #870, #879, #883, #906, #911
• Control-plane security hardening (state-field lock, apply_patch/Critic-verdict gates, argv-only command sinks, env filtering) - #912, #920, #938, #941

Also closed this cycle (manually / duplicate / not-planned; no explicit closing PR)

• #392, #682, #725, #765, #771, #829, #831 (dup), #855, #857, #858, #894, #717, #767

Requirements

(unchanged from v0.8) ROCm platform (MI355X / MI325X / MI300X; MI308X detection); Docker with AMD ROCm support, or Kubernetes with AMD GPU nodes; Cursor IDE + Remote SSH; an LLM provider key (Anthropic or DeepSeek); tmux, jq, Node.js >= 18, Claude CLI. New in v0.9: install from the GitHub-Release wheel (hyperloom_inference_optimizer) plus a bare-metal installer or quick-start Docker image.

Legend: closes = GitHub-verified explicit closer. Issues marked "addresses" are best-effort cross-references (the PR addresses the issue but did not auto-close it).

Hyperloom v0.8

Highlights

Kernel-optimization integrity & GEAK faithfulness

Patch-only kernel wins are no longer discarded by the FULL_BENCHMARK verifier
(full source is reconstructed from the patch); multi-file (L3) kernel
optimizations are preserved end-to-end via recorded kernel-artifact bundles;
GEAK always runs against a freshly re-profiled TraceLens snapshot; and the
kernel-candidate pool cap is decoupled from the dispatch budget.

Profiling / roofline / TraceLens

GPU information is restored in profiler traces (torch-trace "kernel"
category), and a stale TRACELENS_ROOT inherited from the kernel-agent env
file no longer breaks TraceLens discovery.

Orchestrator reliability & long-run durability

reports/final.json is now written crash-safe even on non-graceful /
time_exhausted exits, and orchestrator LLM calls survive slow heavy-reasoning
models (e.g. Kimi-K2.6) via idle-timeout + amplified retry.

Server config & Local-Mode portability

sglang --context-length is clamped to the run's --max-model-len (no more
contradictory server config), and Local Mode portability groundwork removes
Core42 / WekaFS hard-coding (docs).

What's included

Kernel-opt & candidates

• Reconstruct full source from patch so patch-only kernel wins aren't deferred — #681 (closes #680)
• Record & consume kernel-artifact bundles; preserve multi-file (L3) kernels — #782 (closes #696)
• Rerun profiling + TraceLens before GEAK to avoid stale snapshots — #686 (closes #688)
• Decouple kernel-candidate pool cap from dispatch budget — #791† (#667)

Profiling / TraceLens

• Restore torch-trace "kernel" category so profiler traces carry GPU info — #777 (closes #769)

Server config

• Clamp sglang --context-length to --max-model-len — #764 (closes #697)

Reporting

• Crash-safe reports/final.json on non-graceful exit — #763 (closes #464)

Portability (docs)

• Local Mode portability groundwork — remove Core42 / WekaFS hard-coding — #779† (#767; docs only, bare-metal installer #802 in review)

Related improvements (area addressed by a merged PR; issue closed as not-planned)

• GEAK landing-rate: op source paths for vLLM v21 / SGLang 0.5.12 (#706†) and trust verified-correctness from timed-out GEAK partial runs (#750†) — (#742)
• Correct stale inherited TRACELENS_ROOT from the kernel-agent env file — #786† (#722)
• GEAK_v4 E2E patchability follow-up (apply + remeasure, aiter harness routing, dispatch context) — #703† (#720; also advances #717)
• Orchestrator idle-timeout + amplified retry for slow reasoning models — #785† (effectively fixes #679)
• Kernel-artifact-bundle work touching the deterministic-TraceLens / agentic path — #782† (#664)

Requirements

(unchanged from v0.7) ROCm platform (MI355X / MI325X / MI300X; MI308X
detection); Docker with AMD ROCm support, or Kubernetes with AMD GPU nodes;
Cursor IDE + Remote SSH; LLM API key (GEAK); OOB API key + base URL; GBrain
base URL + token (Forge); tmux, jq, Node.js >= 18, Claude CLI.

Hyperloom v0.7

Highlights

Forge: a third autonomous kernel-optimization backend (new track)

v0.7's headline is Forge (Kernel-Forge) — a self-driving kernel-optimization backend that joins GEAK and OOB. It runs an autonomous edit→build→bench loop with kernel_kind-aware fellow routing (Triton / HIP / CK / aiter / hipBLASLt / FlyDSL), an aiter compiled-kernel closed loop, honest compile-only skips for non-rewritable kernels, and its own session-breakdown lane. Forge already produces the majority of detected kernels on MI300X runs.

Deterministic GEMM tuning (new track)

A standalone forge-gemm-tune backend lands as the default GEMM-tuning prelude: it auto-detects MoE/dense + precision/quant, selects the applicable tuners, and each tuner's tuned config is independently E2E-validated and stacked (per-tuner KEEP/REVERT, like kernel-opt) instead of bundled — so one bad tuner can't drag down the set.

Knowledge Base: GBrain-backed, 7-tuple canonical identity

The Recipe KB extends its canonical identity from a 5-tuple to a 7-tuple with config-donor warm-replay, and Forge fellows now read cross-KB knowledge directly from the unified GBrain (KernelForge + GEAK + PTAO), with KB-usage provenance surfaced in the session breakdown.

Long-horizon durability + specialist autonomy

Long-run optimization is hardened end-to-end: crash-window recovery, integrate-fault retries instead of first-crash discard, duplicate-optimizer corruption protection, orchestrator/CLI decoupling, and opened-up GPU specialist exploration with serving-disjoint leases — so multi-day runs stay productive and self-correcting.

GEAK / kernel-dispatch reliability, end-to-end

A full RCA sweep on the trace→shape→dispatch pipeline: correct GEAK dispatch attribution (no more mis-bucketing non-GEAK attempts), faithful harnesses with real per-arg dtypes, trace-anchored shape pinning, candidate-artifact retry on shared-storage visibility lag, empty-queue clean skip, and forwarding of GEAK scoring/profiler knobs across the Ray boundary.

Profiling / roofline + TraceLens v0.7

A deterministic (no-LLM) trace-analysis route, TraceLens v0.7.0, profile-scoped OSL control with a serialization-safe capture cap, expert-parallel flag handling for MoE roofline, and eager-boot fallback for the SGLang profile-cuda-graph path.

Reliability: sandbox-hang elimination & real-cluster hardening

LLM streaming reads are now bounded client-side and timed-out subprocess trees are reaped by process group, eliminating the "pod Running but idle" sandbox hang. Plus setup_env race / USER_DATA_PATH corruption fixes, user-uncommitted-change protection before destructive reverts, and invalid/premature zero-gain rejection.

CI: structural model pre-filter & throughput

A shared model-compatibility pre-flight (multimodal / Gemma2 / Phi3-longrope / dual-chunk / ModelOpt-FP8 / FlashInfer / gated / missing-tokenizer) skips doomed models before a session is created, alongside HF-token rotation with 429 backoff, larger daily pools, and 48h long-horizon budgets.

What's included in this release

Forge (Kernel-Forge) backend

• Forge autonomous-loop backend for kernel optimization ( #562 )
• Complete Forge backend with RCA fixes + aiter compiled-kernel closed loop ( #613 )
• GBrain KB integration + compiled fellows default-on ( #630 )
• kernel_kind routing + honest compile-only skip + aiter op_test harness fixes ( #644 )
• Forge as its own session-breakdown lane (geak/oob/forge) ( #580 )
• Stability env defaults baked in + auto-recover stale forge branch ( #566 )
• TraceLens-GEAK .py -> .cu resolver ( #626 )

Deterministic GEMM tuning

• Integrate forge GEMM tuning backend ( #637 )
• Make forge GEMM tuning run on post-#637 sessions ( #659 )
• GEMM tuning source attribution + breakdown/artifact cleanup ( #656 )
• Top-level gemm_tuning breakdown section + session_meta exporter fix ( #625 )
• Fix hardcoded a8w8 block-scale GEMM tuning for non-block-scale targets ( #588 )

Knowledge Base

• Extend canonical identity to 7-tuple + config-donor warm-replay ( #587 )
• Optional substrate-KB assessment enrichment for proposals ( #541 )
• KB-usage provenance + decision-process attribution ( #612 )

Long-horizon orchestration & autonomy

• Long-run durability + autonomous-orchestration overhaul ( #546 )
• Harden long-run durability and open up specialist exploration ( #617 )
• Open up GPU specialist autonomy + fix serving-disjoint leases ( #648 )
• STEPPING: bug fixes, optimizations, and orchestrator/CLI decoupling ( #585 )
• Explore optimization — warm rebench, measurement, phase flags ( #552 )
• Retry kernel-integrate faults instead of discarding on first crash ( #582 )
• Prevent a duplicate optimizer from corrupting a session ( #606 / #592 )
• Optimizer stability fixes; prevent invalid & premature zero-gain results; harden abnormal-run handling ( #604 / #649 / #675 )

GEAK / kernel-dispatch reliability

• Correct GEAK dispatch attribution ( #616 / #601 / #602 )
• Faithful GEAK harnesses — real per-arg dtypes + tolerate remapped analysis.md ( #610 / #609 / #611 )
• Forward GEAK scoring/profiler knobs across Ray + aiter harness shape-pinning ( #633 )
• Candidates retry + empty-queue skip ( #653 / #600 / #598 )
• GEAK harness generation, 1-GPU pinning, USER-NTID gateway header ( #591 / #559 / #557 )
• Pin GEAK to v3.2.1 then v3.2.2 ( #579 / #668 )

Profiling / roofline / TraceLens

• Deterministic (no-LLM) trace-analysis route ( #558 )
• TraceLens v0.7.0 ( #663 ); installer for v0.6.1 ( #577 )
• Profile-scoped OSL control + serialization-safe capture cap ( #673 / #571 / #570 )
• vLLM expert-parallel flag for EP-required MoE roofline ( #671 / #569 )
• SGLang roofline capture_traces/*.json.gz handling ( #575 )
• Eager boot on SGLang profile-cuda-graph seq_lens assert ( #632 )
• TraceLens fixes: round-none, version-string patcher, model normalization ( #618 / #572 / #574 )

Observability & session breakdown

• Proposer attribution + operation_kind across timeline & trace ( #561 )
• Record bypass-discovery runs in kernel_journey ( #563 )
• Emit session breakdown to Langfuse on pre-flight fail-fast ( #641 )

Server-arg / framework injection

• Single reliable server-param injection point ( #573 )
• vLLM patch fallback + server-args propagation ( #652 )
• Dedup vLLM attention-backend + preserve GEAK tunnel URL + custom orchestration model ( #547 / #520 / #521 / #340 )

Reliability & real-cluster hardening

• Bound LLM stream reads and reap process groups to stop sandbox idle hangs ( #646 )
• Resolve setup_env.sh race + USER_DATA_PATH corruption ( #629 )
• Stash user changes before destructive git revert ( #627 / #593 )
• Preflight model gates + Gemma2 roofline shape-discovery workaround ( #581 )
• Preflight skips + baseline/roofline stability ( #564 )

CI / Automation

• Structural model-compatibility pre-filter (multimodal/Gemma2/Phi3/ModelOpt/FlashInfer/gated) ( #642 and follow-ups )
• HF-token rotation with 429 backoff ( #614 )
• Daily pool to 10 models; multimodal submit; sglang routing + MI300X TP rebalance ( #560 / #586 / #608 )
• 48h inferencex-daily budget + long-run prompt hints ( #619 ); restore schedule + 6h task budget ( #676 )
• Filter small / not-found models; raise default target gain to 500% ( #603 / #615 / #631 )
• Tiny-workload paths (sandboxy & hyperloom); 32B/80B TP1 ( #636 / #638 / #662 )
• Candidate pools on /wekafs, cron 2x/day ( #568 ); idna bump; CodeQL scan; added unit tests ( #565 / #596 / #584 )

Docs

• Align public docs and KB/backend wording with current runtime ( #583 )
• Upgrade existing docstrings to Google style ( #595 )
• How-to examples and repo usage docs ( #222 )

Requirements

• ROCm platform (tested on MI355X, MI325X, MI300X; MI308X detection)
• Docker with AMD ROCm support, or Kubernetes with AMD GPU nodes
• Cursor IDE with Remote SSH extension (for local workflows)
• LLM API key for GEAK kernel optimization
• OOB API key + base URL for OOB Agent MCP (Claude Code / Codex / Cursor backends)
• GBrain base URL + token for cross-KB knowledge injection (Forge)
• tmux, jq, Node.js >= 18, Claude CLI (@anthropic-ai/claude-code) for the multi-agent runtime

Hyperloom v0.6

Highlights

Search Space: looser orchestration + long-horizon runs

v0.6 turns the Orchestration theme from v0.5's "search efficiency" to search space: mechanical guardrails are downgraded to advisory so the optimizer drives itself, free-form and cross-domain specialist dispatch lets it explore beyond the fixed action catalogue, and 2–3 day long-horizon optimization with finer-grained start / stop / checkpoint / resume keeps long runs productive and recoverable.

Quantization agent (new track)

A prompt-driven Quark quantization sub-agent lands as an optimization prelude, with Quark enhancements and quantization-agent proposals — quantization joins kernel-opt as a first-class optimization lever.

Knowledge Base: unified and knowledge-graph-backed

cortex-KB and gbrain converge behind a single Recipe KB interface, GBrain is integrated, a Knowledge Graph is wired in, and a KB-evaluation harness is added so warm-start priors can be measured rather than assumed.

GEAK / kernel-optimization reliability, end-to-end

The trace → kernel-shape → GEAK pipeline is hardened so genuinely-good kernels actually reach GEAK and are not silently dropped: cache-invalidation by target type (aiter cpp_itfs / Triton / inductor), recovery of high-GPU-time kernels missed by analysis.md-only extraction, server-patcher idempotency, and parallel GEAK/OOB ladders with per-attempt cache isolation. rocprof-compute roofline and kernel-level roofline detail land alongside.

TraceLens v0.6, CLI + WebUI, and budget-aware roofline

TraceLens v0.6 ships with open-source MAF backfill (GPU microbenchmark), a TraceLens CLI, WebUI standalone/comparative analysis, and roofline that is time-boxed against the total budget.

Multi-node & scale

Multi-node Optimus support comes online and the Arbor mechanism migration is completed.

Observability overhaul

Live Langfuse tracing, full-trace token and conversation logging, phase/step-level observability, a per-session token-consumption breakdown, and an author-time session-breakdown recorder make long runs inspectable in real time.

Reliability & real-cluster hardening

A broad sweep driven by large-scale cluster-run analysis: fast-fail for immediate arg/config errors, log line-buffering so healthy runs no longer look frozen, pod-local dependency roots decoupled from WekaFS, Ray raylet fd-limits, MI308X detection, attention-backend argument hygiene, GEAK container network path to the LLM gateway, and clearer setup/baseline failure classes.

Docs, licensing & coverage

Repo-wide Google-style docstrings with a published Sphinx documentation site, the license relicensed Apache → MIT, requesting-access / SSO docs, and Python test coverage raised to ~91.5%.

What's included in this release

Orchestration & search space

• Orchestration: search space (v0.6 thematic umbrella) — loosen orchestration + free-form / cross-domain specialist dispatch ( #273 )
• 2–3 day long-horizon optimization ( #271 )
• Finer-grained start / stop / checkpoint / resume ( #285 )
• Run-stability follow-ups, stop_reason vocabulary, structural consolidation, and text-coercible multimodal models on the text path

Quantization

• Quark quantization agent as an optimization prelude ( #270 )
• Quark quantization enhancements ( #445 )
• Quantization agent improvement proposals ( #453 )

Knowledge Base

• Unify cortex-KB and gbrain behind one Recipe KB interface ( #455 )
• Integrate GBrain into Hyperloom ( #448 )
• Integrate Knowledge Graph into Hyperloom ( #446 )
• Knowledge Base evaluation ( #447 )

GEAK / kernel optimization

• Invalidate aiter cpp_itfs runtime cache so good kernels aren't discarded ( #458 )
• Generalize integrate-side cache-invalidation by target type (cpp_itfs / Triton / inductor) ( #484 )
• Recover high-GPU-time "other"-bucket kernels dropped by analysis.md-only extraction ( #514 )
• server-patcher: skip already-applied member instead of fail-softing the set ( #505 )
• Inject --moe-runner-backend triton for MoE baselines on AMD ( #512 )
• Honor KERNEL_OPT_BACKENDS in backend selection ( #418 )
• GEAK Ray cross-Python recovery, CUDA-graph attribution, task-group dedupe ( #432 / #431 / #420 )
• Patchability gate hardening + bind GEAK harness to TraceLens-captured shapes ( #381 / #382 )
• Lower the kernel-opt micro speedup KEEP gate to 1.05 ( #442 )
• Kernel-level roofline details ( #406 )

TraceLens / roofline

• Backfill MAF for open-source TraceLens via GPU benchmark ( #390 )
• TraceLens v0.6.0 release integration ( #474 )
• TraceLens CLI support ( #274 )
• TraceLens Agent WebUI standalone / comparative analysis ( #354 )
• Time-box the roofline torch-profiler phase against total budget ( #467 )

Multi-node & scale

• Multi-node Optimus support ( #440 )
• Complete the Arbor mechanism migration ( #450 )

Observability

• Phase/step-level observability in inference_optimizer ( #266 )
• Session token-consumption breakdown ( #443 )
• Live Langfuse push + token_usage breakdown + /workspace artifact packaging; full-trace token + conversation logging + CLOSE bundle

Reliability & real-cluster hardening

• Fast-fail / escalation for subprocess_nonzero arg-error baselines (umbrella) ( #522 )
• Emit reports/final.json on time-exhaust ( #464 )
• Surface the real EngineCore error in the final summary instead of a benign WARN ( #465 )
• Optimizer proposes invalid --attention-backend value → fast reject ( #519 )
• Duplicate --attention-backend flags in launch command ( #520 )
• GEAK container network path to the LLM gateway ( #521 )
• Intermittent SGLang profiler-patch application ( #516 )
• TRACELENS_ROOT not set / TraceLens root not found ( #518 )
• CUDA-graph profiling working-dir ENOENT ( #523 )
• vLLM EngineCore initialization failure surfacing ( #524 )
• Main run log buffers heavily / appears frozen — line-buffer stdout/stderr ( #468 )
• HYPERLOOM_KERNEL_AGENT_ROOT unset stalls optimization ( #469 )
• geak_prompt_patcher upstream-example drift warning ( #470 )
• Requested FP8 precision silently runs BF16 ( #421 / #435 )
• Ray raylet unstable at default fd-limit; needs nofile>=65536 ( #433 )
• MI308X detection + runner mapping ( #376 )
• Clone open-source deps into a pod-local root decoupled from USER_DATA_PATH

Onboarding / Docs / Licensing

• Requesting-access / Primus-SaFE SSO instructions ( #472 )
• Public documentation site ( #222 )
• Repo-wide Google-style docstrings + Sphinx site ( #398 )
• Relicense Apache → MIT
• Raise overall Python test coverage to ~91.5% ( #221 )

CI / Automation

• Daily top-8 model inference-optimization CI ( #452 )
• Sequential optimize cron over the downloads>100 pool
• CI NFS session-liveness + artifact-grace hardening; Pre-flight CI testing ( #473 )

Closed as Not Planned (skipped this cycle)

• baseline_vllm.yaml uses VLLM_ATTENTION_BACKEND env var ( #462 ) — ops-side prestart_patch.py misconfiguration; the env var is a no-op in vLLM and was never selected
• Baseline slow-start / HIP-OOM retry cascade ( #463 ) — root cause was the closed #462; existing reap/recover/retry already covers it, re-scoped to a follow-up "gen-tok/s sanity floor" enhancement
• Check if GEAK already applied patch ( #392 ) — gated on GEAK's patch-application support landing upstream first; revisit once it stabilizes

Requirements

• ROCm platform (tested on MI355X, MI325X, MI300X; MI308X detection added)
• Docker with AMD ROCm support, or Kubernetes with AMD GPU nodes
• Cursor IDE with Remote SSH extension (for local workflows)
• LLM API key for GEAK kernel optimization
• OOB API key + base URL for OOB Agent MCP (Claude Code / Codex / Cursor backends)
• tmux, jq, Node.js >= 18, Claude CLI (@anthropic-ai/claude-code) for the multi-agent runtime

Hyperloom v0.5

Highlights

Orchestration: vocabulary unification + search efficiency

v0.5 advances the "search efficiency" theme ( #272 ): the optimizer's action/state vocabulary converges onto the unified explore grid-runner and the EXPLORE / specialist fan-out gains parallel headroom. New Atom framework support ( #336 ) and a soft Dynamic Action cross-domain deep-dive ( #335 ) widen the search surface.

GEAK & kernel optimization, deeper

GEAK GEMM tuning ( #331 ) and FlyDSL kernel-optimization integration ( #211 ) land, and kernel-level roofline support + quality fixes ( #330 ) sharpen targeting. Input quality to GEAK is tightened across the board: hot-kernel candidates are filtered to backend-routable kernels only ( #314 ), the kernel-opt prompt drops the bloated full analysis.md ( #307 ), the GEAK budget no longer forces quick-mode timing under GEAK_RUN_MODE=full ( #337 ), and kernel batch parallelism adapts to smaller pods ( #338 ).

Knowledge Base productization

The v0.4 Knowledge Base Service moves toward operations with KB Productization & Data Maintenance ( #333 ) and KB Recipe Ingestion ( #332 ).

Profiling, TraceLens & Dashboard

TraceLens v0.5 ( #358 ); a patched profiler docker image that captures HipGraphLaunch kernels so optimization-loop traces are complete ( #352 ); profiling information for all Hyperloom models ( #346 ); kernel roofline on the dashboard ( #345 ); and a Session Breakdown enhancement spanning auto-collection, alerting, TraceLens/GEAK detail capture, and kernel roofline ( #334 ).

Stability & bug fixes

v0.5 closes a batch of orchestration / runtime defects surfaced by v0.4 runs: local-mode KERNEL phase failing to dispatch GEAK plus TP variants leaking past visible-device scope ( #341 ); integrate_handler early-out on a missing base_tput that was already in SharedState ( #319 ); per-cluster call_timeout_s for the Claude / Codex backends ( #318 ); kernel-agent misreading a mini-swe-agent step-header $X.XX as a budget cap ( #317 ); arbor zombie-process cleanup on failure ( #268 ); and the README cert-install script on RHEL / CentOS hosts ( #328 ).

What's included in this release

Orchestration & search efficiency

• Orchestration: search efficiency (v0.5 thematic umbrella) ( #272 )
• Atom Framework Support ( #336 )
• Dynamic Action Support ( #335 )

GEAK / kernel optimization

• GEAK GEMM Tuning ( #331 )
• FlyDSL kernel-optimization integration ( #211 )
• Kernel-level Roofline support + roofline quality fix ( #330 )
• hot_kernels filtered to GEAK/OOB-routable kernels only ( #314 )
• kernel-opt prompt no longer appends the full analysis.md ( #307 )
• GEAK budget default no longer forces quick-mode under GEAK_RUN_MODE=full ( #337 )
• Kernel batch parallelism default (8) adapted for smaller pods ( #338 )

Knowledge Base

• KB Productization & Data Maintenance ( #333 )
• KB Recipe Ingestion ( #332 )

Profiling / TraceLens / Dashboard

• TraceLens release for v0.5 ( #358 )
• Profiler docker image patch — captures HipGraphLaunch kernels ( #352 )
• Profiling information for all Hyperloom models ( #346 )
• Display Kernel Roofline on Hyperloom Dashboard ( #345 )
• Session Breakdown enhancement: auto-collection, alerting, TraceLens/GEAK detail capture, kernel roofline ( #334 )

Stability / runtime fixes

• Local mode: KERNEL phase never dispatches GEAK + TP variants leak past visible-device scope ( #341 )
• integrate_handler early-outs on missing base_tput even when SharedState has it ( #319 )
• Backends: per-cluster call_timeout_s for Claude / Codex ( #318 )
• kernel-agent misreads mini-swe-agent step-header $X.XX as a budget cap ( #317 )
• arbor leaves zombie CPU/GPU processes on failure ( #268 )
• README cert-install script fails on RHEL / CentOS hosts ( #328 )

Still open / carried forward

These v0.5-labeled items are not yet landed and are candidates to carry into the next cycle:

• Comparative Analysis integration into the TraceLens Agent on WebUI ( #355 )
• TraceLens Agent invocation update + dual-repo from open-sourcing ( #353 )
• Hyperloom–GEAK integration roadblocks: 1.0× per-kernel speedup + post-TraceLens polling + 12h+ FULL LOCAL runs ( #349 )
• GEAK two-level (per-kernel + E2E) speedup validation — artifact handoff ( #348 )

Requirements

• ROCm platform (tested on MI355X, MI325X, MI300X)
• Docker with AMD ROCm support, or Kubernetes with AMD GPU nodes
• Cursor IDE with Remote SSH extension (for local workflows)
• LLM API key for GEAK kernel optimization
• OOB API key + base URL for OOB Agent MCP (Claude Code / Codex / Cursor backends)
• tmux, jq, Node.js >= 18, Claude CLI (@anthropic-ai/claude-code) for the multi-agent runtime

Hyperloom v0.4

Highlights

Hyperloom v2 architecture lands

v0.4 is a substantial v2 leap: end-to-end Model Auto-Optimize, Framework Agent integration, Agent Kernel Arena, the Self-Evolving Skills & Memory layer ramping past the v0.3 proposal-only stage, and the first-iteration Hyperloom Knowledge Base Service. Multi-Node CI/CD comes online as a first-class capability.

Robustness Agent overhaul

A foundation rewrite ships 13 independent signal detectors (preflight, kernel-pipeline, gpu-leak, decision-audit, critic-health, budget, state-integrity, repeated-payload, aiter-jit, external-deps, progress, event, local-health), an action ladder for graduated response, run finalize / postmortem, and a persistent state store. Closes long-standing issues like SQLite corruption on multi-day runs, validate_stack retry / TP miscalibration, and arbor orchestrator silent exits.

TraceLens ↔ GEAK integration tightened

Multi-root InferenceX patcher with post-profile trace structure validation; SGLang 0.5.11 patch parity (alongside 0.5.9); TraceLens prose + source-function aggregation surfaced to GEAK during kernel rewrite; duplicate-markdown and standalone-upload bugs resolved. Server patcher robustness landed in two follow-up rounds.

GEAK Quick vs Full mode

GEAK now supports a Quick (latency-first) vs Full (deeper search) mode, and on the recipe side learns to recommend kernel-fusion opportunities. Old 1.50x early-exit and hard-coded homogeneous-mode hints in the kernel-optimization prompt are obsoleted.

Critic Agent (first iteration)

A new Critic Agent backend ships with decision reviewer, prompt builder, robustness priors, and web-search support for evidence retrieval during decision review.

CI / Automation

Issue ball-tracker and stale-issue auto-management workflows ship; CI scheduling is hardened (top-2000 candidate pool, GLM5 remote mode, Windows-safe NFS_ROOT, per-run optimization-result publishing).

What's included in this release

Hyperloom v2 architecture

• Hyperloom v2: Multi-Node Support ( #162 )
• Hyperloom v2: Multi-Node CI/CD ( #163 )
• Model Auto-Optimize ( #164 ) and Visualization ( #165 )
• Hyperloom Framework Agent ( #166 )
• Hyperloom v2: Integrate with Framework Agent ( #167 )
• Agent Kernel Arena Integration ( #168 )
• Self-Evolving Skills & Memory v0.4 ( #169 )
• Hyperloom Knowledge Base Service ( #171 )

TraceLens / Profiling

• Multi-root InferenceX patcher + post-profile trace structure validator ( #210 )
• TraceLens v0.3.1 per-version SGLang patch subdirs (0.5.9 / 0.5.11)
• Surface TraceLens prose + source-function aggregation to GEAK ( #204 )
• Follow-up to #194 / PR #200: 6 robustness gaps in TraceLens server patcher ( #205 )
• TraceLens triple-duplicate markdown fix ( #203 / #209 )
• TraceLens Standalone Analysis upload error ( #220 )
• TraceLens analysis FileNotFoundError on pre-restructure checkouts ( #250 )
• New TraceLens v0.3.1 ( #255 ) — superseded by per-version patches

Robustness Agent

• Action ladder + RCA engine + finalize / postmortem + persistent state store
• Signal detectors: preflight, kernel-pipeline, gpu-leak, decision-audit, critic-health, budget, state-integrity, repeated-payload, aiter-jit, external-deps, progress, event, local-health
• SQLite DB corruption fix during long runs ( #242 )
• validate_stack stop retrying after N consecutive failures ( #245 )
• validate_stack wrong TP when user pins GPU subset via ROCR_VISIBLE_DEVICES ( #248 )
• arbor dfs orchestrator silent exit fix ( #275 )
• arbor recipe filename uses model name (not dir basename) ( #254 )

GEAK

• Integrate GEAK Quick vs Full mode ( #196 )
• Recommend kernel-fusion opportunities for GEAK optimization ( #195 )
• Obsolete 1.50x early-exit + hard-coded homogeneous-mode hints from kernel-optimization prompt ( #260 )

Critic Agent

• Critic Agent backend (decision reviewer + prompt builder + robustness priors)
• Web search support for evidence retrieval during decision review ( #170 )

Onboarding / Local Mode / Install

• Local-mode setup script local_setup.sh ( #208 )
• install.sh hardening: /opt/venv preference + ROCm-torch wheel gate + old-pip detection + PYTHON/PATH threading ( #232 )
• Ship optimizer_runs/robustness_monitor.sh.example ( #233 )
• GEAK config rendered by install.sh now includes run.budgets block ( #246 )

Multi-node + resource scheduling

• Time-multiplex GPUs between model and agents — TP=8 single-node optimization ( #253 )

UI & Reporting

• Final report and log files download flow restored ( #219 )
• OOB: Cursor as a backend option ( #143 )
• Roofline metrics on the dashboard ( #283 )

Errors / Permissions

• Core42 "user not allowed to create workload" ( #202 )
• LLM Gateway Binding failed — Primus.00001 internal error ( #256 )

CI / Automation

• Issue ball-tracker + stale-issue auto-management workflows
• CI: top-2000 candidate pool; GLM5 remote mode (#230); per-run optimization-result publishing; sandbox_image support; harden scheduled model selection / metrics parsing (#235); breakdown collector fixes (#236); tools-injection cleanup + Windows NFS_ROOT (#227); README + .env.template fixes (#228)

Extensibility direction

• Platform stays as opinionated curated baseline; user-supplied skills and tool enhancements run via local mode (resolved #122 + #106). No platform-side plugin-host / dynamic-skill-selection contract.

Closed as Not Planned (skipped this cycle)

• Detailed instructions to run Hyperloop in Local Mode ( #231 ) — superseded by local_setup.sh + README Quickstart rewrite

Requirements

• ROCm platform (tested on MI355X, MI325X, MI300X)
• Docker with AMD ROCm support, or Kubernetes with AMD GPU nodes
• Cursor IDE with Remote SSH extension (for local workflows)
• LLM API key for GEAK kernel optimization
• OOB API key + base URL for OOB Agent MCP (Claude Code / Codex / Cursor backends)
• tmux, jq, Node.js >= 18, Claude CLI (@anthropic-ai/claude-code) for the multi-agent runtime

Hyperloom v0.3

Highlights

Multi-Agent architecture: Sprint + Marathon unified

The optimizer's execution backbone is rebuilt: the previous single-agent harness is replaced by a multi-agent pipeline that unifies Sprint and Marathon under one orchestration model. Workloads now flow through a P0+P1+P2 + kernel-agent layout with each agent in its own Claude CLI process and JSONL-based IPC, inheriting the 24h-stable, context-isolated runtime first introduced by Marathon in v0.2 — but now applied to Sprint as well, with shared scheduling, memory, and trace plumbing.

TraceLens enters the E2E optimization loop

v0.2 introduced TraceLens as a standalone analysis surface; v0.3 takes the next step and wires TraceLens directly into the end-to-end inference-optimization loop. Profiling, trace splitting, agent invocation, and output parsing all happen inline now — the optimizer reasons over fresh trace evidence between actions instead of relying on stale or summarized data. The integration is pinned against release/hyperloom_integration_v0.3. A new TraceLens Agent Debug Mode also exposes the full StreamJSON event stream for offline replay and diff against local runs.

GEAK gets memory: RAG + Cross-Sessions

Two long-requested capabilities land together. GEAK RAG enablement lets GEAK retrieve past optimization knowledge during kernel-rewrite reasoning. Across-Sessions Memory for GEAK carries learned heuristics, kernel patterns, and outcome data from one optimization session to the next, breaking the "every run starts from scratch" pattern. A new internal Memory Service provides the storage primitives both features sit on, with a Long-Term Memory layer for cross-session knowledge retention.

Self-Evolving Skills (first iteration)

Skills are no longer fully static. The first version of self-evolving skills ships in v0.3: skills accumulate session-level evidence and propose their own incremental updates. The v0.3 surface is intentionally narrow (proposal + manual review); the deeper regression-aware auto-update loop is scoped to v0.4.

Roofline-aware kernel optimization priority

The kernel-classification heuristic that decides "what to optimize first" is overhauled. Instead of a fixed 5-tier kernel-class priority (triton > aiter_ck > framework > comm > hipblaslt), v0.3 picks targets via roofline analysis combined with E2E time share. This directly addresses the regression where MoE / aiter kernels were either silently skipped (vendor mis-classification) or mis-prioritized away from real bottlenecks. Hyperloom now also passes complete kernel metadata to GEAK — shape, dtype, backend, runtime args, env vars, and kernel-specific parameters — replacing the previous path-only contract.

Multi-Node support comes online

Hyperloom now supports optimization across multiple nodes, with an accompanying Multi-Node CI/CD pipeline that exercises the same A/B testing mechanism single-node runs already use. This unlocks training and large-model workloads that don't fit on a single node and keeps the multi-node path continuously validated.

---

What's included in this release

New integrations and capabilities:

• Hyperloom Single-Agent → Multi-Agent transition with unified Sprint + Marathon orchestration (#113)
• Multi-agent skill layout (P0 + P1 + P2 + kernel-agent integration)
• Multi-Node Support (#18)
• TraceLens Agent invocation in E2E mode (#124) with inline output parsing (#125), trace splitting before analysis (#127), and Torch profiler setup integrated into the loop (#126)
• TraceLens internal pinned to release/hyperloom_integration_v0.3 (#148)
• TraceLens Agent Debug Mode (#78) — full StreamJSON exposure for offline debugging
• GEAK RAG enablement (#36)
• GEAK Across-Sessions Memory
• Memory Service (#92) — internal storage primitives
• Long-Term Memory (#112)
• Cross-session memory in GEAK with KB reuse from Hyperloom (#140)
• Self-Evolving Skills (first iteration — proposal-only) (#111)
• Decision-making improvement for kernel optimization priority based on roofline + E2E time share (#116)
• Improper kernel categorization fix — GEAK no longer misses high-priority MoE / aiter kernels (#144)
• Provide Complete Kernel Metadata for GEAK Invocation — pass shape, dtype, backend, runtime args, env vars, and kernel-specific parameters in the GEAK call (#175)
• Code-quality instructions for the patching skill (#46)
• Download All Files UI button — one-click download of all session artifacts (#128)

CI/CD and automation:

• Multi-Node CI/CD pipeline (#17)
• Validated API-mode Remote Memory Backend in the V2 environment (#114)

Bug fixes and stability:

• Sessions got stuck mid-optimization (#118)
• Hyperloom UI proceeded with optimization even when TraceLens profiling was missing (#120)
• Agent stuck at "Download model from HuggingFace" (#121)
• Marathon File content (25521 tokens) exceeds maximum allowed tokens (25000) crash (#132)
• LLM losing all previous context after report generation completes (#82)
• Marathon-suggested optimizations were poor / non-actionable (#105)
• Inference-optimization skill skipped MoE kernels that were mis-tagged as vendor (#89)
• Job scheduled on a node with no GPUs available (#131)
• OCI MCP SSE endpoints returning Primus SaFE HTML instead of text/event-stream (#134)
• Lack of dedicated resources impacting analysis result (#56)
• Session Files Count Varying in TraceLens Standalone Analysis (#79)
• Optimizations proposed were questionable and could not be merged upstream (#53)
• Requested model was ignored and well-known models were optimized instead (#52)
• Baseline / benchmark_serving fails on Qwen3-Next FP8: tokenizer all_special_tokens_extended (#147)
• Dockerfile defaults block MI355X + intellikit double-install fights GEAK (#182)
• GEAK Ray GPU isolation broken in LOCAL mode (#181)
• model_class: litellm defaults route claude-* to Anthropic causing 401 against AMD gateway (#184)
• --exit-immediately was not passed when invoking the GEAK CLI; GEAK was hanging in TUI and SIGTERMed before persisting patches (#188)
• GEAK prompt in task.md hardcoded MI300X/gfx942 hardware notes despite the optimizer knowing the actual GPU type (#189)
• Differences in profiling between TraceLens and Hyperloom — steady-state window formulas, num_prompts override, splitter --R, vLLM/SGLang flag drift (#194)

---

Requirements

• ROCm platform (tested on MI355X, MI325X, and MI300X)
• Docker with AMD ROCm support, or a Kubernetes cluster with AMD GPU nodes
• Cursor IDE with the Remote SSH extension (for local workflows)
• LLM API key for GEAK kernel optimization
• OOB API key and base URL for OOB Agent MCP (Claude Code / Codex backends)
• tmux, jq, Node.js >= 18, and Claude CLI (@anthropic-ai/claude-code) for the multi-agent runtime

Hyperloom v0.2

Highlights

TraceLens/OOB/Magpie standalone comes online

Hyperloom now ships dedicated standalone workflows for TraceLens, OOB, and Magpie — each usable independently of the full end-to-end optimization pipeline. The TraceLens Standalone UI supports three input modes — Default (training / non-vLLM/SGLang eager), Inference with Eager (vLLM/SGLang), and Inference with Graph Capture (vLLM/SGLang with a capture folder) — and produces structured analysis reports under /workspace/hyperloom/standalone_analysis.md. The OOB and Magpie standalone paths follow the same invocation pattern, each running its own agent with the appropriate CLI parameters and collecting per-run artifacts accordingly.

Sandbox Queue for user-facing workload scheduling

When a user's Hyperloom request exceeds available cluster resources, the run is no longer rejected — it is placed into a Sandbox Queue, and the user sees their live queue position directly in the UI. This replaces the previous "resource full → fail fast" behavior with deterministic FIFO scheduling against the finite sandbox pool (currently enabled on the oci-slc cluster).

Agent runtime hardening for long-running sessions

The executor's permission model has been rebuilt: the previous wildcard allow-list (which did not match Claude Code built-in tools) is replaced by an explicit allow-list covering Bash, Write, Edit, and other required tools. Agent invocation prompts now enforce strict step order, mandate independent Task subagents for Step 6 and Step 7 (context isolation), and require findings files to follow the shared template. As a result, LLM-heavy categories such as kernel fusion are no longer silently skipped, and subagent hangs caused by permission denial are resolved.

CI/CD with inference A/B testing

Hyperloom ships an inference A/B test workflow integrated into its single-node CI/CD pipeline, allowing optimization proposals to be validated against a baseline automatically before promotion. Combined with the new auto-labeling system (GitHub Models LLM plus a rule-based engine) and project-board automation, the full optimization loop from issue filing to validated CI results is now end-to-end traceable.

Marathon Inference Launcher (3-pane tmux architecture)

Hyperloom introduces a new Marathon Inference Launcher that runs the inference-optimization pipeline as three independent Claude CLI agents in tmux panes (orchestrator, kernel-manager, watchdog), coordinating via JSONL files on shared NFS. This replaces the previous Python harness, provides independent context windows per agent (surviving 24h runs), and supports auto-restart through the CLI --continue loop.

White-box visibility: Root Cause & Pending Cause Agents

Two new supervisory agents make Hyperloom white-box at both ends of the user experience. The Root Cause Agent watches the Marathon optimization loop for failures, diagnoses the cause, and writes actionable guidance back into the next retry prompt — failures stop being binary pass-or-discard and become constrained retries. The Pending Cause Agent does the analogous thing on the queue side: when a user's run is waiting, it surfaces a concrete reason why. Together they replace two previously opaque states — "something failed" and "still waiting" — with explainable, user-controllable answers.

---

What's included in this release

New integrations and capabilities:

• TraceLens Standalone Analysis UI with three input modes (Default / Inference Eager / Graph Capture)
• TraceLens profiler configuration for quality downstream analysis (steady-state windowing, shape/callstack/graph-capture annotations)
• Hyperloom Local-Local mode for simpler development iteration
• SaFE Plugin, Claw MCP, Magpie Plugin, Sandbox Phase 2, and Sandbox Queue
• Marathon Inference Launcher with the new 3-pane tmux architecture
• Parallel GEAK + LLM kernel optimization race strategy
• GEAK tracing skill for LLM cost attribution
• OOB (Out-of-Box) optimization support
• Claw mode (SaFE RayJob) support in the inference-optimization skill
• MLPerf training optimization skill
• Root Cause / Pending Cause Analysis workstream

CI/CD and automation:

• Single-node inference optimization CI/CD pipeline
• Inference A/B test workflow
• Auto-labeling system combining GitHub Models LLM with a rule-based engine
• Project board automation workflow
• Pinning of IntelliKit to a specific SHA for reproducible Docker builds

Usability and UI:

• Share chat access with the team via link
• Refreshed user-interaction patterns
• Better visibility of the chat 'delete' button on the light theme
• 308x support with configurable model services/clients and user-specified benchmarks
• Clearer code-quality instructions for the patching skill

Bug fixes and stability:

• TraceLens integration now pinned to release/hyperloom_integration_v0.2; restored generation of outputs such as CustomCollective_fwd.csv
• Subagent file-permission denials in the executor resolved via explicit tool allow-list (32 Bash + 16 Write denials eliminated)
• Large file and zip-folder upload supported in TraceLens Standalone UI with automatic decompression on the server
• Removed a source of leaked HF tokens
• BF16 / FP16 precision now supported in addition to FP8 / FP4
• Fixed a case where a requested model was silently replaced by a well-known model
• Fixed false optimization-progress reports when workloads never actually started
• Fixed processes getting stuck during baselining and GEAK tasks
• Fixed multi-node training hang on MI355X after 2 hours
• Resolved session file count inconsistencies and session output UI issues in Standalone Analysis
• Addressed questionable / non-mergeable optimization patches produced by the agent

---

Requirements

• ROCm platform (tested on MI355X and MI308X)
• Docker with AMD ROCm support, or a Kubernetes cluster with AMD GPU nodes
• Cursor IDE with the Remote SSH extension (for local workflows)
• LLM API key for GEAK kernel optimization
• OOB API key and base URL for OOB Agent MCP (Claude Code / Codex backends)
• tmux, jq, Node.js >= 18, and Claude CLI (@anthropic-ai/claude-code) if using the Marathon Launcher