Skip to content

Hyperloom v0.3

Choose a tag to compare

View all tags
@chaojhou chaojhou released this 14 May 04:27
v0.3
b5f41c0

Highlights

Multi-Agent architecture: Sprint + Marathon unified

The optimizer's execution backbone is rebuilt: the previous single-agent harness is replaced by a multi-agent pipeline that unifies Sprint and Marathon under one orchestration model. Workloads now flow through a P0+P1+P2 + kernel-agent layout with each agent in its own Claude CLI process and JSONL-based IPC, inheriting the 24h-stable, context-isolated runtime first introduced by Marathon in v0.2 — but now applied to Sprint as well, with shared scheduling, memory, and trace plumbing.

TraceLens enters the E2E optimization loop

v0.2 introduced TraceLens as a standalone analysis surface; v0.3 takes the next step and wires TraceLens directly into the end-to-end inference-optimization loop. Profiling, trace splitting, agent invocation, and output parsing all happen inline now — the optimizer reasons over fresh trace evidence between actions instead of relying on stale or summarized data. The integration is pinned against release/hyperloom_integration_v0.3. A new TraceLens Agent Debug Mode also exposes the full StreamJSON event stream for offline replay and diff against local runs.

GEAK gets memory: RAG + Cross-Sessions

Two long-requested capabilities land together. GEAK RAG enablement lets GEAK retrieve past optimization knowledge during kernel-rewrite reasoning. Across-Sessions Memory for GEAK carries learned heuristics, kernel patterns, and outcome data from one optimization session to the next, breaking the "every run starts from scratch" pattern. A new internal Memory Service provides the storage primitives both features sit on, with a Long-Term Memory layer for cross-session knowledge retention.

Self-Evolving Skills (first iteration)

Skills are no longer fully static. The first version of self-evolving skills ships in v0.3: skills accumulate session-level evidence and propose their own incremental updates. The v0.3 surface is intentionally narrow (proposal + manual review); the deeper regression-aware auto-update loop is scoped to v0.4.

Roofline-aware kernel optimization priority

The kernel-classification heuristic that decides "what to optimize first" is overhauled. Instead of a fixed 5-tier kernel-class priority (triton > aiter_ck > framework > comm > hipblaslt), v0.3 picks targets via roofline analysis combined with E2E time share. This directly addresses the regression where MoE / aiter kernels were either silently skipped (vendor mis-classification) or mis-prioritized away from real bottlenecks. Hyperloom now also passes complete kernel metadata to GEAK — shape, dtype, backend, runtime args, env vars, and kernel-specific parameters — replacing the previous path-only contract.

Multi-Node support comes online

Hyperloom now supports optimization across multiple nodes, with an accompanying Multi-Node CI/CD pipeline that exercises the same A/B testing mechanism single-node runs already use. This unlocks training and large-model workloads that don't fit on a single node and keeps the multi-node path continuously validated.

What's included in this release

New integrations and capabilities:

  • Hyperloom Single-Agent → Multi-Agent transition with unified Sprint + Marathon orchestration (#113)
  • Multi-agent skill layout (P0 + P1 + P2 + kernel-agent integration)
  • Multi-Node Support (#18)
  • TraceLens Agent invocation in E2E mode (#124) with inline output parsing (#125), trace splitting before analysis (#127), and Torch profiler setup integrated into the loop (#126)
  • TraceLens internal pinned to release/hyperloom_integration_v0.3 (#148)
  • TraceLens Agent Debug Mode (#78) — full StreamJSON exposure for offline debugging
  • GEAK RAG enablement (#36)
  • GEAK Across-Sessions Memory
  • Memory Service (#92) — internal storage primitives
  • Long-Term Memory (#112)
  • Cross-session memory in GEAK with KB reuse from Hyperloom (#140)
  • Self-Evolving Skills (first iteration — proposal-only) (#111)
  • Decision-making improvement for kernel optimization priority based on roofline + E2E time share (#116)
  • Improper kernel categorization fix — GEAK no longer misses high-priority MoE / aiter kernels (#144)
  • Provide Complete Kernel Metadata for GEAK Invocation — pass shape, dtype, backend, runtime args, env vars, and kernel-specific parameters in the GEAK call (#175)
  • Code-quality instructions for the patching skill (#46)
  • Download All Files UI button — one-click download of all session artifacts (#128)

CI/CD and automation:

  • Multi-Node CI/CD pipeline (#17)
  • Validated API-mode Remote Memory Backend in the V2 environment (#114)

Bug fixes and stability:

  • Sessions got stuck mid-optimization (#118)
  • Hyperloom UI proceeded with optimization even when TraceLens profiling was missing (#120)
  • Agent stuck at "Download model from HuggingFace" (#121)
  • Marathon File content (25521 tokens) exceeds maximum allowed tokens (25000) crash (#132)
  • LLM losing all previous context after report generation completes (#82)
  • Marathon-suggested optimizations were poor / non-actionable (#105)
  • Inference-optimization skill skipped MoE kernels that were mis-tagged as vendor (#89)
  • Job scheduled on a node with no GPUs available (#131)
  • OCI MCP SSE endpoints returning Primus SaFE HTML instead of text/event-stream (#134)
  • Lack of dedicated resources impacting analysis result (#56)
  • Session Files Count Varying in TraceLens Standalone Analysis (#79)
  • Optimizations proposed were questionable and could not be merged upstream (#53)
  • Requested model was ignored and well-known models were optimized instead (#52)
  • Baseline / benchmark_serving fails on Qwen3-Next FP8: tokenizer all_special_tokens_extended (#147)
  • Dockerfile defaults block MI355X + intellikit double-install fights GEAK (#182)
  • GEAK Ray GPU isolation broken in LOCAL mode (#181)
  • model_class: litellm defaults route claude-* to Anthropic causing 401 against AMD gateway (#184)
  • --exit-immediately was not passed when invoking the GEAK CLI; GEAK was hanging in TUI and SIGTERMed before persisting patches (#188)
  • GEAK prompt in task.md hardcoded MI300X/gfx942 hardware notes despite the optimizer knowing the actual GPU type (#189)
  • Differences in profiling between TraceLens and Hyperloom — steady-state window formulas, num_prompts override, splitter --R, vLLM/SGLang flag drift (#194)

Requirements

  • ROCm platform (tested on MI355X, MI325X, and MI300X)
  • Docker with AMD ROCm support, or a Kubernetes cluster with AMD GPU nodes
  • Cursor IDE with the Remote SSH extension (for local workflows)
  • LLM API key for GEAK kernel optimization
  • OOB API key and base URL for OOB Agent MCP (Claude Code / Codex backends)
  • tmux, jq, Node.js >= 18, and Claude CLI (@anthropic-ai/claude-code) for the multi-agent runtime
Assets 2
Loading