Skip to content

Hyperloom v0.9

Choose a tag to compare

View all tags
@chaojhou chaojhou released this 17 Jul 10:10
v0.9
7b2579e

Highlights

Ray-managed single-node execution

Every GPU/serving unit (baseline, profile/roofline, explore, sweep, conc_sweep, and needs_gpu specialists) now runs on a Ray-managed GPU lease, with Ray owning GPU queueing, device isolation, and a whole-machine serving mutex. The path fails fast on an infeasible cluster, times out stuck specialist scheduling, detects dead actors, and keeps the raylet stable across per-round server reboots. Multi-node is unchanged (gated off).

Accuracy-gate & eval-result integrity

lm-eval output is wired to a session-scoped EVAL_RESULT_DIR; a baseline that produces no accuracy verdict now hard-stops instead of optimizing an unvalidated baseline; leaked eval results are salvaged back from the InferenceX checkout / local-disk mirror; and session-breakdown attribution no longer fabricates credit from a seeded stack.

Provider-direct LLM configuration

Hyperloom runs against Anthropic + DeepSeek directly (provider-only paths), with env-driven, consistent gateway auth/endpoint resolution and case-insensitive Anthropic-endpoint handling; the Critic can run over the native provider endpoint.

Model-path & workload-default consistency

A single --model value (local path or HF repo id) resolves identically across baseline, roofline, and the kernel agent; prompt-stated ISL/OSL/CONC/TP are honored as flags instead of silently defaulting; and the model_arch freshness guard is org-aware across HF-cache snapshot paths.

Kernel / Forge / GEAK

forge-fusion is adopted end-to-end with hardened subprocess timeouts, GEAK v4 installs via pip/one-click, advertised kernel backends are aligned with the runtime, and a TraceLens-free bypass benchmark harness ships as a Magpie drop-in for text-gen + xDiT.

Framework agent

The FRAMEWORK phase gains cross-framework rating + PR-KB discovery, a flag-gated config-exploration subphase, and a candidate-free local-exploration arm so a dry PR feed no longer wastes the phase.

What's included

Accuracy gate & eval integrity

  • Wire EVAL_RESULT_DIR for lm-eval output + guard stale eval on RUN_EVAL-off - #885 (addresses #909)
  • Stop the run on baseline accuracy failure; revert after baseline - #923
  • Fix lm-eval cleanup deleting Magpie traces - #907 (closes #904)
  • Salvage leaked eval results / eval-dest redirect on the baseline path - #925, #942 (closes #927; addresses #934)
  • Session-breakdown attribution de-fabrication (A1/A2/A3) - #921

Model path & workload defaults

  • Resolve HF repo-id --model to a local dir across all in-process readers - #886
  • Config-driven vLLM --block-size + resolve repo-id in compat preflight - #900
  • Honor prompt-stated workload arg defaults (ISL/OSL/CONC/TP) - #908 (addresses #903; issue open)
  • Case-insensitive Anthropic endpoint + org-aware model_arch guard - #936 (closes #929, #930)

Provider-direct LLM

  • Single-gateway fallback + env-driven gateway auth/endpoint - #841, #867
  • Provider-only execution paths + native-provider Critic - #860, #864

Ray-managed execution

  • Unify single-node GPU/serving on Ray + raylet-stability fix - #910
  • Ray robustness: infeasible fail-fast, sched-timeout, dead-actor detect - #937
  • Enable macro loops for short runs - #940

Kernel / Forge / GEAK / TraceLens

  • forge-fusion E2E adoption + subprocess-timeout hardening - #838, #878
  • GEAK v4 one-click / pip git+ install - #839, #888
  • Align advertised kernel backends with runtime - #897
  • TraceLens-free bypass benchmark harness (text-gen + xDiT) - #801, #872
  • Canonicalize TraceLens graph-capture attribution before GEAK dispatch - #806 (closes #805)

Framework agent

  • Cross-framework rating + PR-KB discovery - #803
  • Flag-gated config-exploration subphase - #811
  • Candidate-free local-exploration arm - #937

Install & portability

  • Bare-metal installer + configurable ROCm installs - #802, #827
  • Quick-start Docker image + local demo/examples - #813, #847, #932
  • Portable per-revision dependency cache - #925 (advances #767)
  • Fix vLLM container start / docker exit - #866 (closes #837)

Reliability, cleanup & hardening

  • Phase-budget charge-back + conc-sweep boost + token efficiency - #919, #899, #853
  • Repo-wide de-dup / dead-code / comment / test consolidation - #863, #870, #879, #883, #906, #911
  • Control-plane security hardening (state-field lock, apply_patch/Critic-verdict gates, argv-only command sinks, env filtering) - #912, #920, #938, #941

Also closed this cycle (manually / duplicate / not-planned; no explicit closing PR)

Requirements

(unchanged from v0.8) ROCm platform (MI355X / MI325X / MI300X; MI308X detection); Docker with AMD ROCm support, or Kubernetes with AMD GPU nodes; Cursor IDE + Remote SSH; an LLM provider key (Anthropic or DeepSeek); tmux, jq, Node.js >= 18, Claude CLI. New in v0.9: install from the GitHub-Release wheel (hyperloom_inference_optimizer) plus a bare-metal installer or quick-start Docker image.

Legend: closes = GitHub-verified explicit closer. Issues marked "addresses" are best-effort cross-references (the PR addresses the issue but did not auto-close it).

Assets 3
Loading