Skip to content

Hyperloom v1.1.5

Latest

Choose a tag to compare

@xiaofei-zheng xiaofei-zheng released this 28 Oct 07:41
7c0708e

Hyperloom v1.1.5

This is a patch release on top of 1.1.4. AgentX optimization now runs on Magpie's native AgentX contract. New FRAMEWORK_AGENT and KERNEL_AGENT work must fit what is left of its phase budget, and a phase transition lets running work finish instead of cancelling it. The release adds GPU power budgets and opt-in power settings, an Experience KB you can operate with the hyperloom-kb command, a custom specialist the Coordinator runs itself, the PrimaTune predictor, Radeon AI PRO R9700 support with vLLM, a Claude–Codex relay for stalled forge-loop searches and a Claude subscription credential path. Explore also stops re-measuring launches the session already has an answer for.

Sessions started by 1.1.4 or earlier cannot be resumed on this build. coordinator.db now records schema version 7 and has no in-place migration, so optimize --resume-from on an older session exits 2 before touching it (#1967), and kernelforge forge-loop --resume refuses a 1.1.4 checkpoint. Finish in-flight sessions under the build that started them. The workspace Experience KB also starts empty, the accuracy eval can no longer be turned off, and several options, environment variables and defaults move; see Breaking changes before upgrading. The per-change history lives in docs/release-notes.md.

Highlights

1. AgentX runs on Magpie's native AgentX contract

Under HYPERLOOM_AGENTX=1, Hyperloom turns --model, --framework, --precision, --tp, --ep and --conc into a Magpie AgentX config, or takes one from --benchmark-config PATH (start from assets/configs/agentx_sglang.yaml). It archives the config as benchmark.yaml in the session and resumes from that copy. Magpie owns server start, trace replay, profiling and cleanup, and a KEEP needs a valid, unprofiled Magpie report for the session's own workload. Native AgentX is single-node SGLang or vLLM at one concurrency per session and exits 2 at launch on a flag it cannot honour. Hyperloom's own AIPerf client is removed, while the MLPerf backend keeps its entry point (#1922). A baseline measured with the prefix cache off is refused (#1884). A candidate can no longer change the speculative draft under simulated acceptance (#1885). GEAK serves the engine the baseline ran (#1792), and on interactivity-graded sessions the stack ledger measures adoptions on the graded axis (#2022).

2. Phase budgets bind, running work finishes, and failures surface

Work whose estimate does not fit what is left of the FRAMEWORK_AGENT or KERNEL_AGENT phase is refused with rule=phase_time_budget, and running work stops at the phase end. The LLM's extend_explore_budget and extend_kernel_budget hints no longer lengthen a phase (#1873). A phase transition waits for running tasks to finish and be booked instead of cancelling them, so a GEAK rebench still running when KERNEL ends is kept or settled (#1860). Orchestration can no longer run an action its phase does not offer (phase_incompatible), and once a gain is validated a higher re-baseline no longer replaces its baseline anchor (#1886). While a baseline is measured and the budget still funds another macro cycle, a skip_to_close is refused as skip_to_close_contradicted and the run keeps optimizing (#2185). Broad exception handlers across the optimizer and KernelForge now catch only the errors their code can raise, so failures 1.1.4 stepped past are recorded on the task, tick or CLOSE step above them (#1986, #2004, #2042, #2049).

3. GPU power budgets and opt-in power settings

--max-power-w N caps the summed measured-phase mean power of the serving GPUs, and --max-per-gpu-power-w GPU_ID W caps one card. Every KEEP path refuses a candidate over a budget or without a per-card reading, and a baseline that breaks a budget stops the run with baseline_over_power_budget (#1770). With --apply-gpu-power-settings, the session sets the declared --gpu-power-cap-w / --gpu-perf-level itself at launch and restores the originals when it ends, including on SIGTERM or SIGHUP. This needs root and keeps its records under /var/tmp/hyperloom/gpu_power_settings/. The power checks and the sampler now translate the visible-device mask to amd-smi card numbers, so on MI300X and MI355X they read the session's own cards, and power readings are not comparable with 1.1.4 (#2032). gpu_power.json also reports each card's sustained clock and power-throttle share (#2038).

4. Explore stops re-measuring answers the session already has

Before each variant runs, explore skips one that only restates the launch it would run on (baseline_noop), or one that already has a KEEP or REVERT on the same base launch, anchor throughput and workload (already_measured). Restatements are judged against the stack the variant would launch on (#1903, #1916, #1918). A delegated grid whose every variant is already covered is refused as explore_grid_already_covered (#1916). Synthetic-benchmark sessions refuse context-copying speculative drafts such as vLLM ngram / suffix or SGLang NGRAM, whose gain does not hold on real text (#1904). In FRAMEWORK_AGENT, a reviewed integrate_patch now dispatches ahead of queued explore grids (#1838). A specialist domain holds at most one queued or running single-domain specialist (#2041).

5. Coordinator-run custom specialist, PrimaTune predictor and source-level wins

With --custom-specialist-prompt-file and --custom-specialist-description set, the Coordinator queues the custom specialist on every FRAMEWORK_AGENT entry and benches its proposals before a plateau exit. HYPERLOOM_CUSTOM_SPECIALIST_ONLY=1 makes FRAMEWORK_AGENT run only that specialist (#1876). --primatune-endpoint URL asks a PrimaTune service for launch configurations at each decision point of an SGLang or vLLM session. --primatune-mode picks shadow (the default once an endpoint is set, which only logs the answers), active (which queues the answers ahead of the specialists' proposals) or off (#1798). When the configuration arm plateaus, a second static_recon_specialist pass runs once per macro cycle, and the orchestration prompt points at a patch-mode specialist for a source change rather than another grid (#1891).

6. An Experience KB you can operate

Each Experience KB service has a kb_id. The new hyperloom-kb command (health, push, pull, rebind, label, restore, exclude, list, export and more) operates the service that HYPERLOOM_KB_URL names. exclude hides an Experience from every read, a pull first labels the state it replaces so restore can undo it, and every service answers /livez, /readyz and /metrics (#1744). HYPERLOOM_KB_HOME names the local service's data home, so hosts that share a USER_DATA_PATH can each run their own (#1832, #1863). Read and list summaries carry each Experience's identity, the conditions it was measured under (#1910). Experiences move to schema_version 2, so upgrade a global KB together with the workspaces that sync with it (#1744).

7. Kernel work: forge-loop relay, GEAK hand-offs and GEMM tuning

A new forge-loop campaign builds one agent profile per registered provider (claude, codex). After three non-improving iterations it hands the whole search to another available profile, and agent_relay records every switch (#1945). On an interactivity-graded AgentX session, the GEAK hand-off names e2e_norm_intvty_p50 and carries the session's KEEP rule (#1606). Before delegating, KERNEL stops the session's leftover servers and waits for the handed-off cards to drain. It also holds back the time GEAK's rebench needs, so a GEAK result is no longer lost to an occupied GPU or a spent budget (#1606, #2040). GEMM tuning reads the formats nested in Quark and compressed-tensors checkpoints, so fmoe_ck now runs on Quark MoE checkpoints, and it reads aiter's fused-MoE dispatch lines again (#1826, #1862). The TraceLens Agent runs from a run.yaml that TraceLens validates before the agent starts (#1952).

8. Radeon AI PRO R9700, Claude subscriptions and new default models

An R9700 (gfx1201), detected by its product name or given as r9700, runs single-node vLLM through Magpie's gfx12 runner (#1782). hyperloom-setup can use a Claude Pro or Max subscription: you run claude setup-token, and a launch whose only Anthropic credential is CLAUDE_CODE_OAUTH_TOKEN now starts (#1937, #1940). The default Claude model is claude-opus-5-5, and preflight falls back to claude-opus-5 where the gateway catalog lacks it (#1941). The default Codex model is gpt-6-astra (#1945).

Breaking changes

An operator upgrading from 1.1.4 has to plan for these. The full tables are under "Before upgrading from 1.1.4" in docs/release-notes.md.

  • Sessions started by 1.1.4 or earlier cannot be resumed (#1967). storage/coordinator.db is now at schema version 7 and there are no in-place migrations, so optimize --resume-from on an older session exits 2 before touching it, with coordinator.db is at schema version 6, older than this build's 7: sessions created before this release cannot be resumed. Finish in-flight sessions under 1.1.4. kernelforge forge-loop --resume also refuses a campaign written by 1.1.4 (run state cumulative has unknown fields: api_errors, orchestration_errors); start a fresh campaign.
  • The workspace Experience KB starts empty (#1744). The service now keeps database.sqlite3, <kb_id>/records/ and <kb_id>/files/ and leaves a 1.1.4 home's kb.sqlite3, sync.sqlite3 and canonical/ unread, so the first launch or ensure after the upgrade starts a new KB and Experiences recorded under 1.1.4 no longer reach prompts. Experiences are now schema_version 2 (hyperloom-kb.collect.v2 mappings), and 1.1.4 and 1.1.5 services reject each other's records, so upgrade a global KB together with every workspace that syncs with it. POST /v1/pull requires schema_ref. hyperloom-setup no longer installs the hyperloom-global-kb skill or asks about a global KB; set HYPERLOOM_GLOBAL_KB_URL / HYPERLOOM_GLOBAL_KB_TOKEN yourself.
  • The accuracy eval can no longer be turned off (#1984). --no-eval is removed, and RUN_EVAL exported in the shell, pinned through --extra-env or set in a benchmark YAML template is ignored (an AgentX template keeps its own value). A host that set RUN_EVAL=false to get past a broken eval must fix the eval: a baseline whose eval fails can stop the run with baseline_accuracy_failed, and a kernel-stack validation, kernel-lane re-baseline or warm-recipe replay whose eval fails now fails that round instead of retrying without the eval.
  • Removed flags and commands fail with unrecognized arguments:
    • --pr-feed-window-days, --local-kb-root (set KNOWLEDGE_LOCAL_ROOT), kernelforge forge-loop --return-after-read-kb and hyperloom multi-node kernel-bench (#1967);
    • --no-eval (#1984);
    • install.sh --only-aiperf (#1922).
  • AgentX optimization runs on Magpie's native AgentX contract, and Hyperloom's AIPerf client is removed (#1922). With HYPERLOOM_AGENTX=1 and no MLPerf backend, or with --benchmark-config, a launch exits 2 on --isl, --osl, --profile-osl, --max-model-len, --reference-script, --quantize, --quantize-scheme, --enable-conc-sweep, --nodes other than 1, a framework other than SGLang or vLLM, and --extra-env pins of ISL, OSL, RANDOM_RANGE_RATIO, MAX_MODEL_LEN, EP_SIZE or any AIPERF_*, AGENTX_* or AGENTIC_* key. Start one session per concurrency, pass expert parallelism as --ep, quantize the checkpoint beforehand, and put other Magpie settings in a --benchmark-config YAML (start from assets/configs/agentx_sglang.yaml). Any HYPERLOOM_AGENTIC_BACKEND value other than mlperf now selects native Magpie AgentX; the MLPerf backend is unchanged.
  • Environment variables that are no longer read. Nothing refuses them (except the AGENTX_* names passed through --extra-env under native AgentX), so a host that still exports one starts normally and behaves as though it were unset. Check launch scripts, .env and the kernel-agent env file:
    • RUN_EVAL (#1984);
    • AGENTX_DATASET, AGENTX_CANONICAL_DATASET, AGENTX_PROFILE_WINDOW_S, AGENTX_TRACE_FIRST_FILE_TIMEOUT_S, AGENTX_TRACE_FLUSH_TIMEOUT_S, HYPERLOOM_ALLOW_UNVERIFIED_SUBMISSION, AIPERF_BIN, INSTALL_AIPERF and the other AIPERF_* installer variables (#1922). A native AgentX run also ignores INFERENCEX_PATH / INFERENCEX_REF; set benchmark.inferencex_path in the YAML or MAGPIE_INFERENCEX_PATH to use your own checkout;
    • HYPERLOOM_SGLANG_REPO_URL, HYPERLOOM_SGLANG_REF, HYPERLOOM_SGLANG_INDEX_URL, HYPERLOOM_VLLM_ROCM_INDEX_URL, HYPERLOOM_ENABLEMENT_ORIGIN_ALLOWLIST and HYPERLOOM_ENABLEMENT_INDEX_ALLOWLIST (#1868). Enablement Rung 3 now installs the runtime the specialist requests and no longer restricts git origins or pip indexes; only --enablement off keeps it from running;
    • HYPERLOOM_LOCAL_KB_ROOT (set KNOWLEDGE_LOCAL_ROOT) and TRACELENS_SOURCE_ROOT (#1967).
  • KernelForge MCP servers and evaluation modules move (#1907). python -m kernelforge.mcp_server.tools.bench, kernelforge.mcp_server.pr_stdio_server and kernelforge.mcp_server.probe_stdio_server become kernelforge.evaluation.benchmark.runner, kernelforge.knowledge.pr_knowledge.mcp_server and kernelforge.orchestrator.specialist_probe_server. KernelForge builds its own server commands, so only a hand-written MCP configuration or code importing a moved module (for example kernelforge.loop.scoring, kernelforge.durable_io) needs updating. tools/kbmine is removed (#1906); it was never part of the package.
  • Re-run install.sh after upgrading.
    • Magpie is pinned to 6a1ee26f9053363d3750b43e3d5afb5d1c4ce84e, and install.sh and preflight replace an installed Magpie at any other revision unless MAGPIE_PATH names your own tree (#1922). The minimum revision rises from e6833b81 (v0.2.0) to 1eb3df69de9cb26cb5c8109e43371dd1abb9bbed; with an older MAGPIE_REF or MAGPIE_PATH, a DeepSeek-V4 round whose checkpoint the installed transformers cannot resolve fails with client_tokenizer_unpatchable (#2054). The AIPerf client is no longer installed (#1922).
    • TraceLens is pinned to 614c597bc0c66cabf6a15152037324acc7386090, and preflight requires TraceLens_generate_perf_report_from_config instead of TraceLens_generate_perf_report_pytorch_inference on PATH; without it a session that runs kernel work or the roofline exits 2 with ERROR: TraceLens CLI(s) not on PATH (#1952).
    • An interactivity-graded AgentX session that delegates to GEAK needs a GEAK that accepts E2E_METRIC=e2e_norm_intvty_p50 and frees its GPUs on exit; GEAK main, the default GEAK_REF, has both. Move a pinned GEAK_REF, otherwise such a session gets no GEAK candidate (#1606, #2040).
    • Multi-node preflight always probes lm_eval[api] and installs the pinned build when missing (#1984).
  • GPU power settings are checked on the cards the mask names (#2032). --gpu-power-cap-w / --gpu-perf-level and the power sampler translate ROCR_VISIBLE_DEVICES, then HIP_VISIBLE_DEVICES (or CUDA_VISIBLE_DEVICES), to amd-smi cards through their KFD node; 1.1.4 read mask numbers as amd-smi indices, so on MI300X/MI355X it checked and sampled other cards and its power readings are not comparable. A mask that is empty, not plain comma-separated numbers (for example GPU UUIDs), repeats a card or names one that does not exist now makes a declared assertion exit 2. --gpu-perf-level accepts only the levels amd-smi set takes, so the AMDSMI_DEV_PERF_LEVEL_<LEVEL> spelling fails parsing.

Defaults that change without operator action:

  • Phase budgets are hard limits for FRAMEWORK_AGENT and KERNEL_AGENT work (#1873). Work that does not fit what is left of the phase is refused (rule=phase_time_budget), skipped or stopped at the phase end; a patch specialist needs 60 minutes of the phase, Forge GEMM tuning and fusion return insufficient_budget instead of running their 5 h / 2 h defaults, and HYPERLOOM_GEMM_TUNING_TIMEOUT_SEC no longer forces tuning into a slice that cannot fund it. The LLM's extend_explore_budget / extend_kernel_budget hints are ignored.
  • The default Claude model is claude-opus-5-5 (#1941). Preflight walks claude-opus-5, claude-opus-4-8, claude-opus-4-7, claude-opus-4-6 when the gateway does not serve it. Set CLAUDE_MODEL=claude-opus-5 to keep 1.1.4's model; the slurm proxy.env.template still does.
  • The default Codex model is gpt-6-astra, with gpt-5.6-sol, gpt-5.5 and gpt-5.4 as preflight fallbacks, where 1.1.4 used gpt-5.6-sol (#1945).
  • New forge-loop campaigns hand a stalled search to the other agent provider when one passes its startup check, instead of to the Supervisor; FORGE_AGENT_CLI is used only when it is the primary provider's CLI, and forge-loop --resume refuses an --agent-backend that differs from the campaign's primary (#1945).
  • Explore and sweep variants keep the operator's --no-enable-prefix-caching, so such sessions measure variants with prefix caching off, as their baselines, and report lower gains than under 1.1.4 (#1798).
  • An AgentX baseline served with the prefix cache off stops the run with stop_reason=baseline_failed (agentx_baseline_prefix_cache_off). Drop --disable-radix-cache / --no-enable-prefix-caching from --server-args; on ATOM pass --enable_prefix_caching (#1884).
  • Synthetic-benchmark sessions refuse a variant that introduces or tunes a context-copying speculative draft (vLLM ngram / ngram_gpu / suffix, SGLang NGRAM) as speculative_context_copy_on_synthetic (#1904).
  • Preflight's validate_server_args step stops a launch whose --max-model-len or $MAX_MODEL_LEN exceeds the model's declared context, or, on vLLM, whose --server-args carry --gpu-memory-utilization outside 0 < value <= 1 or --max-model-len above that context. --tp takes only a positive integer (#2015).
  • A new Quark or compressed-tensors session without --precision records the format the checkpoint nests (for example fp4, fp8) instead of quark / compressed-tensors, so its workload fingerprint and Experience KB records no longer match earlier sessions (#1862).
  • With --custom-specialist-prompt-file and --custom-specialist-description, the Coordinator runs the custom specialist on every FRAMEWORK_AGENT entry and orchestration can no longer dispatch it (#1876).
  • An orchestrator skip_to_close is refused while a baseline is measured and the budget still funds another macro cycle, so such runs keep optimizing where 1.1.4 closed as global_converged (#2185).
  • An optional intent parameter sent as null now inherits the stack's value instead of clearing it, so the same proposal can launch a different server command (#1923).
  • vLLM baselines retried after a CUDA-graph capture failure, and roofline profiles with CUDA graphs off, run with --compilation-config.cudagraph_mode NONE instead of --enforce-eager, so inductor stays on and their numbers differ from 1.1.4 (#1470).
  • GEAK's deadline on every hand-off is shorter by the time its rebench needs, and GEAK is skipped when less than GEAK_MIN_RUN_S (600 s) would remain (#2040).

Report and JSON consumers also need these updates:

  • state.json drops custom_specialist_dispatched (#1876) and enablement.build_novelty (#1868), and gains custom_specialist_only (#1876), analysis_failures (#1890, #1893) and predictor_asked_keys (#1798).
  • AgentX manifest.json workload records benchmark_config instead of corpus_loader, corpus_entries, corpus_duration_s, isl_distribution, osl_distribution and prefix_cache_hit (#1922).
  • session_breakdown.json drops the always-empty robustness and close.robustness, and robustness_escalated leaves the stop-reason vocabulary (#1967). On interactivity-graded AgentX sessions the stack-ledger baseline_tput, contribution_pct, cumulative_gain_pct, attributed_gain_pct, chain_total_gain_pct and reconciliation_gap_pct are measured on the graded axis and are not comparable with 1.1.4 (#2022).
  • The v1 workflow-evaluation contract and schema are removed (#1967). The v2 contract gains baseline_over_power_budget (#1770) and drops robustness_escalated, so new sessions stamp digest 9887b33c30aeada4359b3b3d38b817b9dec1655df84e84a98911e5a1a08871c7; the v2 hint enum drops extend_explore_budget and extend_kernel_budget (#1873).
  • Reports, summaries and the roofline snapshot label serving throughput tok/s instead of tok/s/GPU; throughput_tok_s_per_gpu keeps its name and value (#1939, #1947).
  • The roofline trace-validation summary drops recommended_steady_state_mode, recommended_splitter_mode, modes_that_would_fail, steady_state_forecast and hot_kernel_list_would_be_suppressed, self-certification files no longer carry a split forecast, and an empty steady-state chunk is reported as steady_state_chunk_low_quality instead of steady_state_chunk_empty (#1612).
  • A multi-node kernel apply records jit_build_backup as {"status": "remote"} without per_node / per_host, and multinode.per_node is gone; read multinode.records_by_host (#1844). Backup-ledger rename records carry revert_action: "restore" instead of "restore_old", and snapshot manifest.json entries drop binary (#1861).
  • forge_experiments/pr_refs/index.md renames merged to state, and new snapshot.json entries record state, draft, merged_at and base_ref instead of is_merged (#1846).
  • Provenance reads the GPU from TARGET_GPU_TYPE, so MI325X and MI308X sessions record mi325x / mi308x instead of mi300x (#1782), and setup.py develop checkouts or isolated vLLM venvs record their real framework version instead of unknown_version (#1827); their KB keys no longer match 1.1.4 knowledge for the same setup.
  • Log lines that move: explore dedup: payload=... becomes explore grid: payload=... (leading baseline_noop=N) (#1918); KernelForge's [kb] experience write skipped (...) becomes experience KB publication failed for <commit>: ... (#2049); enablement no longer emits localization_rung5_deferred or localization_fetch_failed (#1858).
  • The enablement constraints file is rocm_constraints.txt (was torch_constraints.txt) (#1868).
  • A Recipe KB still only in the legacy $USER_DATA_PATH/kb (or /workspace/hyperloom/kb) location is no longer read or migrated (#1967).

Known issues

  • On a Magpie AgentX session, a KEEP that removes recipe options can make every later bench fail. A KEEP carrying remove_args or unset_envs (for example an explore winner that drops the hierarchical-cache flags) switches current_best.args_mode to replace, so later candidates reach Magpie with launch_overrides.replace_args: true and lose every recipe option Magpie does not protect. On a recipe that pins golden acceptance, such as DeepSeek-V4-Pro with speculative decoding, every later explore, roofline and integrate_patch bench then fails within seconds as magpie_nonzero_invalid_measurement (Changing AgentX golden acceptance inputs requires a new resolved recipe). The fix (#2186) is on main and ships in the next release.

What's included

CLI, setup, preflight and resume

  • fix: four defects found validating a vLLM 0.29 source-overlay run on MI355X - #1827
  • fix(cli): run model preflight off the event loop so an OAuth-only launch starts - #1937
  • fix(reference-script): recognize the sglang serve entrypoint - #1938
  • docs(setup): add a Claude subscription (claude setup-token) credential path - #1940
  • fix(llm): default to claude-opus-5-5 from one owner; keep claude-opus-5 on the ladder - #1941
  • fix(resume): never record a framework tree as the InferenceX path; refuse to launch from a tree that is not one - #1976

Orchestration, phases and specialists

  • feat(predictor): ask PrimaTune for optimization proposals in FRAMEWORK_AGENT - #1798
  • fix(dispatcher): dispatch reviewed patches ahead of queued explore grids in FRAMEWORK_AGENT - #1838
  • fix(orchestrator): let running work finish at a phase transition - #1860
  • [CC] The no-git patch channel reports and records what it actually did - #1861
  • fix(loop): phase budgets are the operator's, and new FRAMEWORK/KERNEL work must fit them - #1873
  • feat(custom-specialist): make the Coordinator own the custom specialist; add HYPERLOOM_CUSTOM_SPECIALIST_ONLY - #1876
  • fix(orchestrator): refuse out-of-phase actions and keep the baseline anchor once a gain is validated against it - #1886
  • fix: route a kernel request on the target PolicyGate admitted - #1889
  • feat(optimize): bias the OPTIMIZE phase toward source-level wins - #1891
  • fix(gaps): count each failure once on a gap row - #1899
  • fix(orchestration): limit Claude built-ins to declared tools - #1912
  • fix(intent): a payload field written as null is not supplied - #1923
  • perf(critic): cache the static Anthropic review prefix - #1946
  • [RSI] Refuse proposals whose AITER_CONFIG file is not on disk when they are admitted; keep run_grid's check as backstop - #1982
  • [RSI] Deny send_message with a non-string to instead of crashing the events INSERT - #1983
  • fix(specialist): a catalogue domain holds at most one queued or running scope=domain specialist - #2041
  • [RSI] send_message: accept a list of recipients and write one message per recipient (follow-up to #1983) - #2044
  • release/v1.1.5: cherry-pick the contradicted skip_to_close refusal and the initial-roofline hold line (#2175) - #2185

Measurement, explore and grading

  • fix(server-args): repair quote-stripped JSON whose value is an absolute path - #1900
  • fix(explore): skip a variant or warm replay that only restates the baseline launch - #1903
  • fix(explore): refuse a context-copying speculative draft on a synthetic benchmark - #1904
  • fix(explore): do not re-measure a variant the session already judged on the same stack - #1916
  • fix(explore): judge a baseline restatement against the stack it launches on - #1918
  • Remove the accuracy eval opt-out - #1984
  • fix(baseline): validate launch limits and classify CacheConfig failures - #2015
  • fix(bypass): a boot-only round boots the server and runs no client - #2021

GPU power and hardware

  • feat(power): --max-power-w total and --max-per-gpu-power-w budgets - #1770
  • feat(gpu): support the Radeon AI PRO R9700 (gfx1201) with vLLM - #1782
  • feat(power): opt-in --apply-gpu-power-settings, restored however the session ends - #2032
  • feat(power): report each round's power-throttle share and sustained clock - #2038

AgentX and Magpie

  • fix(agentx): hand GEAK the server launch the agentic recipe recorded - #1792
  • fix(agentx): refuse a baseline measured with the prefix cache off - #1884
  • fix(agentx): pin the draft and its acceptance when the baseline simulates acceptance - #1885
  • Use Magpie's native AgentX contract for optimization - #1922
  • fix(stack): attribute AgentX adoptions on the graded axis - #2022
  • fix(conc-sweep): send the boot phase only to a backend that reads it, so Magpie can reuse the server - #2039
  • fix(magpie): stop patching client scripts for a tokenizer hook Magpie now carries - #2054

Enablement

  • fix(enablement): keep enablement rounds, the kept stack, and the framework tree intact - #1858
  • [FIX] e2e-run bug fixes and an agent-requested rung-3 runtime - #1868
  • [fix]Enablement: run later phases on the promoted runtime checkout; add a ROCm install path for SGLang checkouts - #2050

Kernel, profiling, TraceLens and KernelForge

  • fix(roofline): drop only cuda-graph capture, not inductor, for vLLM - #1470
  • fix(geak): make an AgentX hand-off to GEAK end in a result the session can keep. (remainder; the round server teardown shipped in 1.1.4) - #1606
  • Cleanup trace splitter code - #1612
  • fix(gemm_tune): route Quark and compressed-tensors checkpoints on the formats they nest - #1826
  • feat(pr_knowledge): carry PR state, draft, merge time and base branch - #1846
  • fix(gemm-tune): route MoE on the serving log and label Quark sessions by format (#1826 follow-up) - #1862
  • fix(roofline): remember a deterministic analysis failure and do not re-run it on the same input - #1890
  • fix(trace_analyze): serve a remembered recurring failure instead of re-analysing the same trace - #1893
  • fix(kernelforge): keep the workspace guard out of the loop's own output tree - #1909
  • feat(kernelforge): relay search across agent profiles - #1945
  • Drive the TraceLens Agent from a YAML run config - #1952
  • fix(geak): hold back the rebench's own round and let GEAK's GPUs drain before it - #2040

Experience KB, reporting and workflow evidence

  • Experience KB: operable state, user-declared fields and files, SQLite for a workspace and PostgreSQL for replicated global KBs, observability, and notes - #1744
  • docs(skills): name HYPERLOOM_KB_HOME wherever the Experience KB home is described - #1863
  • fix(recipe-kb): preserve publication failure causes - #1875
  • chore(tools): remove tools/kbmine - #1906
  • feat(kb): read and list summaries carry each Experience's identity - #1910
  • fix(report): label the serving primary metric tok/s, not tok/s/GPU - #1939
  • fix(critic): describe serving throughput as tok/s, not tok/s/GPU - #1947
  • fix(conc-sweep): join the Pareto chart in concurrency order and mark single-arm rungs - #1954
  • feat(breakdown): count the framework phase's bench runs and show the funnel in the report - #1978
  • fix(breakdown): count a variant by the facts its recorder settled, not by its outcome spelling - #1985

Error handling

  • fix: stop swallowing failures in 43 non-boundary broad handlers - #1986
  • fix: narrow or remove 121 more broad handlers - #2004
  • fix: narrow or remove 71 broad handlers in kernelforge and hyperloom (stacked on #2004) - #2042
  • fix: narrow, remove or log the remaining broad handlers outside the files under restructuring - #2049

Internal architecture and refactors

  • refactor(kernelforge): split cli.py into a cli package - #1823
  • refactor(kernelforge): split forge-fuse stages out of fusion/command.py - #1824
  • refactor(kernelforge): split preflight and prompt text out of task_preparer.py - #1825
  • refactor(forge): bring KernelForge under the complexity ceiling - #1839
  • refactor(optimizer): bring the optimizer and scripts under the complexity ceiling - #1840
  • refactor(orchestrator): remove duplicate JIT and no-git backup paths, fix stack-index-0 provenance lookup - #1844
  • Delete four duplicated admission-gate rules and the complexity they carried - #1853
  • refactor(orchestrator): remove duplicated writeback, autosubmit and config-env branches - #1877
  • refactor: three field tables replace twenty-nine restatements of one decision - #1878
  • refactor: one rule, one place, for warming a specialist and refusing a request - #1881
  • refactor: one abort path ends a variant in the grid runner - #1882
  • refactor(state): normalize current_best once at load and drop its reader-side dict guards - #1883
  • refactor: one table owns every value warmed into a specialist - #1887
  • refactor: one method answers a kernel request, and a delegate is assembled in named steps - #1888
  • refactor(kernelforge): reorganize evaluation, shared tools, MCP servers and the test root - #1907
  • chore: remove dead code, dead tests and stale docs - #1967
  • refactor: carry shared-helper settings as typed objects instead of parallel keywords - #1969
  • refactor(breakdown): pass each breakdown row as one typed record instead of parallel keywords - #1970
  • refactor(kernelforge): hand each command its options as typed groups instead of one parameter per flag - #1971
  • refactor(orchestrator): pass actions, state and trace-analysis inputs as typed objects instead of parallel keywords - #1972
  • refactor(inference-optimizer): pass trace, roofline, multi-node and CLI inputs as typed objects instead of parallel keywords - #1973
  • refactor(kernelforge): pass loop, knowledge and backend inputs as typed objects instead of parallel keywords - #1974
  • refactor(kernelforge): pass rewrite, orchestration and example-driver inputs as typed objects instead of parallel keywords - #1975
  • refactor(orchestrator): pass phase, kernel, knowledge and loop inputs as typed objects instead of parallel keywords - #1977
  • refactor(executors): collapse the duplicated patch, snapshot and eval helpers into named jobs - #1980
  • refactor(executors): split the benchmark run path's oversized functions along the jobs they do - #1991
  • refactor(executors): split the patch, evidence and probe helpers' oversized functions - #1993
  • refactor(bringup): name the three questions classify answers about a boot attempt - #2007
  • refactor(delivery): name the three shapes a restore record's fields must have - #2009
  • refactor(phases): bring the phase state machine and its handlers under the complexity ceiling - #2010
  • refactor(recipe-kb): give each recipe sub-shape its own dict reader - #2012
  • refactor(state): build each prompt section and ledger record through a named unit - #2014
  • refactor(enablement): split the recipe section, projections, sufficiency and keep records into named units - #2016
  • refactor(roles): name what a Claude turn and a Critic turn collect - #2018
  • refactor(kb_store): move derive_kernel_names into kb_store.identity.kernel_names - #2028

CI, review tooling and tests

  • Add a site-neutral single-leg pre-release runner - #1832
  • fix(review-pr): name a truncated open-PR overlap list instead of reading it as no overlap - #1849
  • ci: remove unused pre-release E2E workflow and its scripts, prompts and tests - #1864
  • ci: restore pre-release leg prompts still used by run-leg.sh - #1865
  • chore(code-metrics): teach the gate to measure parameter count - #1913
  • chore(code-metrics): gate parameter count at five - #1915
  • fix(tests): write the perfmodel tests' checkpoint-sized weight shard sparse - #2061

Release

  • chore(release): cut 1.1.5 - #2262

Requirements

AMD Instinct MI300X, MI325X or MI355X, or a Radeon AI PRO R9700 (gfx1201) for single-node vLLM (#1782); Ubuntu 24.04 recommended or Ubuntu 22.04 legacy; ROCm 7.2.x or 10.0; Python >= 3.10; Docker with AMD ROCm support, Kubernetes with AMD GPU nodes, or the supported bare-metal setup; Claude Code IDE with Remote SSH; an Anthropic API key, a Claude Pro or Max subscription token (claude setup-token), a compatible Anthropic gateway, or an OpenAI-compatible gateway; tmux, jq, and curl with access to claude.ai. The installer provides the Claude CLI when none is on PATH; Node.js is needed only for the Codex carrier and must come from the image. Required tools are unchanged from 1.1.4, except that the installer no longer installs the AIPerf client.

The validated images are lmsysorg/sglang-rocm:v0.5.21-rocm10-mi30x-20261008 (MI300X / MI325X) and lmsysorg/sglang-rocm:v0.5.21-rocm10-mi35x-20261008 (MI355X) for SGLang, rocm/vllm:rocm10.0.0_ubuntu24.04_py3.14_pytorch_2.12.0_vllm_0.27.0 for vLLM, and rocm/atom-dev:v0.1.7-rc0 for ATOM on MI355X. The images default to /bin/bash, which exits at once in a detached container, so a long-running Hyperloom container has to override the entrypoint (for example --entrypoint tail).

Validated component versions are listed in the compatibility matrix: TraceLens 1.0.0, GEAK 4.0.0, IntelliKit 0.1.1, AgentKernelArena 0.2.0, Magpie 0.3.0 (install.sh pins 6a1ee26f; at least 1eb3df69 is required).

The wheel is published on PyPI:

pip install hyperloom-inference-optimizer==1.1.5 --target .

Bare-metal installs read HYPERLOOM_WHEEL_TAG, which now defaults to v1.1.5, so this release must carry the rocm-profiler-hotfix-libs.tar.gz asset.

Stability

v1.1.5 is a patch release under semantic versioning. Unlike 1.1.4, it does not resume sessions recorded by earlier builds: the coordinator database moves from schema version 6 to 7 with no in-place migration, and an older session is refused at resume admission before anything in it changes. A KernelForge campaign written by 1.1.4 cannot be continued with forge-loop --resume, a 1.1.4 Experience KB home is not adopted, and 1.1.4 and 1.1.5 KB services reject each other's records. Downgrading a 1.1.5 session to 1.1.4 is not supported. The breaking changes are confined to session resume, Experience KB storage and records, the accuracy-eval opt-out, the removed flags, commands and environment variables, the AgentX launch path, KernelForge module paths, the Magpie, TraceLens and GEAK installs, the defaults and gates listed above, and the report fields listed above.