Repository navigation
hal0 v1.4.0
Highlights
- Memory writes that were silently lost now land, or are refused
honestly. Hindsight's extraction ran with its own defaults (32
concurrent calls, 64000 tokens) on a slot that serves one request at a
time, so on a slow boxPOST /api/memory/addreturned 200 and nothing was
stored. Extraction now has operator-set limits under[memory.graph]
(Settings ▸ Memory), and a write whose extraction slot is too small for
the prompt is refused up front, over REST and/mcp/memoryalike.
(#1834, #1903, #1930) - Open WebUI works with auth on, and a lost admin key is recoverable.
The installer mints a client key and Open WebUI presents it to hal0's
/v1(per connection, never to another service);hal0 auth rotate clientre-points it.sudo hal0 auth reset-keymints and prints a new
admin key, live or with the API down. (#2314) - Open WebUI is fully pre-wired. Document uploads go through RAG on
hal0's embeddings and the image button generates through ComfyUI as soon
as the matching slots are bound. (#2256) - Installs leave evidence. Every run writes an owner-only log, and a
failed run writes a redacted failure report you can attach to an issue.
(#2243, #2307) - Loading a stack says what it will unload before you confirm, and the
result names what was stopped. (#1511) - A fresh runner system (ADR-0028). Every runner has one name, used for
its registry key, its pin, its catalogue row and its package
(ghcr.io/hal0ai/hal0-runner-<key>). The AMD llama.cpp runner is now
Standard (standard), built from pristine upstream llama.cpp for ROCm
and Vulkan. Every runner image is pinned in one table,manifest.json
runners; the installer, the Vulkan-lane gate, the bench harness and
hal0 doctor toolbox-pullall read it. The runner-image catalogue now syncs
by itself at startup and duringhal0 update. - The bundled Hindsight memory engine moves to 1.0.0 (from 0.9.2), its
first major release.hal0 updateand aninstall.shre-run upgrade it
with the existing build-aside, snapshot, swap, verify and roll-back pass;
see Migrations for what that does to the engine's database. - Stopping
hal0-apino longer hangs until systemd's SIGKILL. With a
chat request in flight, a restart could leave the API down for about 100 s
while a model-cache refresher swallowed the shutdown cancel. Every
shutdown step is now bounded at 3 s, and the installer writes
TimeoutStopSec=30intohal0-api.service. (#2727) - Text-to-speech runs from the model you pulled. The Kokoro slot launches
from the bundlehal0 model pull kokoro-v1installs instead of
downloading about 354 MB on every start, so a box without internet access
can speak;hal0 updatepulls the bundle once and names the remedy if it
is offline. (#2739) - One overflow no longer fails every in-flight request on a slot. The
seedutility,brainandagentprofiles now serve one
sequence (--parallel 1), so concurrent callers queue instead of all
receiving "Context size has been exceeded", Hindsight extraction and
Hermes turns included. (#2692, #2714) - An upgrade that unassigned a slot's model now says so.
hal0 update
prints one line per affected slot with a working remedy
(hal0 slot edit <slot> --model <id>), and/api/slotscarries the
message instead of reading idle. (#2694, #2695, #2696) - Upgraded boxes with a root-owned backups directory finish the runner
conversion. The post-update conversions no longer fail on a
root:hal0 2755/var/lib/hal0/backups;hal0 doctor permsreports and
fixes it. (#2657)
Breaking
-
Exposing a user-installed MCP server to Hermes or the brain profile is
disabled until hal0's MCP proxy mount lands (#2303). Both call a joined
server's URL directly, so hal0 enforces none of its[tools]policy on
that path, and the policy has no wildcard: a tool it does not list is
denied, yet the agent could call it. That covered every exposed server,
including allow-only and empty policies, not only thegated/blocked
case #2343 closed.PATCH /api/mcp/{id}/exposure(and
hal0 mcp expose --hermes/--brain) now refuses turning either target on
for any installed server with409 mcp.exposure_policy_unenforced;
withdrawing exposure still works, andhal0 mcp testsays why the toggle
is unavailable. A server already exposed on an upgraded box is removed
from Hermes's config, the brain profile and the policy mirror at the
next hal0-api start or MCP change. hal0's built-in servers are
unaffected. (#2358) -
hal0 updaterestarts slots on its own and can exit 2 on an update that
applied. When a release rolls the runner image, slots on the old image
are now restarted after the apply, both byhal0-apiat every start
([updates].auto_restart_image_drift, on by default) and by the CLI
(--no-restart-slotsopts out). A slot with a request in flight is skipped.
While any slot still runs the old image,hal0 updateexits 2 ("applied,
convergence outstanding") instead of 0, so a script that treats any
non-zero exit as a failed update needs to accept 2. Becausehal0-api
does this at start, the upgrade to 1.4.0 itself can restart running
slots. (#2096) -
An installed MCP server that would send its
[secrets]or[env]
headers in clear text no longer loads. Astreamable-http/sserecord
whose url is plainhttp://to a non-loopback host, and that carries any
[secrets]key or non-empty[env]value, used to send those values
unencrypted on every probe and Hermes call. 1.4 refuses it: on upgrade such
a record is skipped (hal0.mcp.installed.bad_recordin the journal, and
mcp.record_malformedwhen read directly), it drops out ofhal0 mcp list, and its Hermes entry is removed at the next hal0-api start or MCP
change. To keep it, point the url athttps://or a loopback host, or set
allow_insecure_http = truein its TOML under/etc/hal0/mcp-servers/to
accept the risk. AProxy-Authorizationheader value is refused on any url
and belongs in the proxy environment instead. (#2304) -
Reverse proxies and other DNS names need their origin allowed wherever
the Origin gate runs:require_authon, or (#1822) ADMIN routes reached
from a non-loopback peer on a LAN-bound box with an admin key (#2327). A browser reaching hal0 by a name the installer did
not seed, such as a reverse proxy's domain or a router-assignedbox.lan,
used to pass the Origin gate because itsOriginmatched itsHost. It
now gets403 auth.origin_forbiddenon state-changing requests and WS
close4403on upgrades.hal0 updatecannot know that name, because hal0
never recorded it. Allow it once with
hal0 network public-url set https://hal0.example.comor
hal0 network origins add <origin>(or Settings ▸ Network); the change
applies on the next request with no restart. The refused origin is logged
ashal0.auth.origin_rejected. Access bylocalhost,127.0.0.1, a LAN
IP,hal0.local, the box hostname or an installer-seededpublic_urlis
unchanged, and a box where the gate does not run is unaffected. -
Upgrades now start at 1.3.0. Every release manifest carries
upgrade_from: ">=1.3.0"(declared inmanifest.json), sohal0 update
on a box below 1.3.0 stops withsystem.update_upgrade_path_unsupported
before it downloads anything. Re-running the installer over an existing
install below 1.3.0 stops the same way, before it changes anything. A
nightly counts as a prerelease of its base version. A box below the floor
steps through 1.3.0 by running its installer, or reinstalls; see
"Upgrading from an older release" in
docs/guides/update-and-rollback.mdx. The 0.9.8 upgrade guidance is gone. -
hal0 migrate model-layout,hal0 doctor migrations, themigrations
row ofhal0 doctor all, theHAL0-MIGRATION-PENDINGdiagnosis and the
bundle'sdiagnostics/migrations.jsonare removed. They converted a v0.1
/mnt/ai-modelsstore into a<recipe>/<capability>symlink farm under
/var/lib/hal0/modelsthat nothing reads: registered models point at
their real files. The installer no longer plants the farm on every
install. Links an earlier install planted are left in place and can be
deleted by hand. -
The one-shot converters for state older than 1.3.0 are removed. A box on
1.3.0 or later has already been through each of them:- The v1.0 profile-catalog reset is gone, with its prompt in
hal0 update,POST /api/updates/converge-profiles, thereset_profiles
field ofPOST /api/updates/commit,HAL0_RESET_PROFILES, and the
profile_resetfields of/api/updates/check, the prepare job and the
commit job'sconvergence. A box an operator kept at
meta.schema_version = 1by declining the reset keeps its
profiles.tomland is stamped 2 on its next migration run. hal0 slot migrate-enabled-removaland the boot-time sweep of the slot
enabledkey are gone; the commit job'sconvergenceloses
slot_enabled_swept. A strayenabledkey is now ignored, never
acted on.- Every boot no longer prunes materialised seed profiles, repairs the
pre-rc.11 Hermes MCP header, deletes a model-less slot namedvision,
renames a slot namedstt-npuorembed-npu, or converts a v1
capabilities.toml. Loadingprofiles.tomlno longer injects the eight
demoted seed profiles into a file that lacks them, which removes one of
the two load-time writes #2489 describes. - install.sh no longer warns about a pre-FHS
/opt/hal0, deletes stale
hal0-agent@drop-ins (Hermes provisioning still does), or removes
retired bench scripts.
- The v1.0 profile-catalog reset is gone, with its prompt in
-
Runners. The
rocmfpxrunner (and itsvulkanfpxalias), thestrix
runner and thepromptforgerunner are removed, with no alias. Runners are
standard,cpu,flm,kokoro,moonshineandcomfyui(default) plus
qwen3ttsandcuda(optional). Thefrozentier is gone. Existing
installs are converted automatically (see Migrations). -
ROCmFPX, ROCmFP4, IU4 and ActiveFPX GGUFs are no longer supported. No
shipped runner can load them. Adding, pulling or scanning one is refused
(models.unsupported_quant) and launching one is refused
(slot.unsupported_quant). Use a plain GGUF build of the same model. The
curated ROCmFP4 rows, thehal0-brain-sftQ8/Q4 ROCmFPX variants and the
hal0-quantizeagent skill are removed. The PromptForge specialty support
(companion sidecars,specialty_degradedon slots) and thepromptforge
legacy seed profile are removed with them. -
Env overrides are renamed from
HAL0_TOOLBOX_IMAGE_<KEY>to
HAL0_RUNNER_IMAGE_<KEY>. The old names are no longer read. -
GET /api/system-infono longer returnsbackends[].is_default,
backends[].unsupported_archsorbackends[].supports.specialties; use
backends[].tier. The slot routes no longer returnmodel_fit_warning. -
/api/runner-imagesrows are keyed by runner key (standard,cpu, ...)
instead of package path, carrytitle,summary,family,tier,
backendsandlanes_gatedinstead ofmanifest_key,ownership,
publish,notes,build,runtime_familyandsupported_backends, and
their only badge ispinned. The catalogue readsimages.jsonschema
hal0.runner-images.v2only, and a sync prunes every row the catalogue no
longer lists, downloaded or not, unless the catalogue is empty or holds an
entry this hal0 cannot read. -
The privileged bench seam (
hal0-benchctl) runs only an image the
installed release pins inmanifest.jsonrunners, matched exactly, with
the/opt/hal0/bin/llama-benchentrypoint. AHAL0_RUNNER_IMAGE_<KEY>
override image cannot be benchmarked through it. -
manifest.jsonreplacestoolbox_imagesand_legacy_toolboxeswith the
runnerspin table (_schemahal0.manifest.v2). Its first pins are the
hardware-gatedhal0-runner-*images:standardandcpuon llama.cpp
b11552 (standardgated for ROCm and Vulkan),flm0.9.44 (NPU),kokoro
0.4.9 andmoonshine20251121 (CPU),qwen3tts0.1.1 andcomfyui
0.28.0 (ROCm), plus the upstreamserver-cuda-b11541image forcuda,
which hal0 does not gate. -
The default curated chat model is
qwen3.8-27b(Qwen3.8 27B, UD-Q4_K_XL
with vision, 96K default context). A fresh install seeds it as the primary
chat model on a box with a 32 GB or larger memory pool (smaller boxes keep
qwen3.5-9b/qwen3-4b), and the install-time agent offer is that model. -
Profile-centric config (ADR-0029). A model row holds facts about its
file only (path, architecture, native context, chat template, projector,
whether it has MTP heads). A profile holds every tune: free flags,
ctx_size,enable_thinking,vision(auto/off) andmtp
(auto/off). A slot holds placement only: device, layers, threads, port,
runner, and which model and profile it runs. The launch is composed in that
order (base, model, profile, slot), so the effective command is predictable
and shown layer by layer. Existing installs are converted automatically
(see Migrations). What changed for operators and integrations:- The model API and the MCP
model_*tools no longer accept
or return the model tunesdefaults.profile,extra_args,
context_size,n_gpu_layers,rope_freq_base,enable_thinking,
visionor a preferred profile. Setting one is refused; put it on a
profile instead. - Slot config refuses tuning keys with
slot.tuning_key_denied: the
context window ([model].context_size/ctx_size),parallel,
chat_template,workers,mtp,enable_thinking,vision,
[model].n_gpu_layers,rope_freq_baseand[server].extra_args.
A slot file that still carries one fails to load withslot.unconverted
(the conversion removes them first). PATCH /api/slots/{name}/defaultsis removed, with no alias, and the
slot.edit_defaultsaudit action with it. After the tuning keys left it
could only rebind[model].default. Rebind a slot's model with
PUT /api/slots/{name}/configand{"model": {"default": ...}}, the MCP
slot_edittool, ormodel_swap. A client that still calls the old
route gets a 404.- The slot views drop
ctx_max,chat_template,rope_freq_base,
workers,parallelandllamacpp_args, and addctx_effectiveand
ctx_source(profile,nativeorfallback), plus
conversion_blocked(reason and fix) for a slot the conversion blocked. hal0 slot createandhal0 slot editlose--ctx-size; set the
window withhal0 profile create|edit --ctx-sizeand point the slot at
that profile.- The MCP tool
slot_set_defaultsis removed. hal0 slot migrate-flags,migrate-capsandmigrate-hware removed
with the fold modules behind them, and so is the ownership convergence
panel inhal0 updateand the installer.- A slot with no profile runs its type's seed (
embeddingand
rerankingcarry their mode flag); a profile naming nothing fails with
slot.profile_missinginstead of silently falling back. - The seeds now include
agent(64K context),utility(64K) and
coding, and the shipped slots use them. - A slot file with a key the slot schema does not declare now fails its
typed load, naming the key and the file (failed to validate slot config 'x' at /etc/hal0/slots/x.toml: unknown key 'lru' ...).SlotConfigwas
the last of the slot models (ModelConfigandServerConfigalready did)
to accept and round-trip anything; it no longer has a catch-allextra
table either, so a hand-added table or key is an error rather than kept.
Everything hal0 writes is declared (type,default,served_by,
output_sanityandsystem_promptare now fields, and a dumped config
uses the TOML table namesserver,npuandimage). The retired keys
backend(nowdevice),enabled,lru(nowpriority/pinned) and a
bareimage = "<ref>"(nowimage_pin) are no longer read by the typed
loader; the conversion strips or rewrites them, and a file that still
carries one is reported with its replacement.POST /api/slots,
PUT /api/slots/{name}/config,SlotManager.create/update_configand
stacks apply refuse an undeclared key withvalidation.unknown_keys
(slot.removed_key_deniedfor a retired one) instead of persisting a file
that cannot load. A slot with such a key keeps listing and running; it
shows up inhal0 doctor profilesand the typed-load surfaces.
- The model API and the MCP
-
The
Modelobject losesdefaults(ADR-0029 §6). Its four file facts
are now top-level fields:chat_template,tokenizer_repo,mtp(the
file has MTP heads) andjinja. This applies to every read
(GET /api/models,GET /api/models/{id}, the write responses) and every
write (PUT /api/models/{id},POST /api/models,
POST /api/models/validate, thePOST /api/models/scancommit rows, and
the MCPmodel_edit,model_registerandmodel_validatetools). On a
PUT, a fact you omit keeps its stored value andnullclears it. The old
shape is refused, not translated:- A write that carries a
defaultskey at all is
422 model.defaults_removed. A scan commit row that carries one is
skipped with that code. - A top-level tuning key (
context_size,extra_args,profile,
enable_thinking,vision,n_gpu_layers,rope_freq_base,
preferred_runner) is422 model.tune_key_removed. Before this change
it would have been dropped without an error. - A fact with a value of the wrong type is
400 model.fact_invalid. This
replacesmodel.defaults_invalid, andmodel.defaults_removedreplaces
model.defaults_key_removed.
POST /api/models/feasibilityrows take the context to test from the
request:ctx(a number) orprofile(a profile name, resolved for that
model by the same rule the launch uses). Each result row now reports the
ctxit tested and itsctx_source(request,profile,nativeor
fallback). Stored rows need no conversion: the facts already had their own
columns (tokenizer_repostays in the row'sextrablob), and launch
commands are unchanged. - A write that carries a
Security
-
hal0's own loopback calls (Hindsight memory, the Hermes gateway and kanban,
brain and realtime self-calls, MCP admin, profile generation, health
probes and the CLI) no longer go throughHTTP_PROXY/HTTPS_PROXY/
ALL_PROXYwhenNO_PROXYdoes not cover loopback. Async clients use the
shared client's direct loopback route; sync clients turn the proxy
environment off for loopback URLs (#2526). -
The
/v1dispatcher no longer passes the caller's credentials to the
upstream it picks (#2538). It used to copy every request header except
Authorizationand the hop-by-hop set, so a dashboard request routed to a
remote cloud provider sent that provider the operator'shal0_session
cookie, which hal0 accepts as an admin login, along with anyX-API-Key
andX-hal0-*header. Forwarding is now an allowlist in
hal0.forwarded_headers:content-type,accept,user-agent,
anthropic-version,anthropic-beta,openai-organization,
openai-projectandopenai-beta. The upstream's own key from its
auth_value_envis still added, and multipart audio keeps its boundary.
The dispatcher now asks upstreams for an unencoded body
(accept-encoding: identity), because it relays streams byte for byte
without acontent-encodingheader. The Hermes dashboard plugin proxy uses
the same allowlist, plusIf-None-MatchandIf-Modified-Since; it
already stripped cookies andAuthorization, and now also drops
X-API-Keyand any clientX-hal0-*header. -
The REST Origin gate no longer accepts a request just because its
Origin
matches itsHost(#2327). On a DNS name that match proves nothing: a page
on an attacker's domain can rebind its own name to the box and send both
headers (DNS rebinding). The fallback now accepts only the request's own
origin on an IP-literalHost(v4 or v6), the same rule the agent-chat
WebSocket gate already used (#2326); a named origin passes only through the
allowlist.http://localhost:8080joins the built-in default allowlist,
since browsers resolvelocalhostlocally and no remote page can own it. -
Log, Sentry and failure-report redaction now mask a second secret packed
after a;or,(#2529). InX_KEY=abc;Bearer tokor
HAL0_BEARER_TOKEN=x,HF_TOKEN= hf_vthe first value used to run on to the
next space and take the second secret's name with it, sotokandhf_v
were left visible. A value now ends before a;or,that starts another
secret, and each secret is masked under its own name. A;or,
followed by anything else stays inside the masked value. The same rule
applies to the dashboard's Sentry scrubber, the doctor bundle and the
installer failure report. -
Log, Sentry and failure-report redaction no longer leave a secret's value
visible when the value before it takes in its name (#2593, #2594, #2595):
after an empty value (KEY= X_KEY= v), glued on with no separator
(Bearer a.Authorization: Token b, or concatenated headers such as
Bearer abcX-Api-Key: def), or past a quote that does not close where it
should (--api-key="v, db-password": w). Where the second name begins is
unknown (abcmay be part of the token), so the first mask now runs on
over the second secret's value instead of cutting the name out:
KEY= ***REDACTED***. A quoted secret value that never closes is masked
to the end of its line, a secret after a space in a rawAuthorization
value is masked with it, and a value of a name that is not secret, such as
max_tokens=, ends before a secret inside it. The same applies to the
dashboard's Sentry scrubber, the doctor bundle and the installer failure
report, which now also masks a rawAuthorizationvalue on a line that
holds another header with a known scheme. -
Log and Sentry redaction now mask the whole quoted value of a
*_KEY=,
HAL0_BEARER_TOKEN=orAuthorization: Bearerfield. A value that held
spaces was masked only up to the first one, soapi_key='a b'rendered as
api_key=***REDACTED*** b'and the rest of the secret stayed visible
(#2598). The value now runs to its closing quote on the same line, or to
the end of its line if it never closes; a quote left open no longer pairs
with one on a later line, so the next line is not swallowed. The doctor
bundle and the installer failure report already masked a quoted*_KEY=
value whole. -
Log, Sentry, doctor-bundle and installer failure-report redaction no longer
end a quoted secret at a backslash-escaped quote.api_key="\"a b\"" z
rendered asapi_key=***REDACTED*** b\"" z, and a value such as
'it\'s a b'leaked everything after the escape (#2604). A backslash and
the character after it now count as one unit in both kinds of quote, so an
escaped quote does not close the value and an escaped backslash does not
escape the quote after it. A shell's single quotes have no escapes, but a
log or JSON dump of a value does, and reading one as an escape can only mask
more of the line. -
Doctor-bundle and installer failure-report redaction now mask an
Authorizationvalue that opens with a quote after a known scheme, such as
Authorization: Bearer "abc def"orAuthorization: Basic 'abc def'
(Proxy-Authorizationtoo). Their scheme rule needed a first character that
was no quote, so the line came out unchanged and the credential shipped in
clear (#2605). The quoted text is masked whole with its quotes kept, an
escaped quote does not close it, and one that never closes is masked to the
end of its line. The log pass and the dashboard's Sentry scrubber already
masked a quoted Bearer value; they now also mask a quoted Basic, Token or
other scheme value, which they left visible. -
The doctor bundle's redaction no longer takes quadratic time on a long line
that holds one long secret in text that repeats its own start: 4.5 s for
320 KB ofX_KEY=a.units and 4.7 s for a 120 KBpassword="aaa…that never
closes (#2600). Masking each harvested secret everywhere it appears scanned
for all of them at every position, and each try ran far into a long one. A
secret longer than 128 characters is now masked with one plain replace, ahead
of the scan, which is linear for one such secret; the output is unchanged.
The log redaction and the dashboard's Sentry scrubber were already linear. -
Log, Sentry, doctor-bundle and installer failure-report redaction now mask a
secret whose value opens with a backslash-escaped quote, as in a JSON value
inside a log message or a command line inside a JSON field:password=\"abc def\"leakedabc def\"after the first space or quote, and
{\"password\": \"a b\"}was not masked at all (#2616). The value is read
as quoted and closed by the same delimiter, the quote after exactly as many
backslashes, so\\"and\\\"are delimiters of their own and a longer run
inside the value does not close it. It is masked whole in every place a value
can start:NAME=,NAME":,--flag value,*_KEY=,HAL0_BEARER_TOKEN=
and anAuthorizationorProxy-Authorizationscheme (the header's own quotes
may be escaped too). A value that never closes is masked to the end of its
line. A plain quoted value after a--secretflag and a space, such as
--token "abc def", is now masked to its closing quote too, instead of to the
first space. The installer report drops the escaped delimiters and the
backslashes in front of a secret name's closing quote when it masks. -
The log redaction and the dashboard's Sentry scrubber now start a
*_KEY=
name at the same place. The server tested a Unicode word boundary and the
browser an ASCII one, so a name that follows or starts withı,İ,ſ
(long s) or the Kelvin sign was masked by one and not the other: the browser
leftİprivate_key=&secretvisible, and maskedıapi_key='x y'with its
quotes kept where the server drops them (#2599). Both now use the class the
other name rules already use: the ASCII word characters and those four
letters, whichre.Ifolds ontoi,sandk. A name after any other
character starts there, soéapi_key=,secretand٣api_key=,secretare masked
by both, where neither masked them. The browser also finds names once per run
now: a long run ofıatook quadratic time. -
The installer failure report no longer takes quadratic time on a long line
of glued secrets: 1.2 s for 320 KB ofX_KEY=a.units, 12 s for 1 MB, and
6 s for the whole report at 640 KB (#2618). Reading a harvested value took its
trailing carriage return off with a${line%$'\r'}suffix match, and bash
tries every suffix length against that, so a value as long as the line cost
time quadratic in its length whenever it did not end in a CR. It now checks
the last character and cuts it with a substring, which is linear; the report
is unchanged. -
The installer failure report no longer takes quadratic time on a long value
of plain characters either: 1.3 s for 320 KB ofaafterX_KEY=and 12 s
for 1 MB (#2626). The check that a harvested value ends in a name and its
separator (api_key=) was a regex that bash backtracks over at every
position. A value longer than 8192 characters now goes togrep -Ewith the
same pattern, which runs in time linear in the value; shorter ones keep the
bash test, so the report is unchanged. -
Local inference traffic no longer goes through your HTTP proxy. With
HTTP_PROXY,HTTPS_PROXYorALL_PROXYset andNO_PROXYnot covering
loopback, hal0-api sent every/v1request for a local slot to the proxy in
cleartext: prompts, completions and uploaded audio, although none of it
needs to leave the box. The same happened to slot health and readiness
probes, the omni router's call back into hal0's own/v1, the NPU
speech-to-text and embedding route, the ComfyUI, Kokoro, Moonshine and
Qwen3-TTS clients, and the memory reranker. Requests to127.0.0.1,
localhostand[::1]now go direct. Remote providers still use the proxy,
and the dispatcher's connection limits are unchanged. (#2585) -
The installer no longer writes to guessable file names. Running as
root, it wrote the install log, the failure report, the NodeSource setup
script and the FastFlowLM.debto timestamped or fixed names in/tmpor
in the hal0-owned/var/log/hal0, and a plain redirect orcurl -o
followed any symlink a local user had planted there, so a root install
could overwrite any file. The log and the report now get a random part in
their names (install-<ts>-<random>.log,hal0-install-<random>.log,
hal0-install-report-<ts>-<random>.txt), are created owner-only with
O_EXCL and are written only through the open file. The downloads come from
mktempand are removed when the installer exits, including on an abort.
hal0 doctor bundlestill finds the log and the report. (#2407) -
The
hal0-agent@hermesservice no longer sends a loopback MCP server's
[secrets]headers to an HTTP proxy, and hal0 warns when a header would
follow a redirect. Hermes's own MCP client honoursHTTP_PROXYand
follows redirects, so on a box with a proxy set a server exposed at
127.0.0.2(say) had its header values sent to the proxy in clear text, and
every header butAuthorizationwent on to wherever an endpoint redirected.
hal0 now writesNO_PROXYandno_proxyto/etc/hal0/agents/hermes.env,
which only that service loads, coveringlocalhost,127.0.0.1,::1and
every loopback host an exposed server uses. It keeps the operator's own
entries from hal0-api's environment and/etc/hal0/api.env(set them there,
not by hand inhermes.env), writes an operator's*alone, and re-renders
the file when the host set changes (the agent picks it up on restart). This
extends the 1.4.0 wrapper fix (#2371), which covered only interactive
hermesruns. Hermes has no setting to stop following redirects, so
PATCH /api/mcp/{id}/exposure,POST /api/mcp/{id}/test,
hal0 mcp exposeandhal0 mcp testreturn or print awarningsentry
for an exposed server with a header value other thanAuthorization; the
connect-mcp guide says exposed endpoints must not redirect. (#2330)
Added
-
Memory extraction has limits now, and they are yours to set. The
Hindsight extraction LLM runs on a shared hal0 inference slot — usually one
llama-server serving one request at a time — but ran with Hindsight's own
defaults: 32 concurrent calls, 64000 completion tokens per call, and three
retries at each of two layers. On a slow box one extraction that could not
finish inside the LLM timeout held the slot, timed out, requeued itself and
took the whole retain queue with it; everyPOST /api/memory/addreturned
200 and nothing was ever stored (#1834).[memory.graph]gains
extraction_max_concurrent(default 1),extraction_max_tokens(4096),
extraction_llm_retries(1),extraction_task_retries(2) and
extraction_retry_backoff_s(120). They ride the same hindsight-api
drop-in as the extraction slot and timeout, are echoed by
GET /api/memory/graph/statusasextraction_limits, are editable on
Settings ▸ Data ▸ Memory under Extraction limits, and the shipped
hindsight-api.servicecarries the same values so a fresh install is
capped before anything is saved. A document that still cannot be extracted
now ends under Retry failed instead of occupying the slot for hours.
The two task-retry knobs are Hindsight's worker-wide settings and apply to
every queued memory operation, not only extraction; the generic
PUT /api/settingsnow refusesmemory.graph.*keys with a pointer to
PUT /api/memory/graph, the one writer that applies them; and
install.shrestarts a running hindsight-api when it replaces the unit.
The root-side drop-in validator (hal0-systemctl) learned the five
variables; a box updated throughhal0 updatekeeps its installed wrapper
untilinstall.shis re-run, and until then saving a limit reports the
wrapper as stale rather than applying. -
"Remember me" at login. Ticking it on the dashboard login (or the
in-app sign-in drawer) asks for a 30-day session instead of the 8-hour
default:POST /api/auth/loginaccepts"remember": trueand reports the
lifetime it granted assession_ttl_s. The expiry is signed into the
session cookie by the server, so a browser cannot extend it; the dashboard
keeps only whether the box was ticked, never the key. Off by default.
Opening agent chat used to re-issue an 8-hour cookie unconditionally,
which would have cut a remembered session short; the handshake now renews
a short session but never shortens a longer one. Sessions still cannot be
revoked one at a time:docs/operate/auth.mdxdocuments how to end all of
them at once by deleting the signing secret. -
Open WebUI keeps working with auth on. It used to send the placeholder
sk-hal0-localto hal0's/v1, which an auth-on box refuses, so its chat,
voice and document features answered 401.install.shnow mints a client
key (HAL0_CLIENT_KEY) whenapi.envhas none, without ever writing it to
the install log, andhal0.openwebui.env_writerwrites it into
OPENAI_API_KEYSand the STT/TTS/RAG API keys, but only where the matching
base URL still points at hal0. Those keys are recorded as hal0-managed, so
a key re-pointed at another service never receives hal0's (hal0 withdraws
only a value it wrote itself, per connection for
OPENAI_API_KEYS, so the operator's own key for that service is kept), and
removing the client key falls back to the placeholder. Rotating the client key (hal0 auth rotate client) re-renders Open WebUI's env and restarts it only when
the render changed. No admin key is minted, so a fresh install's auth
posture is unchanged. (#2314) -
sudo hal0 auth reset-key: a way back from a lost admin key. It mints
a new admin key and prints it once. It must run on the box as root, since
the key lives in the root-only/etc/hal0/api.env. Withhal0-apiup it
rotates throughPOST /api/auth/rotateand applies live; with the API down
it writesapi.envdirectly and applies on the nexthal0-apirestart.
On a box with no admin key yet it creates the first one, which arms the
v1.3.0 LAN admin gate on a LAN-bound box; the confirmation prompt says so.
(#2314) -
Open WebUI is now fully pre-wired, not just chat + voice: document
uploads route through RAG the moment an embed-capable slot is bound
(RAG_EMBEDDING_ENGINE/RAG_OPENAI_API_BASE_URL/RAG_EMBEDDING_MODEL
point at hal0's own/v1), and the image button generates through
ComfyUI the moment theimgslot is bound (ENABLE_IMAGE_GENERATION,
COMFYUI_BASE_URL, and aCOMFYUI_WORKFLOW/COMFYUI_WORKFLOW_NODES
pair baked from the SAME translator/v1/images/generationsuses, keyed
to whichever checkpoint the slot is actually bound to). Both blocks are
gated on live capability state and cleared the moment that state stops
being true — never a stale claim of a capability that isn't really there.
hal0 owns one of these keys only while it has something to claim: a value
you hand-set while it has nothing to point at (a self-hosted SearXNG's
WEB_SEARCH_ENGINE/SEARXNG_QUERY_URL, say) is left alone, and only keys
hal0 itself wrote — recorded on a# hal0-managed:header line — are
removed when their capability goes away (#2256). A seam is left for a
future search-provider extension (no search service ships today, so the
web-search keys are never touched). The env file re-renders — and the
companion restarts only when the render actually changed — on capability
apply,embed/imgslot create or delete, and every regular
component-convergence pass. The Services page's Open WebUI card now shows a
wired chip per feature (Chat, Voice, Documents, Images, Web search),
exposed viaGET /api/services. -
Installer forensics: every install now leaves evidence behind instead of
only a terminal scrollback.install.shtees the whole run to
/var/log/hal0/install-<ts>.log(falling back to/tmp/hal0-install-<ts>.log
when not root), prints that path at the start and end, and includes the
latest one inhal0 doctor bundle'slogs/section. An ERR-trapped abort
now also writeshal0-install-report-<ts>.txtnext to the log — redacted
environment (mirrorshal0.api._redact's key-name pattern), port owners,
hal0 unit status, a hardware summary, and a log tail — via the new
installer/lib/failure-report.sh, and the trap prints its path alongside
the existing step-specific recovery advice. Everyinstaller/lib/*.shand
install.sh's own top now carries thePurpose / Expects / Provides / Modder notesheader convention (documented ininstaller/README.md) so a
future decomposition of the 4,000-line script into standalone steps has a
contract to extract against.ui_stepgained an optional per-step time
estimate (EST. TIME: ~30s) and now measures real per-step durations
(UI_STEP_DURATIONS), with total elapsed time printed at the end. A new
--summary-json=PATHflag writes a versioned (hal0.install-summary.v1),
atomically-written machine-readable install summary — versions, dev/no-start
mode, network bind info, hardware class, the brain model pulled, warning/
error counts, and per-step durations — documented in
docs/getting-started/install.mdx. Container-image pulls (the OpenWebUI
background warm-cache pull ininstall.sh, and the dashboard's
runner-image pull job inhal0.registry.runner_pull) now retry transient
failures with backoff (installer/lib/pull-retry.shon the bash side,
is_retryable_pull_error/pull_backoff_delayin Python), classify
auth/404/manifest-unknown/disk-full failures as non-retryable so they fail
fast instead of spinning, and verify the image actually landed in local
storage after an apparent success before declaring victory. (#2243) -
The Doctor panel's next steps are now real, not empty.
health_report.py's
sevenhal0 doctor verifyclassifiers (API, mDNS, runners, capability
slots, memory engine, OpenWebUI, Hermes) used to hard-code
next_steps=[]on every row despite the typedDiagnosis.next_steps/
NextStep(kind="command"|"manual"|"doc")shape already existing — so
GET /api/doctorandhal0 doctor verify --jsonalways answered with an
empty list, and the dashboard'sDiagnosisPanelchips carried the
remediation command only as atitle=tooltip. Every fail/warn branch now
emits at least one realNextStep: the exact command an operator would
run (hal0 serve,hal0 slot restart <name>,systemctl restart hal0-api/hindsight-api/hal0-openwebui/hal0-agent@hermes), a doc
link, or a manual instruction where no single command exists. In the
dashboard,commandchips now show the command text inline in mono (not
only in a tooltip) and offer Copy, plus Run through the existing
typeduseServiceRepair/useSlotRestarthooks when the command maps onto
one of those actions;docchips open the link;manualchips expand.
A new StepsDrawer (numbered steps, same copy/Run/open affordances)
appears on any diagnosis with two or more next steps, reusable as-is for
a future extension-enable-steps surface. A newstatus-copy.tsmodule
gives every slot lifecycle state (offline/pulling/starting/
warming/ready/serving/idle/unloading/error) and companion-service
health word (up/stopped/down) a one-sentence, consequence-first
"what this means for you" line, surfaced as a tooltip beside the existing
precise vocabulary in the slot drawer, the slot card's status dot, and the
Services card — the precise words themselves are unchanged. (#2240) -
Parity documentation pass: an "At a glance" block and an
if-you-know-Ollama/llama.cpp/Open WebUI/LocalAI comparison table in
README.md;docs/getting-started/verify.mdx(the six-step post-install
checklist);docs/reference/support-matrix.mdx(platform/GPU-lane support,
honestly separating installer-level detection from CI- and
fleet-exercised coverage);docs/reference/validation-matrix.mdx(the
α/β/γ test tiers and therc-validaterelease-validation kit mapped onto
a hal0-specific "User Green" definition);docs/reference/generated-files.mdx
(every config file hal0 generates or rewrites, its writer, and whether
operator edits survive it); and an area→required-validation table in
CONTRIBUTING.mdmirroring the PR template's §14.1 high-risk map. (#2242) -
The dashboard steward now has a real memory. Brain chat offers it
memory_add,memory_searchandmemory_list, scoped server-side to the
hal0-brainpersona's namespace (private:hermes__hal0-brain); the model
cannot name another bank, and every othermemory_*tool stays hidden.
Before, every memory tool was filtered out and nothing stored, so "remember
that my cat is called Zorbo" got "I've noted that" with nothing saved. The
steward's ownmemory_addworks under the default[brain_chat] read_only = true, the persona and fallback prompts tell the model to call
memory_addbefore claiming it remembered anything, and a "remember ..." or
"what did I tell you ..." turn answered with no tool call is rerouted to
[brain_chat] tool_model(#2658). -
hal0 profile migratepreviews the move to profile-centric config
(ADR-0029: the model keeps file facts, a profile holds every tune, a slot
holds placement). It reads this box's slots, profiles, stacks and model
registry and prints the profiles it would generate, the slots it would
re-point and every configured value that does not reach the launch today,
such as a slot's[server].extra_args. Those inert values are dropped and
reported, never applied: the conversion keeps what each slot launches.
--applyconverts (it also runs by itself, see Migrations) and
--restore <archive>puts a conversion backup back for a downgrade. -
hal0 profile createandhal0 profile editmanage custom profiles from
the CLI. They set--flags,--ctx-size,--thinking,--vision,
--mtp,--runnerand--intent, and--fromduplicates a seed or a
custom profile first. Seeds stay read-only. -
Hermes command approvals can be switched between
smartandmanualfrom
the dashboard (Settings ▸ Security) or with
hal0 agent command-approvals [smart|manual]; with no argument it shows
the setting. This is Hermes' ownapprovals.mode, its gate on dangerous
shell commands andexecute_codescripts while the terminal tool is on.
It is separate from hal0's approval queue for gated MCP tool calls
(hal0 agent approvals).smartstays the default: yourhal0/agent
model approves a flagged command first, and if that call fails or answers
anything but APPROVE, Hermes falls back to asking you.manualasks you
every time. A change re-renders Hermes' config and restarts
hal0-agent@hermesif it is running; if a step fails, the previous
setting is restored. The setting is stored in the root-owned
/etc/hal0/agents/hermes.toml([hermes] command_approvals), which the
agent cannot write, and the admin MCP tools never expose it. An
approvals.modein/etc/hal0/agents/hermes/overrides.yamlstill wins,
and both surfaces show it as the value in effect. When Hermes'
config.yamldrifts from the expected value, both surfaces say so and
hal0 doctor allshows a warning row. Known limits: Hermes reads the mode
from its ownconfig.yaml, which the agent's user owns, so an agent with
the terminal tool on can edit it directly (re-provisioning restores your
choice). Until ADR-0024 ships, agents also authenticate with the box admin
key, so such an agent can call the API route
(PUT /api/agents/hermes/command-approvals) that changes the setting.
Changed
-
hal0 updatenow restarts slots whose runner image changed. A release
that rolls the runner image used to leave running slots on the old build
until someone ran--restart-slots, so a box reported the new version
while still serving the build the release replaced (#2096). Slots with an
image diff are restarted automatically after an apply; a slot with a
request in flight is skipped and reported as "could not be restarted
automatically: in use". The summary no longer reads "update applied"
while an image restart is pending and the command exits 2. Argv-only drift
is unchanged (banner, opt-in--restart-slots);--no-restart-slots
opts out of the CLI's restart.hal0-apialso does this once at every
start, so a dashboard "Update now" is covered too; set
[updates].auto_restart_image_drift = falseto turn that pass off. The
outcome is reported asauto_restartonGET /api/updates/slot-drift. -
The dashboard login is asked for on every page, and "View read-only"
now applies to one page at a time. On a LAN-bound box with an admin key
and enforcement off, dismissing the login would have switched the whole
browser tab to read-only. It now dismisses it for the page you are on (a
top-level section such as Slots, including its tabs, and across a reload);
moving to another section asks again. With "Require authentication" on,
the login is on every page and cannot be dismissed, as before. -
Docs drift pass: reconciled
ARCHITECTURE.md/CONTRIBUTING.md/CHANGELOG.md
with the code they describe. The Hermes provisioner section now
documents the real 12-step, uncheckpointed_INSTALL_STEPSpipeline
(not the retired 15-phase checkpointed one);BUNDLED_AGENTS's value
and order (("hermes", "pi"), with single-pick applying only between
daemon-kind agents) now matchsrc/hal0/agents/manager.py; the
hal0-agent@.serviceunit'sDocumentation=now points at a tracked
doc (docs/guides/run-agents.mdx) instead of a path that never
existed; the "hal0 keeps no ADR tree" claim is replaced with the real
rule (a publicdocs/adr/plus a gitignoreddocs/internal/adr/),
five ADRs are reconstructed from source + CHANGELOG history
(docs/adr/0004,0012,0013,0020,0023) to close the largest
"ghost citation" gaps —0012and0013carry notes on what the KB-1
key gate and v1.3.0 user-installed MCP servers changed since — and a
mislabeled ADR link indocs/concepts/memory.mdxis fixed. The stale
voice/entry (the package was fully deleted in #620, not merely
emptied) is removed fromARCHITECTURE.md's module tree, and the
"auth was removed" claims now describe the optional KB-1 key gate. No
behavior change. (#2244) -
The pinned OpenWebUI image moves to upstream v0.12.0 (manifest-list digest
sha256:156503890315…, linux/amd64 and linux/arm64). The pin lives in the
same three places as before:hal0.openwebui.image_pin, the
hal0-openwebui.serviceunit andinstaller/install.sh. The v0.12.0
release notes list no breaking changes. The port, data path and every
environment variable hal0 writes intoopenwebui.envare unchanged. New
upstream options such asENABLE_MFAdefault to off, so existing boxes
behave as before. Runhal0 update owuito move a running box. -
The dashboard follows the profile-centric config (ADR-0029). A profile now
holds the whole tune: the profile drawer gains context size (128 to
16,777,216 tokens), thinking (Auto, On, Off), vision (Auto, Off) and MTP
(Auto, Off), and loses backend, device class and quant. The profile screen
also rejects the model-fact flags (--jinja,--chat-template,
--chat-template-file,--mmproj,--no-mmproj,--mmproj-auto),
--spec-draft-deviceand--reasoning-format none. The model drawer keeps
only file facts (chat template, jinja, MTP heads) and shows the native
context window read-only. Its flag editor, "Seed from profile", divergence
diff, context input and Thinking and Vision overrides are gone. The slot
drawer drops the context ceiling, shows the effective context and where it
came from, and adds "Edit profile" and "Duplicate & edit" next to the
profile picker. The stacks editor drops its per-row MTP control, and the
Updates page drops the pending-migrations panel. A slot the conversion
could not convert gets a "Conversion blocked" badge on its card and in its
drawer, with the reason, the fix, and a note when it is still running its
pre-conversion tune. This needs a server with the matching API change. -
ADR-0027:
Runner.tier(default/optional/frozen) replaces
Runner.is_defaultas the one record of whether a runtime is a default.
Default: ROCmFPX, CPU, NPU (FastFlowLM), Kokoro speech, Moonshine
transcription, ComfyUI. Optional: Strix Halo Vulkan, Qwen3 TTS, NVIDIA
CUDA, PromptForge (legacy).GET /api/system-infobackends[]gains
tier, and itsis_defaultnow readstier == "default"(so CPU, FLM,
Kokoro, Moonshine and ComfyUI also reporttrue) until it is removed in
v2.0.0.GET /api/runner-imagesrows gainrunner,title,blurband
tier, andfamiliesgaintitleandtier. The slot and profile
drawers list default runtimes first. -
Every runtime now has a plain-language title and one-line description
saying what it runs on and when to pick it. The AMD llama.cpp runtime is
titled ROCmFPX (it was Standard, a name now reserved for the
upstream runner ADR-0025 proposes). The Runner Images page shows these in
place of theimages.jsonmaintainer notes, which move to a collapsed
"maintainer notes" toggle. -
The Runner Images page calls the per-runtime image choice the launch
image (launch imagechip, Launch images strip, Set as … launch
image button,releasesource chip), so it can no longer be read as the
runtime's tier. -
ADR-0025 open question 1 is answered: the agent slot moves to
Qwen3.8-27B on theupstreamrunner when that runner ships. -
Hermes Agent moves from v2026.7.7.2 (0.18.2) to v2026.7.20 (0.19.0),
pinned at upstream commit3ef6bbd2(#2566). This is the newest release
hal0 can install. From 0.19.1 upstream no longer builds wheels outside its
Nix build, and from 0.21.6 it needs Python 3.14, so going further needs a
new Hermes install method. hal0 now writesapprovals.mode: smart,
which is also the new upstream default. Before a dangerous terminal
command reaches you, a call to yourhal0/agentmodel approves it, denies
it or asks you. If that call fails, returns anything unexpected or times
out (up to about 90 seconds), you are asked as before. Existing boxes switch
to smart approvals the next time Hermes is provisioned. To keep being asked
about every flagged command, runhal0 agent command-approvals manual(see
Added). The plugin contracts hal0 uses (memory
provider, provider profile, plugin context, voice registries, the API
server's runs, sessions and jobs routes, and the kanban schema and run
routes) are otherwise unchanged. Upstream Hermes now gives up on a memory
prefetch after 8 seconds, so a slow Hindsight recall skips that turn's
recalled context instead of delaying the reply. -
ADR-0024: the "approval-route fix only" option is recorded as considered and
not shipped (an agent holding the admin key can log in for a cookie, and an
on-box caller passes the LAN gate);tests/api/test_agent_self_approval.py
pins agent self-approval as a strict xfail until the v1.5 tier lands. -
The bundled Pi agent moves to current upstream:
@earendil-works/pi-coding-agent
0.84.4 to 1.2.0,pi-mcp-adapter2.31.0 to 5.3.0 and
@vectorize-io/hindsight-coding-agents0.4.3 to 0.8.1. Thehal0-provider
extension follows Pi's new custom-provider stream types, and thehindsight
extension now loads upstream's own Pi binding (dist/pi.js) instead of the
Prime Agent bundle, so memories retained from Pi are labelledpirather
thanprime-agent(the bank is unchanged) and the last turn's write-back is
flushed when Pi exits. Thehal0Pi theme points at Pi's new schema URL.
Existing installs pick this up when the Pi agent is reinstalled; the first
Pi start after the bump turns Pi's built-in MCP off in
~/.pi/agent/settings.json("-builtin:mcp"underextensions) because the
adapter replaces it. -
GET/PUT /api/memory/banks/{bank}/profileno longer forward to the
memory engine's/profileendpoint, which answers410 Gonefrom
Hindsight 0.10.0 on. hal0 now builds the same response from the bank list
(name, mission, effective disposition), and aPUTwrites the disposition
traits as bank-config overrides. APUTmay now name only some traits.
hal0 memory bank profile setwrites traits and missions in one
bank-config PATCH.hal0 memory mm listand the dashboard ask for
mental-model content explicitly, because the engine's list now returns
metadata only by default. Thememory_mental_model_listMCP tool keeps the
engine's new default. Passdetail: "content"to get each model's content.
Removed
- Two agent-session planning files and the standalone HTML migration guide
are no longer tracked.docs/superpowers/plans/2026-08-29-specialty-distributions.md
anddocs/superpowers/specs/2026-08-29-specialty-distributions-design.md
were still committed although.gitignorehas excludeddocs/superpowers/
since #1988.docs/hal0-install-migration-guide.html, the v1.0-era
quick start and migration walkthrough, sat outside the.mdxdocs tree;
docs/getting-started/is the published path. Earlier CHANGELOG entries
that cite these files describe history and are left as written. No
behavior change. - hal0 no longer builds runner or toolbox images. Those images are built
only inHal0ai/hal0-runner-images. The in-repo builder (thetoolbox.yml
workflow,scripts/update-toolbox-digests.sh,packaging/toolbox/,
packaging/runner/and their tests) is gone. The workflow had re-run on a
dependency bump and moved the published
hal0-toolbox-qwen3tts:v1tag to an image that had not passed the gate. - The container launch plan no longer has a free-form
extra_argsfield
(ADR-0029 phase 4b). ComfyUI's host IPC and FLM's unlimited memlock are now
typed fields (ipc,ulimits) and render the samePodmanArgs=lines as
before, so slot units are unchanged. Every ComfyUI and FLM load used to log
acontainer.extra_args_deprecatedwarning; that warning is gone. No slot,
model or API setting ever reached this field, so no configuration changes.
TheContainerSpecalias and launch-plan mounts given as(source, target)
tuples are removed as well.
Fixed
-
hal0 doctor bundleand the MCP audit log no longer keep secrets in
clear. The bundle's journal captures and system probes were scrubbed only
for Bearer tokens and JWTs, so API keys and other credentials in the journal
ended up in a file meant to be attached to public bug reports. Every
free-text capture now gets the same shareable-text pass as the install-log
copy. The MCP audit row (mcp.tool.invoked) now masks secret arguments
before they reach journald:provider_credential_write's value, every value
inmcp_server_config_write'senvblock, and any secret-named key,
including header-style names such asx-api-key. Audit rows written before
this fix are masked when the bundle is exported. (#2434) -
Rotating the client key no longer stops memory writes. When
hal0 auth rotate clientrefreshed hindsight-api's LLM key, the engine kept
the old key until someone restarted it, so every memory extraction call
failed with 401. The rotate now restarts hindsight-api in the background
through the existing privileged helper; if that restart fails, the rotate
still succeeds and its note says how to restart by hand. (#2448) -
NPU models load after a fresh install. The installer created
.config/flmas root, so hal0-api, which runs as the hal0 user, could not
point FLM's hardcoded cache at[models].flm_store. Pulls then landed in
~/.config/flm/models, reported success with a path that did not exist,
and the NPU slot never saw the weights. The installer now gives those
directories to hal0, a pull whose store link cannot be made fails with
model.flm_store_unlinkedbefore downloading, and the reported path is
checked on disk.sudo hal0 doctor perms --fixrepairs existing boxes: it
fixes the ownership, moves stranded models into the store (it stops and
reports a name that exists in both places, deleting nothing) and relinks
the cache. Re-running the installer does the same. (#2446) -
The voice TTS picker offers Qwen3-TTS only where it can run. The
gpu-rocmQwen3-TTS row was listed on every host, including GPU-less boxes
and containers with/dev/kfdbut no render node, where the slot can never
start. It now needs the full ROCm lane, the same check as thegpu-rocm
badge and ComfyUI. A saved Qwen3-TTS selection on a host without the lane
can still be disabled. (#2447) -
The dashboard log view and the agent's
logs_tailtool show logs again.
hal0-api runs as thehal0user, which could not read the journal, so
/api/logsreturned an empty list on every box and read as "no logs". The
installer now gives the hal0-api unit, and only that unit, the
systemd-journalgroup;hal0itself is not added to the group, so the
agent units do not get it. In exchange,/api/logsand its stream accept
only hal0's own units (hal0-*,hal0.target,hindsight-api,
hermes-gateway) and refuse anything else with a 400. Every API route that
reads the journal, including slot and ComfyUI logs and the agent and MCP
activity feeds, now masks secrets in what it returns. When journalctl still
cannot read the journal, the response carries ahintsaying why instead of
an empty list, and a newhal0 doctor allrow warns when the running
hal0-api lacks the group. Existing boxes pick this up from the installer
re-run under Migrations. (#2435) -
A slot that was still loading when hal0-api started now becomes ready.
hal0-api adopts a running slot at startup; if its model server was still
loading, the slot was leftwarmingfor good, never registered for
routing, andhal0/<name>requests fell through to the fallback slot.
Adopted slots are now polled for health and promoted to ready as soon as
the server answers, and a slot that was recorded as warming before a restart
is adopted the same way. (#2442) -
Installs no longer warn that the podman seams are broken. The seam
probe passed its verb and argument as one word under the installer's
IFS, so every install printed two falsebad cmdwarnings for
hal0-podman-roandhal0-podman-rw. (#2436) -
hal0 doctor allno longer fails "Moonshine weights" on a box that does
not use Moonshine. The check ran whenever the seeded profile existed, so
every fresh install showed a red FAIL. It now runs only for slots that would
actually launch on Moonshine, decided the same way slot launch decides it,
and checks the profile each of those slots uses. (#2437) -
The installer writes its failure report on every failed exit. Only an
ERR-trapped abort wrote one, so the most common refusals (no GPU, not root,
dieandexit 1paths) left nothing to attach to a bug report. An EXIT
trap now writes the report for any non-zero exit. (#2438) -
The installer's failure report keeps numbers it does not need to hide.
A numeric value under a secret-looking key, such as
extraction_max_tokens = 4096, was treated as a secret, so every4096in
the report was masked, including context sizes and socket queue columns.
Numeric values, quoted or not, are no longer harvested as secrets. (#2439) -
hal0 mcp expose <id> --hermesno longer claims success on an upgraded
record. A record exposed to Hermes before 1.4.0 kepthermes = true, and
re-asserting it returned 200 although nothing was exposed. Any request that
asks for Hermes or brain exposure is now refused with
mcp.exposure_policy_unenforcedwhile that exposure is disabled, whatever
the record held before. (#2440) -
hal0 memory ops retry --all-faileddrains every failed operation. It
stopped at the first operation the memory engine refused with 409 (one
already requeued with its batch), so the rest were never retried. It now
skips those, keeps going, and reports how many were retried, skipped and
failed. (#2441) -
ComfyUI custom-node files no longer appear as chat models. The model
scan registered any.ggufor.safetensorsunder a ComfyUI
custom_nodes/tree as a chat LLM. Those trees are now skipped, including
sharded weight sets, and a scan removes rows it had registered
automatically from a folder it now skips. That includes a sharded model
inside a model root's own folder named like a ComfyUI accessory directory
(for exampleembeddings/orloras/). Rows you added by hand, pulled, or
that a slot, stack, capability or chat setting references are kept, and the
clean-up is skipped when any of those configs fails to load.
make release-testalso takesHAL0_TEST_LLM_MODELto pin the model its
LLM rows use. (#2443) -
hal0 agent bootstrap hermes --dry-runno longer performs a real
install.--dry-runonly skipped writing the report, and--skip-phase
did nothing. Both flags are now refused with exit 2 and a pointer to
hal0 agent status hermes, since the installer has no preview mode. (#2445) -
The ROCm lane is offered only where a slot can open it. The picker,
the hardware recommender, the install seed and the default device for
hal0 slot createtreated a successfulrocm-smias enough, so a
container with/dev/kfdand ROCm tools but no render node was still
offered ROCm and failed at slot load. All four now require/dev/kfdand a
render node. A box with both nodes but norocm-sminow defaults to ROCm
rather than Vulkan forhal0 slot create, matching the seed. (#2452) -
Five stale
make harnessrows are fixed. These δ-tier rows had
drifted from intended behaviour: an mtime check that the installer's
by-designapi.envrewrite always tripped, two agent rows that expected a
data directory the provisioner now owns, a memory teardown fixture with a
stale agent id, and a bootstrap row that now pins the--dry-runrefusal.
(#2444) -
hal0 mcp testshows each tool's real verdict again. While exposure
to Hermes and the brain profile is disabled (#2358), no installed server is
in the agent's policy mirror, so every tool readunknown_server. Until
#2303 the preview now classifies against the server's own[tools]policy
(allow/gated/blocked, elseunknown_tool), in the same order the
agent client uses. (#2415) -
make release-testnow asserts that slots unload. The γ release gate
recorded each row after load and chat, but the onlyslot unloadcalls were
in the EXIT-trap cleanup, which ignored failures and wrote no row, so a
release that left slots wedged on unload still passed. A newunloadrow
runs after the chat rows: every slot the run loaded must exit 0 from
hal0 slot unloadand then showofflineinslot list --json, or the row
fails and names the slot; it skips when nothing was loaded. The cleanup
stays as a safety net and the report schema is unchanged. (#2351) -
The installer's failure report no longer copies secrets out of the
install log. The report masked environment values by key name only and
then copied the install log tail verbatim, so any token the installer had
echoed ended up in a file presented as safe to share. Now every value of a
secret-named key (from the installer's variables,api.envand
hal0.toml, including every line of a multi-line"""or'''
string and every element of an array) is masked wherever it appears in the report, along with
Bearer tokens, a rawAuthorization:header value, credentials in URLs
and--tokenflags. Run-together and
camelCase names such asapikey,apiKeyandaccessKeynow count as
secret-named here and in hal0's config redaction, and a value seen as
NAME: value(a JSON field, YAML key or header) is masked wherever it
reappears, like one seen asNAME=value. A number, short
word or variable name logged as a value (max_tokens=4096,
key=OPENAI_API_KEY) is not taken for a secret, so it is not blanked out
across the rest of the report. If that pass
cannot run, the report keeps only the failed step. The report also gains
systemctl --failed,podman info/podman images, the redacted
api.env/hal0.tomland the hal0-api journal, runs every diagnostic
under a 10-second timeout so a hung Podman cannot stall the abort, and is
now readable by root only.hal0 doctor bundlelists the latest install
log and report underinstall_artifactsinmanifest.jsonand includes
the report; both the report and the install log it copies now go through
the same secret masking as the installer's report, including URL
passwords,--tokenflags,hf_/sk-tokens and values reused
elsewhere in the log. The same masking now also covers/api/logs, the
MCP log tools, event messages and activity rows, which missed values such
asHF_TOKEN=,apikey=and"apiKey":(#2307). -
Interactive
hermesand the user-level gateway now bypass the proxy for
loopback. Only thehal0-agent@hermesunit loads
/etc/hal0/agents/hermes.env, so thehermesandhal0-hermeswrappers
passed on the caller's environment untouched: withHTTP_PROXYset, MCP
calls to loopback servers, and their[secrets]headers, could go through
the proxy. Both wrappers now addlocalhost,127.0.0.1,::1toNO_PROXYand
no_proxy, keeping any existing entries, and leave a bare*alone. This
finishes the #2330 fix. (#2371) -
A user-installed MCP server with
gatedorblockedtools is no longer
handed to Hermes or the brain profile. Both call the server's URL
directly, and nothing on that path enforces the[tools]policy yet
(#2303), so an agent could call the very tools the operator restricted
whilehal0 mcp testreported them blocked.PATCH /api/mcp/{id}/exposure(andhal0 mcp expose) now refuses turning
hermes/brainon for such a server with
409 mcp.exposure_policy_unenforced; withdrawing exposure still works.
A server already exposed, or one that gains agated/blockedentry
later, is left out of the join and logged, so the next sync (or
hal0-api's startup reconcile) removes it from Hermes's config. This is an
interim guard until hal0 sits on the agent's call path. (#2343) -
A container with
/dev/kfdbut no render node is no longer treated as
able to run ROCm. Four places took/dev/kfdalone as proof of the ROCm
lane, but a ROCm slot also opens a/dev/dri/renderD*node. On an LXC
where only kfd was forwarded, preflight reports no usable GPU and offers a
CPU-only install, yet every llama.cpp seed still came outgpu-rocm(#2313);
the capabilities picker offered the GPU (ROCm) row and ComfyUI (#2354);
the hardware recommendation saidrocm(#2355); andhal0 updatemoved
stalegpu-vulkanslots togpu-rocm, a device they cannot open, instead
ofcpu(#2356). All four now ask one check that needs both nodes, as does
applying a profile whose runtime is ROCm-only. -
A stable or preview release now fails its build when
CHANGELOG.mdhas
no section for it.scripts/gen_release_notes.pyused to fall back to a
git log when the## [<version>]header was missing or empty, so the
release still published butrelease.jsoncarried empty highlights,
breaking and migrations lists andhal0 updateshowed no callout. It now
exits non-zero, writes nothing, and names the tag and the header to add.
Nightly releases still use the git log. A stable tag also no longer picks
up its own pre-release section (v1.4.0used to match## [1.4.0-rc.1]),
which would have shipped the rc's notes and slipped past that check.
(#2255, #2345) -
The install log is now owner-only (0600).
installer/lib/logging.sh
created/var/log/hal0/install-<ts>.log, and its/tmpfallback, mode
0644 and tees installer output into it unredacted, so every local user
could read it. Both paths are now created underumask 077and pinned to
0600.hal0 doctor bundlerun as a non-root user can no longer read a
root-owned log and leaveslogs/install.logout of the bundle; run it with
sudoto include it. (#2361) -
hal0 memory statusno longer reportsWrites landingwhile retains are
stuck. The verdict only watched the engine'sfailedcounter for growth,
so retains wedged inpending/processing— which never fail — read green
on a store that had never held a fact, and a brief failure spike went green
again when its 600-second hold ran out with nothing recovered. Writes now
reportFAILING — retain_operations_stalledwhen a retain operation has
been in flight for over 600 seconds (the threshold the memory engine's own
worker logs as stuck) and no retain has completed in that time; the first
completed retain clears it. Other engine work, such as consolidation, does
not count either way (#1833). -
A queued benchmark for a model that can't be resolved now fails visibly.
When a queued model reference matched no registry model, or was a basename
that several models share (such asmodel.gguf), the bench worker planned
a suite that selected nothing and dropped the item without a trace. It now
takes the item off the queue with afailedoutcome and a note
("unknown model" or "ambiguous model reference"), and
GET /api/benchmarks/queuelists the recent ones underfailed. (#2387) -
An installed
sseMCP server exposed to Hermes or the brain profile is
now opened as SSE. The join wrote onlytype: sse, but the pinned Hermes
picks its MCP client bytransportand reads notypekey, so every
joined SSE server was driven as Streamable-HTTP and failed at connect or
tools/list. The join now writestransport(sseorhttp) to both
Hermes'sconfig.yamland the brain profile, and hal0-api's startup
reconcile rewrites an SSE join already on disk without it. (#2331) -
Agents saving memories over MCP get the same honest refusal as the web
API when the memory extraction model's window is too small. The
memory_addtool on/mcp/memory(and thememory_addtool on
/mcp/admin) now runs the context-window checkPOST /api/memory/add
gained in #1903, and returns amemory.extraction_ctx_too_smallerror
naming the slot and both window sizes instead of reporting a write that the
memory engine would then drop (#1930). -
Pull and status now see the image of a slot with no profile. A slot
with noprofileand noimage_pinwhose type routes it to a fixed
runtime — a baretype = "tts"slot falls back to Kokoro, a bare
type = "image"slot to ComfyUI — launches that runtime's image, but
"pull image" found nothing to pull and/api/slotsreported the slot as
not-configured(no expected image). Both now resolve the image the slot
launches. A bare llama slot with no profile or pin still reports
not-configured. (#2389) -
hal0 agent install pimaps the right checkout into the shared
coding-agent::hal0-monomemory bank. The seeded
~/.hindsight/coding-agent.jsonmapped one fixed maintainer path into that
bank, so on every other install the mapping never matched and the hal0 repo's
coding memory silently landed in a per-project bank instead. The driver now
maps the hal0 git checkout it is running from, and seeds no mapping on a
packaged install that has no checkout. An existingcoding-agent.jsonis
still never overwritten. (#2315) -
The OAuth "Connected accounts" panel is reachable from the dashboard.
v1.3.0 shipped it only inside the old Connections page, which the
dashboard redirects to Slots ▸ Endpoints, so no menu led to it and
connecting an account neededhal0 oauth. It now lives at
Settings ▸ Integrations ▸ Connected Accounts (#settings/accounts,
also reachable as#settings/oauthand from the command palette). (#2267) -
hal0 slot migrate-* --apply --stop-servicesno longer leaves hal0
down when the run refuses or fails. The flag stoppedhal0-apiand every
hal0-slot@*unit before the run's own checks, and nothing ever started
them again, so amigrate-flagsdivergent-tune refusal, a declined prompt,
or a backup or write error took the API and every slot offline until
someone restarted them by hand.migrate-flags,migrate-hw,
migrate-capsandmigrate-id-keyingnow stop nothing until the
preflight and the confirmation prompt pass, and start every unit they
stopped again when the run refuses, fails, is interrupted, or only
partly applies. A successful run still leaves them stopped for you to
restart, and now names them. (#2325) -
hal0 capabilities migrate --applyno longer clears NPU selections when
flm listfails (#2333). A failed hostflm list -jprobe (binary not
runnable for the invoking user, a timeout, a non-zero exit, unparseable
output) used to read as "FLM serves nothing", so every NPU (FLM) selection
was classified as "model not in catalog", and--applycleared its model
and disabled it. The FLM catalog probe now keeps "no answer" apart from an
empty answer (flm_catalog()returnsNone), andmigrateleaves NPU
selections untouched and lists them as skipped when the probe did not
answer. Other selections still migrate, and a model thatflm list
answers without is still cleared as before. -
An FLM pull runs
flm list -jonce, not once per progress tick
(#2379). While a pull's install path was unresolved, every progress tick
(about once a second) re-ran the uncachedflm list -jprobe, for the
whole download when a tag's path never resolves. The pull now keeps the
first non-emptyflm listanswer for its lifetime and probes again only
while the answer is empty. -
An FLM pull no longer stalls hal0-api while it measures the model
directory (#2380). To report progress, an FLM pull walks the model's
directory and sums its file sizes: once at the start, on every progress
tick, and once at the end. Each walk ran on the event loop, so on a large
or slow models directory every other request, SSE stream and WebSocket
waited for it, for the whole download. The walks now run on a worker
thread. -
The benchmark roster shows every model stored as
<dir>/model.gguf.
The roster grouped benchmark records by the gguf file name, so models
pulled into their own directory (all namedmodel.gguf) collapsed into
one row, and that row carried the other models' run count and last-run
date. Where several files share a name the roster now tells them apart by
full path, and it only matches a registry model by file name when that
name is unique among both the registry's models and the benchmarked ones.
A model recorded under both a registry id and an old v1 path-like id, or
re-pointed at a new file under the same id, still shows as one row (#1825). Queuing such a row with "+"
also benchmarks that model, not whichevermodel.ggufthe registry
listed first (#2346). -
make harnessno longer reportsharness OKwhen it never loaded a
slot. The δ harness installs with--dev, wherehal0 slot loadcannot
start a systemd slot unit, soruntime-slot-loadis recordeddeferred
and the chat round-trip is skipped — yetscripts/harness.shcounted only
failrows and printed a bareharness OK. A deferred slot load now fails
the run unless it is allowed with--allow-deferredor
HAL0_HARNESS_ALLOW_DEFERRED=1, and an allowed run names the unverified
rows on its OK line. The report schema and row names are unchanged. γ
(make release-test) remains the tier that loads real slots; making
--devslot load work is #2377. (#2349) -
An omni
generate_imagecall no longer strands its own caller on a
single-GPU box. The image render switches the GPU to exclusive image
mode, which unloads the calling LLM slot; the loop's next chat round then
failed withgpu.image_mode, the rendered image was never returned, and
the GPU stayed parked in image mode for the idle window until someone
switched it back by hand (#2191). The loop now restores LLM mode once per
tool round, after every image in the round has rendered and ComfyUI's
queue is empty (so another request's render is never cut short), so the
caller is loaded again when asked to fold the images in; the reload per
image round is the cost of sharing GPU memory. The restore itself cannot
be cut short by the client leaving. A caller that was never evicted
(NPU/CPU, or a GPU that did not flip) is left alone. If image mode is
pinned or the restore fails, the loop does not ask the unloaded model
again: it returns a completion that lists every image the request
rendered, with every tool result it produced, and the reason (hal0.omni.caller_slot_unavailable, with the
verbatim tool results alongside). A request cancelled mid-render restores LLM mode in
the background instead of leaving the GPU parked. -
A slow or hung
flm listno longer freezes hal0-api (#2334). The FLM
catalog probe runs hostflm list -jsynchronously with a 30 s timeout
whenever its 5-minute cache is cold (after boot, after the TTL, after
every FLM pull), and async request handlers called it on the event loop:
GET /api/models,GET /api/capabilitiesand capability changes,
GET /api/npu/occupancy, hardware live stats, slot load/swap and the FLM
model list,/v1requests for an-FLMmodel id, the slot manager's FLM
tag resolution, and the FLM pull job. While it ran, every other request,
SSE stream and WebSocket waited. Those paths now probe a cold cache on a
worker thread (flm_served_models_async()/flm_id_to_tag_async());
sync callers such as the CLI are unchanged. -
hal0 doctor alldescribes the LAN admin gate correctly. On a
LAN-bound box with an admin key and auth off, itsauthrow said mutating
routes need a sign-in "but reads and inference stay open". The gate works
per route class: ADMIN reads (settings, memory, logs, approvals) are gated
too, and CLIENT routes (/v1inference,/mcp/memory) stay open, writes
included. The row now says that. With no admin key, it and the Settings ▸
Security page now say every route is open to the LAN and point atsudo hal0 auth reset-keyto mint one, instead ofhal0 auth rotate admin,
which never prints the key. (#2314) -
Loading a stack now tells you which running slots it will unload.
Applying a stack replaces the running lineup: every running slot the stack
does not name is unloaded. The Load dialog never said so, and the result
toast read "loaded" even when it had just stopped several other slots. The
dry-run (POST /api/stacks/{slug}/apply?dry_run=true) now returns an
unloadslist, the Load dialog shows it before you confirm, and the toast
names what was unloaded. The dialog also no longer claims slots with a
missing model are skipped, and the import dialog no longer claims they
import disabled: both now say those slots will fail to load until the model
is available. (#1511) -
The dashboard session cookie is marked
Securewhen the browser reached
hal0 over TLS.hal0-apilistens on plain HTTP and the documented
deployment terminates TLS at a reverse proxy, but the session cookie was
always set withoutSecure, so a browser that signed in over HTTPS would
still send it in cleartext to a plain-HTTP address on the same host.
Login, logout and the agent-chat handshake now setSecurewhen the
request itself ishttpsor a proxy says so withX-Forwarded-Proto.
Plain-HTTP installs are unchanged. (Flagged in review of #2338; the gap
predates it, but a 30-day "remember me" session made it matter more.) -
A Runner Images default for the voice, speech, NPU and image-generation
families now takes effect.[slots].default_imagesacceptscomfyui,
flm,kokoro,moonshineandqwen3tts, and the dashboard reported the
override as in effect, but only llama slots read it; those five providers
still launched the registry default. They now resolve their image in the
same order as llama slots — slotimage_pin, then the family default, then
the registry default — through one shared helper. The image pull, slot
status, drift check and load-time GPU preflight now resolve that same ref
too: for these slots (the shippedimg,qwen3tts,ttsandflm
slots included) they used to derive a llama runner image from the slot's
device, so the pull fetched the wrong image, status reported an image
mismatch and the drift check flagged an image change that was not there,
with or without a family default. (#2234) -
A LAN-bound box with an admin key no longer leaves the dashboard half
signed out. Since v1.3.0 such a box refuses ADMIN-class requests from
off-box callers even with "Require authentication" off — and that rule is
per route class, so it covers ADMIN reads (settings, memory, logs, the
activity stream, approvals, services), not only changes. The dashboard only
handled a refused change: pages rendered empty while their reads 401'd
every poll, the top bar gave no hint of a session, and the admin-key drawer
appeared only if you happened to try a mutation. NowGET /api/auth/status
reports two per-caller fields —admin_gated(are this caller's ADMIN
requests enforced at all) andadmin_sign_in_required(would one be
refused right now), computed by the same predicates the enforcement
middleware runs — and the dashboard keys on them: the login screen appears at
load (worded for this posture, with a View read-only way past it), a
top-bar chip shows Sign in or Admin · Log out, a session that
lapses mid-use brings the login screen back instead of silent 401s, and
signing in refetches every panel rather than only the retried action.
Whenever the login screen takes over — after a log-out or a lapsed
session — the dashboard now discards everything it fetched under that
session, so the next person at that browser cannot read the previous
admin's cached pages through View read-only. The Security page no
longer claims that reads stay open in this posture. Boxes with no admin
key, and browsers on the box itself, see no change. -
The memory bank's Web view no longer greys out the whole graph when you
pick a tag or a fact type. The view looked for a fact's tag and type on
each graph node, but the memory engine only sends those in the graph
response's per-fact side table, so on a real bank every dot failed the
filter and dots were never coloured by type. It now reads them from that
table, so the matching facts stay lit and each dot takes its type's
colour. (#1996) -
Security: the
hal0CLI no longer hands this box's API key to another
host. The CLI attached the key it found in/etc/hal0/api.env(the admin
key, or the client key on a single-key box) to whateverHAL0_API_URL
named, so pointing the CLI at another machine, or at a mistyped address,
sent that machine this box's admin key in theAuthorizationheader. A key
found inapi.envnow goes only to a loopback API (127.0.0.1,::1or
localhost;127.0.0.2, look-alike hostnames and URLs carrying
user@credentials do not count). A key you exported yourself as
HAL0_ADMIN_KEYorHAL0_CLIENT_KEYis still sent to a remote target,
since exporting it is a deliberate choice. The CLI also no longer follows
redirects, so a3xxcannot carry the key to a different address. When a
remote target answers 401 and a discovered key was held back, the error
says so and points atHAL0_ADMIN_KEY; it never prints the key. This
covershal0 chatand thehal0 setupprobes too. (#2499) -
Security: the
hal0CLI no longer sends local API requests through your
HTTP proxy. WithHTTP_PROXYorALL_PROXYset andNO_PROXYnot
covering loopback, every CLI request to the local API went to the proxy in
cleartext, bearer key included, so a corporate or system proxy saw this
box's key. Requests to a loopback API now skip the proxy environment and go
straight to the box; requests to a remote API still honour it, and
SSL_CERT_FILE/SSL_CERT_DIRstill apply. (#2503) -
Security: benchmarks no longer send the box key through your HTTP proxy
or along a redirect. The bench planner, runner, slot A/B harness and
hal0 bench uploadsent their loopback requests, bearer key included,
throughHTTP_PROXYwhenNO_PROXYdid not cover loopback, and the upload
token was copied onto any redirect. Loopback bench requests now skip the
proxy and refuse a redirect to another host, the upload token is never
forwarded on a redirect, and Tool Bench adds the loopback addresses to the
NO_PROXYit passes on while keeping your own entries. Remote targets
still use the proxy. (#2505) -
Queued benchmarks run on a box that requires authentication. With
"Require authentication" on, the bench worker called hal0-api with no key
and got 401: it could not list models, so the queued item was re-queued
twice and then dropped; it read the GPU as busy forever and found no slot;
and the slot-tuning and Tool Bench cells failed the same way. Every bench
call now carries the box key (the client key where the route allows it, the
admin key only for the two slot-changing calls), read from the environment
and then/etc/hal0/api.env, which the worker's user can already read.
The key is attached only when the bench is pointed at the box's own
loopback address and is never forwarded on a redirect, so--apior
HAL0_BENCH_APIaimed at another host sends nothing. (#2478) -
Header-style secret names are masked like their underscore spelling.
The text redaction behindhal0 doctor bundle, live log views and the
installer's failure report matched a secret word only when the rest of the
name used letters, digits and underscores. A hyphen after the secret word
ended the match, soapi-token-prod=…,github-token-ci=…,
token-type: …andanthropic-ratelimit-tokens-*passed through with
their values visible, whileapi_token_prodwas masked. Names now run over
hyphens the same way they run over underscores, in both the Python
redaction and the installer. Names that are locations stay readable
(api-token-file,token-env), as do plain counts such asmax-tokens.
(#2488) -
Plural secret names such as
api_tokensare no longer read as token
counts. The rule that keeps counts likemax_tokensreadable also
skipped any name containingtokens, so a credential stored under
api_tokens,auth_tokensortokens_by_hostwas left unmasked in the
doctor bundle, the MCP audit log and live log views.tokensis now
treated as a count only after a counting word (max_,prompt_,
completion_,total_and the like) or in a count shape (tokens_count,
tokens_per_second);max_tokensandcached_tokensstay readable and
api_tokensis masked. (#2466) -
A fresh install no longer reports itself as a pre-v1.0 box. The seeded
brainslot, bound to a pulled model, was counted byhal0 slot migrate-flagsas an outstanding flag fold, so every new box warned of a
"pre-v1.0 slot/model shape" and the firsthal0 updateexited 2
("convergence outstanding"). That slot already launches with its profile's
flags, so folding them into the model would change nothing. The check now
treats a model as converged when every slot bound to it names a profile
that fits, has no flags of its own, and the model has no tune of its own.
Boxes with real pre-v1.0 slot flags are still detected. (#2476) -
HAL0_PULL_RETRY_DELAYStakes effect in the installer. The documented
space-separated list ("2 4 8") never split under the installer's shell
settings, so the override was read as one bad value and the default
schedule applied silently. It now splits on spaces. (#2470) -
The installer's package install hint stays on one line. A hint for
several missing packages was broken across lines, so pasting it installed
only the first. It is now joined with spaces. (#2471) -
make release-testno longer fails the Vulkan and ROCm rows on a box
that has an NPU model. WithHAL0_TEST_LLM_MODELunset, the default
model was the first installedllmentry, which could be an FLM (NPU)
model that llama-server cannot load. FLM and NPU models are now skipped
when picking the default; an explicitHAL0_TEST_LLM_MODELis still used
as given. (#2477) -
The release check's GitHub-checks gate no longer fails on a green
main, and no longer passes when GitHub cannot be asked. The gate
rejected any check that was notsuccess, so the nightly's two
deliberately skipped jobs failed it, while an error fromgh apiwas
treated as all clear. It now requirespython (3.12),uiand
γ-suite (chromium)to succeed, acceptsskippedandneutralfor
the rest, fails onfailure,cancelledortimed_out, counts only the
latest run of each check, reads every page of results, and fails when the
query errors or a required check is missing. (#2479) -
hal0 doctor verifyandGET /api/doctorno longer report "3 active: embed, img,
voice" on a fresh install: a capability group now counts as active only when
at least one of its selections is enabled, so the "none active" warning and its
enable next step are reachable again (#2736). -
A slot unload whose stop outlives the 30 s terminate budget now converges to
offlineonce the teardown finishes (with a one-time "converged to offline" message) instead of sticking inerror; the upstream is deregistered when the late teardown lands, and_wake_capability_slotruns the convergence so capability requests auto-load the slot after an api restart instead of 404ingdispatch.no_route(#2738). -
tts: the Kokoro slot now launches from the bundle
hal0 model pull kokoro-v1installs, instead of a hardcoded/mnt/ai-models/local/...
path that a fresh install never has. Before, every tts start downloaded
about 354 MB from GitHub into the container's throwaway filesystem, and a
box without internet access could not speak. The pull now installs
kokoro-v1.0.onnxandvoices-v1.0.binunder<store>/tts/kokoro-v1/,
the provider passes that directory as--model_pathfrom the configured
store, and the seededkokoroprofile names no path. A missing bundle
fails the slot withslot.weights_missingnaming the pull command. On
upgrade,hal0 updatepulls the bundle once when kokoro is in use and the
store lacks it, replacing the mislabelledmodel.ggufan earlier pull left.
If the box is offline the update still succeeds, and the remedy
(hal0 model pull kokoro-v1) is reported and named by the slot's error
(ADR-0030, #2739). -
api:
systemctl stop/restart hal0-apino longer hangs until systemd's
SIGKILL when a chat request is in flight. Cancelling that request flips its
slot serving -> ready, so the model-cache refresher was usually mid
/v1/modelsfetch when shutdown cancelled it; the HTTP client swallowed the
cancel and the refresher went back to waiting for events forever, leaving the
API down for ~100 s on most restarts with memory on. The refresher now honours
a swallowed cancel, every lifespan teardown step is bounded at 3 s (a stuck
step is logged ashal0.api.shutdown_step_timed_outand the shutdown
continues), and the installer writesTimeoutStopSec=30into
hal0-api.service(fresh installs and installer re-runs;hal0 updatedoes
not rewrite that unit) (#2727). -
slots: a stopped slot whose state record kept a stale
model_idno
longer claims it is "still serving" a model after its[model].defaultwas
emptied; the "still serving" wording is now only used while the slot is live
(ready/idle/serving), and an offline slot gets the plain "no model assigned"
message (#2715, follow-up to #2694). -
catalog: removed the three
whispercppSTT rows (Whisper-Tiny, Whisper-Base,
Whisper-Large-v3-Turbo) and thewhispercppprovider mapping: hal0 never
shipped a whisper.cpp runtime (ADR-0001), so the picker offered a model no
slot could serve. The omni bundle manifests no longer list them, and a
voice.sttselection whose provider has no runtime profile now fails with
capability.no_engine_for_providerinstead of being rebound to the Moonshine
runner (#2691). -
normalize: a
/v1request carryingresponse_format(json_object/json_schema) or a llama.cppgrammar/json_schemanow gets the defaultchat_template_kwargs.enable_thinking=falselike any other request, so a Qwen3-family thinking-by-default model on the Hindsight extraction slot no longer spends the whole completion budget reasoning and returns 0 facts on every retain. The #2020 exemption now applies only to a model whose registrychat_templateishal0-brain-sft, and a profile's explicitenable_thinking = falseis honoured on grammar requests on every template (#2713). -
slots: healing a type-less slot TOML no longer stamps
type = "llm"on a rerank/embed/tts slot: the type now follows the profile, then the slot name. The output-sanity gate also skips anllm-typed slot whose profile or launch flags carry--reranking/--embedding, andhal0 doctor profiles --fixrewrites an already-healedtype = "llm"on a non-chat profile; the failure message now names that remedy (#2693). -
updater: a slot whose model the runner-system conversion unassigned (a
ROCmFPX/ROCmFP4/IU4/ActiveFPX quant no runner can load, ADR-0028 §7) is now
reported where the operator sees it.hal0 updateprints one line per slot
naming the slot, the dropped model and the fix (hal0 slot edit <slot> --model <id>, withlfm2.5-2.6bas the brain example); before, the warning
reached only the hal0-api journal (#2694). A slot still serving a model
whose[model].defaultis empty now carries amessageon/api/slots
saying so, instead of reading idle withmessagenull. Its render error
says "has no model assigned ([model].default is empty)" with the same fix,
instead of blaming a registry lookup (#2695). The exit-2 verdict no longer
sends a slot whose launch command does not render to
hal0 update --restart-slots, which skips such a slot; it names the slot's
own fix instead (#2696). -
capabilities: saving voice.stt with no device (the dashboard sends only
{model, enabled}and a fresh install seedsdevice="") no longer falls back to agpu-rocmstt slot that 503s on a host without/dev/kfd. The device is now resolved server-side from the advertised backends (non-GPU, supported by the selected model), and a host where nothing fits gets 400capability.no_engine_for_devicebefore any slot file is written (#2690). -
slots: the seed
utilityandagentprofiles now launch llama-server with--parallel 1. Without it llama-server ran-np auto(4 sequences sharing one 65,536-cell unified KV pool), so one combined overflow returned HTTP 500 "Context size has been exceeded" to every in-flight request on the slot, including unrelated Hindsight extraction/consolidation calls and Hermes turns. Capacity is now deterministic; concurrent callers queue instead of failing each other (#2692). -
slots: the seed
brainprofile now launches llama-server with--parallel 1too (#2692 pinned onlyutilityandagent). Without it brain ran-np auto: 4 sequences on one 65,536-cell unified KV pool, so roughly four concurrent steward rounds (~16k tokens each) overflowed the pool and every in-flight brain request got HTTP 500 "Context size has been exceeded". Concurrent brain callers now queue instead (#2714). -
mcp: the hal0-admin
slot_listresult is now a compact roster (name, id, state,loaded, model id, backend, port, container health, image tag) sent as one compact-JSON text block with no duplicatestructuredContent, instead of ~2.9 KB per slot. Hermes 0.19.0 cuts any tool result over its per-result budget (39,321 chars at 65,536 ctx) to a 1,500-char preview onhermes -zturns, so a box with 14+ slots was answered from a truncated first slot.resolved_command, the fullimageref and nested tuning are no longer in the list;slot_status/slot_by_namestill return the full row (#2662). -
ui: a vulkan-only GPU (render node passed through, no
/dev/kfd) now counts as a GPU on the Overview utilization row and health strip and on the Settings Live-hardware and Power & thermal panels, instead of reading "no GPU access" while a vulkan slot is serving; the NPU rows gate on the NPU probe rather than the GPU one (#2674). -
A profile whose
profiles.tomlentry fails validation is no longer
invisible. It is quarantined as before, butGET /api/profilesand
hal0 profile listnow show it as a read-only row with the reason, the
Profiles page lists it under "Needs attention", andhal0 profile delete
(or the page) removes it. Theprofiles.invalidmessage on a slot bound to it
now says to recreate the profile under the same name or delete it, instead of
pointing athal0 profile edit, which cannot see a quarantined entry (#2653). -
A host with an NPU but no FLM runner image no longer lists NPU-only models
in the capability catalogs, and a capability apply naming backendnpuon
it (API orhal0 capabilities set ... --backend npu) is refused with 400
capability.backend_unavailableinstead of returning 200pending_reload
and committing a selection that can never serve (#2659). -
The post-update runner-system and profile-centric conversions no longer
fail on a box whose/var/lib/hal0/backupsa root-run tool created as
root:hal0 2755. Before,hal0-api(User=hal0) could not write the
backup, reported "nothing was changed", and every restart hit the same
error, so the box stayed unconverted. The conversion now makes a
directory it owns2775, and otherwise fails with the directory, its
owner and mode, and the fix.hal0 doctor permsnow reports and fixes the
directory (hal0:hal0 2775),install.shcreates it that way, and the
hal0 update"Migration incomplete" panel names the fix instead of only
saying to restart (#2657). -
A running slot whose launch command no longer renders (for example a
profile-centric slot that still carries[model].context_size) now shows
up inGET /api/updates/slot-driftand thehal0 updatesummary with its
image drift and arender_errorthat says why. Before, the drift check
returned nothing for it, so the summary said "no slots need restart" while
/api/slotsreportedimage_mismatch. The automatic image-drift restart,
the CLI's post-apply restart,hal0 update --restart-slots,slot load's
re-converge, and an unnamedPOST /api/updates/restart-slotsleave such a
slot running and list it underskipped_render_error, since a restart
would fail to render and take it down. Naming the slot in
POST /api/updates/restart-slots(withoutskip_busy) still restarts it
(#2660). -
The brain's live-state grounding guard (#2022) no longer lets "why is/are/do/does"
phrasing hide a live-state question, and now recognises services, health,
memory and hindsight as subjects. "Why is the agent slot offline?" and "Is the
hindsight memory service healthy?" under a brevity clause were answered with
zero tool calls and an invented answer on the shipped 1.1B steward; they are
now rerouted to the tool model like any other ungrounded live-state answer
(#2675). -
hal0 update --rollbacknow waits up to 300 s for the server instead of
giving up after 10 s. The route re-pips the prior tree synchronously (13 s on
a 32-core box), so the CLI used to printdid not respond within 10sand
exit 1 while the rollback completed andhal0.previousflipped, hiding the
reverted-to version and restart panel and inviting a second--rollbackthat
undid the first. If the wait still expires, the CLI says the server may still
be rolling back, how to check, and that re-running--rollbackwould flip
back (#2661). -
A stack row whose slot the profile-centric conversion blocks now still drops
the retired tuning keys (vision,mtp,enable_thinking,
server_extra_args). Only the profile re-point is reverted. Before, the
row kept them,StackSlotEntryrejected the rewrittenstacks.toml, and
every/api/stackscall failed until the block cleared. The migration plan
no longer reports a drop it reverted (#2649). -
Hermes's main model and its auxiliary role map now agree on one slot. On a
box with noagentslot,_resolve_primary_slotand the role resolver both
take the first ready LLM slot in pool order (chat/primaryare no longer
special, per ADR-0023), and log the fallback as
hermes.primary_slot_fallback/role_slots.main_fallbackinstead of
choosing silently (#2635). -
omni: a pinned image mode is now reported at once instead of after the
ComfyUI queue wait, and a chat request waits at most 15 s for the caller
restore (the detached restore keeps its 600 s queue wait), so a busy or
unreadable/queueno longer holds a request for ten minutes and ends in
gpu.queue_busy(#2639). -
The boot-time image-drift restart no longer bounces slots it cannot prove idle. A slot with no
/metricsreading (ComfyUI, Kokoro, Qwen3-TTS, Moonshine, FLM, llama slots without--metrics, or a timed-out scrape) used to count as idle and could be restarted mid-request; it now lands underskipped_busy. The explicithal0 update --restart-slotskeeps its behaviour. The runner-catalogue boot sync task is also cancelled at shutdown. (#2634) -
hal0 auth reset-keynow refuses, with a non-zero exit, whenHAL0_API_URL
points at a non-loopback host. It edits this box'sapi.env, so it used to
rotate the remote box's key (e.g. undersudo -E) while printing this box's
unchanged key as the new one (#2640). -
A box already on the newest nightly no longer offers that same nightly to
itself as an update.GET /api/updates/checkandGET /api/updates/state
now compare the release manifest against the installed release
(current/VERSION) instead of the wheel base version, so the dashboard and
API stop reporting1.4.0→1.4.0-nightly.<stamp>forever (#2629). -
The Hindsight engine upgrade pass no longer downgrades the engine. When the
installedhindsight-apiis newer than the version the running hal0 tree
pins (for example afterhal0 update --rollback), the pass now stops with
statusaheadand a remedy instead of rebuilding an older engine onto the
newer database schema, and the boot-time check logs the same. Update hal0 to
a release pinned at or above the installed engine, or restore
.pg0.pre-<pin>by hand with the matching venv (#2613). -
Two slot TOMLs that declare the same
nameno longer both load. hal0
keeps one (the id-keyed<id>.tomlwhen present, else the first by
filename), logsslot.duplicate_tomlwith both paths, and skips the other,
soGET /api/slotslists the slot once. Deleting a model that only the
skipped stale file referenced no longer unloads the live slot of the same
name (#2542). The skipped file is logged once per process and no longer
marks the slot enumeration partial, so/v1/modelskeeps refreshing while
the stale file is present (#2631). -
The profile-centric conversion no longer strips a blocked custom profile.
A profile (or a stack's embedded profile) whose flags carry a model fact
with no profile home (--mmproj,--chat-template,--chat-template-file)
was reported as blocked but written back without that flag, so the next
start converted its slot as unchanged and launched it without the projector
or template. The entry now stays inprofiles.toml(orstacks.toml)
exactly as stored, is never reused for another slot's generated profile,
and blocks on every run until the flag is moved to the model (#2648). -
A memory-engine upgrade that failed once no longer loses memories on a
second failure. The first failed attempt kept its.pg0.pre-<version>
snapshot. Every later attempt then reused that snapshot instead of taking
a new one, so a second rollback restored it and dropped everything written
since the first rollback. Each attempt now takes a fresh snapshot after
the old engine stops, and replaces the previous one only once the copy is
complete. A rollback that cannot copy the snapshot back no longer starts
the old engine on a half-restored.pg0, which could have initialised an
empty database and served it. It leaves the engine stopped and writes
.pg0.restore-failed. The nexthal0 updaterestores the snapshot before
doing anything else. If that restore fails too, the engine stays stopped
and the update reports how to restore by hand. -
A browser origin you allow now survives an installer re-run. The REST and
WebSocket origin checks only readHAL0_ALLOWED_ORIGINS, which sits in the
installer-managed block of/etc/hal0/api.env, so a reverse-proxy origin
added there by hand, or derived from an install-timeHAL0_PUBLIC_URL, was
gone after the next run. A newhal0.toml[network]section holds
public_urlandextra_origins. hal0 adds both on top of
HAL0_ALLOWED_ORIGINSon the next request, with no restart, and removing
one revokes it the same way. Set them withhal0 network public-url set|clearandhal0 network origins add|remove|list, or on the new
Settings ▸ Network page. Each origin is validated and stored the way a
browser sends it (punycode for an internationalised name, the compressed
form of an IPv6 address); anything else is refused. An install-time
HAL0_PUBLIC_URL, including the bare-host form, is saved to
[network].public_urlonce, never over a set value. (#2382) -
The seed stacks (
saber,forge,pi) are now virtual, like the seed
profiles: hal0 takes them from code on every load, and a same-named entry
in/etc/hal0/stacks.tomlis ignored. Before, the seeds were used only when
stacks.tomldid not exist, so a box that had upgraded kept the copies an
older release wrote, which still named the retiredrocm,vulkanand
rocm-moeprofiles, and applying them failed. The API refuses to edit seed
stacks, so there was no way to fix them. Seed stacks are never written
back, so the next stack create, edit, delete, snapshot or import cleans
those copies out ofstacks.toml.hal0 profile migrateno longer plans
or blocks on a seed-named stack in the file. Custom stacks are unchanged.
(#2584) -
Deleting a custom profile is no longer allowed while a slot file still
names it but fails to load. A slot that is unconverted (slot.unconverted)
or otherwise invalid was left out of the profile in-use scan, so the delete
went through and the slot later failed withslot.profile_missing. The scan
now reads the profile name from the raw slot TOML (either layout) when the
typed load fails, soprofiles.in_useand the Profiles tab'sused_bylist
include those slots. A slot file that is not valid TOML makes the delete
fail with 409profiles.slot_unreadable, naming the file; the profile list
keeps working. (#2574) -
An
image(ComfyUI) slot that names a profile that does not exist no
longer launches silently with the built-in default flags. It now refuses
to launch withslot.profile_missing, as every llama-server slot already
does, so a typo in a profile name cannot quietly start a different tune.
A slot that names no profile still gets the default flags. The Kokoro,
Moonshine and Qwen3-TTS slots already refused, but with the profile
catalog'sprofiles.not_found; they now raiseslot.profile_missingtoo,
so one condition has one code on the launch path (#2573). -
The dashboard profile form called an unset Thinking default "Auto — the
model decides". hal0 treats it as off unless the request asks, so the
option, the profile card and the drawer preview now say "default (off)". -
GET /api/slots/{name}/resolvedno longer invents a llama-server command
for a slot that llama-server does not launch. Kokoro, Moonshine, Qwen3-TTS,
ComfyUI and FLM/NPU slots got a made-upbase/model/profile/slot
argv, and the slot drawer showed it as the command they run. The endpoint
now asks the same provider selection the launch path uses and answers
argv: nullwith emptyprovenancefor those slots, as ADR-0029 says; the
resolved_commandfield ofGET /api/slotsfollows suit, and the drawer
says the command is not composed by llama-server. llama-server slots are
unchanged (#2577). -
SlotManager.create()andupdate_config()now run the same write guard as
the HTTP routes.create()only screened[server].env, so an in-process
caller could write a profile-owned tuning key (ctx_size,parallel,
mtp,[model].context_size,[server].extra_args, ...) that the loader
then refused at launch asslot.unconverted; both methods also now refuse
the removedenabledkey (slot.removed_key_denied). The NPU backend
install route no longer writesenabled = trueinto the slot it creates.
(#2578) -
A profile, or the recorded tune of a slot the profile conversion blocked,
can no longer redirect a slot's model or reach outside it through a short
or alternative llama.cpp flag. The flag screen only knew one spelling of
each flag, so-m /etc/shadowreplaced the slot's model (llama-server
keeps the last-m), and-hf,--model-url,--api-key-file,
--ssl-key-file,--log-file,--path,--rpc,--tools, the
-ag/--agenttools shortcut, the preset switches that download a model
(--gpt-oss-20b-default,--embd-gemma-default, …) and similar flags
passed unscreened. Every
spelling llama.cpp accepts is now screened (short and alternative forms,
flag=value,_for-). Flags that fetch a remote model, read or write
a host file, open or shape a listener, or set auth/TLS are refused
(slot.host_owned_flag_denied), as are bare positional arguments,
including one after a joined--flag=valueor after a switch that takes no
value, such as--metrics(slot.positional_arg_denied).
The profile drawer shows the same refusals inline before Save. The seeds
and the shipped add-ons are unaffected. -
A slot's
[server].envcan no longer set a llama-server launch option.
llama.cpp reads every option from an environment variable as well as from
its command line (LLAMA_ARG_MODEL,LLAMA_ARG_AGENT,LLAMA_ARG_TOOLS,
LLAMA_API_KEY,HF_TOKEN, …), so a slot env could redirect the model or
turn on the built-in tools past the profile flag screen. Every
LLAMA_ARG_*name, plusLLAMA_API_KEY,HF_TOKENand
MTMD_BACKEND_DEVICE, is now refused withslot.llama_env_deniedwhen a
slot is created or edited (API, stacks, in-process writers) and again when
it launches; tuning belongs on the slot's profile. The profile conversion
blocks a slot that carries one, naming the variable. Runtime variables such
asHSA_OVERRIDE_GFX_VERSIONare unaffected. -
The bundled agent's STATE.md capability summary now lists speech-to-text
and reranking slots. It matched slot typessttandrerank, but slots
are typedtranscriptionandreranking, so a Moonshine STT slot or a
rerank slot never appeared in it. Every slot type other thanllmnow has
a label, and a test fails if a new type is added without one. -
The runner-system migration no longer fails on a packaged install with
"could not write the runner-migration backup ([Errno 1] Operation not
permitted)". It tried tochmod 0700the sharedbackupsdirectory, which
is root-owned and group-writable, so thehal0service user could not
change it. An existing directory is now left as the operator set it; the
archive itself is still0600. -
The runner-image catalogue sync now reads the
runnersarray that
hal0-runner-images publishes inimages.json(schema
hal0.runner-images.v2). It looked for animagesarray, so every sync
failed with "images.json has no 'images' array" and the catalogue never
refreshed. -
The runner-system migration now reads a slot's model file, not just its
name, before keeping it. A fork build (ROCmFPX and friends) renamed to a
plain file name used to keep its slot through the migration and was then
refused at launch; it is now unassigned with the same warning as a
name-detected one. Every shard of a split model is read. A missing,
unreadable or truncated file is never an error: the name check still
applies. -
The runner-image catalogue sync no longer leaves cancelled probe tasks
pending when an unexpected error aborts it, which logged "Task was destroyed
but it is pending" noise. -
The demoted workload profiles (
dense,moe,coding,chadrock-dense,
chadrock-moe) no longer pass--reasoning-format none. Upstream llama.cpp
answers HTTP 400 "Failed to initialize samplers" to everyresponse_format
(json_object/json_schema) request when the server runs with that flag,
so any slot still on one of these profiles lost structured output once it
moved to an upstream runner.--reasoning offstays, so the profiles still
mean "no thinking". The published addon envelopes undercommunity/addons/
are regenerated without the flag.hal0 slot migrate-flagsalso drops the
flag instead of folding it into a model's defaults. Saving a profile or a
model'sdefaults.extra_argsthat adds--reasoning-format noneis now
rejected with a 400 (slot.reasoning_format_none_denied) explaining why,
and the flag is dropped from the command line of every slot launch as a
backstop for anything already saved (#2545). -
hal0 updatenow removes root-owned files older releases installed and
nothing replaces: thehal0-slotctlwrapper and its sudoers grant
(installed by 0.8.0-beta.1 to beta.3; it let thehal0user write a
root-owned slot unit), thehal0-comfyuisudoers grant (dropped in #984),
and a pre-quadlet statichal0-slot@.servicetemplate with its drop-ins,
which shadows every quadlet slot unit. Only install.sh and uninstall.sh
removed them before, so a box that had only ever been updated kept them.
The sweep runs on the activation of the next update after this one. -
installer/uninstall.shnow removes thehal0-updateandhal0-slotctl
sudoers grants. It left both behind. -
The Runner Images page no longer lists niche runtimes (PromptForge,
Qwen3 TTS, Strix) under "Default runtime". It groups rows by each runtime's
registry tier — Default runtimes, Optional runtimes, Frozen
runtimes (when any) and Other images for catalogued images no
runtime ships — instead of by which image a runtime currently launches
(ADR-0027). -
Deleting every model no longer brings them all back. The registry
re-importedregistry.tomlwhenever its model table was empty, and
hal0 registry exportandhal0 model import-backupwrote that same
file, so the next registry open after deleting the last model restored
every model in it. Nothing readsregistry.tomlnow. -
A deleted model no longer reappears after a restart. Files kept on disk
(by opting out, or because a safety check refused to remove them) are
recorded in<var_lib>/registry/ignored-paths.json, and the discovery
scan skips them.hal0 model add, scan commit and pulls clear that
record for the path they register. -
The startup auto-scan now drops registry rows whose file is gone. These
rows are kept:- rows that a slot, stack or
hal0.tomlfield references - rows with a pending pull
- FLM rows
- rows under a model root that is empty or missing, such as an unmounted
share
If any of those references fails to load, pruning is skipped for that
boot. - rows that a slot, stack or
-
The scan no longer registers model files under dot directories inside a
model root, such as.Trash-1000/. It also no longer names a Hugging
Face shard set after its hash-named blob, and it no longer registers a
hardlinked copy of a model that is already registered. -
Deleting a model while it is still downloading cancels the download
first, so the finished download can't bring the model back. If the
download doesn't stop within 30 seconds, the delete is refused with
model.pull_active. -
The Delete dialog shows the model's real path and size. It used to claim
it removed the size "from /var/lib/hal0/models", whatever the model's
actual location.
Migrations
- After
hal0 updateto 1.4.0, re-run the installer once:curl -fsSL https://hal0.dev/install.sh | sudo bash. It is an upgrade in place, not a
reinstall:/etc/hal0,/var/lib/hal0and your update channel are kept.
hal0 updateswaps the release, re-installs the venv and refreshes the sudo
wrappers in/usr/lib/hal0/bin, but only the installer rewrites
hal0-api.service,hindsight-api.service, the/usr/local/bin/hermes
wrapper and/etc/hal0/api.env, and four 1.4.0 fixes live there (next
entry). - What the re-run picks up. The memory extraction limits (#1834) are baked
intohindsight-api.service; until the installer runs,hindsight-api
keeps its own defaults (32 concurrent calls, 64000 completion tokens), the
failure this release fixes. Saving Settings ▸ Memory once also applies them.
Only the installer mintsHAL0_CLIENT_KEY(#2314), so until it runs Open
WebUI still sends its placeholder and gets 401 on an auth-on box;hal0 auth rotate clientalso mints the key and re-points Open WebUI, and neither
turns auth on. Thehermesandhal0-hermeswrappers that bypass the proxy
for loopback (#2371) live in/usr/local/bin, which only the installer
writes. The journal read that brings back the dashboard log view (#2435) is
a line inhal0-api.service; until the installer runs, logs stay empty and
hal0 doctor allwarns. - **Before re-running, note any
HAL0_ALLOWED_ORIGINSline you added to
Truncated — full notes in CHANGELOG.md and as RELEASE_NOTES.md inside the release tarball.