Skip to content

Native hook relay intermittently becomes unavailable on 2026.5.26, blocking memory/filesystem tools #87395

Closed

Description

@OhBoyPrintshop

Summary

After updating to OpenClaw 2026.5.26 (10ad3aa) on macOS/darwin, the native hook relay intermittently becomes unavailable and blocks native tools before command execution. The symptom looks like a runtime/native hook relay failure rather than memory database corruption.

Environment

  • OpenClaw: 2026.5.26 (10ad3aa)
  • Platform: macOS / darwin
  • Affected area: native shell/filesystem tool execution, memory file read/write checks, gh/native command usage

Symptoms

  • Native shell/filesystem commands intermittently fail before execution with:
Native hook relay unavailable
  • When this happens, memory file read/write checks are blocked because the native filesystem path cannot execute.
  • gh/native command usage is also blocked by the same pre-execution failure.
  • Browser-based GitHub navigation was blocked by policy in the parent session, so the issue could not be filed through the GitHub UI from that path.

Recovery/diagnostics observed

  • Restarting the gateway alone did not recover the native relay.
  • A full OpenClaw hard restart temporarily recovered native shell/filesystem access.
  • After the hard restart, current workspace memory reads/writes succeeded.
  • The failure recurred minutes later after the temporary recovery.
  • The memory database itself appears healthy when native hook execution works: sqlite PRAGMA integrity_check returned ok.
  • This suggests the durable memory store/files are not the primary failure point; the failure appears to be in the runtime/native hook relay path.

Request

Could maintainers help distinguish this from memory corruption and confirm whether this is a runtime/native hook relay regression in 2026.5.26?

Useful guidance would include:

  • Recommended diagnostics to capture when Native hook relay unavailable recurs.
  • Whether gateway logs, hook registry state, session generation IDs, or app-server state are most relevant.
  • A supported recovery path short of a full OpenClaw hard restart.
  • Whether this is related to recently closed native hook relay issues, or a separate recurrence involving memory/filesystem tool access.

Activity

  1. clawsweeper commented on May 27, 2026

    @clawsweeper
    Contributor

    ClawSweeper status: review started.

    I am starting a fresh review of this issue: Native hook relay intermittently becomes unavailable on 2026.5.26, blocking memory/filesystem tools This is item 1/1 in the current shard. Shard 0/1.

    This placeholder means the worker is alive and reading the current context. I will edit this same comment with the actual review when the claws are done clicking.

    Crustacean status: shell secured, claws on keyboard, evidence pebbles being sorted.

  2. WarrenJones commented on May 27, 2026

    @WarrenJones
  3. OhBoyPrintshop commented on May 27, 2026

    @OhBoyPrintshop
    Author

    Additional doctor/config detail from the same affected machine:

    • openclaw doctor reports: Memory search is explicitly disabled (enabled: false).
    • Confirmed in /Users/hal9000/.openclaw/openclaw.json: agents.defaults.memorySearch.enabled is currently false.
    • plugins.entries["memory-core"].enabled is still true, with memory-core.config.dreaming.enabled also true.
    • Native command execution was working at the time of this check; openclaw doctor completed and did not itself reproduce Native hook relay unavailable.

    My current read is that this is useful configuration context but not the same failure mode as the intermittent native hook relay outage. The relay outage blocked shell/filesystem access before commands could run; the memorySearch.enabled: false setting appears to affect semantic/search behavior rather than direct workspace file reads/writes.

  4. OhBoyPrintshop commented on May 27, 2026

    @OhBoyPrintshop
    Author

    Additional recovery/update from the affected machine, sanitized to include only OpenClaw/runtime details:

    What happened

    • The failure appeared after updating to OpenClaw 2026.5.26 (10ad3aa) on macOS/darwin.
    • Native tool execution intermittently failed before the requested command could run.
    • The visible error was:
    Native hook relay unavailable
    
    • When this happened, workspace filesystem/memory operations from the agent path were blocked because the command never reached execution.
    • Direct memory data did not appear corrupted. When commands worked again, direct workspace memory files were readable/writable, and a SQLite memory integrity check returned ok.

    Recovery attempts/results

    1. Restarted the gateway.

      • Result: gateway came back up, but this alone did not fully resolve the native hook relay failure.
    2. Checked hooks/runtime health after restart.

      • openclaw hooks check later reported hooks ready.
      • Native command execution worked temporarily after a harder restart.
    3. Checked config/doctor state.

      • openclaw doctor previously reported memory search explicitly disabled via agents.defaults.memorySearch.enabled=false.
      • plugins.entries["memory-core"].enabled and dreaming config were present/enabled at that time.
      • My read remains that this config drift is useful context, but likely separate from the relay outage because the relay failure occurred before shell/filesystem commands could execute.
    4. Reverted OpenClaw to an older working version.

      • Current restored version:
    OpenClaw 2026.5.22 (a374c3a)
    
    • After the revert, native commands are working again in the active agent session.
    • Workspace memory files are readable/writable again through the normal command path.
    • openclaw doctor completes on the reverted version.

    Other relevant runtime clue

    • Logs during the recovery period showed repeated protocol mismatch messages from an older TUI/client version (openclaw-tui v2026.4.14) connecting to a newer gateway expecting protocol 4.
    • After using the current web/control UI on the reverted runtime, that mismatch was not observed in the current active path.

    Why no raw log upload

    I am not attaching raw logs from this machine because they may contain private workspace/session/project context unrelated to this runtime bug. The details above are the sanitized recovery facts relevant to the native hook relay failure and restoring OpenClaw to a working state.

  5. rollingstone87 commented on May 28, 2026

    @rollingstone87

    I can reproduce this on 2026.5.26 (10ad3aa):

    1. /reset
    2. Run native shell: sed -n '1,3p' /Users/joe/.openclaw/workspace/skills/voicechat/SKILL.md
      -> succeeds once
    3. Run the same command again
      -> Native hook relay unavailable

    After failure:

    • /tmp/openclaw-native-hook-relays-$(id -u) has no relay records
    • sessions.json gets modelProvider=openai-codex while providerOverride=openai
    • liveModelSwitchPending remains true
    • openclaw doctor reports Legacy openai-codex/* session route state detected

    This looks related to #87331 / fixed by #87272, but the fix is not in the 2026.5.26 npm release I’m running.

  6. AlexanderGolys commented on May 28, 2026

    @AlexanderGolys

    As pointed out by my claw:

    Worse: I just tried to inspect templates with parallel shell calls and the native hook relay wedged again, so I can’t patch it this second. Very funny, OpenClaw. Very normal.

  7. hanyizuo commented on May 28, 2026

    @hanyizuo

    Additional Linux reproduction on 2026.5.26 (10ad3aa), with OpenAI Codex runtime.

    Environment:

    • OpenClaw: 2026.5.26 (10ad3aa)
    • Platform: Linux, Node 22.22.2
    • Gateway service: systemd user service, running
    • Runtime involved: OpenAI Codex

    Observed behavior:

    • Native command execution intermittently fails before command execution with:
    Native hook relay unavailable
    
    • The failure can block even trivial commands such as true or date.
    • The active session still reports elevated/full execution state when the failure occurs, so this does not look like a permission denial.
    • A gateway-level soft restart/reload was not sufficient in my case.
    • A full server/OpenClaw restart temporarily restored execution.
    • The failure later recurred, which suggests a relay/sidecar lifecycle or app-server bridge issue rather than a one-time bad command.

    Related user-facing symptom:

    The chat/runtime may display:

    Codex stopped before confirming the turn was complete. Some work may already have been performed; verify the current state before retrying.
    

    This appeared while commands had partially run or while the agent was trying to continue after restart. It made it hard to tell whether the task had completed.

    What would help:

    • A supported diagnostic command for native hook relay state.
    • A recovery path short of rebooting the server.
    • Clearer error classification so this is not confused with elevated permission denial.
    • If this is tied to the Codex app-server bridge, a watchdog/reconnect path for the native hook relay.
  8. Lore-37 commented on May 28, 2026

    @Lore-37

    AI-assisted analysis — the forensic deep-dive and code citations below were produced in a co-working session with Claude (Anthropic, Opus 4.7). All evidence was captured from my own gateway and I have reviewed every line of the analysis manually; the conclusions are mine.

    Adding a code-level forensic analysis from a separate deep-dive on the same OpenClaw 2026.5.26 (10ad3aa) install (Docker, ghcr.io/openclaw/openclaw with a custom Chromium layer, Lightsail Ubuntu 22.04, Codex stdio + Telegram). I believe this thread captures two different failure modes with the same user-visible error string, and only one of them is addressed by #87272.

    TL;DR — two failure modes

    Failure mode Bridge file on disk? Mechanism Fix status
    Generation UUID stale Yes — file present, generation rotated resumed codex thread uses cached old --generation #87272 merged (waiting for release after 2026.5.26)
    Bridge file deleted No — /tmp/openclaw-native-hook-relays-<uid>/ empty bridge rmSync'd at attempt teardown while shared codex is still alive not yet addressed

    @rollingstone87's note in this thread (/tmp/openclaw-native-hook-relays-$(id -u) has no relay records) and the "harder restart temporarily recovers" symptom map onto the second mode.

    Code path for the bridge-deleted mode (from the shipped 2026.5.26 bundle)

    End-of-attempt teardown in dist/run-attempt-CHLuDYRa.js (≈ L6757):

    if (nativeHookRelay)
      if (shouldDelayNativeHookRelayUnregister)
        scheduleCodexNativeHookRelayUnregister({ relay: nativeHookRelay, hookTimeoutSec: ... });
      else
        nativeHookRelay.unregister();

    And the condition (≈ L6703):

    shouldDelayNativeHookRelayUnregister =
        activeProjector.getCompletedTurnStatus() === "completed"
        && !timedOut
        && !runAbortController.signal.aborted
        && !finalAborted
        && !finalPromptError;

    So any non-clean attempt ending — timeout, abort, runtime error, final-prompt error — sets shouldDelayNativeHookRelayUnregister = false and unregisters immediately, with no grace window.

    unregisterNativeHookRelayBridge then deletes the on-disk record (dist/native-hook-relay-AN6S_wz5.js ≈ L372):

    function unregisterNativeHookRelayBridge(relayId) {
      const bridge = relayBridges.get(relayId);
      if (!bridge) return;
      relayBridges.delete(relayId);
      bridge.server.close();
      if (readNativeHookRelayBridgeRecordIfExists(relayId)?.token === bridge.token)
        rmSync(bridge.registryPath, { force: true }); // <-- bridge file gone
    }

    The architectural mismatch: the codex app-server client is shared/persistent across attempts (SharedCodexAppServerClient, presumably for cold-start amortization), while the relay is registered/unregistered per attempt. After any non-clean attempt teardown, the still-alive shared codex's first subsequent tool call lands on an empty bridge directory and is rejected fail-closed in the PreToolUse hook.

    Because clean completions get a small (10–15 s) grace via scheduleCodexNativeHookRelayUnregister, this also surfaces as intermittent on healthy paths if a follow-up event from the shared codex lands just after the grace timer fires.

    Forensic evidence

    Process tree captured during a stuck period and again after self-recovery — identical PIDs across the recovery, confirming the shared codex itself never restarts:

    PID 1   docker-init
    └─ PID 7   tini
       └─ PID 8   node dist/index.js gateway (persists; pid in bridge file is this one)
          └─ PID 86  codex app-server wrapper (persists across attempts)
             └─ PID 93  vendored codex app-server binary (persists)
    

    State while stuck:

    $ docker exec openclaw-gateway sh -c 'ls /tmp/openclaw-native-hook-relays-1000/'
    (empty)
    

    The agent self-recovers by issuing gateway-tool: restart — verbatim from our gateway log (2026-05-28 10:21 local):

    [gateway-tool] gateway tool: restart requested (delayMs=default,
      reason=Native hook relay is intermittently unavailable and blocking
      file-write tools such as apply_patch, while shell commands still run.
      Restarting gateway to restore the hook relay before persisting worksp)
    [gateway-tool] gateway tool: restart requested (delayMs=1000,
      reason=Second restart attempt because native hook relay remains
      unavailable after first restart and now blocks even simple shell commands.)
    [gateway] signal SIGUSR1 received
    [gateway] received SIGUSR1; restarting
    

    After the in-process restart the bridge file reappears with pid=8 (same gateway PID) and tools work — until the next attempt ends in error and the cycle repeats. The self-restart loop produces user-visible noise but does not address the underlying lifecycle issue.

    Why this is distinct from #87331 / #87272

    #87272 persists generation in the codex thread binding so a resumed thread can keep using its issued generation across gateway restarts. Necessary, but not sufficient here: if the bridge file itself is rmSync'd at attempt teardown while the shared codex is still alive, the persisted generation has nothing to talk to.

    Why this is distinct from #87563

    #87563 prunes bridge files whose pid is dead. In this repro the pid in the bridge file is the current live gateway PID — the file is removed by OpenClaw's own teardown path, not left zombie by a crash.

    Possible fix directions

    A. Minimal / config-shaped: treat shared codex sessions as a case where shouldDelayNativeHookRelayUnregister should be true even on error paths, so the grace window applies and a subsequent attempt re-registers before the timer fires.

    B. Architectural: bind the relay lifecycle to SharedCodexAppServerClient. When the client is shared and still alive, the relay persists with it; unregister fires only on shared-client release. Possibly via a refcount or a "keep-alive" handle from the harness.

    Happy to draft a PR for either direction if maintainers think the angle is right, or to wait for guidance if a fix is already in progress for this specific path. I have local reproduction available (any cron-driven workflow or apply_patch to a non-existent file reliably ends the attempt in error and surfaces the next-attempt block).

    cc maintainers active on adjacent work — apologies if the ping isn't welcome: @amknight (#87272), @steipete (#87331 assignee).

  9. Lore-37 commented on May 28, 2026

    @Lore-37

    Update / partial correction to my previous comment.

    I ran a coordinated real-time observation today — polling /tmp/openclaw-native-hook-relays-1000/ at 100 ms while my agent attempted an apply_patch from Telegram — and the data does not match the "bridge file deleted" mechanism I described earlier. Reporting the correction because it points to a different (and already-fixed) root cause.

    What I actually saw

    11:28:26.276  [telegram] inbound (agent receives the apply_patch request)
    11:28:28.910  bridge file APPEARS
                  4d862c9d…79b.json = {"version":1,
                    "relayId":"codex-6c955fb2…7139c2afdd9",
                    "pid":8, "port":35957,
                    "token":"4302b300-1dbc-4339-8505-0c3c6bc1197c",
                    "expiresAtMs":1780479208835}        ← ~5.7 days TTL, doesn't expire
    11:28:34.200  watcher window ends, bridge still present
    11:28:38      [telegram] outbound  (agent starts replying)
    11:28:48      [telegram] outbound  (agent reports "apply_patch blocked")
    ~11:29        bridge file rmSync'd (post-attempt grace timer)
    

    So the bridge file was present when apply_patch was attempted. My earlier hypothesis (immediate rmSync at attempt teardown stranding subsequent tool calls) does not match this trace.

    Selective-by-tool symptom — possibly novel data

    Within the same attempt window the agent reports:

    • pwd, date, git status, file reads via shell — work
    • apply_patch to a small /tmp/probe.txt — blocked with Command blocked by PreToolUse hook: Native hook relay unavailable

    So the failure is not "the relay is down for everything"; it's per-tool. Shell-exec PreToolUse paths pass while apply_patch's PreToolUse hook is the one being rejected. The bridge HTTP server (pid=8, port=35957) is live and reachable at that moment.

    Best fit: #87331 / #87272 (generation UUID staleness)

    The selective-per-tool behaviour fits @adamamzalag's #87331 repro very well: codex caches the PreToolUse command for each tool with --generation <UUID>; when the relay re-registers (new attempt, TTL renew, restart), the generation rotates and the cached command becomes stale. Tools whose cached command still matches the live generation work; tools whose cached command was issued under the previous generation get the 410 stale registration response that the CLI wrapper renders as "Native hook relay unavailable".

    So this thread is most likely a continuation of #87331, not a separate failure mode as I implied earlier. Apologies for the noise.

    I did not capture a 410 in the gateway info-level log (the hook handler may log it below info, or my filter missed it). Happy to repeat the observation with debug logging if useful.

    Counter-productive mitigation (warning to anyone reading)

    To stabilise my install while diagnosing, I deployed a small host-side watchdog that sent SIGUSR1 to the gateway when the bridge directory appeared empty for more than 90 s. This is actively harmful. Each external SIGUSR1 triggers an in-process gateway restart, which (per the failure mechanism in #87331) rotates the generation UUID and renders every tool command cached under the previous generation stale. My agent's own self-issued gateway-tool: restart calls do the same thing. I have disabled the watchdog.

    Restarting is the opposite of a workaround for this specific bug until #87272 ships.

    #87272 / release cadence

    PR #87272 merged 2026-05-27T18:55 UTC. Stable 2026.5.26 had cut earlier the same day (11:27 UTC) so it does not include the fix. 2026.5.27-beta.1 (2026-05-28T05:54 UTC) does — the release notes call out "preserve native hook relay generations across restarts and fresh fallbacks". Based on the recent beta → stable cadence (~6–14 h), 2026.5.27 stable should land soon and that is the realistic path out for installs currently on 2026.5.26.

    If maintainers can confirm: does #87272's generation-persistence cover the attempt → attempt rotation case (same gateway, same session, just a fresh attempt picking up a new generation) in addition to the resumed-thread-after-restart case described in the PR summary? My read of the diff suggests yes — the persisted generation lives in the thread binding and is reused — but I haven't run the fix yet.

  10. rollingstone87 commented on May 28, 2026

    @rollingstone87

    Follow-up from my earlier reproduction on 2026.5.26 (10ad3aa):

    I upgraded to OpenClaw 2026.5.27 and reran the original repro:

    /reset
    sed -n '1,3p' /Users/joe/.openclaw/workspace/skills/voicechat/SKILL.md
    sed -n '1,3p' /Users/joe/.openclaw/workspace/skills/voicechat/SKILL.md
    

    Result: both native shell calls now succeed; the second call no longer fails with Native hook relay unavailable.

    I also checked the file/write path after the upgrade and did not hit the relay failure. From this machine, the #87331 / #87272 generation-persistence fix appears to cover the 5.26 regression I reported above.

  11. amknight commented on May 28, 2026

    @amknight
    Member

    @rollingstone87 just confirming you're no longer seeing any issues with 2026.5.27-beta.1? The official version should be out soon.

    It would also be great to get some more confirmations from others.

  12. rollingstone87 commented on May 28, 2026

    @rollingstone87

    @amknight confirming from my side: I tested the official OpenClaw 2026.5.27 release, not 2026.5.27-beta.1.

    After upgrading, the original 2026.5.26 repro no longer reproduces for me:

    • repeated native shell calls succeed
    • the second call no longer fails with Native hook relay unavailable
    • the file/write path also did not hit the relay failure in my post-upgrade check

    So from this machine, the #87331 / #87272 generation-persistence fix in 2026.5.27 appears to cover the regression I reported. Happy to run a specific diagnostic or longer repro loop if that would help.

  13. Lore-37 commented on May 28, 2026

    @Lore-37

    Closing the loop — confirmed fixed on 2026.5.27.

    Upgraded my install from 2026.5.26 to 2026.5.27 and re-ran a coordinated trace (100 ms polling on the bridge directory) while my agent executed a four-step cross-turn apply_patch test from Telegram. The case that was failing in my previous trace now passes.

    Test

    • Turn 1: apply_patch creates /tmp/probe.txt with test
    • Turn 1: shell reads back test
    • Turn 2 (separate Telegram message, separate attempt): apply_patch modifies the file to test-cross-turn
    • Turn 2: shell reads back test-cross-turn
    • Turn 2: apply_patch cleanup deletes the file
    • Verification: file absent

    All steps succeeded.

    Watcher trace (relevant excerpt)

    TURN 1 — create + read
      16:50:25.374  inbound (test instructions)
      16:50:47.021  bridge REGISTERED  port=34241  token=9191825e…
      16:50:39 / 16:51:09  agent outbound (turn-1 progress)
      16:51:22.035  bridge UNREGISTERED  (post-turn grace fires)
    
      ─── 88 s with the bridge directory empty ───
    
    TURN 2 — modify + read + delete  (separate attempt)
      16:52:44.306  inbound ("go with the second apply_patch")
      16:52:50.192  bridge RE-REGISTERED  port=41207  token=7a8fa09d…
      16:52:54 / 16:53:32  agent outbound (turn-2 progress)
      16:53:45.052  bridge UNREGISTERED
    

    The same relayId (codex-6c955fb2…) is re-registered between turns with a fresh port and token — i.e. the generation does rotate exactly the way #87331 described. With #87272 in 2026.5.27 the codex thread binding tracks the new generation and the cross-turn apply_patch PreToolUse hook is no longer rejected as stale.

    Net result on this install:

    • 2026.5.26: apply_patch reliably blocked in turn 2 (shell-exec passed; PreToolUse-gated tools failed).
    • 2026.5.27: both turns pass, full round-trip incl. cleanup.

    Thanks @amknight and @steipete for the quick turnaround — happy to provide more detail or repeat the trace if useful.

  14. amknight commented on May 28, 2026

    @amknight
    Member

    Thanks folks for confirming the fix, I'll close this issue now

  15. locked as resolved and limited conversation to collaborators on May 31, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    Priority

    None yet

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions