fix(connect): auto-recover from SSH identity drift after host reboot

[ericksoa]

Summary

• Auto-recover from SSH identity drift after host reboot instead of requiring full re-onboard
• Fix registry recovery gate to trigger for bare nemoclaw <name> (no explicit action)
• Probe live gateway when requestedSandboxName is set, even with empty registry

Fixes #2056

Supersedes #2057 (which had commits from incorrect git identity).

Changes

src/nemoclaw.ts:

1. Import pruneKnownHostsEntries from ./lib/onboard
2. Registry recovery gate (line ~2408): added "" to the action allowlist so bare nemoclaw <name> triggers recovery when registry is empty
3. shouldProbeLiveGateway (line ~477): include requestedSandboxName in the probe condition so recovery works when both registry and session are empty
4. ensureLiveSandboxOrExit identity_drift handler (line ~763): instead of process.exit(1), clear stale openshell-* entries from ~/.ssh/known_hosts using the existing pruneKnownHostsEntries helper and retry the sandbox lookup

test/cli.test.ts:

• Updated assertion for the identity drift test to match new auto-recovery behavior (error message changed from "gateway trust material rotated" to "Could not reconnect")

test/reboot-identity-drift.test.ts (new):

• 6 tests covering healthy reconnect, identity drift detection, and registry recovery from empty state — both bare and explicit command forms

Type of Change

• Code change (feature, bug fix, or refactor)

Verification

• npm run build:cli passes
• npx vitest run test/reboot-identity-drift.test.ts — 6/6 pass
• npx vitest run test/cli.test.ts — all pass (including updated identity_drift assertion)
• npm test — 1690 pass, 1 pre-existing failure (version string mismatch, unrelated)

🤖 Generated with Claude Code

Summary by CodeRabbit

• Bug Fixes

  • Improved SSH identity drift recovery after reboot by clearing stale host keys and attempting reconnection
  • Enhanced gateway probing to better detect live gateways when sandbox names are provided
  • Updated error messaging for reconnection failures

• Tests

  • Added regression test suite for post-reboot SSH identity drift and registry recovery scenarios

[coderabbitai]

📝 Walkthrough

Walkthrough

The pull request implements auto-recovery from SSH identity drift after host reboot by clearing stale SSH known_hosts entries and retrying gateway connections, while broadening registry recovery gating to handle bare sandbox commands.

Changes

Cohort / File(s)	Summary
Identity Drift Recovery src/nemoclaw.ts	Added pruneKnownHostsEntries import; identity_drift recovery path now clears stale SSH host keys from ~/.ssh/known_hosts and retries gateway connection. Updated console messaging to reflect SSH key clearing and reconnect attempt. Broadened shouldProbeLiveGateway logic and fixed registry recovery gating to include empty-string action for bare sandbox commands.
Test Assertions test/cli.test.ts	Updated test expectation for gateway trust rotation scenario to expect "Could not reconnect" message instead of "gateway trust material rotated after restart".
Regression Test Suite test/reboot-identity-drift.test.ts	New Vitest test suite with temporary isolated HOME fixtures, stub openshell executables, and helpers to simulate post-reboot SSH identity drift and registry recovery. Tests both successful sandbox connection and identity drift detection/recovery paths, plus registry recovery when gateway is live but registry is empty.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Poem

🐰 A reboot came, SSH keys turned strange,
But clever nemoclaw rose to the change—
Pruning old entries with practised care,
Reconnecting through drift-laden air,
The sandbox hops on, no re-onboard despair! 🏠✨

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 40.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The PR title accurately describes the main fix: auto-recovery from SSH identity drift after host reboot, which is the primary change across the files.
Linked Issues check	✅ Passed	All coding requirements from issue #2056 are implemented: clearing stale known_hosts entries on identity_drift [#2056], retrying sandbox lookup after cleanup [#2056], and fixing the registry recovery gate to include bare commands [#2056].
Out of Scope Changes check	✅ Passed	All changes directly address the three coordinated fixes outlined in issue #2056; no extraneous modifications or unrelated functionality were introduced.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch fix/2056-reboot-identity-drift-v2

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

🧹 Nitpick comments (1)

test/reboot-identity-drift.test.ts (1)

9-12: Outdated comment: this PR is the fix for #2056.

Line 12 says "Once the fix for #2056 lands, update these tests to assert auto-recovery" — but this PR implements that fix. Consider removing or updating this comment to avoid confusion for future readers.

♻️ Suggested fix

 // Simulates the post-reboot scenario where the gateway restarts with new SSH
 // keys, causing "handshake verification failed" errors. Verifies:
 //   1. The registry recovery gate triggers for bare `nemoclaw <name>` (no action)
-//   2. Identity drift is detected and surfaced (current behavior)
-//
-// Once the fix for `#2056` lands, update these tests to assert auto-recovery.
+//   2. Identity drift is detected, stale SSH keys are cleared, and reconnect is attempted
+//   3. Registry recovery works when the registry is empty but gateway is live

🤖 Prompt for AI Agents

Verify each finding against the current code and only fix it if needed.

In `@test/reboot-identity-drift.test.ts` around lines 9 - 12, The top block
comment in reboot-identity-drift.test.ts contains an outdated note "Once the fix
for `#2056` lands, update these tests to assert auto-recovery"; update or remove
that sentence so it reflects that this PR implements the fix for `#2056`—either
delete the line or replace it with a short note that the tests now assert
auto-recovery (keep the remaining comment lines about registry recovery and
identity drift if still relevant) to avoid confusion for future readers.

🤖 Prompt for all review comments with AI agents

Verify each finding against the current code and only fix it if needed.

Nitpick comments:
In `@test/reboot-identity-drift.test.ts`:
- Around line 9-12: The top block comment in reboot-identity-drift.test.ts
contains an outdated note "Once the fix for `#2056` lands, update these tests to
assert auto-recovery"; update or remove that sentence so it reflects that this
PR implements the fix for `#2056`—either delete the line or replace it with a
short note that the tests now assert auto-recovery (keep the remaining comment
lines about registry recovery and identity drift if still relevant) to avoid
confusion for future readers.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 86ff307d-eef8-4445-a479-aeec71b0014b

📥 Commits

Reviewing files that changed from the base of the PR and between bbbaa0f and 4620c7c.

📒 Files selected for processing (3)

• src/nemoclaw.ts
• test/cli.test.ts
• test/reboot-identity-drift.test.ts