Skip to content

Shellper processes do not survive Tower restart despite detached:true #324

@waleedkadous

Description

@waleedkadous

Bug

Shellper processes are spawned with detached: true and child.unref(), and the Tower graceful shutdown code explicitly avoids calling shellperManager.shutdown() to preserve sessions. Despite this, all shellper processes die when Tower restarts.

Evidence

Tower logs from restart on 2026-02-16:

Found 6 shellper session(s) in SQLite — reconnecting...
Session 549f047e reconnect failed: process 55972 is dead
Session 57096007 reconnect failed: process 77442 is dead
Session 7d25d9be reconnect failed: process 24565 is dead
Session c1020d72 reconnect failed: process 45524 is dead
Session 04316bd1 reconnect failed: process 61241 is dead
Session 0a05886a reconnect failed: process 69573 is dead
Reconciliation complete: 0 shellper, 0 orphan, 0 killed, 6 stale rows cleaned

All 6 shellper PIDs were dead. Zero survived.

Impact

  • Builder sessions are killed on Tower restart
  • This defeats the purpose of the shellper architecture (spec 0104)
  • Architects cannot safely restart Tower when builders are running

Likely Root Cause

Node.js detached: true on macOS may not be sufficient to survive parent process exit. Possible causes:

  • Process group signal propagation (SIGHUP) when Tower exits
  • process.exit(0) in graceful shutdown may trigger cleanup before detach takes effect
  • macOS-specific behavior with spawn() detached option

Relevant Files

  • packages/codev/src/terminal/shellper-process.ts — shellper spawn
  • packages/codev/src/agent-farm/servers/tower-server.ts — graceful shutdown (lines 112-152)
  • packages/codev/src/agent-farm/servers/tower-terminals.ts — reconciliation logic
  • packages/codev/src/terminal/session-manager.ts — reconnect logic

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions