RFC: Session Management — Design Proposal

[thepagent]

RFC: Session Management

Tracking issue: #75
Status: Draft
Author: @thepagent

---

Summary

Comprehensive session management for agent-broker covering lifecycle control, isolation, observability, security, and multi-agent support.

Current State

• SessionPool uses HashMap<String, AcpConnection> keyed by Discord thread_id
• Basic cleanup_idle (TTL-based) and shutdown exist
• No user-facing session control, no API, no metrics
• Write lock held during entire prompt streaming (fix: pool write lock held during entire prompt streaming causes cross-session deadlock #58)

---

1. Session Lifecycle

1a. /close command (#40)

• Discord handler intercepts /close message
• Calls pool.remove(thread_id) → drops AcpConnection → kill_on_drop kills child process
• Replies "Session closed." and archives thread

1b. Session timeout / auto-expiry

• Split current session_ttl_hours (hard max) from new idle_timeout_minutes (e.g. 30 min)
• On idle timeout → post "⏰ Session expired due to inactivity" in thread → remove session

1c. Per-user session limits

• Track HashMap<UserId, Vec<ThreadId>> for active sessions per user
• Exceeding limit → reply "You have too many active sessions. Use /close to free one."

1d. Graceful shutdown

• On shutdown, post "🔄 Broker restarting..." in each active thread before clearing pool
• Phase 2: persist session metadata to disk/S3 for restart recovery

---

2. Session Isolation & Stability

2a. Per-thread working directories (#38)

• Change working_dir to {base_working_dir}/{thread_id}/
• Each session gets its own filesystem namespace
• Cleanup deletes working dir along with session

2b. Cross-session deadlock fix (#58)

• Change from RwLock<HashMap<K, AcpConnection>> to RwLock<HashMap<K, Arc<Mutex<AcpConnection>>>>
• Outer RwLock only protects map insert/remove (released immediately)
• Per-connection Mutex protects streaming — session A no longer blocks session B

pub struct SessionPool {
    connections: RwLock<HashMap<String, Arc<Mutex<AcpConnection>>>>,
}

pub async fn with_connection(&self, thread_id: &str, f: F) -> Result<R> {
    let conn = {
        let conns = self.connections.read().await;
        conns.get(thread_id).cloned()
            .ok_or_else(|| anyhow!("no connection"))?
    };
    let mut guard = conn.lock().await;  // per-session lock only
    f(&mut guard).await
}

---

3. Session State

3a. Session metadata

struct SessionMetadata {
    thread_id: String,
    user_id: String,
    agent_name: String,
    created_at: Instant,
    last_active: Instant,
    message_count: u64,
    status: SessionStatus,  // Active, Idle, Expired
}

• Phase 1: in-memory, used for observability and lifecycle decisions
• Phase 2: serialize to disk/S3 for restart recovery

3b. Context window management

• Track message_count per session, warn user when approaching limits
• Conversation summarization deferred to Phase 2 (requires extra LLM call)

---

4. Session Observability (#39)

4a. Management API

Lightweight HTTP server on separate port (e.g. 9090) using axum:

GET    /sessions              — list all active sessions
GET    /sessions/:thread_id   — session detail
DELETE /sessions/:thread_id   — force terminate
GET    /health                — broker health + pool stats
GET    /metrics               — prometheus-compatible metrics

4b. Metrics

• active_sessions (gauge)
• total_sessions_created (counter)
• session_duration_seconds (histogram)
• messages_per_session (histogram)
• pool_exhaustion_events (counter)

4c. Audit trail

• Structured logging via existing tracing — add fields: thread_id, user_id, event
• Events: session_created, session_prompt, session_closed, session_expired

---

5. Session Security & Access Control

5a. Session ownership

• SessionMetadata records owner_user_id
• /close restricted to session owner or admin
• Other users can still interact in thread (Discord threads are public)

5b. Rate limiting per session

• Per-session sliding window: configurable max_messages_per_minute (default: 10)
• Exceeding limit → reply "⏳ Rate limited, please wait."

---

6. Multi-agent

6a. Session routing

• Extend config from single [agent] to [agents] table with multiple agent configs
• Routing by Discord channel or /agent <name> command
• Pool becomes HashMap<String, (AgentConfig, AcpConnection)>

6b. Session handoff (Phase 2)

• /handoff <agent> → close current connection, respawn with new agent
• Optional: carry conversation summary to new agent system prompt

---

Implementation Phases

Phase	Scope	Complexity
Phase 1	#58 deadlock fix, #40 /close, idle timeout notification, session metadata	Low-Med
Phase 2	#39 management API, metrics, per-user limits, rate limiting	Medium
Phase 3	#38 per-thread working dirs, session ownership, audit trail	Medium
Phase 4	Multi-agent routing, persistence/recovery, handoff	High

---

Open Questions

1. Should the management API require auth (API key / mTLS)?
2. Should we support session resume after agent crash (requires agent-side support)?
3. Multi-agent routing: per-channel config vs. user command vs. both?
4. Rate limiting: per-session or per-user?

---

Comments and feedback welcome.

[ruan330]

Great RFC — this aligns well with what we've been hitting in production. A few notes from running agent-broker with claude-agent-acp across multiple concurrent sessions:

What we've already implemented and tested

Our PR #77 covers the core of Phase 1: Arc<Mutex<AcpConnection>> per-connection locking, alive check, and startup thread cleanup. It's been running in production for the past two days across two instances (bare metal + Docker). Happy to share what we learned:

• The Arc<Mutex> approach works well. No cross-session blocking observed after the change. The code in §2b of this RFC matches our implementation almost exactly.
• Startup cleanup is essential. Without it, stale threads silently exhaust max_sessions after every restart. We archive threads owned by the bot in the ready handler.
• Alive check needs a hard timeout companion. conn.alive() correctly detects process crashes, but can't detect processes that run forever (e.g., flutter run app server). We added a 30-min hard ceiling as a safety net — configurable via config.toml would be better.

On the Open Questions

1. Management API auth:
For localhost (single-machine deployment like ours), no auth is fine. For any networked deployment (K8s pod, remote access), a bearer token from config seems like the minimum. We'd suggest making it optional: [management] auth_token = "${MGMT_TOKEN}", skip validation if empty.

2. Session resume after crash:
In our experience, crashed sessions are not worth resuming. The Claude Code process state (conversation context, tool state) is lost on crash. A clean restart with a fresh session/new is more reliable than trying to recover a corrupted session. The startup thread cleanup approach (archive old threads, start fresh) has been more practical.

3. Multi-agent routing:
We run two instances (bare metal for Flutter/macOS, Docker for sandboxed projects) with separate Discord channels. Per-channel config works well for our use case. A /agent <name> command would be a nice addition for single-instance multi-agent setups.

4. Rate limiting:
Per-session makes more sense to us. In our setup, one user drives all sessions — per-user limits wouldn't help. Per-session limits would prevent a runaway agent from flooding a thread.

One thing we'd add to the RFC

ACP event ordering resilience (detailed in #76): end_turn sometimes arrives before the final agent_message_chunk, causing empty responses. This is specific to claude-agent-acp but may affect other backends. Our fix is a 200ms drain window after receiving end_turn — simple and effective, but a protocol-level solution (sequence numbers) would be more robust long-term.

This could fit under §2 (Session Isolation & Stability) as a new subsection on notification delivery guarantees.

[qijie850]

Solid RFC. A few additions from our side:

Phase 3 — worktree as isolation strategy. Our use case (described in #38) is concurrent agents working on different branches of the same monorepo. sessions/<thread_id>/ solves ad-hoc isolation, but each session still needs to git clone the full repo. A worktree option would let sessions share one .git and check out branches instantly — significant speedup for large repos.

Session continuity across idle cycles. In our setup, sessions get reclaimed after idle timeout, but users expect to return to the same thread and pick up where they left off. We tested ACP session/load but it's not supported by claude-agent-acp (returns Invalid params -32602). A broker-side approach — persisting conversation history and replaying it into a fresh session — would solve this without depending on agent-side support.

Open Question #2 — session resume after crash: Clean restart is more practical than resume for now. The ACP spec lists session/load but current implementations don't support it reliably.

Happy to help with implementation — we're running a patched version with #38 + #40 changes in production.

[m13v]

solid RFC. one area that gets tricky in practice is session recording and replay. if you're doing per-thread working directories and isolated sessions, capturing enough state to reproduce a session later (for debugging or auditing) becomes important.

we found that persisting the raw conversation transcript alongside periodic state snapshots gives you the best balance. the transcript alone isn't enough because you lose the file system state and tool outputs that influenced the agent's decisions. but full filesystem snapshots are too expensive.

for the graceful shutdown recovery in Phase 2, the approach that worked for us was writing session metadata (active thread, last message index, working directory hash) to a journal file on every turn completion. on restart, you can offer to resume from the journal rather than starting fresh. the key is making the journal write atomic (write to temp file, then rename) so a crash mid-write doesn't corrupt the recovery state.

the per-user session limits are a good safeguard. worth adding a soft limit that warns before hitting the hard limit, because users who hit the wall suddenly get frustrated.

[m13v]

our session capture service that records agent sessions with the state snapshot + journal approach described above: https://github.com/m13v/macos-session-replay/blob/main/Sources/SessionReplay/ScreenCaptureService.swift

handles the capture lifecycle, permission management, and the frame delivery pipeline that makes session recording lightweight enough to run continuously alongside the agent.

[chaodu-agent]

Hi @thepagent,

Comprehensive RFC — good to have the full session management vision documented in one place. Several sub-items already have independent issues and PRs in progress:

• fix: pool write lock held during entire prompt streaming causes cross-session deadlock #58 / PR fix: per-connection locking to prevent cross-session deadlock during long prompts #59 / PR fix: robust notification loop — per-connection locking, alive check, drain window, and startup cleanup #77 — deadlock fix (p1)
• feat: /close command to manually terminate a thread session #40 — /close command
• feat: lightweight management API for session observability and lifecycle control #39 / PR feat: lightweight management API for session observability and lifecycle control #57 — management API
• feat: per-thread isolated working directories #38 — per-thread working dirs

PR #79 has the RFC document. We'll discuss phasing with the team. Community feedback welcome on the open questions.