Skip to content

Implement M10 trajectory tracer + M11 run-benchmark CLI; wire SessionMux::set_emitter in runner agent#5

Merged
Billy1900 merged 11 commits into
mainfrom
m10-trajectory-tracer
Jun 6, 2026
Merged

Implement M10 trajectory tracer + M11 run-benchmark CLI; wire SessionMux::set_emitter in runner agent#5
Billy1900 merged 11 commits into
mainfrom
m10-trajectory-tracer

Conversation

@Billy1900

@Billy1900 Billy1900 commented Jun 6, 2026

Copy link
Copy Markdown
Owner
  • migrations/003_trajectories.sql: rollouts, trajectories, trace_events tables
  • arbor-common/src/trace.rs: TraceEvent enum, TraceEmitter, TrajectoryId/RolloutId, query row types (TrajectoryRow, TraceEventRow, RolloutSummary, TrajDiff)
  • arbor-controller/src/tracer.rs: TrajectoryStore with async batch writer, open/close trajectory, create rollout, get/list/export/diff/replay queries
  • state_machine.rs: inject TrajectoryStore into Controller; emit WorkspaceForked on fork_checkpoint (now opened synchronously before spawning drive_restore, fixing a race), CheckpointTaken on drive_checkpoint; drive_create_workspace opens trajectory before create_vm and passes trajectory_id in request; drive_restore accepts trajectory_id parameter
  • session_mux.rs: store command + output tail per session; emit ExecStarted / ExecExited trace events via injected TraceEmitter
  • arbor-runner-agent/src/tracer.rs (new): RunnerTracer with its own PgPool; open_trajectory() creates a new DB row, emitter_for_existing() attaches to a trajectory already opened by the controller; background batch writer flushes trace events to Postgres
  • arbor-runner-agent/src/vm_manager.rs: VmManager gains optional RunnerTracer field; set_emitter() is now called in both create_vm (after guest-agent handshake) and restore so ExecStarted/ExecExited events are actually emitted
  • arbor-runner-agent/src/main.rs: reads optional ARBOR_RUNNER__DATABASE_URL, creates PgPool and RunnerTracer, passes tracer to VmManager::new()
  • arbor-common/src/proto.rs: trajectory_id: Option added (with #[serde(default)]) to CreateVmRequest and VmRestoreRequest so the controller can pass an already-opened trajectory ID to the runner agent
  • arbor-egress-proxy: ProxyState.emitters DashMap; emit NetworkAccess on every allowed CONNECT and plain HTTP request
  • arbor-api: AppState gains tracer field; TrajectoryStore writer spawned in main; new routes/trajectories.rs with GET/POST for trajectories, rollouts, diff, replay, export (chunked NDJSON), workspace trajectory shortcut
  • arbor-cli: new crate with arbor run-benchmark, arbor replay, arbor diff, arbor export subcommands; ArborClient thin reqwest wrapper; comfy-table comparison output with pass/fail coloring and replay hints

- migrations/003_trajectories.sql: rollouts, trajectories, trace_events tables
- arbor-common/src/trace.rs: TraceEvent enum, TraceEmitter, TrajectoryId/RolloutId,
  query row types (TrajectoryRow, TraceEventRow, RolloutSummary, TrajDiff)
- arbor-controller/src/tracer.rs: TrajectoryStore with async batch writer,
  open/close trajectory, create rollout, get/list/export/diff/replay queries
- state_machine.rs: inject TrajectoryStore into Controller; emit WorkspaceForked
  on fork_checkpoint, CheckpointTaken on drive_checkpoint
- session_mux.rs: store command + output tail per session; emit ExecStarted /
  ExecExited trace events via injected TraceEmitter
- arbor-egress-proxy: ProxyState.emitters DashMap; emit NetworkAccess on every
  allowed CONNECT and plain HTTP request
- arbor-api: AppState gains tracer field; TrajectoryStore writer spawned in main;
  new routes/trajectories.rs with GET/POST for trajectories, rollouts, diff, replay,
  export (chunked NDJSON), workspace trajectory shortcut
- arbor-cli: new crate with arbor run-benchmark, arbor replay, arbor diff,
  arbor export subcommands; ArborClient thin reqwest wrapper; comfy-table
  comparison output with pass/fail coloring and replay hints
Copilot AI review requested due to automatic review settings June 6, 2026 16:52
@Billy1900 Billy1900 self-assigned this Jun 6, 2026
@Billy1900 Billy1900 added the enhancement New feature or request label Jun 6, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces the foundations for M10 “trajectory tracing” (DB schema + shared trace types + controller store + API endpoints) and adds a new M11 CLI crate (arbor) with run-benchmark, replay, diff, and export commands. It also updates the README and docs landing page to reflect the new rollout/trajectory positioning and roadmap.

Changes:

  • Add Postgres schema for rollouts/trajectories/trace events and Rust shared trace types (TraceEvent, TraceEmitter, ID newtypes).
  • Add controller-side TrajectoryStore with background writer and basic query/export/diff/replay helpers.
  • Add API routes for trajectories/rollouts and a new CLI crate that drives the REST API for benchmarking and analysis.

Reviewed changes

Copilot reviewed 27 out of 28 changed files in this pull request and generated 11 comments.

Show a summary per file
File Description
README.md Updates product framing + roadmap entries for M10/M11.
docs/index.html Updates marketing copy/examples to reflect rollouts/trajectories.
migrations/003_trajectories.sql Adds rollouts, trajectories, trace_events tables + indexes.
crates/arbor-runner-agent/src/session_mux.rs Adds trace emission hooks for exec start/exit (via injected emitter).
crates/arbor-egress-proxy/src/lib.rs Adds per-workspace emitters and emits NetworkAccess events.
crates/arbor-egress-proxy/Cargo.toml Adds dashmap dependency.
crates/arbor-controller/src/tracer.rs Adds TrajectoryStore (writer + queries + export/diff/replay helpers).
crates/arbor-controller/src/state_machine.rs Injects tracer and emits CheckpointTaken/opens trajectory on fork.
crates/arbor-controller/src/lib.rs Exposes tracer module + TrajectoryStore re-export.
crates/arbor-controller/Cargo.toml Adds dashmap dependency.
crates/arbor-common/src/types.rs Extends RestoreRequest with optional rollout_id.
crates/arbor-common/src/trace.rs Adds shared tracing types and TraceEvent enum.
crates/arbor-common/src/lib.rs Exposes trace module and re-exports trace types.
crates/arbor-common/Cargo.toml Adds tokio dependency.
crates/arbor-cli/src/main.rs Adds CLI entrypoint + subcommands.
crates/arbor-cli/src/client.rs Adds thin reqwest client for Arbor REST API.
crates/arbor-cli/src/commands/mod.rs Declares CLI command modules.
crates/arbor-cli/src/commands/benchmark.rs Implements run-benchmark flow and table output.
crates/arbor-cli/src/commands/replay.rs Implements replay command.
crates/arbor-cli/src/commands/diff.rs Implements diff command output.
crates/arbor-cli/src/commands/export.rs Implements export to NDJSON file.
crates/arbor-cli/Cargo.toml Adds new CLI crate dependencies and arbor bin.
crates/arbor-api/src/routes/trajectories.rs Adds M10 trajectory/rollout endpoints (list/export/replay/diff).
crates/arbor-api/src/routes/mod.rs Registers new trajectories routes.
crates/arbor-api/src/main.rs Adds tracer to AppState and spawns writer task.
crates/arbor-api/Cargo.toml Adds tokio-stream for streaming NDJSON export.
Cargo.toml Adds arbor-cli workspace member + dashmap workspace dependency.
Cargo.lock Updates lockfile for new crates/deps.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread migrations/003_trajectories.sql Outdated
Comment thread crates/arbor-controller/src/tracer.rs Outdated
Comment thread crates/arbor-api/src/routes/trajectories.rs Outdated
Comment thread crates/arbor-egress-proxy/src/lib.rs Outdated
Comment on lines +113 to +115
/// Per-workspace emitter for M10 trace events.
/// Populated by the API layer after fork_checkpoint opens a trajectory.
pub emitters: Arc<dashmap::DashMap<String, TraceEmitter>>,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Implemented in d1f606c. The API fork_checkpoint path now registers the workspace trace emitter into ProxyState (with a short async retry window to handle trajectory-open timing), so proxy NetworkAccess emits are no longer a guaranteed no-op.

Comment on lines +233 to +237
if let Some(ref e) = *emitter.read() {
e.emit(TraceEvent::ExecStarted {
session_id,
command: state.command.clone(),
cwd: String::new(),
Comment thread crates/arbor-controller/src/tracer.rs Outdated
Comment thread crates/arbor-api/src/routes/trajectories.rs
Comment thread crates/arbor-cli/src/commands/benchmark.rs
Comment thread crates/arbor-cli/src/commands/benchmark.rs
Billy1900 and others added 8 commits June 7, 2026 00:58
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
@Billy1900
Billy1900 merged commit 74cb0a3 into main Jun 6, 2026
@Billy1900
Billy1900 deleted the m10-trajectory-tracer branch June 6, 2026 17:07
Copilot AI changed the title Implement M10 trajectory tracer + M11 run-benchmark CLI Implement M10 trajectory tracer + M11 run-benchmark CLI; wire SessionMux::set_emitter in runner agent Jun 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants