feat: GovSignals v4.4.4 hardening (replaces #2, #3)

.claude/skills/span-timeline-events/SKILL.md

@@ -0,0 +1,78 @@
+---
+name: span-timeline-events
+description: Use when adding, modifying, or debugging OTel span timeline events in the trace view. Covers event structure, ClickHouse storage constraints, rendering in SpanTimeline component, admin visibility, and the step-by-step process for adding new events.
+allowed-tools: Read, Write, Edit, Glob, Grep, Bash
+---
+
+# Span Timeline Events
+
+The trace view's right panel shows a timeline of events for the selected span. These are OTel span events rendered by `app/utils/timelineSpanEvents.ts` and the `SpanTimeline` component.
+
+## How They Work
+
+1. **Span events** in OTel are attached to a parent span. In ClickHouse, they're stored as separate rows with `kind: "SPAN_EVENT"` sharing the parent span's `span_id`. The `#mergeRecordsIntoSpanDetail` method reassembles them into the span's `events` array at query time.
+2. The timeline only renders events whose `name` starts with `trigger.dev/` - all others are silently filtered out.
+3. The **display name** comes from `properties.event` (not the span event name), mapped through `getFriendlyNameForEvent()`.
+4. Events are shown on the **span they belong to** - events on one span don't appear in another span's timeline.
+
+## ClickHouse Storage Constraint
+
+When events are written to ClickHouse, `spanEventsToTaskEventV1Input()` filters out events whose `start_time` is not greater than the parent span's `startTime`. Events at or before the span start are silently dropped. This means span events must have timestamps strictly after the span's own `startTimeUnixNano`.
+
+## Timeline Rendering (SpanTimeline component)
+
+The `SpanTimeline` component in `app/components/run/RunTimeline.tsx` renders:
+
+1. **Events** (thin 1px line with hollow dots) - all events from `createTimelineSpanEventsFromSpanEvents()`
+2. **"Started"** marker (thick cap) - at the span's `startTime`
+3. **Duration bar** (thick 7px line) - from "Started" to "Finished"
+4. **"Finished"** marker (thick cap) - at `startTime + duration`
+
+The thin line before "Started" only appears when there are events with timestamps between the span start and the first child span. For the Attempt span this works well (Dequeued -> Pod scheduled -> Launched -> etc. all happen before execution starts). Events all get `lineVariant: "light"` (thin) while the execution bar gets `variant: "normal"` (thick).
+
+## Trace View Sort Order
+
+Sibling spans (same parent) are sorted by `start_time ASC` from the ClickHouse query. The `createTreeFromFlatItems` function preserves this order. Event timestamps don't affect sort order - only the span's own `start_time`.
+
+## Event Structure
+
+```typescript
+// OTel span event format
+{
+  name: "trigger.dev/run",        // Must start with "trigger.dev/" to render
+  timeUnixNano: "1711200000000000000",
+  attributes: [
+    { key: "event", value: { stringValue: "dequeue" } },  // The actual event type
+    { key: "duration", value: { intValue: 150 } },         // Optional: duration in ms
+  ]
+}
+```
+
+## Admin-Only Events
+
+`getAdminOnlyForEvent()` controls visibility. Events default to **admin-only** (`true`).
+
+| Event | Admin-only | Friendly name |
+|-------|-----------|---------------|
+| `dequeue` | No | Dequeued |
+| `fork` | No | Launched |
+| `import` | No (if no fork event) | Importing task file |
+| `create_attempt` | Yes | Attempt created |
+| `lazy_payload` | Yes | Lazy attempt initialized |
+| `pod_scheduled` | Yes | Pod scheduled |
+| (default) | Yes | (raw event name) |
+
+## Adding New Timeline Events
+
+1. Add OTLP span event with `name: "trigger.dev/<scope>"` and `properties.event: "<type>"`
+2. Event timestamp must be strictly after the parent span's `startTimeUnixNano` (ClickHouse drops earlier events)
+3. Add friendly name in `getFriendlyNameForEvent()` in `app/utils/timelineSpanEvents.ts`
+4. Set admin visibility in `getAdminOnlyForEvent()`
+5. Optionally add help text in `getHelpTextForEvent()`
+
+## Key Files
+
+- `app/utils/timelineSpanEvents.ts` - filtering, naming, admin logic
+- `app/components/run/RunTimeline.tsx` - `SpanTimeline` component (thin line + thick bar rendering)
+- `app/presenters/v3/SpanPresenter.server.ts` - loads span data including events
+- `app/v3/eventRepository/clickhouseEventRepository.server.ts` - `spanEventsToTaskEventV1Input()` (storage filter), `#mergeRecordsIntoSpanDetail` (reassembly)

.env.example

@@ -77,9 +77,28 @@ POSTHOG_PROJECT_KEY=
 # DEPOT_TOKEN=<Depot org token>
 # DEV_OTEL_EXPORTER_OTLP_ENDPOINT="http://0.0.0.0:4318"
 # These are needed for the object store (for handling large payloads/outputs)
-# OBJECT_STORE_BASE_URL="https://{bucket}.{accountId}.r2.cloudflarestorage.com"
-# OBJECT_STORE_ACCESS_KEY_ID=
-# OBJECT_STORE_SECRET_ACCESS_KEY=
+#
+# Default provider
+# OBJECT_STORE_BASE_URL=http://localhost:9005
+# OBJECT_STORE_BUCKET=packets
+# OBJECT_STORE_ACCESS_KEY_ID=minioadmin
+# OBJECT_STORE_SECRET_ACCESS_KEY=minioadmin
+# OBJECT_STORE_REGION=us-east-1
+# OBJECT_STORE_SERVICE=s3
+#
+# OBJECT_STORE_DEFAULT_PROTOCOL=s3  # Only specify this if you're going to migrate object storage and set protocol values below
+# Named providers (protocol-prefixed data) - optional for multi-provider support
+# OBJECT_STORE_S3_BASE_URL=https://s3.amazonaws.com
+# OBJECT_STORE_S3_ACCESS_KEY_ID=
+# OBJECT_STORE_S3_SECRET_ACCESS_KEY=
+# OBJECT_STORE_S3_REGION=us-east-1
+# OBJECT_STORE_S3_SERVICE=s3
+#
+# OBJECT_STORE_R2_BASE_URL=https://{bucket}.{accountId}.r2.cloudflarestorage.com
+# OBJECT_STORE_R2_ACCESS_KEY_ID=
+# OBJECT_STORE_R2_SECRET_ACCESS_KEY=
+# OBJECT_STORE_R2_REGION=auto
+# OBJECT_STORE_R2_SERVICE=s3
 # CHECKPOINT_THRESHOLD_IN_MS=10000
 
 # These control the server-side internal telemetry

CLAUDE.md

@@ -80,6 +80,10 @@ pnpm run changeset:add
 
 When modifying only server components (`apps/webapp/`, `apps/supervisor/`, etc.) with no package changes, add a `.server-changes/` file instead. See `.server-changes/README.md` for format and documentation.
 
+## Dependency Pinning
+
+Zod is pinned to a single version across the entire monorepo (currently `3.25.76`). When adding zod to a new or existing package, use the **exact same version** as the rest of the repo - never a different version or a range. Mismatched zod versions cause runtime type incompatibilities (e.g., schemas from one package can't be used as body validators in another).
+
 ## Architecture Overview
 
 ### Request Flow

apps/supervisor/Containerfile

@@ -25,7 +25,8 @@ RUN --mount=type=cache,id=pnpm,target=/root/.local/share/pnpm/store pnpm fetch -
 FROM deps-fetcher AS dev-deps
 ENV NODE_ENV development
 
-RUN --mount=type=cache,id=pnpm,target=/root/.local/share/pnpm/store pnpm install --frozen-lockfile --offline --ignore-scripts
+# TEMP --no-frozen-lockfile and remove --offline for overrides for CVEs
+RUN --mount=type=cache,id=pnpm,target=/root/.local/share/pnpm/store pnpm install --no-frozen-lockfile --ignore-scripts
 
 FROM base AS builder
 

apps/supervisor/package.json

@@ -14,9 +14,11 @@
   },
   "dependencies": {
     "@aws-sdk/client-ecr": "^3.839.0",
+    "@internal/compute": "workspace:*",
     "@kubernetes/client-node": "^1.0.0",
     "@trigger.dev/core": "workspace:*",
     "dockerode": "^4.0.6",
+    "p-limit": "^6.2.0",
     "prom-client": "^15.1.0",
     "socket.io": "4.7.4",
     "std-env": "^3.8.0",