Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .llm/tmp/run/auth-s8-audit-observability/commits.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# AS8 commits

- 17b27819: feat(auth): add audit telemetry observability
40 changes: 40 additions & 0 deletions .llm/tmp/run/auth-s8-audit-observability/context-pack.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
# AS8 — Auth Audit Observability — context-pack.md

## Current State

- Worktree: `/home/codex/repos/netscript-as8`
- Branch: `feat/prime-time/auth-s8-audit-observability`
- Base noted by plan: `origin/main` at `6f1c40f0`
- PR target: #103

## Implemented

- Added auth telemetry constants, outcome/error taxonomies, salted HMAC subject hashing, principal
redaction, and a no-op-safe `createAuthTelemetry()` factory under
`packages/plugin-auth-core/src/telemetry/`.
- Exported `@netscript/plugin-auth-core/telemetry` and root public telemetry symbols.
- Injected auth telemetry at the service composition root and wrapped all v1 handlers with
parent-aware child spans.
- Added durable stream trace propagation for auth lifecycle events and session records.
- Extended logger sensitive-field fragments for session/access/refresh/JWT token key variants.
- Added tests for hashing/redaction, per-operation auth span attributes, negative token leakage, and
durable trace propagation.

## Validation Summary

- Root `deno task check`: PASS.
- Scoped check/lint/fmt wrappers for `packages/plugin-auth-core` and `plugins/auth`: PASS.
- `packages/plugin-auth-core`: `deno task check` PASS, `deno task test` PASS (26/0), full export
`deno doc --lint` PASS, publish dry-run PASS.
- `plugins/auth`: `deno task check` PASS, `deno task test` PASS (17/0).
- `deno.lock` unchanged.

## Remaining

- Commit and push with explicit refspec:
`git -C /home/codex/repos/netscript-as8 push origin HEAD:refs/heads/feat/prime-time/auth-s8-audit-observability`
- Append `commits.md` after commit.
- PR comment is requested by the harness; this session is operating under the prompt's
GitHub-API-blind caveat, so use the available approved PR-comment mechanism from the supervisor if
GitHub app access is unavailable here.

14 changes: 14 additions & 0 deletions .llm/tmp/run/auth-s8-audit-observability/drift.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
# AS8 — Auth Audit Observability — drift.md

## 2026-06-21 — process artifact gap

- **Severity:** minor/process
- **Plan reference:** `plan.md` requires persisted PLAN-EVAL before implementation.
- **Observed:** This worktree contained `research.md`, `plan.md`, and `commits.md` only when the
generator started. `plan-eval.md`, `worklog.md`, `context-pack.md`, and `drift.md` were absent.
- **Decision:** The implementation brief explicitly stated PLAN-EVAL had passed and directed the WSL
Codex generator to implement. The generator did not self-certify PLAN-EVAL and did not fabricate an
evaluator verdict artifact; this drift entry records the missing local artifact.
- **Closure:** Supervisor/evaluator should restore or attach the separate-session PLAN-EVAL artifact
before IMPL-EVAL closes the run.

109 changes: 109 additions & 0 deletions .llm/tmp/run/auth-s8-audit-observability/evaluate.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,109 @@
# IMPL-EVAL — AS8 Auth Audit Observability

- Slice: AS8
- Branch: `feat/prime-time/auth-s8-audit-observability`
- Base: `main` (`6f1c40f0`)
- Impl commit: `17b27819`
- HEAD: `b38d9607` (plus harness trace commit `2b852e3a` from prior run)
- Run id: `auth-s8-audit-observability`
- Evaluator: OpenHands (`openrouter/qwen/qwen3.7-max`), run `27901147563-1`
- Scope: additive audit telemetry on `packages/plugin-auth-core` (package) + wiring in `plugins/auth` (plugin)

## Verdict

**PASS**

Prior evaluator run (`run-27900718714-1`) computed PASS and reported all 8 gates green but exhausted
its iteration budget before rendering the formal artifact and PR comment, so the harness had no
terminal record. This session independently re-ran the gates below to confirm before rendering the
terminal verdict.

## Gate Results

| # | Gate | Scope | Exit | Outcome |
|---|------|-------|------|---------|
| 1 | `run-deno-check.ts` | `packages/plugin-auth-core` (22 files) | 0 | PASS |
| 2 | `run-deno-check.ts` | `plugins/auth` (29 files) | 0 | PASS |
| 3 | `run-deno-lint.ts` | `packages/plugin-auth-core` (22 files) | 0 | PASS |
| 4 | `run-deno-lint.ts` | `plugins/auth` (29 files) | 0 | PASS |
| 5 | `run-deno-fmt.ts` | `packages/plugin-auth-core` (22 files, ts/tsx) | 0 | PASS |
| 6 | `run-deno-fmt.ts` | `plugins/auth` (29 files, ts/tsx) | 0 | PASS |
| 7 | `deno test ... telemetry_test.ts --unstable-kv` | core telemetry unit | 0 | PASS (3/3) |
| 8 | `deno test plugins/auth/tests --unstable-kv -A` | auth plugin test surface | 0 | PASS (17/17) |

Notes on Gate 8: the scoped wrapper in the prompt (`deno test plugins/auth/tests --unstable-kv`)
omits permissions. The plugin's own `test` task in `plugins/auth/deno.json` declares
`deno test --unstable-kv --allow-all`, and several suites (scaffold manifest reader,
better-auth service module, debug import-surface) require read/env permissions to import. Re-running
with `-A` per the plugin's own task config yielded 17/17 pass; this is configuration/permission
evidence, not a test defect — consistent with the prior run's green outcome.

## Slow-Types (closes prior-run remaining risk #3)

```
deno doc --lint mod.ts src/config/mod.ts src/contracts/v1/mod.ts src/domain/mod.ts
src/ports/mod.ts src/presets/mod.ts src/streams/mod.ts src/testing/mod.ts
src/telemetry/mod.ts
```

From `packages/plugin-auth-core/` (the full 9-entry `exports` map including the new
`./telemetry` barrel which uses `export *`):

- `Checked 9 files`
- Exit: **0** (0 diagnostics)

Confirmed independently. The new telemetry surface (AuthAttributes enum, createAuthTelemetry API,
hashSubject, redactAuthPrincipal) types resolve fully for JSR slow-types; no ambiguous-type
findings.

## Cast Audit

Search of S8-changed files for unsanctioned `as` / `any`:

- `packages/plugin-auth-core/src/telemetry/*.ts` — no `as unknown`/`any`
- `plugins/auth/services/src/routers/v1-handlers.ts`, `v1-types.ts` — no `as unknown`/`any`
- `plugins/auth/streams/producer.ts`, `schema.ts` — only sanctioned `X as CORE_X` import-rename
bindings (not casts) and documentation prose

D5 traceparent propagation in `plugins/auth/streams/producer.ts` flows through a domain-typed
`SerializedTraceContext` interface, NOT through a cast — matches the sagas exemplar.

Zero-cast policy: **PASS**

## Redaction + D5

Telemetry tests independently verify:

- `hashSubject` produces a 64-char salted HMAC and does not include the raw subject material in
the output, with salt-dependent divergence.
- `redactAuthPrincipal` removes `accessToken` and nested `refresh_token` claims from the serialized
principal output while preserving non-secret claims (email, displayName) and counts.
- `createAuthTelemetry.traceOperation` records `AuthSpanNames.ME` span with sanitized
`SUBJECT_HASH` attribute and `AUDIT_LOG` + `PRINCIPAL_RESOLVED` events, with no raw subject or
access-token material in the serialized span output.

The auth-service integration test (`auth handlers emit audit-safe telemetry attributes per
operation`) exercises the wired telemetry through a real kv-oauth registry round-trip and asserts
the same redaction properties on the recorded span.

D5 traceparent: persisted as a first-class `traceparent?: string` field on the
`AuthSessionProjectionV1` and `AuthEventV1` event schemas (`streams/schema.ts:37,70`), populated
from the OTel active-span headers or propagated via the explicit `SerializedTraceContext` port.
Not a cast.

## Lock Hygiene

- `deno.lock` diff `origin/main..HEAD`: **0 commits** touched. No churn.
- No junk artifacts or accidental tracked files in the implementation slice.
- `17b27819` (impl) and `b38d9607` (commits.md append) are the only substantive commits; the prior
evaluator run left `2b852e3a` (harness trace record) which is benign metadata, not source churn.

## Rationale

All 8 scoped gates are green with raw exit 0; the full 9-entry `@netscript/plugin-auth-core`
export map clears `deno doc --lint` independently (closing the only open risk flagged by the
prior run); zero-cast and redaction/D5 properties are asserted by substantive unit and integration
tests; lock hygiene is clean. The implementation is additive and mirrors the sagas + workers
telemetry seam (domain-typed attributes, redaction port, createAuthTelemetry DI seam, durable-event
trace propagation), with no source churn outside the targeted telemetry module surface. Verdict is
`PASS`.
149 changes: 149 additions & 0 deletions .llm/tmp/run/auth-s8-audit-observability/plan-eval.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,149 @@
# AS8 — Auth Audit Observability — plan-eval.md

> PLAN-EVAL artifact (Plan-Gate hard stop). Session: `openhands/minimax-M3`, separate session.
> Branch: `feat/prime-time/auth-s8-audit-observability` @ `3b9c75b1`.
> Base: `main` @ `6f1c40f0` (auth umbrella + framework-prime-time both merged there via #73).
> Inputs: `.llm/tmp/run/auth-s8-audit-observability/{research.md,plan.md}`.
> Verdict: **PASS**.

## 1. Inputs and branch state

- `research.md` and `plan.md` both present and committed on `feat/prime-time/auth-s8-audit-observability` @ `3b9c75b1` (re-baseline commit, `docs(harness): AS8 auth-audit-observability plan-of-record (re-baselined to main)`).
- Working tree clean (only untracked `.llm/tmp/run/openhands/pr-103/`). No tracked file modifications.
- Plan-of-record section §1 confirms umbrella re-base to `main`; commit log shows the re-baseline.

## 2. Archetype + scope selection (plan-gate §1)

| Item | Plan claim | Evidence on `main` | OK |
|------|------------|---------------------|----|
| Primary archetype ARCHETYPE-5 (plugin) | `plugins/auth` exists at top level; ARCHETYPE-5 doctrine file lists `plugins/auth` as the canonical example | `plugins/auth/{mod.ts,contracts.ts,verify-plugin.ts,streams/,services/,tests/}` layout matches | ✅ |
| Supporting ARCHETYPE-2/3 — `packages/plugin-auth-core` | Package layout is `src/{config,contracts,domain,ports,presets,public,streams,testing}` — consistent with archetype-3 doctrine | `packages/plugin-auth-core/src/` matches | ✅ |
| Reuse `@netscript/telemetry` + `@netscript/logger` | Both packages present | `packages/telemetry/`, `packages/logger/` exist | ✅ |
| Scope overlay `SCOPE-service.md` | Plan §1 references overlay; deliverables are service + plugin wiring (withRPC, instrumentation factory), not CLI/browser/runtime expansion | `plugins/auth/services/src/main.ts:74` `.withRPC({ traceContext: true })` confirms service scope | ✅ |

Archetype and scope selection are correct.

## 3. Contract-first ordering (plan-gate §1)

- §3 deliverables order: `AuthAttributes`/`AuthSpanNames`/`AuthErrorCode` typed constants → `createAuthTelemetry()` factory → service wiring → tests. Contract-first. ✅
- §8 design checkpoint re-states the same contract-first order and sagas-parity DI seam. ✅
- Mirrors `plugins/sagas/src/telemetry/otel-saga-tracer.ts` + `plugins/sagas/services/src/main.ts:93` `createSagaTelemetry()` injection, verified on `main`. ✅

## 4. Zero-cast compliance (plan-gate §2)

- §5 and research.md §7 both enumerate the rule: **no new `any` / `as unknown as`** in handler / business-logic / contract code. Neither sanctioned exemplar (Class A router-composition, Class B external-boundary) is in scope here.
- Auth service is not a router of the Class A shape (the Class A exemplar is `plugins/sagas/services/src/router.ts:39-44,71-75` per `arch-debt.md:889`, not auth). AS8 touches `services/src/router.ts` only for telemetry propagation, not router composition.
- No SDK adapters are touched (backends stay pure), so Class B is not in scope either.
- All telemetry reuses `@netscript/telemetry` primitives — no hand-rolled span `any` hacks.

Zero-cast compliance is satisfied.

## 5. Gate set adequacy (plan-gate §3)

| Gate | Plan coverage | Verdict |
|------|---------------|---------|
| `deno task check` (+ `--unstable-kv` on KV-touching checks) | §4 | ✅ |
| `run-deno-check.ts` / `run-deno-lint.ts` / `run-deno-fmt.ts` scoped to `packages/plugin-auth-core` + `plugins/auth` (`--ext ts,tsx`) | §4 | ✅ |
| `deno task test` (auth + plugin-auth-core) | §4 | ✅ |
| `jsr-audit` A1 doc-lint full export set on `packages/plugin-auth-core` | §4 | ✅ |
| `scaffold.runtime` deferred to umbrella merge-readiness (per AGENTS.md trigger template, not per-loop) | §4 | ✅ |
| `arch:check` (doctrine) | Implicit in `deno task` chain; covered by `jsr-audit` A1 export-set check | ✅ |

Gate set is adequate for a plan-of-record; no scope gaps.

## 6. Anchor resolution (plan-gate §4)

All file:line anchors in `research.md` and `plan.md` resolve on `main` @ `6f1c40f0`:

| Anchor | Resolves? | Note |
|--------|-----------|------|
| `packages/auth-better-auth/src/better-auth.ts:136-162` | ✅ | `authenticate()` with bare `better_auth_session_lookup_failed` reason confirmed. |
| `packages/auth-workos/src/workos-authenticator.ts:147-187` | ✅ | `authenticate()` with bare `workos_session_cookie_missing` / `workos_authentication_failed` reasons confirmed. |
| `packages/service/src/builder/service-builder-impl.ts:244-267` (ctx.traceHeaders) | ✅ | File 510 LOC. |
| `packages/service/src/primitives/handlers.ts:72-114` | ✅ | File 221 LOC. |
| `packages/logger/constants.ts:11-19` (`SENSITIVE_FIELD_FRAGMENTS`) | ✅ | Verified list: `password`, `token`, `secret`, `key`, `auth`, `credential`, `apikey`. |
| `packages/logger/orpc-plugin.ts:376-394` (`redactSensitiveFields`) | ✅ | Verified at 376-394. |
| `packages/telemetry/deno.json:6-14` (exports map) | ✅ | Verified exports `./`, `./config`, `./tracer`, `./context`, `./attributes`, `./instrumentation`, `./registry`, `./orpc`. |
| `packages/telemetry/src/context/mod.ts:13-14` (re-export of `extractContext`) | ✅ | File 16 LOC. |
| `packages/telemetry/src/context/payload-context.ts:8-22` (`createJobTraceEnv`) | ✅ | File 48 LOC. |
| `packages/telemetry/src/context/w3c.ts:74-92` (`injectContext`) + `:120-149` (`extractContext`) | ✅ | File 162 LOC. |
| `packages/telemetry/src/instrumentation/worker.ts:202-287` (`traceJobExecution`) | ✅ | File 570 LOC. |
| `packages/telemetry/src/public/mod.ts:9-27` | ✅ | File 27 LOC. |
| `plugins/sagas/services/src/main.ts:93` (createSagaTelemetry injection) | ✅ | `import { createSagaTelemetry } from '../../src/telemetry/otel-saga-tracer.ts'` at line 33; `.withInstrumentation({ instrumentation: createSagaTelemetry(), … })` at line 93. |
| `plugins/sagas/src/telemetry/otel-saga-tracer.ts:22-44` | ✅ | File 102 LOC. |
| `plugins/workers/services/src/main.ts:62` (`withRPC({ traceContext: true })`) | ✅ | `.withRPC({ traceContext: true })` confirmed on workers `main.ts` (auth's `.withRPC` lives at `plugins/auth/services/src/main.ts:74`). |
| `plugins/workers/services/src/routers/jobs.ts:93-119` | ✅ | File 122 LOC. |
| `plugins/workers/worker/job-dispatcher.ts:27-28` + `:58-66` (extract + persist traceparent) | ✅ | File 297 LOC. |
| `plugins/auth/services/src/main.ts:74` (`.withRPC({ traceContext: true })` on auth) | ✅ | Confirmed. |

All 18 anchors resolve. No drift.

## 7. Net-new telemetry surface (plan-gate §5)

- `packages/plugin-auth-core/src/` currently contains: `config/`, `contracts/`, `domain/`, `ports/`, `presets/`, `public/`, `streams/`, `testing/`. **No `telemetry/` directory exists.** ✅
- Plan §3 introduces `packages/plugin-auth-core/src/telemetry/` (additive, no conflicts). ✅

Surface is genuinely net-new — no duplication or migration risk.

## 8. Decision soundness

### D2 — salted HMAC subject-hash

- `HMAC-SHA-256(subject, deployment_salt)`. Salt from deployment config (env/appsettings), **never derived from subject**. Same subject → same hash within a deployment → audit correlation. Not reversible. Not cross-deployment-linkable. ✅
- Sound, well-motivated design. Uses Web Crypto `crypto.subtle.sign('HMAC', …)` (no hand-rolled hashing).

### D3 — mandatory tested redaction

- `SENSITIVE_FIELD_FRAGMENTS` extension adds `['sessionid','accesstoken','refreshtoken','jwttoken']`. These are app-specific fragments, distinct from existing 7. ✅
- `redactAuthPrincipal()` hashes `subject` and strips token-bearing claims before any span attribute / log field. ✅
- §3 negative assertions over serialized output guarantee "no raw subject/token" in span attributes or log fields. ✅
- Sound — closes research.md §5 baseline gap.

### D4 — per-operation child spans

- `auth.signin|callback|signout|session|me`, parented via `getParentContextFromHeaders(ctx.traceHeaders)` — exactly the workers `job-dispatcher.ts:27-28` pattern, and `withRPC({ traceContext: true })` already in place at `plugins/auth/services/src/main.ts:74`. ✅
- Connection to inbound request trace is established by existing service-builder plumbing, not new cast-worthy code. Sound.

### D5 — durable-event traceparent propagation against AS4 emit seam

- Auth durable events emit through `plugins/auth/streams/producer.ts` (verified `upsert`/`delete` via `DurableStreamProducer` from `@netscript/plugin-streams-core`); producer interface has `upsert`/`delete`/`flush`/`close`.
- Plan calls for `injectContext()` onto event payload/headers + persist `traceparent`/`tracestate` on the event record, mirroring `plugins/workers/worker/job-dispatcher.ts:58-66` execution-state persistence. ✅
- **Contingency noted in plan §6:** if the AS4 emit seam cannot carry `traceparent` without a typed change, the plan **explicitly requires** a scoped `arch-debt.md` entry rather than smuggling a cast. ✅
- Sound; the contingency is the right call — it preserves zero-cast compliance and keeps debt visible.

### D6 — typed error taxonomy via ErrorHandlingPlugin

- `ErrorHandlingPlugin` lives at `packages/telemetry/src/orpc/error-plugin.ts` (referenced from `plugins/auth/tests/services/auth-service_test.ts` and `packages/service/src/primitives/handlers.ts`). ✅
- Typed codes `AUTH_INVALID_CREDENTIALS`, `AUTH_SESSION_EXPIRED`, `AUTH_PROVIDER_ERROR`, `AUTH_CALLBACK_INVALID` map from backend string reasons at the service seam; backends stay unchanged externally. ✅
- Sound, machine-readable audit outcomes.

### D7 — middleware seam out of scope

- `packages/service/src/auth/auth-middleware.ts` excluded with explicit follow-up trigger (only if AS7 fitness flags it). ✅
- Good slice discipline; keeps AS8 bounded to the auth plugin's owned surface.

## 9. Doctrine and debt

- Contract-first ✅ (Section 3)
- No reinvention ✅ (reuses `@netscript/telemetry` + `@netscript/logger`)
- Additive-only public surface ✅ (Section 3, Section 5)
- No new debt entries required for AS8. §6 is a forward-looking contingency for D5, not a current obligation. ✅
- Plan does **not** introduce a doctrine violation.

## 10. Drift check

- No drift detected between `plan.md` / `research.md` and the repo state on `main` @ `6f1c40f0`. All anchors resolve; surface is net-new; gate set is scoped correctly; out-of-scope middleware is explicitly bounded; umbrellas (auth #73 + framework-prime-time #86) are already merged at base.

## 11. Verdict

**PASS.**

The plan is contract-first, zero-cast compliant, archetype/scope-correct, anchor-verified, net-new surface, with an adequate scoped gate set and a sound contingency path for the durable-event propagation. The generator may proceed on `feat/prime-time/auth-s8-audit-observability` (base `origin/main` @ `6f1c40f0`) without further planning cycles.

## 12. Implementation constraints (carried forward to IMPL-EVAL)

- No source modifications during PLAN-EVAL (this session). ✅ (no commits to source; only `plan-eval.md` written).
- No `deno.lock` churn.
- No junk files.
- All gate evidence must be persisted at `.llm/tmp/run/auth-s8-audit-observability/gates/`.
- If D5 hits the AS4 typed-change contingency, file a scoped `arch-debt.md` entry **before** the cast is smuggled in.
Loading
Loading