Skip to content

Harden provider runtime observability and secret safety (codex) - #2369

Merged
numnx merged 2 commits into
feature/CODUX-143-qs-coverage-observability-and-ci-reliability-audit-3from
task/feature-codux-143-qs-coverage-observability-and--t06-codex-mr9l879j
Jul 6, 2026
Merged

Harden provider runtime observability and secret safety (codex)#2369
numnx merged 2 commits into
feature/CODUX-143-qs-coverage-observability-and-ci-reliability-audit-3from
task/feature-codux-143-qs-coverage-observability-and--t06-codex-mr9l879j

Conversation

@numnx

@numnx numnx commented Jul 6, 2026

Copy link
Copy Markdown
Collaborator

🧩 Task: Harden provider runtime observability and secret safety

Automated task PR opened by Code UX for T06.

📋 Summary

Sprint: Sprint 143 · QS: Coverage Observability and CI Reliability Audit · Task: T06

Strengthen Code UX coverage observability and CI reliability through enforceable tests, workflow hardening, structured observability guarantees, security validation, and aligned operational documentation without lowering coverage thresholds.

📝 Full Task Prompt
## Objective
Strengthen provider runtime observability and security validation so invocation records and logs remain useful without leaking secrets or raw transcripts.

## Scope
- `src/infrastructure/providers/cli/provider-runner.ts`
- `src/infrastructure/providers/cli/provider-execution-loop.ts`
- `src/infrastructure/providers/cli/provider-telemetry-watcher.ts`
- `src/infrastructure/providers/cli/docker-runner.ts`
- `src/infrastructure/providers/cli/workspace-manager.ts`
- `src/repositories/execution-repository.ts`
- `tests/backend/infrastructure/providers/cli/provider-runner.test.ts`
- `tests/backend/infrastructure/providers/cli/provider-execution-loop.test.ts`
- `tests/backend/infrastructure/providers/cli/provider-telemetry-watcher.test.ts`
- `tests/backend/infrastructure/providers/cli/docker-runner.test.ts`
- `tests/backend/infrastructure/providers/cli/workspace-manager.test.ts`
- `tests/backend/repositories/execution-repository.test.ts`
- `docs/operations/logging-and-correlation.md`
- `docs/operations/runbook.md`
- Current verification gap: provider telemetry tests cover some secret cases, but the audit needs broader guarantees for invocation records, Docker env handling, and metadata-only logging.
- Desired guarantee: provider invocations are recorded with correlation-aware metadata, Docker arguments do not contain secret values, debug logs are level-controlled, and raw transcripts or usage JSON never appear in structured log metadata.

## Implementation Requirements
1. Extend provider telemetry watcher tests to assert `logPurpose: "invocation"`, active `correlationId`, provider/session/invocation identifiers, token counters, and absence of raw transcript/API key values in success and failure logs.
2. Extend Docker runner or workspace manager tests to prove provider environment secrets are written through env files or controlled mounts and never appear in process argument arrays or structured logs.
3. Add execution repository coverage for creating, updating, and querying provider invocation rows with failure metadata and provider invocation IDs preserved.
4. Add debug log level tests if needed in `tests/backend/shared/logging/logger.test.ts` or provider tests so file logging honors `DEBUG_LOG_FILE_LEVEL` independently of console filtering.
5. Update `docs/operations/logging-and-correlation.md` and `docs/operations/runbook.md` with the provider invocation record fields, log sanitization guarantees, and focused validation commands.

## Constraints
- Do not log raw provider transcripts, API keys, provider env values, raw usage JSON payloads, or full command prompts.
- Do not require Docker to be installed for tests; mock spawned commands and filesystem writes.
- Preserve existing provider invocation schema and public dashboard fields unless a test exposes a bug.
- Keep telemetry metadata bounded and stable for CI output.

## Verification
- `pnpm run test:backend -- tests/backend/infrastructure/providers/cli/provider-runner.test.ts tests/backend/infrastructure/providers/cli/provider-execution-loop.test.ts tests/backend/infrastructure/providers/cli/provider-telemetry-watcher.test.ts tests/backend/infrastructure/providers/cli/docker-runner.test.ts tests/backend/infrastructure/providers/cli/workspace-manager.test.ts tests/backend/repositories/execution-repository.test.ts tests/backend/shared/logging/logger.test.ts`
- `pnpm run test:backend:coverage`
- `pnpm run lint`
- Success means provider telemetry is correlation-aware and metadata-only, secrets stay out of logs/process args, and invocation records preserve operational diagnostics.

✅ QA Review

⏳ QA review runs after this PR is opened. Check the task card in the dashboard for the outcome once it lands.

🤖 Model & Provider

Provider Codex CLI
Model gpt-5.5

⏱️ Timing

Started 2026-07-06 19:03:02 UTC
Finished 2026-07-06 19:15:18 UTC
Duration 12m 16s

📊 Token Usage

Metric Value
Input tokens 138,414
Cached input tokens 4,116,864
Output tokens 13,199
Total tokens 4,268,477
Invocations 1
Tool calls 63
Active time 10m 50s

Included usage estimate (subscription/local login): $3.15

🌿 Branch Info

Base: feature/CODUX-143-qs-coverage-observability-and-ci-reliability-audit-3
Head: task/feature-codux-143-qs-coverage-observability-and--t06-codex-mr9l879j


🤖 Generated by Code UX

Code UX added 2 commits July 6, 2026 19:15
…i-reliability-audit-3 into task/feature-codux-143-qs-coverage-observability-and--t06-codex-mr9l879j
@numnx
numnx merged commit 19e329a into feature/CODUX-143-qs-coverage-observability-and-ci-reliability-audit-3 Jul 6, 2026
@numnx
numnx deleted the task/feature-codux-143-qs-coverage-observability-and--t06-codex-mr9l879j branch July 6, 2026 19:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant