Skip to content

feat(certification): improve chat reliability and outcomes - #6239

Merged
bokelley merged 3 commits into
mainfrom
refine-user-flow
Aug 6, 2026
Merged

feat(certification): improve chat reliability and outcomes#6239
bokelley merged 3 commits into
mainfrom
refine-user-flow

Conversation

@bokelley

@bokelley bokelley commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

Improves the certification chat experience with durable retries and resume, atomic turn leases, completion-capacity protection, verified contributions, clearer credential recovery, and configurable model selection.
Adds success instrumentation and an admin dashboard for turn recovery, resume-to-completion, evidence and credential outcomes, capacity blocks, contributions, and model-by-module performance.
The root cause was that certification progress, streaming delivery, and terminal state were handled independently, allowing interrupted or concurrent requests to lose context or misattribute outcomes.
Validated with expert security, learning-design, and concurrency reviews; focused tests, migration checks, TypeScript and production Docker builds, local migration 536, and real-browser learner/admin UI checks all pass (the full precommit server suite hit its 240-second host-load timeout, and its lone reported test passed independently).

@bokelley
bokelley marked this pull request as ready for review August 6, 2026 06:15
@bokelley bokelley changed the title [codex] Improve certification chat reliability and outcomes feat(certification): improve chat reliability and outcomes Aug 6, 2026
Comment thread server/src/services/certification-experience.ts Outdated

@aao-secretariat aao-secretariat Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ladon verdict: Approve

Approve — certification chat reliability rework (idempotent web-turn leases, interrupted-turn recovery/replay, resume-to-conversation, bounded per-day cost reserve with cross-replica lease, verified GitHub contributions, experience metrics + admin dashboard).

Gate checks

  • Platform-only change (server/**); no static/schemas/source/**, docs/**, dist/**, or task/compliance surface touched → changeset/spec-drift/oneOf/immutability rules N/A.
  • Migrations 535/536 are new files (added) under a high-risk glob — inherently low risk; reviewer flagged no medium-or-higher concern on them. Judged as production-facing (auto-run on startup) and found sound.
  • No V1/V2 contamination, no destructive/deleted files, no gated_paths (false), no no-auto-approve team match.
  • Certification/exam completion remains reachable only through Addie tool calls; no REST self-report route introduced.

Findings

  • 1 medium (certification-experience.ts:326 — GitHub issue-verification fetch has no timeout on a user-facing route) and 1 low (omitted). No critical/high.

Decision path: No critical/high (row 1 no). gated_paths false (row 2 no). No deleted high-risk file (row 3 no). The single medium is an operability/timeout gap, category is not data-loss/schema/infra (row 4 no). High-risk matches are on modified files, but the medium finding is on certification-experience.ts, which is not itself in the high_risk_reasons list (row 5 no). No prior decision (row 6 no). No team gate (row 7 no). Only 1 medium, fewer than 3 (row 8 no). Falls through to row 9 → approve.

Recommend addressing the missing fetch timeout as a follow-up before this path sees heavy load.

Medium findings

  • server/src/services/certification-experience.ts:326 — GitHub issue-verification fetch has no timeout on a user-facing route

@aao-secretariat aao-secretariat Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ladon verdict: Approve

Approve — delta resolves the prior review's sole finding with no new issues.

The prior run (approve) flagged one medium: the certification GitHub issue-verification fetch had no timeout on a user-facing route. This delta resolves it — the fetch is now bounded by AbortSignal.timeout(10_000), authenticates via resolveGitHubToken() (fails closed to null, never throws), and converts fetch/JSON-parse failures and rate-limit responses into retryable user errors. New unit tests cover timeout, rate-limit, secondary rate-limit, body-timeout, and last-request success paths.

Checked against decision rules:

  • No critical/high findings (row 1 n/a).
  • gated_paths is false (row 2 n/a).
  • high_risk is true, but all high-risk migration files are (added) scaffolding and the modified server/src/addie/**, server/src/routes/** files carry no medium-or-higher findings (rows 3–5 n/a). No deletions.
  • Two new migrations (535, 536) run automatically on startup; reviewer surfaced no data-loss/schema/infra medium (row 4 n/a).
  • Prior decision was approve, not escalate (row 6 n/a).
  • No no-auto-approve team match (row 7 n/a).
  • Zero medium findings this run (row 8 n/a).

Falls through to row 9: approve. Net improvement over prior pass — the sole prior finding is fixed and no new findings emerged.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant