stage1: consolidate items 2-4 and fix webhook notification schema gate - #23
Conversation
Why: Stage 1 item 1 requires CI merge gate stability; GitHub Actions started failing before checks due conflicting pnpm versions between workflow input and packageManager pin. Tests: pnpm verify:phase-boundary; pnpm ci:check Evidence: .github/workflows/comment-copilot-ci.yml; .github/workflows/isolation-check.yml; docs/ops/stage-1-evidence.md (Item 1 note update); failing run logs 22670447675 and 22670447688 show ERR_PNPM_BAD_PM_VERSION root cause
Why: Stage 1 item 1 CI gates still failed after pnpm version unification because workflow cache and execution paths pointed to a non-existent comment-copilot/ subdirectory in this repo. Tests: pnpm verify:phase-boundary; pnpm ci:check Evidence: .github/workflows/comment-copilot-ci.yml; .github/workflows/isolation-check.yml; docs/ops/stage-1-evidence.md (Item 1 note); failing run logs 22674824106 and 22674823812 showed unresolved cache-dependency-path
Why: Stage 1 item 1 CI gates still failed because setup-node cache mode requires a tracked dependency path; this repository does not track a pnpm lockfile path for cache lookup in CI. Tests: pnpm verify:phase-boundary; pnpm ci:check Evidence: .github/workflows/comment-copilot-ci.yml; .github/workflows/isolation-check.yml; docs/ops/stage-1-evidence.md (Item 1 note); failing run logs 22674931922 and 22674931920 showed unresolved cache dependency path
Why: Establish a single global instruction for continuous autonomous execution and explicitly reference it from Stage 1 boundary rules. Tests: pnpm verify:phase-boundary; pnpm ci:check Evidence: docs/dev-phase-policy.md (Execution Continuity section); docs/dev-phase-ops-hardening.md (Autonomous Agent Rules pointer)
Why: Complete Stage 1 Item 2 scope by wiring webhook latency/failure observability and external error-tracking forwarding across Instagram, TikTok, and Stripe webhook handlers. Tests: pnpm --filter @copilot/web test:webhooks:e2e:ci; node scripts/report-webhook-latency.mjs /tmp/stage1_item2_webhook_observability.log; pnpm verify:phase-boundary; pnpm ci:check Evidence: Updated docs/dev-phase-ops-hardening.md Item 2 tracker and docs/ops/stage-1-evidence.md Item 2 row; artifacts at apps/web/test-results/webhooks.e2e.junit.xml, /tmp/stage1_item2_webhook_observability.log, /tmp/stage1_item2_webhook_latency_report.txt.
Why: Complete Stage 1 Item 3 runbook deliverables (webhook replay, provider outage, token/billing, and triage escalation flow) with dated exercise evidence while keeping owner signoff pending.
Tests: pnpm verify:phase-boundary; pnpm ci:check; APP_URL=http://localhost:3100 pnpm smoke:stripe:webhook; pnpm exec convex run billing:getUsageSummary "{\"accountId\":\"j5746ef9edrcmn7mase0qcm0t1822tb7\"}" --typecheck disable --codegen disable
Evidence: Added docs/ops/incident-runbook-exercise-2026-03-04.md with artifact paths under /tmp/stage1_item3_incident_runbook_exercise_20260304; updated docs/ops/stage-1-evidence.md Item 3 artifacts; updated docs/dev-phase-ops-hardening.md status tracker and README links.
…ments Why: Stage 1 scope item 3 requires clear incident runbook rehearsal evidence and explicit required checks aligned with phase policy. Tests: pnpm verify:phase-boundary; pnpm ci:check Evidence: Updated docs/ops/incident-runbook-exercise-2026-03-04.md (walkthrough status matrix) and docs/ops/stage-1-evidence.md (item 3 required tests); artifacts under /tmp/stage1_item3_incident_runbook_exercise_20260304/*
Why: Advance Stage 1 Item 4 by adding a single production deploy/environment checklist (web + Convex + notification worker) and rollback rehearsal evidence artifacts. Tests: pnpm verify:phase-boundary; pnpm ci:check; pnpm sync:web:env; APP_URL=http://localhost:3100 pnpm smoke:stripe:webhook; curl -sS http://localhost:3100/api/health/orchestration; pnpm exec convex run devSeed:getFirstAccountId --typecheck disable --codegen disable Evidence: Added docs/ops/production-deploy-checklist.md and docs/ops/deploy-checklist-dry-run-2026-03-04.md with artifact links under /tmp/stage1_item4_deploy_dryrun_20260304_v3; documented optional local convex-dev schema-drift finding at /tmp/stage1_item4_deploy_dryrun_20260304/convex-dev.log; updated docs/ops/stage-1-evidence.md Item 4 and docs/dev-phase-ops-hardening.md status tracker.
Why: Stage 1 scope item 4 requires a repeatable deploy/environment checklist validation flow with recorded dry-run evidence and rollback-ready artifacts. Tests: ARTIFACT_DIR=/tmp/stage1_item4_deploy_checklist_verify_20260304 APP_URL=http://localhost:3100 VERIFY_CONVEX=0 pnpm verify:deploy:checklist; pnpm verify:phase-boundary; pnpm ci:check Evidence: Updated docs/ops/production-deploy-checklist.md, docs/ops/deploy-checklist-dry-run-2026-03-04.md, docs/ops/stage-1-evidence.md, docs/dev-phase-ops-hardening.md; artifacts at /tmp/stage1_item4_deploy_checklist_verify_20260304/* and /tmp/stage1_item4_deploy_dryrun_20260304_v3/*
Why: Stage 1 item 4 deploy checklist validation should fail fast on unhealthy runtime endpoints and avoid non-portable tooling dependencies. Tests: ARTIFACT_DIR=/tmp/stage1_item4_deploy_checklist_verify_20260304_hardened APP_URL=http://localhost:3100 VERIFY_CONVEX=0 pnpm verify:deploy:checklist; pnpm verify:phase-boundary; pnpm ci:check Evidence: Runtime verification artifacts at /tmp/stage1_item4_deploy_checklist_verify_20260304_hardened/*; script update in scripts/verify-deploy-checklist.sh
Why: Stage 1 item 4 requires repeatable rollback rehearsal evidence; manual restart steps were brittle and used broad process matching. Tests: ARTIFACT_DIR=/tmp/stage1_item4_deploy_rehearsal_20260304_v2 APP_URL=http://localhost:3100 pnpm rehearse:deploy:rollback; pnpm verify:phase-boundary; pnpm ci:check Evidence: Updated docs/ops/production-deploy-checklist.md, docs/ops/deploy-checklist-dry-run-2026-03-04.md, docs/ops/stage-1-evidence.md; artifacts at /tmp/stage1_item4_deploy_rehearsal_20260304_v2/* and /tmp/stage1_item4_deploy_checklist_verify_20260304/*
Why: Stage 1 item 4 rollback rehearsal evidence should be deterministic; reusing non-empty artifact directories can mix runs and weaken deploy evidence quality. Tests: ARTIFACT_DIR=/tmp/stage1_item4_deploy_rehearsal_20260304_v2 APP_URL=http://localhost:3100 pnpm rehearse:deploy:rollback (expected fail); ARTIFACT_DIR=/tmp/stage1_item4_deploy_rehearsal_20260304_v3 APP_URL=http://localhost:3100 pnpm rehearse:deploy:rollback; pnpm verify:phase-boundary; pnpm ci:check Evidence: Updated docs/ops/deploy-checklist-dry-run-2026-03-04.md and scripts/rehearse-deploy-rollback.sh; artifacts at /tmp/stage1_item4_deploy_rehearsal_20260304_v3/* and guard-fail logs /tmp/stage1_item4_deploy_rehearsal_guard_fail.log + .exit
Why: Strengthen Stage 1 Item 4 evidence by recording successful local verification and rollback rehearsal command runs for the deploy checklist workflow. Tests: ARTIFACT_DIR=/tmp/stage1_item4_verify_local APP_URL=http://localhost:3100 VERIFY_RUNTIME=0 VERIFY_CONVEX=0 pnpm verify:deploy:checklist; ARTIFACT_DIR=/tmp/stage1_item4_rehearse_local APP_URL=http://localhost:3100 pnpm rehearse:deploy:rollback; pnpm verify:phase-boundary; pnpm ci:check Evidence: Updated docs/dev-phase-ops-hardening.md Item 4 tracker status and docs/ops/stage-1-evidence.md Item 4 artifacts/notes with /tmp/stage1_item4_verify_local and /tmp/stage1_item4_rehearse_local outputs.
Why: Fix Stage 1 observability gate regression where Convex dev failed schema validation for existing notificationEvents.eventType=webhook_processing_failed data. Tests: pnpm exec convex dev --once --typecheck disable --tail-logs disable; pnpm verify:phase-boundary; pnpm ci:check Evidence: Convex dev once completed successfully after schema update; notification schema/worker/docs updated in convex/schema.ts, convex/notifications.ts, apps/worker/src/notificationWorker.ts, docs/notification-worker.md.
…ollow-up Why: Keep Stage 1 evidence accurate after resolving the Convex schema mismatch for notificationEvents.eventType=webhook_processing_failed. Tests: pnpm exec convex dev --once --typecheck disable --tail-logs disable; pnpm verify:phase-boundary; pnpm ci:check Evidence: Updated docs/ops/stage-1-evidence.md Item 2 artifacts/notes and docs/ops/deploy-checklist-dry-run-2026-03-04.md Convex follow-up section with /tmp/stage1_item2_convex_dev_once_20260304.log.
|
Added follow-up evidence for the resolved Convex schema mismatch ().\n\n- Updated Item 2 row with command + artifact: \n- Updated to mark the prior risk as resolved with revalidation evidence\n- Re-ran required gates on this branch:
Phase boundary check passed for active stage stage-1 (docs/dev-phase-ops-hardening.md) and
Phase boundary check passed for active stage stage-1 (docs/dev-phase-ops-hardening.md)
Scope: 3 of 4 workspace projects
RUN v4.0.18 /Users/kevinlau/Documents/Dev/MyMarketPlaceGuard/tempFiles/comment-copilot/apps/web ✓ tests/reviews.cleanup.integration.test.ts (1 test) 3ms Test Files 10 passed (10)
RUN v4.0.18 /Users/kevinlau/Documents/Dev/MyMarketPlaceGuard/tempFiles/comment-copilot/apps/web ✓ tests/oauth.integration.test.ts (32 tests) 35ms Test Files 1 passed (1) JUNIT report written to /Users/kevinlau/Documents/Dev/MyMarketPlaceGuard/tempFiles/comment-copilot/apps/web/test-results/oauth.junit.xml
OAuth JUnit quality gate passed: tests=32, failures=0, errors=0, min=32
RUN v4.0.18 /Users/kevinlau/Documents/Dev/MyMarketPlaceGuard/tempFiles/comment-copilot/apps/web stdout | tests/webhooks.e2e.integration.test.ts > Webhook Ingestion E2E Integration > ingests instagram webhook and triggers workflow when signature is valid stdout | tests/webhooks.e2e.integration.test.ts > Webhook Ingestion E2E Integration > ingests tiktok webhook and triggers workflow when signature is valid stdout | tests/webhooks.e2e.integration.test.ts > Webhook Ingestion E2E Integration > does not start instagram workflow when ingest reports duplicate delivery stdout | tests/webhooks.e2e.integration.test.ts > Webhook Ingestion E2E Integration > does not start tiktok workflow when ingest reports duplicate delivery ✓ tests/webhooks.e2e.integration.test.ts (8 tests) 11ms Test Files 1 passed (1) JUNIT report written to /Users/kevinlau/Documents/Dev/MyMarketPlaceGuard/tempFiles/comment-copilot/apps/web/test-results/webhooks.e2e.junit.xml
OAuth JUnit quality gate passed: tests=8, failures=0, errors=0, min=4
RUN v4.0.18 /Users/kevinlau/Documents/Dev/MyMarketPlaceGuard/tempFiles/comment-copilot/apps/web ✓ tests/inbox-send-telemetry-report.integration.test.ts (2 tests) 128ms Test Files 1 passed (1)
Isolation check passed. (both pass). |
|
Follow-up (clean summary):
|
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: efd29147f1
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Why: Close the remaining Item 2 evidence gap by recording live Stripe webhook observability output and alert-routing artifacts.
Tests: APP_URL=http://localhost:3100 pnpm smoke:stripe:webhook; node scripts/report-webhook-latency.mjs /tmp/stage1_item2_stripe_observability_20260304/dev-web.log; pnpm verify:phase-boundary; pnpm ci:check
Evidence: Updated docs/ops/stage-1-evidence.md Item 2 row with /tmp/stage1_item2_stripe_observability_20260304/{dev-web.log,stripe-smoke.log,webhook-latency-report.txt}.
|
Added Item 2 live Stripe-route observability evidence to this PR. What was added:
Verification from these artifacts:
Required gates re-run and passing:
|
Why: Owner signoff approved for Stage 1 items 2-4; close Stage 1 boundary/evidence trackers with Done/PASS/Approved states. Tests: pnpm verify:phase-boundary; pnpm ci:check Evidence: Updated docs/dev-phase-ops-hardening.md status tracker and docs/ops/stage-1-evidence.md metadata/item rows to Stage Status Done, Exit Gate Approved Yes, Overall Signoff Approved.
|
Owner signoff applied and Stage 1 is now marked closed in this PR. Updates pushed:
Validation rerun after these updates:
This PR should now be merge-ready for Stage 1 completion. |
1 participant
Summary
This PR consolidates Stage 1 implementation/evidence progress across Items 2-4 and fixes the Convex schema gate regression caused by persisted
notificationEvents.eventType="webhook_processing_failed"records.Why
Scope Included
pnpm verify:deploy:checklistpnpm rehearse:deploy:rollbackwebhook_processing_failed.Validation / Required Checks
Executed on this branch and passing:
pnpm verify:phase-boundarypnpm ci:checkpnpm exec convex dev --once --typecheck disable --tail-logs disableARTIFACT_DIR=/tmp/stage1_item4_verify_local APP_URL=http://localhost:3100 VERIFY_RUNTIME=0 VERIFY_CONVEX=0 pnpm verify:deploy:checklistARTIFACT_DIR=/tmp/stage1_item4_rehearse_local APP_URL=http://localhost:3100 pnpm rehearse:deploy:rollbackEvidence Updated
docs/dev-phase-ops-hardening.mddocs/ops/stage-1-evidence.mddocs/ops/incident-runbook-exercise-2026-03-04.mddocs/ops/deploy-checklist-dry-run-2026-03-04.mdNotes
In Progress/PENDINGwhere owner signoff is still pending.