Skip to content

feat: closed-loop engagement tuning (Phase I)#33

Closed
psdjungpulzze wants to merge 2 commits into
feat/peter-phase-h-completionfrom
feat/peter-phase-i-engagement
Closed

feat: closed-loop engagement tuning (Phase I)#33
psdjungpulzze wants to merge 2 commits into
feat/peter-phase-h-completionfrom
feat/peter-phase-i-engagement

Conversation

@psdjungpulzze
Copy link
Copy Markdown
Contributor

@psdjungpulzze psdjungpulzze commented May 6, 2026

Phase I — Engagement / Effectiveness Loop

The last of the six loop archetypes. Adds ENGAGE signal emission to notification mark-read and a weekly tune-engagement job that aggregates per-tenant engagement scores into `loops_calibrations`.

Depends on: #32 (Phase H — completion). Branched off Phase H.

What's in this PR

  • `src/app/api/v1/notifications/[id]/read/route.ts` — emits ENGAGE on the first read (`wasUnread` guard) so we count actual engagement rather than every revisit. Notification has no `organizationId` field, so we attribute via the user's primary OrgMembership (oldest by `joinedAt`). Multi-org users get best-effort attribution.
  • `src/worker/jobs/tune-engagement.ts` (new) — pure derivation `computeEngagementScale` (count/day) + per-tenant orchestration. Aggregates ENGAGE by feature, writes `engagement-rate:` calibrations to `loops_calibrations`. Skips features with no signals; updates existing rows in place.
  • `src/worker/index.ts` — registers the job; cron `30 5 * * 1` (Monday 05:30, after threshold tuning at 05:00).

Cold-start safety

  • `LOOPS_ENABLED !== "true"` → emit + job both no-op
  • User has no org membership → no emission (best-effort attribution)
  • No ENGAGE signals → no calibrations written
  • Read-site consumers (callers of `getCalibration("engagement-rate:")`) get null on cold start

Test plan

  • 12 new tests: 6 math (rate normalization, defensive zero/negative handling, rounding) + 6 orchestration (LOOPS_ENABLED gate, multi-tenant iteration, per-tenant fault isolation, feature aggregation, 30-day lookback, update-in-place vs create)
  • `npm run typecheck` clean
  • `npm test` — 456 passing vs Phase H baseline 444 (+12); same 8 pre-existing failures unchanged

Loops architecture status after this PR

Archetype Status Wired in
Suggestion → Acceptance full loop B + C + D
Prediction → Outcome Calibration full loop (consumer pending) G
Vocabulary / Mapping client ready (UKB extension pending) F
Threshold / Trigger full loop (consumer pending) B + H
Ranking / Routing full loop E
Engagement / Effectiveness full loop (consumer pending) B + I (this PR)

All six archetypes have producer + storage + aggregation/learning jobs wired. Phase J wires the remaining production consumers (cost forecast → `budget.ts`, threshold → auto-flag severity, engagement → digest) so every loop steers behavior end-to-end.

Wires Archetype 6 (Engagement / Effectiveness) — the last of the six
loop archetypes. Adds ENGAGE signal emission to the notification mark-
read endpoint and a weekly tune-engagement job that aggregates ENGAGE
signals per (tenant × feature) into per-tenant engagement-rate
calibrations.

- src/app/api/v1/notifications/[id]/read/route.ts — emits ENGAGE on
  the *first* read (wasUnread guard), so we count actual engagement
  rather than every revisit. Notification has no organizationId field,
  so we attribute via the user's primary OrgMembership (oldest by
  joinedAt). Multi-org users get best-effort attribution.
- src/worker/jobs/tune-engagement.ts (new) — pure derivation
  computeEngagementScale (count/day) plus per-tenant orchestration.
  Aggregates ENGAGE by feature, writes engagement-rate:<feature>
  calibrations to loops_calibrations. Skips features with no signals,
  updates existing rows in place.
- src/worker/index.ts — registers the job in maintenanceProcessor;
  cron 30 5 * * 1 (Monday 05:30, after threshold tuning at 05:00).
- src/worker/jobs/tune-engagement.test.ts (new) — 12 unit tests:
  pure math (rate normalization, defensive zero/negative handling,
  rounding) and orchestration (LOOPS_ENABLED gate, multi-tenant
  iteration, per-tenant fault isolation, feature aggregation,
  30-day lookback, update-in-place vs create).

Cold-start safety preserved end-to-end:
  - LOOPS_ENABLED off → emit + job both no-op
  - User has no org membership → no emission (best-effort attribution)
  - No ENGAGE signals → no calibrations written
  - Read-site consumers get null on cold start via the standard
    getCalibration helper

Closes the closed-loop architecture: every archetype now has a full
working loop. Producers + consumers + calibration/threshold/engagement
jobs all wired.
@psdjungpulzze psdjungpulzze marked this pull request as ready for review May 6, 2026 08:27
@psdjungpulzze psdjungpulzze deleted the branch feat/peter-phase-h-completion May 6, 2026 08:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant