Skip to content

feat(newsletters): writer/owner access + lfx-v2-email-service integration#804

Merged
niravpatel27 merged 17 commits into
mainfrom
feat/newsletters-combined
May 28, 2026
Merged

feat(newsletters): writer/owner access + lfx-v2-email-service integration#804
niravpatel27 merged 17 commits into
mainfrom
feat/newsletters-combined

Conversation

@niravpatel27
Copy link
Copy Markdown
Contributor

Summary

Combines the two parallel newsletter PRs into one shippable unit, plus the recipient + AI-gen fixes uncovered during dev testing.

  • Writer/owner access — Newsletters nav + route guard + Express auth opens up to writers/owners of the active foundation/project (was ED-only).
  • lfx-v2-email-service integration — Send + analytics now go through lfx-v2-email-service via NATS. One send_email per recipient sharing a UUID group_id; analytics derived from per-recipient EmailRecipientRecord[] (totals + daily-opens chart from opened_at_list).
  • Recipient resolution in Express — fixes the FGA filter that returned zero recipients when the Go newsletter-service called query-service with its Heimdall-minted JWT. Express now resolves committee members directly using the user's Auth0 bearer (same path the committee tab already uses).
  • AI /generate open to writers — was ED-only; writers need it to compose.

Dependencies (already deployed in dev)

Supersedes

Test plan

  • Lint / build / format / license / post-commit reviewer pair clean across all commits on both source branches
  • Smoke-tested in dev: writer can navigate to newsletters, compose, generate (AI), preview recipients (4 members resolved correctly), send test email (delivered via SES)

Switches newsletter email delivery and engagement analytics from the
Go newsletter-service's (stubbed) send path over to lfx-v2-email-service
via NATS.

Send flow:
  1. Resolve recipients from selected committees.
  2. Mint a UUID group_id in Express (one per newsletter dispatch).
  3. Fan out one send_email NATS call per recipient (5-wide concurrency
     limit) on lfx.email-service.send_email, all sharing that group_id.
  4. PATCH the Go newsletter-service: POST /newsletters/drafts/{id}/send
     with {groupId} body and If-Match header to flip status=sent and
     persist the group_id on the row.

Analytics flow:
  1. Fetch the newsletter to read its persisted groupId.
  2. Query lfx.email-service.get_email_status with {group_id}.
  3. Aggregate per-recipient EmailRecipientRecord[] into the existing
     NewsletterAnalytics shape — totals + daily-opens series derived
     from opened_at timestamps. Frontend is unchanged.

Edge cases handled before any email goes out:
  - Already-sent newsletters are rejected (prevents duplicate-send on
    double-click; the Go service's status='draft' check would catch it
    too, but only after emails were already dispatched).
  - Empty resolved recipient list is rejected with a clear validation
    error.
  - Pre-deploy sent newsletters with no groupId fall back to an empty
    analytics payload so the analytics page still renders.

Test sends bypass group_id (the email-service auto-generates one) so
they don't pollute newsletter analytics.

Files:
  - shared/src/enums/nats.enum.ts: add 3 email-service NATS subjects.
  - shared/src/interfaces/email-service.interface.ts: new — request/
    response types for the 3 subjects.
  - shared/src/interfaces/newsletter.interface.ts: add groupId? to
    Newsletter and groupId to NewsletterSendResult.
  - server/services/email-service.client.ts: new — NATS client mirroring
    impersonation.service.ts (codec/JSON, MicroserviceError on timeout
    and on the email-service {error} envelope).
  - server/services/newsletter-service.client.ts: drop the old send /
    sendDraft / testSend / getAnalytics methods; add markSent which
    PATCHes the Go service with {groupId} + If-Match.
  - server/controllers/newsletter.controller.ts: orchestrate per-
    recipient fan-out + analytics aggregation. Bounded-concurrency
    worker loop; pre-dispatch validations; analytics fallback for
    pre-deploy rows.

Depends on lfx-v2-newsletter-service PR #9 (group_id storage). Without
that PR deployed, markSent will succeed superficially but the group_id
won't be persisted, and the analytics page will fall through to the
empty-analytics path.

Signed-off-by: Nirav Patel <npatel@linuxfoundation.org>
Extracts the send + analytics business logic from the controller into a
new NewsletterService, and hardens several edge cases surfaced by the
post-commit review pair.

Code-reviewer findings addressed:
- Critical: business logic in controller. The five private methods
  (dispatchNewsletter, fanOutEmails, buildEmptyAnalytics,
  buildAnalyticsFromRecords, aggregateDailyOpens) move to a new
  services/newsletter.service.ts so the three-file pattern is honored.
  The controller is back to validation + logging lifecycle + delegation.
- Critical (partial): duplicate-send risk on markSent failure. Adds a
  3-attempt exponential-backoff retry around the markSent call (only
  retries 5xx; 4xx errors surface immediately). This rides out the
  transient network / upstream 5xx case. Full operator-retry idempotency
  still requires the Go service to support persisting a group_id on the
  draft BEFORE the fan-out — tracked as a follow-up; the comment in
  dispatchNewsletter documents the gap.
- Important: getStatusByGroup 502 on empty/missing records would surface
  as a failed analytics page. getAnalytics now catches the
  EMAIL_SERVICE_ERROR MicroserviceError and falls through to the empty
  analytics payload, with a warning log carrying the underlying message.
- Important: totalOpens/uniqueOpens vs dailyOpens used different
  predicates. Both now use `r.opened && r.opened_at` so the headline
  count always sums to the chart buckets.
- Important: the email-service `{error}` envelope check was too broad
  (`'error' in parsed`) — a successful record with a legitimate `error`
  field would be misclassified. Tightened to require a single-key object
  with a string `error` field.
- Important: logger.error inside dispatchNewsletter ran from the
  controller catch chain. Now using logger.warning per attempt in the
  service; the central apiErrorHandler logs the final error.
- Important: `payload as CreateNewsletterDraftRequest` cast in send().
  Construct the create-draft body explicitly from validated fields so a
  future divergence between NewsletterSendPayload and
  CreateNewsletterDraftRequest surfaces as a type error.

Learnings-reviewer findings addressed:
- Important: recipient email leaking into structured logs. Dropped `to`
  from the email-service debug-send log and from the test-send success
  log. group_id remains as the correlation key (no PII).

Files:
- services/newsletter.service.ts: new — owns dispatchNewsletter,
  getAnalytics, plus markSentWithRetry / isRetryable / sleep,
  fanOutEmails, buildEmptyAnalytics, buildAnalyticsFromRecords,
  aggregateDailyOpens.
- services/email-service.client.ts: tightened envelope detection
  (isErrorEnvelope helper); dropped recipient email from debug log.
- controllers/newsletter.controller.ts: delegates send / sendDraft /
  getAnalytics to NewsletterService. testSend success log no longer
  records the recipient email. send() constructs
  CreateNewsletterDraftRequest explicitly.

Signed-off-by: Nirav Patel <npatel@linuxfoundation.org>
Adapts to lfx-v2-email-service PR #8 which replaces the single
`opened_at` timestamp on EmailRecipientRecord with `opened_at_list`
(per-event array, deduped by SNS MessageId) plus `open_count` and
`last_opened_at`, and adds `unique_opened` to the engagement response.

API contract changes (consumed):
- EmailRecipientRecord:
    - opened_at removed
    - opened_at_list: OpenEvent[] added (one entry per unique open
      event, keyed server-side by SNS MessageId for dedup)
    - open_count: number added (== opened_at_list.length)
    - last_opened_at?: string added (most-recent open, advances forward)
- EmailServiceEngagementResponse:
    - opened semantic now sums open_count across recipients (total opens
      including repeats)
    - unique_opened: number added (distinct recipients with >=1 open)

Analytics aggregation switched to use the new fields:
- totalOpens = sum(record.open_count). Was hardcoded == uniqueOpens
  because the old single-opened_at schema couldn't distinguish them.
- uniqueOpens = count(record.opened === true). Unchanged semantically;
  still the headline "people who opened" number for the open-rate.
- openRate = uniqueOpens / delivered (reach, not frequency).
- dailyOpens chart now buckets every event in opened_at_list, with
  per-day `opens` (event count) and `uniqueOpens` (distinct email_ids
  that opened that day). Previously the chart only had one bucket per
  recipient because we only had one timestamp per record.
- lastEventAt uses record.last_opened_at (newer fallback) instead of
  the removed opened_at.
- Graceful degradation against pre-deploy email-service responses:
  open_count ?? 0 reads zero (so totalOpens is 0 until the new schema
  lands), opened_at_list ?? [] makes the chart empty. uniqueOpens still
  comes through correctly from record.opened.

Post-commit reviewer findings on bef38aa rolled in:
- isRetryable now treats raw transport errors (DNS, ECONNRESET, "fetch
  failed" — anything MicroserviceProxyService re-throws unwrapped) as
  retryable. Previously only 5xx MicroserviceError retried; transport
  blips fell through to "giving up" on the first attempt, contradicting
  the commit-message intent. 4xx stays non-retryable.
- markSentWithRetry warnings now use `err: error` instead of
  `error: error.message` so Pino's err serializer preserves the stack
  trace and any custom properties. Same swap in getAnalytics's
  email-service-error fallback.
- Final-attempt warning message keeps the operator-recovery hint
  ("emails delivered, group_id captured in logs for manual recovery")
  that SREs grep for. Intermediate attempts stay brief.
- Renamed the misleading `retryable` log field to two flags:
  `error_retryable` (intrinsic property of the error) and `will_retry`
  (loop-state — false on the final attempt even for retryable errors).

Note on lfx-v2-email-service PR #8: until that PR is deployed,
EmailRecipientRecord responses still use the pre-change shape (no
opened_at_list / open_count). The aggregation falls back to empty
dailyOpens and zero totalOpens but still renders the rest of the
analytics correctly. Once #8 deploys, the chart populates.

Signed-off-by: Nirav Patel <npatel@linuxfoundation.org>
…empty

Upstream lfx-v2-email-service serializes EmailRecipientRecord with
`json:"opened_at_list,omitempty"`, so the field is absent from the JSON
whenever a recipient has never opened (the common case until SES open
events arrive). Declaring it as required `OpenEvent[]` on the TS side
gave consumers a false guarantee that the field is always present and
could be dereferenced directly.

NewsletterService.aggregateDailyOpens already uses `?? []` defensively,
so this is purely a type-correctness fix to keep the TS contract in
sync with the upstream Go contract.

Found by the post-commit code-reviewer subagent on 1876ee0.

Signed-off-by: Nirav Patel <npatel@linuxfoundation.org>
`open_count === opened_at_list?.length ?? 0` parses as
`(open_count === opened_at_list?.length) ?? 0` in JS (=== binds tighter
than ??), which evaluates to `false` for the absent-array case rather
than the documented invariant. The parens make the JSDoc example match
the intent: when the field is absent, treat its length as zero, then
compare to open_count.

Found by the post-commit code-reviewer subagent on 468e139.

Signed-off-by: Nirav Patel <npatel@linuxfoundation.org>
The Newsletters entry in the left navigation was gated solely on the
Executive Director persona. Writers and owners of a foundation/project
need to see and use it too — they're the persona running the actual
communications work in many projects.

Changes:

- main-layout.component.ts: split the foundation-lens ED-only block so
  the Communications section (Newsletters) is gated on a new
  `canSeeNewsletters` computed signal (ED OR ProjectContextService
  .canWrite() against the active foundation/project context). Metrics
  stays ED-only. Project-lens visibility uses the same signal in place
  of the previous ED-only ternary.

- guards/newsletter-access.guard.ts (new): replaces executiveDirector-
  Guard on the four newsletters routes. Fast-paths ED persona
  (synchronous, SSR-safe); falls through to ProjectService.getProject
  + project.writer check against the active context's slug. Falls
  back to /foundation/overview on denial to match the prior redirect
  behavior.

- newsletters.routes.ts: switches the four canActivate arrays from
  executiveDirectorGuard to newsletterAccessGuard.

`canWrite()` is keyed off the active lens's selected foundation/
project and reads `project.writer`, which the backend's FGA check
sets true for both explicit writer grants and owner-equivalent
roles — so this single signal covers "writer or owner".

Signed-off-by: Nirav Patel <npatel@linuxfoundation.org>
The post-commit reviewer pair caught that the previous commit only
swapped the child route guards in newsletters.routes.ts. The parent
routes in app.routes.ts (foundation/newsletters, project/newsletters,
and the bare /newsletters lens-redirect path) still carried
executiveDirectorGuard, so Angular's parent-before-child evaluation
order rejected non-ED users at the parent before the new
newsletterAccessGuard ever ran. Net effect: writers/owners still got
bounced to /foundation/overview — feature shipped dead.

Changes in this commit:

- app.routes.ts: replace executiveDirectorGuard with
  newsletterAccessGuard on the three newsletter parent routes
  (lines 209, 258, 294). Also reorder the guards so
  projectQueryParamGuard / lensRedirectGuard run BEFORE
  newsletterAccessGuard — that way the active context is seeded
  from the URL before the access check reads it. The two ED-only
  routes (health-metrics, marketing-impact) keep
  executiveDirectorGuard unchanged.

- newsletter-access.guard.ts: resolve the project slug from the
  route's ?project= query param first, falling back to the active
  context only when the URL doesn't carry one (e.g. the /newsletters
  lens-redirect path). Makes the guard robust against deep-link /
  hard-reload races where the active context hasn't synced yet.
  Also surface lookup errors via console.error before the redirect
  so a 5xx / transport hiccup is triageable in Datadog RUM instead
  of looking identical to a legitimate deny.

- main-layout.component.ts: extract `canSeeNewsletters` to a private
  initializer function per `.claude/rules/component-organization.md`
  §3 — complex computed signals route through `init*` methods
  grouped at the bottom of the class.

Signed-off-by: Nirav Patel <npatel@linuxfoundation.org>
Two fixes, both surfaced by walking through the writer-side test path
that the post-commit reviewers flagged:

1) routes/newsletters.route.ts — drop the requireExecutiveDirector
   middleware that was blanket-applied to every /api/newsletters/*
   endpoint. That middleware caused a 403 the moment a writer-or-owner
   user hit the list endpoint, because it gated on the ED persona only.
   Authorization is enforced authoritatively downstream by lfx-v2-
   newsletter-service (FGA via the forwarded user bearer token), and
   the corresponding frontend route guards prevent unauthorized users
   from reaching these endpoints in the first place. /generate, which
   doesn't proxy to the Go service, stays behind the global auth
   middleware + /api rate limit.

2) app.routes.ts — swap the canActivate order on the two parent
   newsletter routes from [projectQueryParamGuard, newsletterAccessGuard]
   to [newsletterAccessGuard, projectQueryParamGuard]. The previous
   order let projectQueryParamGuard resolve `?project=<slug>` and mutate
   active foundation/project context BEFORE the access check ran, so
   non-permitted users still leaked a context-set side effect even
   though the navigation was eventually redirected. Since the access
   guard now reads the slug directly from the query param (commit
   fcec67e), it doesn't need the context to be seeded first, so the
   swap is safe.

Found by the post-commit learnings reviewer on fcec67e against
`server-request-handling/guard-runs-before-its-prerequisite` (KB
empirical citation: PR #701).

Signed-off-by: Nirav Patel <npatel@linuxfoundation.org>
The previous commit (bca114d) dropped requireExecutiveDirector from the
entire /api/newsletters/* router. That was correct for the endpoints
that proxy to lfx-v2-newsletter-service — the Go service enforces FGA
authoritatively. But /generate doesn't proxy: it calls AiService
(LiteLLM) directly, with no downstream authz to fall back on. Removing
the gate would let any authenticated user POST to /generate and burn
LiteLLM tokens with arbitrary prompts at no rate other than the global
/api budget.

Re-applies requireExecutiveDirector to the /generate route only (not
the whole router). Net effect:
  - Writers/owners keep access to list / draft CRUD / send / analytics
    (the broadening from this branch is preserved).
  - AI generation stays ED-only until /generate accepts a contextUid
    and the controller can do a writer/owner check against the active
    project, which is the right end state but requires a payload
    change in the shared interface + the generate drawer component.

Found by the post-commit code-reviewer subagent on bca114d.

Signed-off-by: Nirav Patel <npatel@linuxfoundation.org>
The Go newsletter-service was calling lfx-v2-query-service with the
Heimdall-minted JWT it received from its own ruleset — not the user's
original Auth0 token. Query-service's Heimdall couldn't validate that
JWT as OIDC, fell through to anonymous, and FGA returned zero rows
even for committees the user can see in the standard committee tab.

Switch recipient resolution to Express. Express already has the user's
Auth0 bearer and uses it via MicroserviceProxyService for every other
query-service call (including the existing committee-members fetch
that powers the committee tab), so FGA evaluates against the real
principal and returns the actual members.

Changes:
- NewsletterService.resolveRecipients(req, committeeUids): new method
  that fans out to CommitteeService.getCommitteeMembers per committee,
  dedupes by lowercased email, drops invalid addresses. Matches the
  Go service's prior filtering behavior so the email-service fan-out
  downstream doesn't need per-recipient validation.
- dispatchNewsletter switched from newsletterClient.getRecipients to
  the new resolveRecipients. /api/newsletters/recipient-count and
  /api/newsletters/recipients controllers do the same.
- NewsletterServiceClient: drop getRecipients / getRecipientCount —
  no longer used; the Go endpoints are still up for any future direct
  caller but lfx-v2-ui no longer routes through them.

Side benefit: newsletter-service no longer needs COMMITTEE_SERVICE_URL
for the recipient path. That env var stays for any direct callers of
the Go service but isn't a runtime dependency of this UI flow anymore.

Signed-off-by: Nirav Patel <npatel@linuxfoundation.org>
Writers compose newsletters in the UI now (the nav + route + send
authorization broadening), so blocking them from /generate defeats
the feature. The original ED gate on /generate was a cost-bounding
measure for an endpoint the downstream Go service doesn't FGA-check;
the global authMiddleware (must be a logged-in user) plus the /api
rate limit remain as protection. A tighter writer/owner check tied
to a project contextUid is still a worthwhile follow-up.

Signed-off-by: Nirav Patel <npatel@linuxfoundation.org>
Copilot AI review requested due to automatic review settings May 28, 2026 02:26
@niravpatel27 niravpatel27 requested a review from a team as a code owner May 28, 2026 02:26
@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented May 28, 2026

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: d733147a-9609-4ff0-b6cf-172b109d5ae6

📥 Commits

Reviewing files that changed from the base of the PR and between 5b69b70 and a3ba6e0.

📒 Files selected for processing (2)
  • apps/lfx-one/src/app/modules/newsletters/newsletter-list/newsletter-list.component.ts
  • apps/lfx-one/src/app/modules/newsletters/newsletter-manage/newsletter-manage.component.ts

Walkthrough

This PR restructures newsletter authorization and email operations. Frontend routes transition from executive-director-only gating to a new newsletterAccessGuard that checks both executive-director persona and project writer permissions. Backend reorganizes email dispatch by extracting newsletter orchestration into a new NewsletterService that coordinates with a new EmailServiceClient for NATS-based email sending. Newsletter domain models add groupId tracking for engagement analytics aggregation.

Changes

Newsletter Authorization and Email Service Integration

Layer / File(s) Summary
Frontend newsletter access guard
apps/lfx-one/src/app/shared/guards/newsletter-access.guard.ts
Implements newsletterAccessGuard that grants immediate access to executive-director persona, otherwise resolves the target project slug and checks writer permission via ProjectService.getProject() before allowing navigation to /foundation/overview on denial.
Frontend route guard integration and sidebar visibility
apps/lfx-one/src/app/app.routes.ts, apps/lfx-one/src/app/modules/newsletters/newsletters.routes.ts, apps/lfx-one/src/app/layouts/main-layout/main-layout.component.ts, apps/lfx-one/src/app/modules/newsletters/newsletter-manage/newsletter-manage.component.ts, apps/lfx-one/src/app/modules/newsletters/newsletter-list/newsletter-list.component.ts
Replaces executiveDirectorGuard with newsletterAccessGuard across newsletter routes (foundation, project, top-level). Sidebar "Communications → Newsletters" visibility updated to reactive canSeeNewsletters() signal that grants access to executive-director OR project writers; "Metrics" section remains executive-director-only. Adds UI warning and navigation behavior for partial send persistence failures and seeds the list tab from query params.
Email service interfaces and newsletter domain updates
packages/shared/src/interfaces/email-service.interface.ts, packages/shared/src/interfaces/newsletter.interface.ts, packages/shared/src/enums/nats.enum.ts, packages/shared/src/interfaces/index.ts
Defines email service NATS contracts: EmailServiceSendRequest/Response, EmailRecipientRecord, OpenEvent, engagement analytics, and error envelopes. Adds three NATS subjects for email operations. Extends Newsletter and NewsletterSendResult with groupId and markSentFailed for send tracking and analytics.
NewsletterService client surface
apps/lfx-one/src/server/services/newsletter-service.client.ts
Simplifies newsletter-service client surface and adds markSent() to POST /newsletters/drafts/${id}/send with groupId and If-Match header; removes several direct send/analytics helper methods.
NewsletterService orchestration
apps/lfx-one/src/server/services/newsletter.service.ts
Orchestrates complete newsletter operations: resolveRecipients() fetches committee members with lowercased email deduping and invalid-email filtering; dispatchNewsletter() validates status, resolves recipients, generates groupId, fan-outs email sends with bounded concurrency, retries sent-state persistence via markSent with exponential backoff on retryable errors; getAnalytics() retrieves email-service records and aggregates delivery counts, unique openers, daily open buckets (UTC), and lastEventAt timestamp; enrichListWithEngagement() overlays engagement on list rows.
Newsletter controller and backend routes refactoring
apps/lfx-one/src/server/controllers/newsletter.controller.ts, apps/lfx-one/src/server/routes/newsletters.route.ts, apps/lfx-one/src/server/services/newsletter-service.client.ts
Controller refactored to use NewsletterService and EmailServiceClient for recipient resolution, preview sends, dispatch, and analytics. testSend now dispatches preview emails via EmailServiceClient directly without recipient PII in logs. Server routes drop persona-based middleware, delegating authorization to frontend guards and downstream FGA.

Sequence Diagram(s)

sequenceDiagram
  participant Controller
  participant NewsletterService
  participant EmailServiceClient
  participant NewsletterClient
  Controller->>NewsletterService: dispatchNewsletter(newsletter)
  NewsletterService->>NewsletterService: resolveRecipients(committeeUids)
  NewsletterService->>EmailServiceClient: sendEmail(recipient payload) [fan-out]
  EmailServiceClient-->>NewsletterService: send responses (per-recipient)
  NewsletterService->>NewsletterClient: markSent(draftId, { groupId }, If-Match)
  NewsletterClient-->>NewsletterService: updated Newsletter (groupId persisted)
  NewsletterService-->>Controller: dispatch result (counts, failures, groupId)
  Controller->>NewsletterService: getAnalytics(newsletter)
  NewsletterService->>EmailServiceClient: getStatusByGroup(groupId)
  EmailServiceClient-->>NewsletterService: EmailRecipientRecord[]
  NewsletterService-->>Controller: aggregated analytics
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Possibly related PRs

Suggested labels

enhancement, deploy-preview

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly summarizes the two main changes: writer/owner access expansion and lfx-v2-email-service integration, making it specific and directly related to the changeset.
Description check ✅ Passed The description is well-detailed and directly related to the changeset, covering the main features, dependencies, test plan, and superseded PRs.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/newsletters-combined

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Consolidates two parallel newsletter PRs (writer/owner access + lfx-v2-email-service integration) and adds fixes uncovered during dev testing. Routes newsletter send/analytics through lfx-v2-email-service over NATS, moves recipient resolution into Express (to bypass the Heimdall-JWT FGA issue with the Go service), and broadens newsletter access from Executive Director-only to ED OR writer/owner of the active foundation/project.

Changes:

  • New EmailServiceClient (NATS) + NewsletterService orchestrator: per-recipient send_email fan-out with shared group_id, analytics aggregation from EmailRecipientRecord[], mark-sent with retry; controller and Go-service client trimmed accordingly.
  • Recipient preview/count/test-send/analytics moved off the Go newsletter-service onto the BFF, using CommitteeService.getCommitteeMembers for FGA-correct recipient resolution.
  • Newsletter access broadened beyond ED: new newsletterAccessGuard, sidebar canSeeNewsletters signal driven by ProjectContextService.canWrite(), and requireExecutiveDirector removed from the Express route.

Reviewed changes

Copilot reviewed 13 out of 13 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
packages/shared/src/interfaces/newsletter.interface.ts Adds groupId to NewsletterSendResult and Newsletter.
packages/shared/src/interfaces/email-service.interface.ts New shared contracts for the email-service NATS subjects.
packages/shared/src/interfaces/index.ts Re-exports the new email-service interfaces.
packages/shared/src/enums/nats.enum.ts Adds three lfx.email-service.* NATS subjects.
apps/lfx-one/src/server/services/email-service.client.ts New NATS client wrapping send/status/engagement with MicroserviceError mapping.
apps/lfx-one/src/server/services/newsletter.service.ts New orchestration service: recipient resolution, fan-out, mark-sent retry, analytics aggregation.
apps/lfx-one/src/server/services/newsletter-service.client.ts Drops send/recipients/analytics proxy methods; adds markSent posting {groupId} with If-Match.
apps/lfx-one/src/server/controllers/newsletter.controller.ts Wires controller to NewsletterService/EmailServiceClient; analytics now fetches draft then aggregates.
apps/lfx-one/src/server/routes/newsletters.route.ts Removes ED-only middleware; documents new auth model and that /generate lacks an FGA hook.
apps/lfx-one/src/app/shared/guards/newsletter-access.guard.ts New guard: ED fast-path, otherwise project.writer === true for the resolved slug.
apps/lfx-one/src/app/modules/newsletters/newsletters.routes.ts Swaps executiveDirectorGuard for newsletterAccessGuard on child routes.
apps/lfx-one/src/app/app.routes.ts Uses newsletterAccessGuard for foundation/project/flat newsletter routes.
apps/lfx-one/src/app/layouts/main-layout/main-layout.component.ts Newsletter nav visibility now driven by canSeeNewsletters (ED OR canWrite).

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread apps/lfx-one/src/app/shared/guards/newsletter-access.guard.ts Outdated
Comment thread apps/lfx-one/src/server/services/email-service.client.ts
Comment thread apps/lfx-one/src/app/app.routes.ts Outdated
Comment thread apps/lfx-one/src/server/controllers/newsletter.controller.ts
Comment thread apps/lfx-one/src/server/services/newsletter.service.ts
The newsletter list table read totalRecipients / uniqueOpens / openRate
straight from the Go newsletter-service row. Those fields are written
once at send time from the Go service's own resolveRecipients call,
which hits the same FGA-filtering issue as the recipient dropdown —
for writers, the persisted counts are zero even though Express
actually delivered the emails. Net effect: list table showed
"Recipients: 0, Open Rate: 0%" while the per-newsletter analytics
page (which reads from email-service) showed the real numbers.

Overlay the three engagement fields with live values from
lfx-v2-email-service's `get_email_engagement_analytics` (one NATS
call per sent newsletter with a groupId, fanned out in parallel).
Drafts and pre-integration sent rows pass through unchanged. If
email-service can't return engagement for a row, log a warning and
fall back to the Go-stored value rather than erroring the whole
list.

Signed-off-by: Nirav Patel <npatel@linuxfoundation.org>
Address review comments from copilot[bot]:

- newsletter-access.guard.ts: remove unnecessary `eslint-disable-next-line
  no-console` — `no-console: off` plus a no-restricted-syntax rule that
  explicitly allows `console.error/warn/info/trace` make the disable dead
  (per copilot[bot])
- app.routes.ts: fix guard-import ordering — newsletterAccessGuard now
  comes after lensRedirectGuard alphabetically (per copilot[bot])
- email-service.client.ts: relax `isErrorEnvelope` — drop the strict
  "exactly one key" check so the email-service can add correlation
  fields (request_id, code, etc.) to error envelopes without us
  misclassifying them. New check: requires `error: string` AND the
  absence of any documented success field (email_id, total_sent,
  delivered, opened, open_count, opened_at_list). Tighter than
  `'error' in parsed` (so a per-recipient error field doesn't read as
  a full-request failure), looser than length===1 (so new envelope
  fields don't break us) (per copilot[bot])
- newsletter.service.ts + newsletter.interface.ts + newsletter-manage.
  component.ts: add `markSentFailed` flag on NewsletterSendResult.
  When markSentWithRetry exhausts its retries after emails were
  delivered, return the result with markSentFailed=true instead of
  throwing — the controller returns 200 and the frontend shows a
  distinct "Sent — status update failed" warn toast asking the
  operator NOT to retry (since that would cause duplicate emails).
  The previous behavior surfaced a generic error toast that hid the
  emails-delivered state and invited duplicate-send retries (per
  copilot[bot])

Resolves 4 review threads; 1 thread (getDraft on sent newsletters)
flagged as false positive — the Go `repo.Get(ctx, id)` does no status
filter and serves both drafts and sent newsletters via the same path.

Resolves 5 review threads.

Signed-off-by: Nirav Patel <npatel@linuxfoundation.org>
Copilot AI review requested due to automatic review settings May 28, 2026 02:53
@niravpatel27
Copy link
Copy Markdown
Contributor Author

Review Feedback Addressed

Commit: 5b69b70bdd85cc87e4a05a17ac8f2ed06441837a

Changes Made

  • apps/lfx-one/src/app/shared/guards/newsletter-access.guard.ts: removed unnecessary eslint-disable-next-line no-console (no-console: off + no-restricted-syntax already allows console.error) (per copilot[bot])
  • apps/lfx-one/src/app/app.routes.ts: fixed guard-import alphabetical order (per copilot[bot])
  • apps/lfx-one/src/server/services/email-service.client.ts: relaxed isErrorEnvelope so error envelopes can carry correlation fields like request_id / code without misclassification; now checks for error: string AND absence of any documented success field (per copilot[bot])
  • packages/shared/src/interfaces/newsletter.interface.ts + apps/lfx-one/src/server/services/newsletter.service.ts + apps/lfx-one/src/app/modules/newsletters/newsletter-manage/newsletter-manage.component.ts: added markSentFailed flag on NewsletterSendResult. When markSentWithRetry exhausts after emails went out, the response is now 200 with markSentFailed: true (not a generic 5xx). Frontend shows a distinct warn toast: "Sent — status update failed. Don't retry — contact support to reconcile." — prevents the operator from triggering duplicate sends (per copilot[bot])

No Change Needed

  • apps/lfx-one/src/server/controllers/newsletter.controller.ts:398: getDraft for sent newsletters — verified Go's PostgresNewsletterRepo.Get has no status filter (WHERE n.id = ?), so the endpoint serves both drafts and sent newsletters. The endpoint name is misleading but the implementation is status-agnostic. Rename flagged as follow-up. (flagged by copilot[bot])

Threads Resolved

5 of 5 unresolved threads addressed in this iteration.

After a clean send, route back to the list with `?tab=sent` so the
operator immediately sees the row they just sent in the Sent tab
instead of having to switch tabs from Drafts (where it no longer
belongs).

- goToList() now accepts an optional `tab` arg and adds it as a
  query param.
- The send-success handler in newsletter-manage passes `'sent'`
  when the result fully succeeded. On `markSentFailed` (emails
  delivered but the Go-side status flip exhausted retries — the
  row is still a draft on the backend), pass `'draft'` instead so
  we don't mislead the operator about where to find the row.
- onCancel() and the load-error path stay on the default Drafts
  tab (no query param).
- Newsletter list constructor reads `?tab=` from the route and
  seeds `statusTab` accordingly; defaults to 'draft' when absent.

Signed-off-by: Nirav Patel <npatel@linuxfoundation.org>
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 15 out of 15 changed files in this pull request and generated no new comments.

Comment thread apps/lfx-one/src/app/shared/guards/newsletter-access.guard.ts
Comment thread apps/lfx-one/src/server/services/newsletter.service.ts
Comment thread apps/lfx-one/src/server/services/email-service.client.ts
Copy link
Copy Markdown
Contributor

@dealako dealako left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi Nirav! This is a solid integration that's clearly been thought through carefully — the FGA-bypass root cause analysis is correctly diagnosed and the fix (Express resolving recipients with the user's Auth0 bearer) is the right solution. The partial-success contract for markSentWithRetry is particularly well-handled. Below is my full independent pass plus reconciliation with Copilot and MRashad26's feedback.


PR Overview

Combines #802 (writer/owner access) and #803 (email-service integration) into one shippable unit. Extends newsletter access from ED-only to ED or writer/owner of the active foundation/project, routes email delivery through lfx-v2-email-service over NATS with a shared group_id, moves recipient resolution to Express to bypass the Heimdall-JWT FGA filter, and enriches the Sent-tab list with live engagement data from the email-service.

15 files changed, CI clean, DCO passed, CodeQL clean, MERGEABLE.


Independent Findings

apps/lfx-one/src/server/services/newsletter.service.ts:216 — [minor] enrichListWithEngagement unbounded NATS fan-out

const enrichments = await Promise.all(
  newsletters.map(async (n) => { ... this.emailServiceClient.getEngagement(...) ... })
);

fanOutEmails caps concurrency at 5 workers; enrichListWithEngagement uses a bare Promise.all over all sent newsletters with a groupId. As the Sent list grows (e.g., 30+ newsletters), this fires 30+ simultaneous NATS requests. The graceful fallback on error (lines 224–228) limits blast radius, and pagination limits initial load, but a concurrency cap here would match the rest of the service's throttling discipline.

apps/lfx-one/src/app/layouts/main-layout/main-layout.component.ts:372 — [nit] Stale "ED-only" comment

// Project-lens Communications section (ED-only); appended dynamically in sidebarItems().
private readonly projectCommunicationsSection: SidebarMenuItem = {

The comment predates the writer-access change. Should read "ED or writer/owner".

apps/lfx-one/src/app/modules/newsletters/newsletters.routes.ts:16–37 — [nit] Double guard invocation on child routes

newsletterAccessGuard runs on the parent routes in app.routes.ts (canActivate: [newsletterAccessGuard, projectQueryParamGuard]) and again on every child route (list, create, :id/edit, :id/analytics). In the non-ED path this fires projectService.getProject() twice per navigation. Removing newsletterAccessGuard from the child routes (parent already gates the subtree) would eliminate the duplicate HTTP call and the double writer-check.

apps/lfx-one/src/server/routes/newsletters.route.ts:34–39 — [minor] /generate no per-project writer scope — follow-up needs a JIRA

The inline comment ("A tighter writer/owner check tied to a project contextUid is a follow-up") is clear, but I don't see a linked JIRA ticket in the PR description or the commit messages. The global authMiddleware + API rate limit is a reasonable interim gate — any logged-in user can currently call /generate with any contextName — but the follow-up should be tracked. Please add a JIRA reference before merge.


AI Bot Reconciliation

Copilot left 5 inline comments (all addressed in 5b69b70b):

Copilot comment My assessment Status
eslint-disable-next-line no-console unnecessary Agree — no-console: off in eslint.config.js, console.error also allowed by no-restricted-syntax. Dead comment. ✅ Resolved — removed in 5b69b70b
isErrorEnvelope exact-key-count brittle Agree — a future request_id correlation field on error envelopes would silently misclassify. ✅ Resolved — relaxed to documented-success-field absence (email-service.client.ts:157)
Import ordering for newsletterAccessGuard Agree — alphabetical by filename is the established pattern. ✅ Resolved — moved after lensRedirectGuard in 5b69b70b
getDraft reused for analytics endpoint (potential 404 on sent newsletters) Checked the Go side with Nirav's explanation: PostgresNewsletterRepo.Get has no status filter, so GET /newsletters/drafts/{id} serves all statuses. The route name is a Go-side naming quirk, not a behaviour gap. ✅ No code change needed — Nirav's explanation is correct
markSentWithRetry exhaustion surfaces as an error, masking delivered sends Agree — the right fix is the partial-success contract, not re-throwing. ✅ Resolved — markSentFailed flag implemented, dispatchNewsletter catches and warns instead of re-throwing (newsletter.service.ts:131–161)

MRashad26 flagged multi-line JSDoc blocks in three files (newsletter-access.guard.ts:12, newsletter.service.ts:28, email-service.client.ts:13). I agree these violate CLAUDE.md's "one short line max" rule. That said, most of the method-level JSDoc documents genuinely non-obvious WHY — the FGA bypass rationale, the retry strategy and operator-recovery contract, and the backwards-compat fallback for the older schema. These would be a real loss if stripped to a single line. The class-level docstrings are the clearest violations and worth trimming; the method-level blocks on complex orchestration methods have earned their length. MRashad26 approved despite flagging — which I think is the right call.

CodeRabbit returned clean (no inline comments, status SUCCESS). No Cursor Bot comments detected.


Security Assessment (Phase 1 + 2)

All automated checks pass: no secrets, no injection patterns, no Math.random() for tokens (crypto.randomUUID() used correctly), no PII in logs (recipient email explicitly excluded from log metadata at email-service.client.ts:37).

Auth chain: authMiddleware at server.ts:187 gates the entire /api/* tree before the newsletter router mounts at line 222. Newsletter CRUD forwards the user's Auth0 bearer to the Go service for downstream FGA. Recipient resolution correctly uses the user bearer (not M2M). IDOR on GET /:id/analytics is protected by the Go service's per-user FGA on getDraft. group_id generated with crypto.randomUUID() — cryptographically sound.

One acknowledged gap: /generate has no per-project authz scope (see [minor] finding above). Not blocking given the interim rate-limit gate, but needs a tracked follow-up.


Revision Tracking

Commits after the initial Copilot pass:

  • 5b69b70b — addressed all 5 Copilot inline comments
  • 3fb0b403 — enriches list rows with live email-service engagement (Sent-tab totals fix)
  • cadf643a — drops ED gate on /generate so writers can use AI gen (correctly reversed after earlier over-restriction in cccd9514)
  • a3ba6e08 — navigates to Sent tab after successful send (UX improvement)

Summary

1 minor (follow-up JIRA for /generate), 1 minor (unbounded Promise.all in enrichListWithEngagement), 2 nits (stale comment, double guard). No blockers. Architecture is sound, error handling is thoughtful, auth layering is correct.

Decision: ✅ Approved with minor comments

@niravpatel27 niravpatel27 merged commit dbf5fc8 into main May 28, 2026
9 of 10 checks passed
@niravpatel27 niravpatel27 deleted the feat/newsletters-combined branch May 28, 2026 13:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants