Skip to content

feat: background template reloading p1 - baseline decomposition #2148

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 4 commits into from
Sep 12, 2025
Merged

feat: background template reloading p1 - baseline decomposition #2148

merged 4 commits into from
Sep 12, 2025

Conversation

cstockton
Copy link
Contributor

Summary
Currently every transactional email fetches its template inside transactions. This adds request latency, blocks database transactions, and causes user visible failures when the template service is slow or unavailable.

This pr will address these issues in 3 phases:

  1. Baseline decomposition (this PR)

    • Untangle the mailer into layered, single-purpose packages (mailmeclient, noopclient, validateclient, taskclient, templatemailer).
    • Simplify API construction with a single newMailer() factory inside the api package to composes the pipeline.
    • No behavioral changes, this is strictly restructuring to make follow ups easier.

  2. Template cache + interface refactor (upcoming)

    • Fix the race conditions in the template caching system.
    • Add methods for the background template reloader to refresh the cache.
    • Redefine the low-level mail client interface to only accept (to, subject, body, headers) while templatemailer owns all templating logic.
    • Introduce some configuration parameters for tuning.

  3. Introduce background template reloading

    • Add a worker at startup similar to the previous PR for background workers.
    • Periodically update the cache at a configurable interval.
    • Notify the template reloader to update when config reloads occur.
    • Remove blocking calls to the template cache unless a cache miss occurs.
    • Never drop a template from cache before having an updated entry.

feat: decompose the mailer package
f41e35d 
- changes the flow of object construction, not behavior
- API builds mailer as: noop|mailme -> validate -> tasks -> templatemailer
- remove MailerOptions & mailerClientFunc options
- makes it easier to form a mental model around the mailer package
- sets the stage for the upcoming template reloading change
@cstockton cstockton requested a review from a team as a code owner August 28, 2025 20:51
@coveralls
Copy link

coveralls commented Aug 28, 2025 •
edited
Loading

Pull Request Test Coverage Report for Build 17658835204

Details

  • 285 of 644 (44.25%) changed or added relevant lines in 10 files are covered.
  • No unchanged relevant lines lost coverage.
  • Overall coverage decreased (-0.3%) to 67.645%
Changes Missing Coverage Covered Lines Changed/Added Lines %
internal/api/mail.go 0 1 0.0%
internal/mailer/noopclient/noopclient.go 7 13 53.85%
internal/mailer/templatemailer/templatemailer.go 60 66 90.91%
internal/api/options.go 0 9 0.0%
internal/mailer/validateclient/validateclient.go 17 37 45.95%
internal/mailer/taskclient/taskclient.go 5 28 17.86%
internal/mailer/mailmeclient/mailmeclient.go 0 44 0.0%
internal/api/apiworker/apiworker.go 0 57 0.0%
internal/mailer/templatemailer/template.go 191 384 49.74%
Totals Coverage Status
Change from base Build 17648046512: -0.3%
Covered Lines: 12745
Relevant Lines: 18841
💛 - Coveralls

hf
hf approved these changes Aug 29, 2025
View reviewed changes
Copy link
Contributor

@hf hf left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good!

@cstockton cstockton changed the title feat: background template reloading feat: background template reloading p1 - baseline decomposition Aug 31, 2025
@cstockton cstockton mentioned this pull request Aug 31, 2025
Merged
cstockton and others added 2 commits September 11, 2025 15:15
@cstockton
feat: background template reloading p2 - mailer template cache & inte…
48d98ce 
…rface refactor (#2150)

**Summary**
This PR builds on
[auth#2148](#2148) to deliver phase
2, a robust template cache plus a simplified mail client interface.
Templates are now rendered once per type, cached with controlled
refresh, and then delivered via clients that only send pre-rendered
subject/body. This reduces latency and removes templating logic from
SMTP clients while preserving current behavior. It will also pave the
way for alternative Mailer implementations.

**Template cache + interface refactor**
* Replace `TemplateMailer` with `templatemailer.Mailer` that owns all
templating and a shared cache. Entries have a TTL and a shorter “recheck
interval”; on fetch/parse errors we serve the last good or fall back to
built-in defaults.
* Centralize default subjects/bodies and validate them at init. Typed
template keys are: `invite`, `confirmation, recovery`, `email_change`,
`magic_link`, `reauthentication`.
* Move headers logic into the mailer; keep `$messageType` substitution
and the legacy “reauthenticate” header name for compatibility.
* Change `mailer.Client.Mail` to `(ctx, to, subject, body, headers,
typ)` and update all clients accordingly (SMTP/Noop/Validate/Tasks).
* Persist a single mailer instance in `API`; add `WithMailer` for DI;
remove on-demand construction.

---------

Co-authored-by: Chris Stockton <chris.stockton@supabase.io>
@cstockton
Merge branch 'master' into cs/background-template-reloading
a6bb7bb
@cstockton cstockton merged commit 746c937 into master Sep 12, 2025
5 checks passed
@cstockton cstockton deleted the cs/background-template-reloading branch September 12, 2025 13:40
@github-actions github-actions bot mentioned this pull request Sep 12, 2025
Merged
cemalkilic pushed a commit that referenced this pull request Sep 23, 2025
@github-actions
chore(master): release 2.180.0 (#2151)
3511eb4 
🤖 I have created a release *beep* *boop*
---


##
[2.180.0](v2.179.0...v2.180.0)
(2025-09-23)


### Features

* add OAuth client type
([#2152](#2152))
([b118f1f](b118f1f))
* add phone to sms webhook payload
([#2160](#2160))
([d475ac1](d475ac1))
* background template reloading p1 - baseline decomposition
([#2148](#2148))
([746c937](746c937))
* config reloading with fsnotify, poller fallback, and signals
([#2161](#2161))
([c77d512](c77d512))
* enhance issuer URL validation in OAuth server metadata
([#2164](#2164))
([a9424d2](a9424d2))
* implement OAuth2 authorization endpoint
([#2107](#2107))
([5318552](5318552))
* **oauth2:** add `/oauth/token` endpoint
([#2159](#2159))
([a89a0b0](a89a0b0))
* **oauth2:** add admin endpoint to regenerate OAuth client secrets
([#2170](#2170))
([0bd1c28](0bd1c28))
* **oauth2:** return redirect_uri on GET authorization
([#2175](#2175))
([b0a0c3e](b0a0c3e))
* **oauth2:** use `id` field as the public client_id
([#2154](#2154))
([86b7de4](86b7de4))
* **openapi:** add OAuth 2.1 server endpoints and clarify OAuth modes
([#2165](#2165))
([1f804a2](1f804a2))
* password changed email notification
([#2176](#2176))
([fe0fd04](fe0fd04))
* support `transfer_sub` in apple id tokens
([#2162](#2162))
([8a71006](8a71006))


### Bug Fixes

* ensure request context exists in API db operations
([#2171](#2171))
([060a992](060a992))
* **makefile:** remove invalid @ symbol from shell commands
([#2168](#2168))
([e6afe45](e6afe45))
* **oauth2:** switch to Origin header for request validation
([#2174](#2174))
([42bc9ab](42bc9ab))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
issuedat pushed a commit that referenced this pull request Sep 30, 2025
@cstockton @issuedat
feat: background template reloading p1 - baseline decomposition (#2148)
73537ca 
**Summary**
Currently every transactional email fetches its template inside
transactions. This adds request latency, blocks database transactions,
and causes user visible failures when the template service is slow or
unavailable.

This pr will address these issues in 3 phases:

1. **Baseline decomposition (this PR)**

* Untangle the mailer into layered, single-purpose packages
(`mailmeclient`, `noopclient`, `validateclient`, `taskclient`,
`templatemailer`).
* Simplify API construction with a single `newMailer()` factory inside
the api package to composes the pipeline.
* No behavioral changes, this is strictly restructuring to make follow
ups easier.

2. **Template cache + interface refactor (upcoming)**

   * Fix the race conditions in the template caching system.
* Add methods for the background template reloader to refresh the cache.
* Redefine the low-level mail client interface to only accept `(to,
subject, body, headers)` while `templatemailer` owns all templating
logic.
   * Introduce some configuration parameters for tuning.

3. **Introduce background template reloading**

* Add a worker at startup similar to the previous PR for background
workers.
   * Periodically update the cache at a configurable interval.
   * Notify the template reloader to update when config reloads occur.
* Remove blocking calls to the template cache unless a cache miss
occurs.
   * Never drop a template from cache before having an updated entry.

---------

Co-authored-by: Chris Stockton <chris.stockton@supabase.io>
issuedat pushed a commit that referenced this pull request Sep 30, 2025
@github-actions @issuedat
chore(master): release 2.180.0 (#2151)
8d191ff 
🤖 I have created a release *beep* *boop*
---


##
[2.180.0](v2.179.0...v2.180.0)
(2025-09-23)


### Features

* add OAuth client type
([#2152](#2152))
([b118f1f](b118f1f))
* add phone to sms webhook payload
([#2160](#2160))
([d475ac1](d475ac1))
* background template reloading p1 - baseline decomposition
([#2148](#2148))
([746c937](746c937))
* config reloading with fsnotify, poller fallback, and signals
([#2161](#2161))
([c77d512](c77d512))
* enhance issuer URL validation in OAuth server metadata
([#2164](#2164))
([a9424d2](a9424d2))
* implement OAuth2 authorization endpoint
([#2107](#2107))
([5318552](5318552))
* **oauth2:** add `/oauth/token` endpoint
([#2159](#2159))
([a89a0b0](a89a0b0))
* **oauth2:** add admin endpoint to regenerate OAuth client secrets
([#2170](#2170))
([0bd1c28](0bd1c28))
* **oauth2:** return redirect_uri on GET authorization
([#2175](#2175))
([b0a0c3e](b0a0c3e))
* **oauth2:** use `id` field as the public client_id
([#2154](#2154))
([86b7de4](86b7de4))
* **openapi:** add OAuth 2.1 server endpoints and clarify OAuth modes
([#2165](#2165))
([1f804a2](1f804a2))
* password changed email notification
([#2176](#2176))
([fe0fd04](fe0fd04))
* support `transfer_sub` in apple id tokens
([#2162](#2162))
([8a71006](8a71006))


### Bug Fixes

* ensure request context exists in API db operations
([#2171](#2171))
([060a992](060a992))
* **makefile:** remove invalid @ symbol from shell commands
([#2168](#2168))
([e6afe45](e6afe45))
* **oauth2:** switch to Origin header for request validation
([#2174](#2174))
([42bc9ab](42bc9ab))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@hf hf hf approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@cstockton @coveralls @hf