Skip to content

Added Sentry monitoring for Billing app load failures#27826

Merged
aileen merged 5 commits into
mainfrom
codex/billing-app-load-monitor
May 12, 2026
Merged

Added Sentry monitoring for Billing app load failures#27826
aileen merged 5 commits into
mainfrom
codex/billing-app-load-monitor

Conversation

@aileen
Copy link
Copy Markdown
Member

@aileen aileen commented May 12, 2026

Summary

  • Added a Billing iframe readiness monitor that reports to Sentry if the Billing app does not send a valid message within 10 seconds
  • Starts monitoring when the Billing iframe src is assigned, matching the preload path before Ghost(Pro) is clicked
  • Clears the monitor when the Billing iframe sends its first accepted message

ref INC-284

Testing

  • corepack pnpm exec eslint app/services/billing.js app/components/gh-billing-iframe.js tests/unit/services/billing-test.js --cache
  • corepack pnpm exec ember exam --file-path tests/unit/services/billing-test.js --split 1 --parallel 1

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented May 12, 2026

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

Walkthrough

Adds a billing app load monitor: BillingService gains timeout/retry constants, tracked state, lifecycle cleanup, start/timeout/retry/reload/mark-as-loaded helpers, and one-time Sentry failure reporting. GhBillingIframe starts the monitor after setting iframe src and marks the app loaded on postMessage. Unit tests cover timeout-to-retry-to-failure flow, iframe reload on retry, Sentry report contents and tagging, monitor clearing on success, and no reporting when sentry_dsn is null.

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The pull request title accurately and concisely summarizes the main change: adding Sentry monitoring for Billing app load failures.
Description check ✅ Passed The pull request description clearly relates to the changeset, explaining the Billing iframe readiness monitor, monitoring behavior, and testing instructions.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch codex/billing-app-load-monitor

Comment @coderabbitai help to get the list of available commands and usage tips.

@aileen aileen marked this pull request as ready for review May 12, 2026 08:02
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
ghost/admin/tests/unit/services/billing-test.js (1)

66-74: ⚡ Quick win

Add a regression test for “loaded once, then monitor again”

Please add a case that calls markBillingAppLoaded(), then starts monitoring again for a new load attempt, and verifies a new timeout can be scheduled/fired.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@ghost/admin/tests/unit/services/billing-test.js` around lines 66 - 74, Add a
new unit test in billing-test.js that first calls
service.markBillingAppLoaded(), then calls service.startBillingAppLoadMonitor()
again to start a new load attempt, assert that service.billingAppLoadTimeout is
set (non-null) after starting monitor, simulate/fire the timeout (or advance
fake timers) and then assert that a report was created (testkit.reports() length
increased) and that service.billingAppLoadTimeout is cleared (null) after the
timeout handler runs; reference the existing helpers startBillingAppLoadMonitor
and markBillingAppLoaded and assert on billingAppLoadTimeout and
testkit.reports() to validate the regression.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@ghost/admin/app/services/billing.js`:
- Around line 63-77: The current logic sets billingAppLoaded = true in
markBillingAppLoaded so startBillingAppLoadMonitor returns early on subsequent
iframe loads and never re-arms monitoring; fix by making the monitor re-armable:
either remove the billingAppLoaded early-return in startBillingAppLoadMonitor
(so it only checks billingAppLoadTimeout) or change markBillingAppLoaded to
avoid permanently setting billingAppLoaded (e.g., do not set billingAppLoaded =
true or explicitly reset it back to false after calling
clearBillingAppLoadMonitor); update references to startBillingAppLoadMonitor,
markBillingAppLoaded, clearBillingAppLoadMonitor, billingAppLoaded, and
billingAppLoadTimeout accordingly.

---

Nitpick comments:
In `@ghost/admin/tests/unit/services/billing-test.js`:
- Around line 66-74: Add a new unit test in billing-test.js that first calls
service.markBillingAppLoaded(), then calls service.startBillingAppLoadMonitor()
again to start a new load attempt, assert that service.billingAppLoadTimeout is
set (non-null) after starting monitor, simulate/fire the timeout (or advance
fake timers) and then assert that a report was created (testkit.reports() length
increased) and that service.billingAppLoadTimeout is cleared (null) after the
timeout handler runs; reference the existing helpers startBillingAppLoadMonitor
and markBillingAppLoaded and assert on billingAppLoadTimeout and
testkit.reports() to validate the regression.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 23a496b9-4329-4d08-8878-f0adf6ab29ad

📥 Commits

Reviewing files that changed from the base of the PR and between 39ee9fb and a478950.

📒 Files selected for processing (3)
  • ghost/admin/app/components/gh-billing-iframe.js
  • ghost/admin/app/services/billing.js
  • ghost/admin/tests/unit/services/billing-test.js

Comment thread ghost/admin/app/services/billing.js
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@ghost/admin/tests/unit/services/billing-test.js`:
- Around line 49-53: The test is flaky because it relies on a timing-sensitive
assertion that reportBillingAppLoadFailure is false immediately after awaiting
reloadBillingIframe; change this to assert call order instead: after awaiting
both waitUntil(...) for reloadBillingIframe and waitUntil(...) for
reportBillingAppLoadFailure, assert that reloadBillingIframe was called before
reportBillingAppLoadFailure (e.g. using
sinon.assert.callOrder(reloadBillingIframe, reportBillingAppLoadFailure) or by
comparing their firstCall.callId/timestamps) to make the expectation
deterministic; update the assertions around reloadBillingIframe,
reportBillingAppLoadFailure, and waitUntil to reflect this call-order check.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 44823a89-8349-4301-8f6b-237cac348b00

📥 Commits

Reviewing files that changed from the base of the PR and between a478950 and b1074f9.

📒 Files selected for processing (2)
  • ghost/admin/app/services/billing.js
  • ghost/admin/tests/unit/services/billing-test.js

Comment thread ghost/admin/tests/unit/services/billing-test.js Outdated
@aileen aileen force-pushed the codex/billing-app-load-monitor branch from d66096d to a35b193 Compare May 12, 2026 09:37
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@ghost/admin/app/services/billing.js`:
- Around line 68-76: The monitor currently stops being startable forever because
startBillingAppLoadMonitor() returns early when billingAppLoadFailureReported is
true; change the logic in startBillingAppLoadMonitor to not permanently block
retries by clearing or ignoring billingAppLoadFailureReported when initiating a
new monitor (e.g., reset billingAppLoadFailureReported = false before starting)
and ensure billingAppLoadTimeout and billingAppLoadAttempts are handled
consistently with billingAppLoaded; update references to
billingAppLoadFailureReported, billingAppLoadTimeout, billingAppLoaded, and
billingAppLoadAttempts in startBillingAppLoadMonitor and any failure-reporting
paths so a single reported failure does not disable future monitor attempts in
the same session.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: f6eabd2f-08db-4ea5-8ed6-e62c6e5e29f1

📥 Commits

Reviewing files that changed from the base of the PR and between d66096d and a35b193.

📒 Files selected for processing (3)
  • ghost/admin/app/components/gh-billing-iframe.js
  • ghost/admin/app/services/billing.js
  • ghost/admin/tests/unit/services/billing-test.js
🚧 Files skipped from review as they are similar to previous changes (2)
  • ghost/admin/app/components/gh-billing-iframe.js
  • ghost/admin/tests/unit/services/billing-test.js

Comment thread ghost/admin/app/services/billing.js
@aileen aileen force-pushed the codex/billing-app-load-monitor branch from a34d32e to 5bf1f39 Compare May 12, 2026 10:58
@aileen aileen requested a review from sagzy May 12, 2026 10:59
Comment thread ghost/admin/app/services/billing.js Outdated
Comment thread ghost/admin/app/services/billing.js Outdated
@sagzy
Copy link
Copy Markdown
Contributor

sagzy commented May 12, 2026

@aileen LGTM, added a few nits but nothing blocking

aileen added 4 commits May 12, 2026 15:27
ref [INC-284](https://app.incident.io/ghost/incidents/284)

The Billing app iframe could fail to become ready without any telemetry, leaving reports of the Ghost(Pro) route not loading invisible to the team. Added a 10 second readiness monitor around `GhBillingIframe` startup that reports a standardized Sentry event when no valid Billing iframe message arrives, while clearing the monitor once the app communicates successfully.
ref [INC-284](https://app.incident.io/ghost/incidents/284)

The Billing app readiness monitor now performs one iframe reload after a short backoff before reporting a failure to Sentry. This avoids reporting transient startup failures while still surfacing persistent cases where the Billing app never sends a valid message back to Admin.
ref [INC-284](https://app.incident.io/ghost/incidents/284)

The Billing app load monitor now re-arms cleanly after a previous readiness message so a later iframe load can still be observed. The retry test now asserts call order between iframe reload and failure reporting, which avoids relying on timing-sensitive intermediate state.
ref [INC-284](https://app.incident.io/ghost/incidents/284)

The Billing app load monitor now treats a previous failure report as completed state rather than a permanent block, so a later iframe load in the same session can be monitored again. Added a regression test covering the failure-to-new-load path.
@aileen aileen force-pushed the codex/billing-app-load-monitor branch from 5bf1f39 to f40de34 Compare May 12, 2026 11:27
ref [INC-284](https://app.incident.io/ghost/incidents/284)

The Billing app load failure report now uses the standardized `billing_monitor` Sentry context and is captured as an exception so it matches the updated monitoring catalog. Updated the billing service test to assert the new context shape.
@aileen aileen enabled auto-merge (squash) May 12, 2026 11:47
@aileen aileen merged commit f5741c1 into main May 12, 2026
41 checks passed
@aileen aileen deleted the codex/billing-app-load-monitor branch May 12, 2026 12:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants