Skip to content

✅ [RUM-15484] bootstrap integration test infrastructure#91

Merged
bcaudan merged 13 commits into
mainfrom
bcaudan/integration-tests
Apr 21, 2026
Merged

✅ [RUM-15484] bootstrap integration test infrastructure#91
bcaudan merged 13 commits into
mainfrom
bcaudan/integration-tests

Conversation

@bcaudan

@bcaudan bcaudan commented Apr 9, 2026

Copy link
Copy Markdown
Collaborator

Motivation

The existing E2E tests use a minimal fixture app, which doesn't catch issues that only surface in specific customer toolchains (different bundlers, packagers, module formats, or dev vs packaged environments). This PR adds integration tests that validate the SDK against realistic Electron app setups.

Changes

  • Add 4 integration apps covering the most common customer toolchains: forge-webpack, forge-vite, electron-vite, and electron-builder-vite — each covering a different bundler/packager combination and SDK preload loading strategy
  • Add a Playwright fixture (integrationFixture.ts) handling app launch in dev and packaged modes, with a local intake server capturing RUM events
  • Add integration scenarios testing view event emission, renderer error propagation, and crash reporting across restart — run against every app × mode combination
  • Add test:integration:init / test:integration scripts and a dedicated CI job
  • Apps install the SDK from a packed tarball (yarn pack) to mirror the npm install experience
  • Fix a potential race condition where preload registration could happen before app is ready or after the creation of windows

Test instructions

yarn test:integration:init   # build SDK tarball, install and package all integration apps
yarn test:integration        # run all integration scenarios (24 tests across 8 projects)

To test a specific app/mode:

yarn test:integration --project=forge-webpack-dev
yarn test:integration --project=electron-vite-packaged

Checklist

  • Tested locally (playground)
  • Added unit tests for this change.
  • Added e2e/integration tests for this change.
  • Updated related documentation.

@bcaudan bcaudan force-pushed the bcaudan/integration-tests branch 8 times, most recently from 30273ad to f1bccc2 Compare April 9, 2026 17:03
@bcaudan bcaudan force-pushed the bcaudan/integration-tests branch from f1bccc2 to e17a135 Compare April 9, 2026 17:15
@bcaudan bcaudan changed the title Bcaudan/integration tests ✅ [RUM-15484] bootstrap integration test infrastructure Apr 9, 2026
@bcaudan

bcaudan commented Apr 9, 2026

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e17a1355c0

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread e2e/scenarios/context.scenario.ts Outdated
Comment thread e2e/integration/apps/electron-vite/src/main/index.ts
@bcaudan bcaudan force-pushed the bcaudan/integration-tests branch from e17a135 to 0e864ef Compare April 10, 2026 07:45
bcaudan added 6 commits April 10, 2026 10:40
- Add Playwright config generating dev/packaged projects
- Add test:integration:* scripts to package.json
- Add installation script to build all apps before tests
- Add integrationFixture.ts handling dev/packaged app launch + intake server
- Add integration scenario skeleton (view, renderer error, crash reporting)
- Add GitLab CI job running in parallel with e2e
- Exclude integration/apps from root lint and typecheck
@bcaudan bcaudan force-pushed the bcaudan/integration-tests branch 2 times, most recently from abaeb22 to ac79dc7 Compare April 10, 2026 08:49
@bcaudan bcaudan force-pushed the bcaudan/integration-tests branch from ac79dc7 to 0822d47 Compare April 10, 2026 08:51
@bcaudan bcaudan marked this pull request as ready for review April 10, 2026 08:56
@bcaudan bcaudan requested a review from a team as a code owner April 10, 2026 08:56

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 0822d4709b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread e2e/integration/lib/integrationFixture.ts Outdated
bcaudan added 3 commits April 10, 2026 11:35
since we do it manually for crash test, it would generate a useless extra instance

@cdn34dd cdn34dd left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall looks good, just a couple of minor tweaks.

Comment thread src/bridge/registerPreload.spec.ts Outdated
Comment thread src/bridge/registerPreload.spec.ts
Comment thread src/bridge/registerPreload.ts
Comment thread src/bridge/registerPreload.spec.ts Outdated
Comment thread e2e/integration/scenarios/integration.scenario.ts
Comment thread e2e/integration/scenarios/integration.scenario.ts
bcaudan added 2 commits April 21, 2026 14:46
The tests were written with `waitForEventCount('view', 2)` but only ever
received 1 bridge view event reliably. They appeared to pass on main
because the old `waitForEventCount` silently returned whatever events
were available on timeout instead of throwing — so the count=2 assertion
was effectively never enforced.

Our branch made `waitForEventCount` throw on timeout, correctly surfacing
the issue. Investigation showed:
- browser-rum sends view event 1 immediately during `datadogRum.init()`
- view event 2 (load-complete update) is deferred by a 3000ms throttle
  (`THROTTLE_VIEW_UPDATE_PERIOD`, `leading: false`) and arrives after
  `flushTransport()` has already run, so it never reaches the intake
  within the test window

count=1 is the correct, honest expectation: it reliably verifies that
bridge events flow from renderer → IPC → main process → intake.
@bcaudan bcaudan requested a review from cdn34dd April 21, 2026 13:26
@bcaudan bcaudan merged commit 587f84d into main Apr 21, 2026
15 checks passed
@bcaudan bcaudan deleted the bcaudan/integration-tests branch April 21, 2026 13:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants