[Bug]: Tests running in Electron do not show snapshots in the timeline

[cedricbloem]

Version

1.44.1

Steps to reproduce

1. Create a basic Playwright project
   a. YARN: yarn create playwright
   b. NPM: npm init playwright@latest
2. Install Electron as a devDependency
   a. YARN: yarn add -D electron
   b. NPM: npm install --save-dev electron
   c. In my case, I installed version v31.0.2
3. Set your Playwright config and test up with the code down below
4. Run Playwright test in UI
   a. YARN: yarn playwright test --ui
   b. NPM: npx playwright test --ui
5. Run the example.spec.ts test "has title"
6. Your test should launch Electron, visit the Playwright website in the Electron app and pass
7. Notice how you have actions, but no snapshots tied to those actions.

Code

./playwright.config.ts

import { defineConfig, devices } from '@playwright/test';

export default defineConfig({
  testDir: './tests',
  reporter: 'html',
  use: {
    trace: 'on',
  },
  projects: [
    {
      name: 'electron',
      use: { ...devices['Desktop Chrome'] },
    },
  ],
});

./tests/example.spec.ts

import { _electron, test, expect } from '@playwright/test';

test('has title', async () => {
  const electronApp = await _electron.launch({ args: ['main.js'] });

  // Get the first window that the app opens, wait if necessary.
  const window = await electronApp.firstWindow();

  // Direct Electron console to Node terminal.
  window.on('console', console.log);

  await window.goto('https://playwright.dev/');

  // Expect a title "to contain" a substring.
  await expect(window).toHaveTitle(/Playwright/);

  // Exit app.
  await electronApp.close();
});

./main.js

const { app, BrowserWindow } = require('electron');

app.whenReady().then(() => {
  const win = new BrowserWindow({
    width: 800,
    height: 600,
  });
  win.loadURL('about:blank');
})

app.on('window-all-closed', e => e.preventDefault())

Expected behavior

I expect to see snapshots in the timeline of the trace viewer.
I expect the selected action to show a snapshot on "Action", "Before" and "After".

Actual behavior

I only see one blank snapshot at the start of the test, I do not see a snapshot of the page visited during the test (or any other action).

Additional context

No response

Environment

System:
  OS: Windows 11 10.0.22621
  CPU: (20) x64 12th Gen Intel(R) Core(TM) i9-12900H
  Memory: 25.07 GB / 63.67 GB
Binaries:
  Node: 21.1.0 - C:\Program Files\nodejs\node.EXE
  Yarn: 1.22.19 - ~\AppData\Roaming\npm\yarn.CMD
  npm: 10.2.0 - C:\Program Files\nodejs\npm.CMD
npmPackages:
  @playwright/test: ^1.44.1 => 1.44.1

[mxschmitt]

Debugging notes, while Electron integration into @playwright/test isn't fully there yet, #8208 I still experienced a bug around closing the Electron application:

This was throwing because ElectronApplication closes the BrowserContext which disposes the APIRequestContext which triggers this via the instrumentation but we never started tracing before.

diff --git a/packages/playwright/src/index.ts b/packages/playwright/src/index.ts
index 3fbaf3358..85f5883ef 100644
--- a/packages/playwright/src/index.ts
+++ b/packages/playwright/src/index.ts
@@ -597,7 +597,8 @@ class ArtifactsRecorder {
     if ((tracing as any)[this._startedCollectingArtifacts])
       return;
     (tracing as any)[this._startedCollectingArtifacts] = true;
-    if (this._testInfo._tracing.traceOptions())
+    // This was throwing because ElectronApplication closes the BrowserContext which disposes the APIRequestContext which triggers this via the instrumentation but we never started tracing before.
+    if (this._testInfo._tracing.traceOptions() && (tracing as any)[kTracingStarted])
       await tracing.stopChunk({ path: this._testInfo._tracing.generateNextTraceRecordingPath() });
   }
 }
diff --git a/tests/page/wheel.spec.ts b/tests/page/wheel.spec.ts
index c8c634e16..87bf1a1bb 100644
--- a/tests/page/wheel.spec.ts
+++ b/tests/page/wheel.spec.ts
@@ -13,7 +13,7 @@
  * See the License for the specific language governing permissions and
  * limitations under the License.
  */
-import type { Page } from 'playwright-core';
+import { _electron, type Page } from 'playwright-core';
 import { test as it, expect, rafraf } from './pageTest';
 
 it.skip(({ isAndroid }) => {
@@ -48,23 +48,29 @@ async function expectEvent(page: Page, expected: any) {
   expect(received).toEqual(expected);
 }
 
-it('should dispatch wheel events @smoke', async ({ page, server }) => {
-  await page.setContent(`<div style="width: 5000px; height: 5000px;"></div>`);
-  await page.mouse.move(50, 60);
-  await listenForWheelEvents(page, 'div');
-  await page.mouse.wheel(0, 100);
-  await page.waitForFunction('window.scrollY === 100');
-  await expectEvent(page, {
-    deltaX: 0,
-    deltaY: 100,
-    clientX: 50,
-    clientY: 60,
-    deltaMode: 0,
-    ctrlKey: false,
-    shiftKey: false,
-    altKey: false,
-    metaKey: false,
-  });
+const test = it;
+it('should dispatch wheel events @smoke', async ({ trace }) => {
+  const electronApp = await _electron.launch({ args: ['/Users/maxschmitt/Developer/playwright/tests/electron/electron-window-app.js'] });
+
+  // Get the first window that the app opens, wait if necessary.
+  const window = await electronApp.firstWindow();
+  if (trace)
+    await window.context().tracing.start({ screenshots: true, snapshots: true });
+
+  await window.goto('https://playwright.dev/');
+
+  // Expect a title "to contain" a substring.
+  await expect(window).toHaveTitle(/Playwright/);
+
+  if (trace) {
+    await window.context().tracing.stop({ path: test.info().outputPath('electron-trace.zip') });
+    test.info().attachments.push({
+      name: 'trace',
+      path: test.info().outputPath('electron-trace.zip'),
+      contentType: 'application/zip'
+    });
+  }
+  await electronApp.close();
 });
 
 it('should dispatch wheel events after context menu was opened', async ({ page, browserName, isWindows }) => {

[mxschmitt]

We fixed the regression in 1.45.1. For trace support, I'm folding into #8208 For now we recommend starting tracing manually and attaching it e.g. like here.