[Bug]: When using --ui the test just say "Failed" with no context at all

[adrianwix]

Version

1.46.1

Steps to reproduce

• Run playwright with --ui
• Run the desired test

Expected behavior

• Test runs and it success or fails and shows why

Actual behavior

It just says "Failed" with absolutely no information

Additional context

context.close();
_page.close();

Was making it fail when using --ui but not when using --debug. For the love of god is anything, ANYTHING fails inside your library please report it to the user. Not a "Failed" message that says nothing. I only made it work because of intuition.

Environment

System:
    OS: macOS 15.0.1
    CPU: (8) arm64 Apple M2
    Memory: 64.47 MB / 16.00 GB
  Binaries:
    Node: 20.10.0 - ~/.nvm/versions/node/v20.10.0/bin/node
    Yarn: 4.3.1 - ~/.nvm/versions/node/v20.10.0/bin/yarn
    npm: 10.2.3 - ~/.nvm/versions/node/v20.10.0/bin/npm
  IDEs:
    VSCode: 1.98.0 - /usr/local/bin/code
  Languages:
    Bash: 3.2.57 - /bin/bash
  npmPackages:
    @playwright/test: ^1.46.1 => 1.46.1

[agg23]

Please provide a complete reproduction example. Additionally, why are you using such an old version of Playwright? Does it still show this issue on 1.51?

[adrianwix]

The error still happens with 1.51.

In this case the I had a timeout of 30000 ms and the test was still running during it.

I am afraid that for a completely reproducible env I need to create a repo with an Express backend, Hasura, React FE, etc, etc which would take me 1 or 2 days.

I image that it would be easier to go the pat of the repo that decides on showing the "Failed" message it finding out which Error is not handled properly

[agg23]

Most likely it's a completely unexpected error. Can you look in the browser console in the UI? If that doesn't have anything, I would suggest turning on most logging with setting DEBUG=pw:*. This will be a large file, but it might tell us what is happening.

[adrianwix]

Before showing "Failed" I manage to see that some error was throw in the section that says "After Hook" but the test that I am running has no explicit after hook. Nor they are defined globally

Let me try that. Should I just run DEBUG=pw:*. yarn playwright test?

[agg23]

Let me try that. Should I just run DEBUG=pw:* yarn playwright test?

Yes please. Without the "." after star though.

[adrianwix]

This breaks before I manage to run the tests.

test.txt

[agg23]

Something is causing what is probably the UI Chromium instance to dump an extremely large amount of information to stdout. Do the tests execute fine when not in UI mode? Did adding DEBUG=pw:* prevent UI mode from opening?

[adrianwix]

Adding DEBUG=pw:* didn't prevented the UI from opening but the moment I click "Run test" it crashed.

This is not necessarily the error from yesterday by the way. The random "Failed" that I have found it:

Case 1

I wanted to use a new page and context to ignore the Auth state that was store in the Setup method so I was running

context.close();
_page.close();

This was making it fail when using --ui but not when using --debug.

Case 2

I had test.setTimeout(30000) and the test ran for more than 30 seconds which made it failed. When it was doing something in the "After Hook" it crashed and showed "Failed" without any information.

Here after removing the timeout the test passed without issues.

More context

I can actually show you some setup files without any hassle. Maybe this helps:

import { defineConfig, devices } from '@playwright/test';
import dotenv from 'dotenv';
import path from 'path';

// We now set VITE_E2E_TEST through npm scripts instead

const configFile =
  process.env.VITE_ENV === 'development'
    ? '.env'
    : process.env.VITE_ENV === 'preview'
      ? '.env.preview'
      : process.env.VITE_ENV === 'staging'
        ? '.env.staging'
        : '.env';

if (!configFile)
  throw Error('VITE_ENV must be one of development, preview, staging');

console.log(`Loading environment variables from ${configFile}`);

dotenv.config({
  path: path.resolve(__dirname, configFile),
});

export default defineConfig({
  timeout: 60000,
  testDir: './e2e',
  /* Run tests in files in parallel */
  fullyParallel: true,
  /* Fail the build on CI if you accidentally left test.only in the source code. */
  forbidOnly: !!process.env.CI,
  /* Retry on CI only */
  retries: process.env.CI ? 2 : 0,
  /* Opt out of parallel tests on CI. */
  workers: 1,
  /* Reporter to use. See https://playwright.dev/docs/test-reporters */
  reporter: 'html',
  /* Shared settings for all the projects below. See https://playwright.dev/docs/api/class-testoptions. */
  use: {
    /* Base URL to use in actions like `await page.goto('/')`. */
    baseURL: process.env.PREVIEW_BASE_URL
      ? process.env.PREVIEW_BASE_URL
      : process.env.BASE_URL,

    /* Collect trace when retrying the failed test. See https://playwright.dev/docs/trace-viewer */
    trace: 'on-first-retry',
    screenshot: 'only-on-failure',
    permissions: ['clipboard-read'],
  },

  /* Configure projects for major browsers */
  projects: [
    { name: 'setup', testMatch: /.*\.setup\.ts/ },
    {
      name: 'chromium',
      testMatch: /.*\.spec\.ts/,
      use: {
        ...devices['Desktop Chrome'],
        storageState: '.auth/user.json',
      },
      dependencies: ['setup'],
    },
  ],

  /* Run your local dev server before starting the tests */
  ...(process.env.VITE_ENV === 'development'
    ? {
        webServer: {
          command: 'VITE_E2E_TEST=true yarn start',
          url: 'http://localhost:3000',
          reuseExistingServer: !process.env.CI,
          stdout: 'pipe',
          stderr: 'pipe',
          timeout: 60000,
        },
      }
    : {}),
});

auth.setup.ts

import { test as setup } from '@playwright/test';
import { login } from './utils/auth.utils';

const authFile = '.auth/user.json';

setup('authenticate', async ({ page, baseURL }) => {
  // Perform authentication steps. Replace these actions with your own.
  if (!baseURL) throw new Error('Missing BASE_URL env variable.');

  await login({
    page,
    baseURL,
    credentials: {
      email: '',
      password: '',
    },
  });
  // End of authentication steps.

  await page.context().storageState({ path: authFile });
});