Skip to content

[Bug]: Last failed flag has unexpected behavior with no failed tests #36465

Open
@miguelangaranocurrents

Description

@miguelangaranocurrents

Version

1.52.0

Steps to reproduce

The issue is produced in 2 stages:

  1. A first execution that has a global error before starting executing the tests that produces a .last-run.json file with no failed tests but with failed status.
  • To create a global error you can put this code in a index.js file
console.log('Starting simple application...');

// Simple math operations
const num1 = 10;
const num2 = 5;
const sum = num1 + num2;
const product = num1 * num2;

console.log(`Sum: ${sum}`);
console.log(`Product: ${product}`);

// Array operations
const fruits = ['apple', 'banana', 'orange'];
console.log('Fruits:', fruits);

// This will cause an error - trying to access a property on undefined
const user = undefined;
console.log('User name:', user.name); // This line will throw an error

console.log('This line will never be reached due to the error above');
  • Add a webServer config in the playwright.config.js file:
webServer: {
    command: "node index.js",
    port: 3000,
    timeout: 120 * 1000,
    reuseExistingServer: !process.env.CI,
  },
  1. A second execution using --last-failed reads the .last-run.json file and doesn’t executes any tests.
  • Remove the webServer property from playwright.config.js.
  • Execute the tests again

Expected behavior

Option A:

When a global error happens in a run, Playwright should not generate a .last-run.json file.
This option makes sense because if no tests were executed, there won't be any failed or passed tests, hence, no need for generating the file.
This will help to execute all tests in a next run using --last-failed flag.

Option B:

Let the .last-run.json file to be generated in the first execution with a global error, but in the second execution using --last-failed flag, if the "status": "failed" and empty failedTests array in the .last-run.json file, execute all the tests required.

Actual behavior

Option A:

Having a tests execution with a global error before starting to execute tests, should not generate .last-run.json file.

Option B:

Executing tests using --last-failed flag with a .last-run.json file that contains "status": "failed" and empty failedTests array, should execute all the tests.

Additional context

No response

Environment

System:
    OS: macOS 15.5
    CPU: (8) arm64 Apple M1 Pro
    Memory: 116.73 MB / 32.00 GB
  Binaries:
    Node: 20.14.0 - ~/.nvm/versions/node/v20.14.0/bin/node
    Yarn: 1.22.19 - /usr/local/bin/yarn
    npm: 10.7.0 - ~/.nvm/versions/node/v20.14.0/bin/npm
    pnpm: 10.12.1 - ~/Library/pnpm/pnpm
    bun: 1.1.12 - ~/.bun/bin/bun
  Languages:
    Bash: 3.2.57 - /bin/bash
  npmPackages:
    @playwright/test: ^1.52.0 => 1.52.0 
    playwright: ^1.38.1 => 1.52.0

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions