Incremental progress reporting

[dgozman]

e2e tests are slower, so I'd like to see test cases (or at least their failures) being reported as they run.

[JoelEinbinder]

jestjs/jest#9819 is working on this.

[dgozman]

Another thing to consider: currently, many users write their tests sharing a page per describe and using it to do some steps. Since we insist on test isolation, it would be nice to have some kind of "test steps" that could be reported back to the test runner, especially when I am focusing on the test or suite.

[JoelEinbinder]

Another thing to consider: currently, many users write their tests sharing a page per describe and using it to do some steps. Since we insist on test isolation, it would be nice to have some kind of "test steps" that could be reported back to the test runner, especially when I am focusing on the test or suite.

Interesting! step alongside it! I like it. We'd report each step as its own test case to jest, and guarantee that they run serially with the same state.

[JoelEinbinder]

steps('should work', ({page}) => {
  step('setup the page', async() => {
    await page.goto('https://joel.tools/')
  });
  step('click the button', async() => {
    await page.click('button');
    expect(await page.evaluate('clicked')).toBe(true);
  });
  step('click the other button button', async() => {
    await page.click('other-button');
    expect(await page.evaluate('otherClicked')).toBe(true);
  });
});

it('should work', ({page}) => {
  await step('setup the page', async() => {
    await page.goto('https://joel.tools/')
  });
  await step('click the button', async() => {
    await page.click('button');
    expect(await page.evaluate('clicked')).toBe(true);
  });
  await step('click the other button button', async() => {
    await page.click('other-button');
    expect(await page.evaluate('otherClicked')).toBe(true);
  });
});

it('should work', ({page}) => {
  step('setup the page');
  await page.goto('https://joel.tools/')
  
  step('click the button');
  await page.click('button');
  expect(await page.evaluate('clicked')).toBe(true);
  
  step('click the other button button');
  await page.click('other-button');
  expect(await page.evaluate('otherClicked')).toBe(true);
});

Which do you like?

[dgozman]

The last one. However, it would probably be it('should work', ({page, step}) => {...})

[celeryclub]

Has any thought been given to an automatic log of commands performed by Playwright? I implemented this in a project. It's modeled after Cypress's in-browser command log. The dots are yellow while pending and then green on success or red on failure.

https://share.getcloudapp.com/OAubBAbv

My current implementation is a bit hacky (see microsoft/playwright#1654), but I'm curious what you think of this general idea.

[dgozman]

@celeryclub Yep, we are thinking about this. The new Logger api in Playwright should be able to provide such a log, and test runner will be able to consume it.