Integration/e2e testing doubts

[Tansito]

We are trying to use the tools for testing that you offer in @open-wc/testing and @web/test-runner but we have a couple of doubts meanwhile we write the tests. We are trying to test two use cases:

• Just get an h1 to see that the application starts correctly
• Obtain a meta (for example) to check that the SEO works as it should

An example of a simple test that we are writing is:

describe('AppTest', () => {
  let element: AppIndex;
  beforeEach(async () => {
    element = await fixture(html`
      <app-index></app-index>
    `);
  });

  it('renders a h1', () => {
    const h1 = element.shadowRoot!.querySelector('h1')!;
    expect(h1).to.exist;
    expect(h1.textContent).to.equal('About');
  });
});

Our app-index is our shell and we are facing two problems here:

• the first one is that we are having problems loading dynamic imports using the vaadin-router
• the second one is that with this test we are not starting the application if not we are only instantiating the app-index I think.

So we are a little stuck trying to figure out how we can run the application navigate to an end-point and test there something like:

describe('AppTest', () => {
  beforeEach(async () => {
    browser().goTo('/about');
  });

  it('renders a h1', () => {
    // Just test here something in the about page
  });
});

Thanks community!

[daKmoR]

if you have a state property for those pages like

<app-index page="about">

then you could test that.

However, it seems more like you want to have end to end testing which is a browser actually opening your running application and you do some expects on what the browser sees.

for that using for example playwright + node moch seems more appropriate.

see some examples at the playwright homepage
https://playwright.dev/#version=v1.5.1&path=docs%2Ftest-runners.md&q=test-runners

the way that would work with web dev server would be to run those in parallel (for example with https://www.npmjs.com/package/concurrently)

"scripts": {
  "e2e": "concurrently \"web-dev-server\" \"node e2etest.js\"",
}

The playwright-cli even has a tool which records your interactions and creates a test file automatically
https://www.infoq.com/news/2020/10/playwright-records-scripts-tests/

another popular tool would be https://www.cypress.io/

[LarsDenBakker]

To clarify, web test runner runs your tests in the browser itself and not with a remote controlled browser like selenium, puppeteer or playwright.

You can test it the way @daKmoR explains, but if it's a single page app you can still test your app with WTR. You need to make sure your router is initialized properly, and you make changes to the URL in a way your router expects. This is different for each router.

You also need to make sure you initialize your application properly like you mentioned. Again this is different for each application. What is loaded in your index.html?

[Tansito]

Yes! I think what @daKmoR said can work. We had cypress in some projects but we wanted to test your setup.

About your comments @LarsDenBakker what we have in our index.html is just the shell for the application (ref here), I don't know if that solves your doubt.

Thanks for the help team!

[WickyNilliams]

web test runner runs your tests in the browser itself and not with a remote controlled browser like selenium, puppeteer or playwright.

Reading through the docs over the last few days, I feel this point could be made clearer. I haven't seen anywhere where this is prominently mentioned - please correct me if I'm wrong! Whilst it's implied e.g. by use of DOM APIs in tests, and the need for commands to communicate with the server, it isn't spelled out.

Perhaps it could be mentioned on the front page of the docs.

[Tansito]

We were trying different configurations investigating a little bit about playwright, mocha, etc, and we finished with the next branch. In this branch we were trying to launch the wrt but it's showing us the next error:

As a test we tried too configure without web-test-runner the tests here and it worked so we don't know what we are doing wrong exactly. Can you give us some light?

[LarsDenBakker]

Your module could not be loaded by the browser. If you open the browser you will get a more descriptive error message. You can do this by running the test runner in watch mode, and pressing "D".

Looking at your code, I think it's because you configured mocha to use TDD while you're using describe in your tests. TDD uses suite and test while BDD uses describe and it.

Looking at your tests I think there is still a misunderstanding. Web test runner runs your tests in the browser, so your javascript code is executing in the same environment as your source code. In the browser there is no global page object where you can call goto on. If you are building a traditional website, navigation will reload the page and thus unload the javascript code including the tests. If you are building a SPA you could still make this work, but you have to use a navigation that works for your router.

So this really depends on whether you're looking to run unit/component tests, or e2e tests on a real environment.

[Tansito]

Yes, our idea is to create an e2e test on a real environment. We tried to follow a similar configuration that we have here with ts-mocha and playwright trying to follow the steps explained before by @daKmoR but moving that idea to the web-test-runner, but it's obvious that i'm struggling at it 😅 .

Reading more in depth your comment @LarsDenBakker if I understand well the difference between the approach that we have between this last-branch with ts-mocha/playwright and the one with the web-test-runner implementation is that with ts-mocha and playwright we are launching the tests through node but with web-test-runner the approach is different because we are executing the tests directly in the browser. Now I think I got finally what you are trying to say, thanks Lars!

PD: thanks for the clarification between TDD and BDD, I will never forget it apart from now 😂