Migration Guide

This document outlines how to migrate a repo using @open-wc/testing helpers and/or the first generation @brightspace-ui/visual-diff library to @brightspace-ui/testing.

Before starting, install @brightspace-ui/testing as a devDependency and include it in the repository's package.json. Similarly, once the migration is complete, @open-wc/testing can be removed.

Unit Tests

Unit tests using the @open-wc/testing helpers can easily be migrated to use @brightspace-ui/testing's versions with a few minor changes.

Step 1: Adjust Imports

In test import statements, search and replace from '@open-wc/testing' with from '@brightspace-ui/testing'.

Before:

import { fixture, oneEvent } from '@open-wc/testing';

After:

import { fixture, oneEvent } from '@brightspace-ui/testing';

Step 2: Sinon Fake Timers

If any tests are using Sinon's useFakeTimers, ensure that the scope of what gets faked is limited.

For example, if a test requires that setTimeout be faked, pass { toFake: ['setTimeout'] } to Sinon. This prevents Sinon from faking requestAnimationFrame, which will block calls to the new fixture.

Before:

import { useFakeTimers } from 'sinon';

describe('my tests', () => {

  let clock;
  beforeEach(() => clock = useFakeTimers());
  afterEach(() => clock.restore());

});

After:

import { useFakeTimers } from 'sinon';

describe('my tests', () => {

  let clock;
  beforeEach(() => clock = useFakeTimers(
	toFake: ['setTimeout']
  }));
  afterEach(() => clock.restore());

});

Step 3: No need to wait for `updateComplete`

The fixture() in @brightspace-ui/testing differs from the @open-wc/testing version in that it waits for ALL nested Lit components' updateComplete to resolve.

Therefore, any tests which were previously manually await-ing various elements' updateComplete no longer need to do so. 🎉

Before:

const elem = await fixture(html`
  <some-elem>
    <some-other-elem></some-other-elem>
  </some-elem>
`);
await elem.shadowRoot.querySelector('nested-elem').updateComplete;
await elem.querySelector('some-other-elem').updateComplete;

After:

const elem = await fixture(html`
  <some-elem>
    <some-other-elem></some-other-elem>
  </some-elem>
`);

However, waiting for all nested updateCompletes means fixture() can be a bit slower. Therefore, tests that were relying on waiting for an event fired during the initial render will likely fail due to the event now firing during the fixture() call.

Those tests will need to be reworked to not rely on events fired during render().

Step 4: Use Browser Interaction Helpers

For tests which were manually calling elem.click() or elem.hover(), use the clickElem() and hoverElem() helpers. These helpers will trigger a real browser interaction using the mouse.

Before:

it('should focus', async() => {
  const elem = await fixture(html`<button>hello</button>`);
  elem.click();
});

After:

import { clickElem } from '@brightspace-ui/testing';

it('should focus', async() => {
  const elem = await fixture(html`<button>hello</button>`);
  await clickElem(elem);
});

If tests were manually focusing and/or dispatching keyboard events, use the focusElem() and sendKeysElem() helpers.

Note: sendKeysElem() will focus on the element before sending the keys, so calling focusElem() beforehand isn't necessary.

it('should press ENTER', async() => {
  const elem = await fixture(html`<input type="text">`);
  elem.focus();
  const eventObj = document.createEvent('Events');
  eventObj.initEvent('keypress', true, true);
  eventObj.keyCode = '13';
  elem.dispatchEvent(eventObj);
});

After:

import { focusElem, sendKeysElem } from '@brightspace-ui/testing';

it('should focus', async() => {
  const elem = await fixture(html`<button>hello</button>`);
  await sendKeysElem(elem, 'press', 'Enter');
});

Step 5: Use `d2l-test-runner` Binary

Switch to d2l-test-runner from web-test-runner/wtr binaries in any package.json scripts.

By default, d2l-test-runner will:

Run in Chrome, Firefox and Safari (except vdiff just runs in Chrome)
Enable the --node-resolve flag
Look for tests in ./test/**/*.test.js (using the test group)
Suppress ResizeObserver errors (a common reason repos use a config file)

Due to these new defaults, if the repo is using a web-test-runner.config.js, it may no longer be necessary and could be removed. If the config file is necessary, rename it to d2l-test-runner.config.js.

Before

  "scripts": {
    "test:unit": "web-test-runner --files \"./test/**/*.test.js\" --node-resolve"
  }

After:

  "scripts": {
    "test:unit": "d2l-test-runner"
  }

vdiff Tests

Breaking changes were made between the @brightspace-ui/visual-diff library and @brightspace-ui/testing's vdiff functionality. These changes will require a migration, which is unfortunately only partially automated.

Overview of changes:

File/directory name changes (automatic migration)
- Suite names which were previously repeated in the directory and filename have been removed
- Leading d2l- in suite names has been removed
- Browser name (always chromium previously) is now included in the path
- Local snapshots are now stored in a .vdiff directory at the root of the project (which should be listed in .gitignore)
- CI goldens are still stored alongside the tests in a golden directory, but without screenshots/ci
Fixtures were defined in HTML files and tests in JavaScript files. Both are now defined in JavaScript. (manual migration)
Puppeteer was previously used to interact with the browser. It has been replaced with Playwright, but the browser can now be interacted with directly. (manual migration)

Strategy for Large Repositories

For repositories with many vdiff tests, it may be easier to migrate a subset of tests at a time.

By working in this way, both the old and new vdiff libraries continue to run in parallel as tests are gradually migrated from one to the other. Then once all tests are migrated, the old infrastructure can be removed.

The automated migration tools described below support an optional file pattern to make a gradual migration process possible.

In CI, the old and new GitHub actions would also run in parallel during the transition period. Refer to the instructions to get the new version of the vdiff GitHub Action configured in the repository.

Step 1: Automatically Migrate Local Goldens

Before migrating a set of tests, it's helpful to first generate a local set of goldens using the old @brightspace-ui/visual-diff library.

d2l-test-runner vdiff migrate-local ./**/<location>/*

The optional location parameter can be used to migrate a subset of tests.

This will:

Install the old @brightspace-ui/visual-diff library
Run the old library to generate local goldens. Optional: pass a location to run a subset of tests.
Move the local goldens to their new location in the .vdiff directory

Step 2: Manually Migrate Tests

For each test file being migrated:

Rename the JavaScript file from *.visual-diff.js to *.vdiff.js
Move the imports from the old *.visual-diff.html file to the top of the new JavaScript file. Do not import @brightspace-ui/core's typography.js as it will be included automatically.
Add import { expect, fixture, html } from '@brightspace-ui/testing';
Move the defined fixtures from the HTML file to fixtures in the JavaScript file. Remove ids which were used solely for identifying the fixture for getRect.
Delete the old *.visual-diff.html file
Remove all the old Puppeteer, browser and VisualDiff setup code
Replace async function() { with async () => {
Rewrite everything using page.$eval to access the element directly (just like unit tests do)
Replace calls to getRect and screenshotAndCompare with await expect(elem).to.be.golden(). If rect was not passed in to take a screenshot of the whole page, use await expect(document).to.be.golden() instead.
Make use of the focusElem, clickElem, sendKeysElem, oneEvent, etc. helpers where possible

Before (my-elem.visual-diff.html):

<html>
  <head>
    <script type="module">
      import '@brightspace-ui/core/components/typography/typography.js';
      import '../my-elem.js';
    </script>
  </head>
  <body class="d2l-typography">
    <div>
      <my-elem id="default"></my-elem>
    </div>
  </body>
</html>

Before (my-elem.visual-diff.js):

import puppeteer from 'puppeteer';
import { VisualDiff } from '@brightspace-ui/visual-diff';

describe('my-elem', () => {

  const visualDiff = new VisualDiff('my-elem', import.meta.url);

  let browser, page;

  before(async() => {
    browser = await puppeteer.launch();
    page = await visualDiff.createPage(browser);
    await page.goto(`${visualDiff.getBaseUrl()}/test/my-elem.visual-diff.html`, { waitUntil: ['networkidle0', 'load'] });
    await page.bringToFront();
  });

  beforeEach(async() => {
    await visualDiff.resetFocus(page);
  });

  after(async() => await browser.close());

  it('default', async function() {
    await page.$eval('#default', (elem) => {
      const innerElem = elem.shadowRoot.querySelector('button');
      button.focus();
    });
    const rect = await visualDiff.getRect(page, '#default');
    await visualDiff.screenshotAndCompare(page, this.test.fullTitle(), { clip: rect });
  });

});

After (my-elem.vdiff.js):

import '../my-elem.js'
import { expect, fixture, focusElem, html } from '@brightspace-ui/testing';

describe('my-elem', () => {

  it('default', async() => {
    const elem = await fixture(html`<my-elem></my-elem>`);
    await focusElem(elem.shadowRoot.querySelector('button'));
    await expect(elem).to.be.golden();
  });

});

Much less code, right?

Step 3: Test Migration Locally

Test the migration by running d2l-test-runner vdiff, filtering by file using --filter <file-name> if appropriate.

If tests fail, run d2l-test-runner report to view an HTML report visualizing the differences.

Tips:

If possible, line up the snapshot element size and viewport size to match the old values. This will result in a much clearer picture of any changes - or there may be none at all!
The Daylight fonts and typography are now included automatically. If those weren't added previously, this will result in some font changes. It may be helpful to turn these off with inline styles (font-family: auto; letter-spacing: normal;, maybe color: black; font-size: 20px;) while developing or to get an initial "clean" CI run to verify there are no actual unexpected changes. They can then be removed before merging the new goldens.

Step 4: Migrating CI Goldens

Finally! The last step is to automatically migrate the repo's CI goldens. To do this, run:

d2l-test-runner vdiff migrate ./**/<location>/*

Again, the optional location parameter can be used to migrate a subset of tests.

Commit the changes, which will be limited to goldens being moved/renamed and the addition of .vdiff to the repo's .gitignore.

Follow the instructions to get the new version of the vdiff GitHub Action configured in the repository. Remember that for large repos both the old and new tests can be run together to enable a gradual migration.

Troubleshooting

Ensure that all calls to fixture() in tests are always contained inside it, before or beforeEach blocks. Otherwise the tests will not run and will fail silently.
If a test works when only that file is run (using filter or grep), but not when multiple test files are run at the same time, it's likely highlighting a race condition in the test. Running tests concurrently means each test runs a bit slower.
When in watch mode, if a test is covering the control toolbar (for example, a dialog's backdrop is blocking them) you can run d2lTest.run() in the developer console (d2lTest.runAll() and d2lTest.skip() coming soon)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

migration-guide.md

migration-guide.md

Migration Guide

Unit Tests

Step 1: Adjust Imports

Step 2: Sinon Fake Timers

Step 3: No need to wait for `updateComplete`

Step 4: Use Browser Interaction Helpers

Step 5: Use `d2l-test-runner` Binary

vdiff Tests

Strategy for Large Repositories

Step 1: Automatically Migrate Local Goldens

Step 2: Manually Migrate Tests

Step 3: Test Migration Locally

Step 4: Migrating CI Goldens

Troubleshooting

Files

migration-guide.md

Latest commit

History

migration-guide.md

File metadata and controls

Migration Guide

Unit Tests

Step 1: Adjust Imports

Step 2: Sinon Fake Timers

Step 3: No need to wait for updateComplete

Step 4: Use Browser Interaction Helpers

Step 5: Use d2l-test-runner Binary

vdiff Tests

Strategy for Large Repositories

Step 1: Automatically Migrate Local Goldens

Step 2: Manually Migrate Tests

Step 3: Test Migration Locally

Step 4: Migrating CI Goldens

Troubleshooting

Step 3: No need to wait for `updateComplete`

Step 5: Use `d2l-test-runner` Binary