Testing Conventions

---

tags: [conventions]

Components should have an automated test for any incoming features or bug fixes. Make sure all tests pass as PRs will not be allowed to merge if there is a single test failure.

We encourage writing expressive test cases and code that indicates intent. Use comments sparingly when the aforementioned can't be fully achieved. Keep it clean!

Please see Stencil's doc for more info on end-to-end testing. See one of our test examples here.

Choosing which tests to write

End-to-end tests

In most cases, when working with components, you will need to write end-to-end tests.

Unit (spec) tests

If you are adding or updating shared utilities or shared modules you should make sure that there are unit tests covering those use cases.

Visual regression tests

You don't have to worry about writing visual regression tests. We cover those automatically using Chromatic in our build pipeline.

Writing Tests

Important

New tests should be added as browser mode tests instead. Vitest's browser mode is being prioritized to help move away from Puppeteer/Node tests.

Helpers and utilities

There are helpers and utilities to make testing common workflows easier.

commonTests.ts contains helpers for common tests that you can import and use for your component. For example, every component should have an accessible test. To use the test in your component, import the helper and assert that the component is accessible.

import { accessible } from "../../tests/commonTests/browser";

describe("accessible", () => {
  accessible(() => mount(<calcite-example />));
});

import { accessible } from "../../tests/commonTests/browser";

describe("accessible", () => {
  describe("default", () => accessible(() => mount(<calcite-other-example />));

  describe("other mode", () => accessible(() => mount(<calcite-other-example mode="other" />));
});

Important

Each common test should have a parent describe block. When using the same helper more than once, wrap each configuration in its own describe block. This avoids name clashes because test helpers use fixed names for internal tests and groups.

Here is an example of the helper test usage in accordion.e2e.ts.

There are many more useful, common tests that you should use in specific scenarios. In most IDEs or text editors, you can search the function name of a common test to find usage examples in component's end-to-end test files.

In addition to the common tests, there are also test utilities in utils.ts. These utilities are created to avoid duplicate code in component end-to-end test files. If you find yourself creating the same test function for different components, then it should be moved to utils.ts. A good example is getElementXY. Determining the screen location of an element can be very important when testing interactive components. Here are some end-to-end tests where the utility is used in color-picker.e2e.ts, input-e2e.ts, shell-panel.e2e.ts, and slider.e2e.ts.

Prevent unnecessary logging in the build

This is only necessary if a component's test will produce a lot of console messages in a test run.

As a best practice when writing tests, prevent emitting console warnings by stubbing them. Depending on the tested component, this may also apply to other console APIs.

Console warnings can end up polluting the build output messaging that makes it more difficult to identify real issues. By stubbing console.warn, you can prevent warning messages from displaying in the build. See color.e2e for an example.

Unstable Tests

If you notice that a test fails intermittently during local or CI test runs, it is unstable and must be skipped to avoid holding up test runs, builds and deployments.

To skip a test, use the skip method that's available on tests, or suites and submit a pull request. Once that's done, please create a follow-up issue by choosing the unstable test template and filling it out.

Determining Coverage

• Make sure the test focuses on the high level fix or feature.
• Existing coverage can be omitted from focused tests.
  • For example, if another test already verifies that an event is emitted, a test focused on a different behavior does not need to assert on that event.
• Unless validating event driven state, use vi.waitUntil to assert on visual or DOM state. Otherwise, explicitly wait for the relevant event.
• Make sure tests fail when they should so bugs are not missed. Some options:
  1. Write the test first, then implement the fix or feature until the test passes.
  2. Once the test passes, temporarily comment out or revert the fix or feature and verify the test fails.
  3. Copy the test to dev and verify it fails there as well.

Expected code paths

Tests need to be deterministic, so avoid adding code to handle an incomplete state (e.g., handling missing shadow root when querying internal elements). These cases should be strict and fail.

Browser mode guidelines

Accessing elements for tests

Use the following to determine whether you use a locator or query the DOM directly in your test:

• use locators when testing user interactions
• use DOM queries when testing the component’s API

Note

These are general guidelines, not strict rules. Depending on the test, both approaches can still be used. Prefer whatever keeps the test simpler to maintain while remaining stable.

Locators

Whenever possible, use one of the semantic locators provided by Vitest instead of querying by CSS selector with getBySelector.

Note

getBySelector is a custom locator added to ease migration of Node E2E tests, which are selector based.

Isolated testing

In certain edge cases, tests might need to run in complete isolation to ensure coverage. For example, a test may need lazy-loading and custom element registration to occur in a clean environment.

These tests should have their own files using the following naming scheme:

<component>.<test-case>.isolated.browser.e2e.tsx

This ensures the test runs in its own iframe with a clean environment.

Typing

• Leverage generics in DOM APIs to preserve semantics.

  const { el } = await mount(
    <calcite-panel>
      <div id="some-el">Some content</div>
    </calcite-panel>
  );
  
  const someEl = document.querySelector<HTMLDivElement>("#some-el"); // typed as HTMLDivElement | null

Note

TypeScript properly infers the element type when querying by tag.

• Use the non-null assertion operator when an element is known to exist, such as elements provided by the test setup and queried directly after mount.

  const { el } = await mount(
    <calcite-panel>
      <div id="some-el">Some content</div>
    </calcite-panel>
  );
  
  const someEl = el.querySelector<HTMLDivElement>("#some-el")!; // typed as HTMLDivElement

• Avoid defensive code in tests. It can make deterministic setup issues harder to catch.

  const { el } = await mount(
    <calcite-panel>
      <div id="some-el">Some content</div>
    </calcite-panel>
  );
  
  const someEl = el.querySelector<HTMLDivElement>("#some-el")!;
  someEl.parentElement!.disabled = true; // contrived example to demonstrate non-null assertion