Testing Conventions

---

tags: [conventions]

Components should have an automated test for any incoming features or bug fixes. Make sure all tests pass as PRs will not be allowed to merge if there is a single test failure.

We encourage writing expressive test cases and code that indicates intent. Use comments sparingly when the aforementioned can't be fully achieved. Keep it clean!

Please see Stencil's doc for more info on end-to-end testing. See one of our test examples here.

Choosing which tests to write

End-to-end tests

In most cases, when working with components, you will need to write end-to-end tests.

Unit (spec) tests

If you are adding or updating shared utilities or shared modules you should make sure that there are unit tests covering those use cases.

Visual regression tests

You don't have to worry about writing visual regression tests. We cover those automatically using Chromatic in our build pipeline.

Writing Tests

Helpers and utilities

There are helpers and utilities to make testing common workflows easier. commonTests.ts contains common tests that you can import and use for your component. For example, every component should have an accessible test. To use the test in your component, import the helper and assert that the component is accessible.

import { accessible } from "../../tests/commonTests";

it("is accessible", async () =>
  accessible(`<calcite-example>${content}</calcite-example>`));

Here is an example of the helper test usage in accordion.e2e.ts.

There are many more useful, common tests that you should use in specific scenarios. In most IDEs or text editors, you can search the function name of a common test to find usage examples in component's end-to-end test files.

In addition to the common tests, there are also test utilities in utils.ts. These utilities are created to avoid duplicate code in component end-to-end test files. If you find yourself creating the same test function for different components, then it should be moved to utils.ts. A good example is getElementXY. Determining the screen location of an element can be very important when testing interactive components. Here are some end-to-end tests where the utility is used in color-picker.e2e.ts, input-e2e.ts, shell-panel.e2e.ts, and slider.e2e.ts.

Prevent unnecessary logging in the build

This is only necessary if a component's test will produce a lot of console messages in a test run.

As a best practice when writing tests, prevent emitting console warnings by stubbing them. Depending on the tested component, this may also apply to other console APIs.

Console warnings can end up polluting the build output messaging that makes it more difficult to identify real issues. By stubbing console.warn, you can prevent warning messages from displaying in the build. See color.e2e for an example.

Unstable Tests

If you notice that a test fails intermittently during local or CI test runs, it is unstable and must be skipped to avoid holding up test runs, builds and deployments.

To skip a test, use the skip method that's available on tests, or suites and submit a pull request. Once that's done, please create a follow-up issue by choosing the unstable test template and filling it out.

Browser mode guidelines

Accessing elements for tests

Use the following to determine whether you use a locator or query the DOM directly in your test:

• use locators when testing user interactions
• use DOM queries when testing the component’s API

Note

These are general guidelines, not strict rules. Depending on the test, both approaches can still be used. Prefer whatever keeps the test simpler to maintain while remaining stable.

Determining coverage

• Make sure test focuses on the high level fix/feature
• Existing coverage can be omitted in focused tests
  • For example, if there's a test covering when an event is emitted, a test focusing on another aspect does not need to assert on the event
• Unless dealing with event-related state, use vi.waitUntil to assert on visual/DOM state, otherwise, explicitly wait on events

Expected code paths

Tests need to be deterministic, so avoid adding code to handle an incomplete state (e.g., handling missing shadow root when querying internal elements). These cases should be strict and fail.