Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PoC for component testing #1137

Closed
wants to merge 3 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -81,6 +81,7 @@ dist/
/backstop_data/bitmaps_test/
/backstop_data/bitmaps_reference/
/backstop_data/html_report/
/screenshots/**/failed/**

# GitHub Actions #
###############
Expand Down
39 changes: 38 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,44 @@ To contribute to Stacks documentation or its CSS library, you’ll need to build
Having trouble getting these steps to work? Open [an issue](https://github.com/StackExchange/Stacks/issues/new) with a `setup` label.

## Testing Stacks
Stacks has implemented visual regression testing with [Backstop](https://github.com/garris/BackstopJS). To test if your new feature introduces visual regressions, run `npm run test` in a new Terminal window while Stacks is running. After the tests have run, a new browser window with any regressions will show. If the regressions are desired, you can run `npm run approve` to establish the new baseline.

### Component Tests
🚧 Work in Progress 🚧

Component tests are written with [DOM Testing Library](https://testing-library.com/docs/dom-testing-library/intro).
Please follow the library's principles and documentation to write tests.

Stacks uses [Web Test Runner](https://modern-web.dev/docs/test-runner/overview/) and [Playwright](https://modern-web.dev/docs/test-runner/browser-launchers/playwright/) to run tests in a real browser context

Execute the component tests suite via running:
```sh
npm test
```
or if you prefer watch mode run:
```sh
npm run test:watch
```

### Visual Regression Tests
🚧 Work in Progress 🚧

This [Web Test Runner plugin](https://www.npmjs.com/package/@web/test-runner-visual-regression) is used to run visual regression tests. [DOM Testing Library](https://testing-library.com/docs/dom-testing-library/intro).
Visual regression tests end with this suffix `*.visual.test.ts`.

Execute the visual regression tests suite via running:
```sh
npm run test:visual
```

Update the visual baseline via:
```sh
npm run test:visual:update
```

Failing tests (including diffs) can be found under `screenshots/[browser]/failed/` folders.

### Visual Regression Tests (Legacy)
Stacks has implemented visual regression testing with [Backstop](https://github.com/garris/BackstopJS). To test if your new feature introduces visual regressions, run `npm run test:regression` in a new Terminal window while Stacks is running. After the tests have run, a new browser window with any regressions will show. If the regressions are desired, you can run `npm run approve` to establish the new baseline.

Individual routes to test are found in [backstop.json](/backstop.json)

Expand Down
62 changes: 62 additions & 0 deletions lib/test/tooltip.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
import { html, fixture, expect } from "@open-wc/testing";
import { screen, waitForElementToBeRemoved } from "@testing-library/dom";
import userEvent from "@testing-library/user-event";
import "../ts/index";

const user = userEvent.setup();

describe("tooltip", () => {
it("should make the tooltip element visible on hover (after a delay)", async () => {
const trigger = await fixture(html`
<button
class="s-btn s-btn__filled"
role="button"
data-controller="s-tooltip"
title="tooltip content"
data-s-tooltip-placement="bottom-start"
>
Hover tooltip popover
</button>
`);

expect(screen.queryByRole("tooltip")).to.be.null;

await user.hover(trigger);
const tooltip = await screen.findByRole("tooltip");
expect(tooltip).to.be.visible;

await user.unhover(trigger);
await waitForElementToBeRemoved(() => screen.queryByRole("tooltip"));
Comment on lines +22 to +29
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tested out both the s-tooltip.ts and s-tooltip-1.3.0.ts being imported and looks like the tests pass/fail as we'd expect 🎉

With that said, I'm not sure why this one fails when using s-tooltip-1.3.0.ts

        await user.hover(trigger);
        const tooltip = await screen.findByRole("tooltip");
        expect(tooltip).to.be.visible;

I would expect tooltip to be visible under these circumstances. Can you help me understand why this fails? t's very likely that I'm missing something super obvious but I wanna make sure I understand this test and that it's behaving as we expect for the reasons we'd expect.

Copy link
Contributor Author

@giamir giamir Oct 27, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you look at the generated output you can spot that in this version we had another a11y bug where we were not updating correctly the aria-hidden attribute. When the user hovered on the tooltip aria-hidden remained set to true: you can check this also in the code pen you shared with me.
Perhaps this is a confirmation bias but this, in my opinion, highlights how powerful the testing library selectors are to spot a11y problems too.
This is the error output for me:

TestingLibraryElementError: Unable to find role="tooltip"

      Ignored nodes: comments, script, style
      <body>


        <div>


          <button
            aria-describedby="--stacks-s-tooltip-85tbv7uk"
            class="s-btn s-btn__filled"
            data-controller="s-tooltip"
            data-s-tooltip-placement="bottom-start"
            role="button"
          >

                      Hover tooltip popover

          </button>
          <div
            aria-hidden="true"
            class="s-popover s-popover__tooltip pe-none is-visible"
            data-popper-placement="bottom-start"
            id="--stacks-s-tooltip-85tbv7uk"
            role="tooltip"
            style="position: absolute; inset: 0px auto auto 0px; margin: 0px; transform: translate(0px, 48px);"
          >
            tooltip content
            <div
              class="s-popover--arrow"
              style="position: absolute; left: 0px; transform: translate(70px, 0px);"
            />
          </div>


        </div>
      </body>

You can see there that the aria-hidden is set to true and it should not. This is the same exact behavior we had for users until your refactor I think. 🙂

});

it("should not flicker when the host contains an SVG and the user hover on it", async () => {
await fixture(html`
<button
class="s-btn"
role="button"
title="tooltip content"
data-controller="s-tooltip"
data-s-tooltip-placement="bottom-start"
>
<svg
data-testid="svg"
aria-hidden="true"
class="bg-red-500"
width="18"
height="18"
viewBox="0 0 18 18"
></svg>
👈 Shouldn't flicker 🤷‍
</button>
`);
const button = screen.getByRole("button");
const svg = screen.getByTestId("svg");

await user.hover(button);
const tooltip = await screen.findByRole("tooltip");
expect(tooltip).to.be.visible;

await user.hover(svg);
expect(tooltip).to.be.visible;
});
});
31 changes: 31 additions & 0 deletions lib/test/tooltip.visual.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
import { html, fixture } from "@open-wc/testing";
import { screen } from "@testing-library/dom";
import userEvent from "@testing-library/user-event";
import { visualDiff } from "@web/test-runner-visual-regression";
import "../ts/index";

const user = userEvent.setup();

describe("tooltip", () => {
it("should not introduce visual regressions", async () => {
const wrapper = await fixture(html`
<div style="height: 100px; display: inline-block;">
<button
class="s-btn s-btn__filled"
role="button"
data-controller="s-tooltip"
title="tooltip content"
data-s-tooltip-placement="bottom-start"
>
Hover tooltip popover
</button>
</div>
`);

const trigger = screen.getByRole("button");
await user.hover(trigger);
await screen.findByRole("tooltip");

await visualDiff(wrapper, "tooltip");
});
});
27 changes: 20 additions & 7 deletions lib/ts/controllers/index.ts
Original file line number Diff line number Diff line change
@@ -1,8 +1,21 @@
// export all controllers *with helpers* so they can be bulk re-exported by the package entry point
export { ExpandableController } from './s-expandable-control';
export { hideModal, ModalController, showModal } from './s-modal';
export { TabListController } from './s-navigation-tablist';
export { attachPopover, detachPopover, hidePopover, BasePopoverController, PopoverController, showPopover } from './s-popover';
export { TableController } from './s-table';
export { setTooltipHtml, setTooltipText, TooltipController } from './s-tooltip';
export { UploaderController } from './s-uploader';
export { ExpandableController } from "./s-expandable-control";
export { hideModal, ModalController, showModal } from "./s-modal";
export { TabListController } from "./s-navigation-tablist";
export {
attachPopover,
detachPopover,
hidePopover,
BasePopoverController,
PopoverController,
showPopover,
} from "./s-popover";
export { TableController } from "./s-table";
export { setTooltipHtml, setTooltipText, TooltipController } from "./s-tooltip";
// uncomment and comment the line above to run the tests against version 1.3.0 of the tooltip and see them failing
// export {
// setTooltipHtml,
// setTooltipText,
// TooltipController,
// } from "./s-tooltip-1.3.0";
export { UploaderController } from "./s-uploader";