Add docs about how to use Ladle and Playwright for local visual snapshot testing

[tajo]

No description provided.

[kosciak9]

What

Hi, we have achieved that (with CI support) in our project. What we have is visual regression from screenshots, automatically generated from meta.json. It's not interacting with UI, but for these sort of validation we have integration tests via Vitest. Snapshots are supposed to catch a major CSS / UI shifts or changes. We believe it's a good time investment / results ratio.

How to

1. Setup Playwright - https://playwright.dev/docs/intro. Nothing special needs to be done for this feature - we have gone with a somewhat default and recommended settings (3 major browsers + iPhone and Pixel emulators).
2. Add code that will extend test with stories list:

import { test as base } from "@playwright/test";
import got from "got";

interface StoriesMeta {
  stories: {
    [key: string]: {
      name: string;
      levels: string[];
    };
  };
}

const LADLE_URL = "http://localhost:61000" as const;

export const test = base.extend<{ storiesMeta: StoriesMeta; ladleUrl: string }>(
  {
    // had to, cannot do (_, use) as first argument has to be destructured (not
    // sure why though)
    /* eslint-disable-next-line @typescript-eslint/no-unused-vars */
    storiesMeta: async ({ page: _ }, use) => {
      const storiesMeta = await got(`${LADLE_URL}/meta.json`)
        .json()
        .catch(() => {
          /* eslint-disable-next-line no-console */
          console.error("Unable to setup stories meta");
        });
      use(storiesMeta as StoriesMeta).catch(() => {
        /* eslint-disable-next-line no-console */
        console.error("Unable to setup stories meta");
      });
    },
    ladleUrl: LADLE_URL,
  }
);

export { expect } from "@playwright/test";

2. Setup Ladle and add any story you want.
3. Add a test - .js extension was due to some TS / ESModule problems I encountered, maybe it will not be required in your environment:

// importing via ".js" extension due to convoluted setup.
// only required in playwright tests!
import { expect, test } from "./test.js";

test("visual regression of Ladle stories", async ({
  page,
  storiesMeta,
  ladleUrl,
}) => {
  for (const storySlug of Object.keys(storiesMeta.stories)) {
    const story = storiesMeta.stories[storySlug];
    await test.step(story.name, async () => {
      // console.log(story, storySlug);

      await page.goto(`${ladleUrl}/?mode=preview&story=${storySlug}`);
      await page.waitForSelector("[data-storyloaded]");
      await expect(page).toHaveScreenshot(`${storySlug}.png`);
    });
  }
});

4. Now every run will open each story and take snapshot of it - then compare with base one. On first run you have to generate base snapshots (this is detailed in Playwright docs).

I would advise to generate snapshots via same docker image as you run your CI - this way you'll not have pixel differences.

5. Run playwright on your CI:

// scripts in package.json
{
  "scripts": {
    "e2e:dev": "playwright test -c src/playwright",
    "e2e:ci": "start-server-and-test 'pnpm ladle dev' http://localhost:61000 'pnpm e2e:dev'"
  }
}

# we use GitLab CI
stories visual regression tests:
  image: mcr.microsoft.com/playwright:v1.23.4-focal
  stage: test
  script:
    - pnpm e2e:ci
  needs:
    - linting
  artifacts:
    when: always
    paths:
      - test-results/**
    expire_in: 2 days

[tajo]

Nice, I have something similar WIP here: #178

Just need to make some updates and write an article about it.

[tajo]

https://ladle.dev/blog/visual-snapshots