How to unify fonts and color schemes across systems in the context of snapshot testing?

[jwedel]

This is not directly a "bug report" but rather a question for best practices. Maybe the answer could be added to the README.

What we've seen is, that between Windows, Mac and Linux, there are differences in

• Used Default Font (Windows -> Arial, Mac -> Helvetica Neue, Linux -> ?)
• Color Scheme (Same rendered pixel has different RGB values)

As Mac, Linux and Windows don't have a common default font, it's pretty hard to run the snapshot comparison on all environments with the same result, obviously. Also, on Mac there is the ColorSync which changes the RGB values of a rendered pixel.

Both leads to failing snapshot tests.

When increasing the failureThreashold, it starts missing real issues. E.g. changing the color of a curve in a chart will not be detected anymore.

Are there any configuration options known, maybe for Cypress or Chrome that will lead to similar rendering results on all systems?

Thanks!

[simonsmith]

Hey @jwedel, I moved it to a discussion

Have you tried Docker for this? I've had a similar issue on my projects (difference between Mac and Linux CI machines) and I generate my snapshots using run mode in Docker. I then rely on open mode for testing

[jwedel]

Thanks so much for the detailed comment! I will certainly try it!

[jwedel]

@simonsmith I just tried it and ran into one issue. How do you connect Cypress to the host? Also the linked article does not cover this. Do you start your app outside the container and than expose the host network or do you run the app inside the container?

[simonsmith]

Usually it's triggered from inside the container, so it might run a yarn script that starts a mock server and then runs the tests. Here's an example of what those scripts look like:

{
    "cy:run": "cypress run",
    "test:e2e:run": "LOG_LEVEL=silent npm-run-all --parallel --race start:e2e start:e2e:server 'cy:run {@}' --",
    "start:e2e": "API_URI=localhost:3069 yarn start",
    "start:e2e:server": "ts-node cypress/server/e2e.ts"
}

Our CLI script uses the following function to run Docker:

docker_run() {
  exec \
    docker run -it --rm \
      --network host \
      -e CYPRESS_updateSnapshots=$CYPRESS_updateSnapshots \
      -v $PWD:/e2e \
      -w /e2e \
      --user 1000:1000 \
      --entrypoint="yarn" \
      $CYPRESS_IMAGE \
      $@
}

And then later:

docker_run test:e2e $@

So the yarn scripts are started in the container itself

Looks like the --network flag is what you need?

[jwedel]

For anyone interested:

We could not get it to run the app inside the container as we use nx and nx apparently uses platform-specific binaries. So if you do

• yarn install (Mac)
• yarn run start:mocks (Inside Linux container)

The latter fails. As we did not want to run a yarn install inside the container, we were looking for alternatives. We got it running by changing the baseUrl (--config baseUrl=http://host.docker.internal:4200) to connect to the host port 4200 where our application runs.

So we start the app on the host and then use the cypress base image to run the tests inside the container.

The created snapshots now work pretty well also on CI where Linux is used.

So, big thanks again to @simonsmith for the valuable hints!

[simonsmith]

@jwedel Great to hear! Thanks for adding your findings too 👌🏻