Test workflow is failing with Jest snapshot errors

[mohanraj-r]

Test workflow is failing with Jest snapshot errors saying they are not up-to-date - but the updated snapshots have been checked in and the tests pass locally after a clean build.

Github CI run

$ jest --coverage --ci
...
Snapshot Summary
 › 11 snapshots failed from 4 test suites. Inspect your code changes or run `yarn test -u` to update them.

Test Suites: 4 failed, 3 passed, 7 total
Tests:       11 failed, 27 passed, 38 total
Snapshots:   11 failed, 7 passed, 18 total
Time:        5.62 s

Local run of the same branch with no other changes

$ jest --coverage --ci
...
Test Suites: 7 passed, 7 total
Tests:       38 passed, 38 total
Snapshots:   18 passed, 18 total
Time:        5.852 s

• checked out the branch to a diff clean location in my machine and tried out the CI commands - no issues 😕
• tried running the build locally in ubuntu container - runs fine without any issues
• switching to mac-os from ubuntu for the workflow doesn't fix the issue :(
• Resolved this issue by switching to CircleCi. Tests all pass now.

Reporting here because I like Github actions and would like to continue using it.

[bryanmacfarlane]

Not sure it's an issue with setup-node (since it just installs node) but I'll keep it open to investigate. Might also be a VM / hosted pool machine issue? I did see this before on one of our repos ...

[semoal]

I'm having a similar issue with typescript, same node version, npm, and typescript on my local machine than the ci machine, when i run the ci, i'm seeing errors that on my local machine doesn't appear.

[rodrigondec]

+1

[premkumar2306]

+1

[rodrigondec]

Found a workaround. My jest was failing because I was using a arrow funcion on my vue template.

The arrow function was translated wrongly by babel istanbul plugin. Replacing it with a component method worked 😅

Ref to similar situation:

• Handle function name inference istanbuljs/babel-plugin-istanbul#63

[BjornGrylls]

Our local node version is version 15.2. All our tests passes.
Github tests version 10.x, 12.x and 14.x. None of Github tests passes.
We then changed Github to only test version 15.x. Now all tests passes.

[m4rvr]

Same problem here. Works locally but somehow fails when running the jobs. It looks like GitHub adds some extra spaces or lines somehow?

EDIT:
Works now. I use Windows were the generated snapshot contained backslashes, but the actions use slashes. Didn't notice that. :)

[misterrodger]

I got it to work by matching the runs-on environment and version to my local, as well as matching the node-version to my local. So for me, this worked:

name: Node Continuous Integration

on:
  push:
    branches: [master]

jobs:
  run_tests:
    runs-on: macos-latest
    steps:
      - uses: actions/checkout@v2
      - uses: actions/setup-node@v2.1.5
        with:
          node-version: 14.x
      - run: npm ci
      - run: npm test
      - run: npm run build

[elliedori]

@misterrodger You may be aware of this, but running Github Actions with a macOS runner costs 10x that of using the Linux runner. It's awesome you found a fix for your issue, but worth calling out the cost difference - you'll eat up your Github minutes much, much faster.

https://docs.github.com/en/github/setting-up-and-managing-billing-and-payments-on-github/about-billing-for-github-actions

[kara-ryli]

FWIW I was also debugging this for my CI build. My npm test just ran this:

{
  "scripts": {
    "test": "tsc && jest -u && git diff && git status && git reset --hard && jest"
  }
}

What was interesting was the git status:

2021-09-15T04:46:36.8432929Z Changes not staged for commit:
2021-09-15T04:46:36.8433672Z   (use "git add/rm <file>..." to update what will be committed)
2021-09-15T04:46:36.8434485Z   (use "git restore <file>..." to discard changes in working directory)
2021-09-15T04:46:36.8435533Z 	deleted:    src/components/__snapshots__/header.spec.tsx.snap
2021-09-15T04:46:36.8436063Z 
2021-09-15T04:46:36.8436527Z Untracked files:
2021-09-15T04:46:36.8437176Z   (use "git add <file>..." to include in what will be committed)
2021-09-15T04:46:36.8437965Z 	src/components/__snapshots__/Header.spec.tsx.snap
2021-09-15T04:46:36.8438450Z 

So I'm guessing this issue is a bug in jest related to casing -- as you can see, the version of Jest I created the snapshot with downcast the file name, even though the tested file was capitalized.

It's very similar to jestjs/jest#9865 and jestjs/jest#8900 but I think it's its own issue.

The good news (for me at least), was that fixing the casing discrepancy fixed the issue, and the latest version of Jest generated files with the correct casing (could have been updating, could have been me switching from Intel to ARM for dev -- who knows). So if you come to this issue via search, I'd check out these possible solutions.

[shubhanus]

In case someone has renamed the filenames that may not been updated Linux takes things case sensitive by mac os doesn't
Either remove snapshots and run all test cases or manually change snapshot file name

[M-Scott-Lassiter]

I am having a similar issue (failed test run link). If I should open a new issue instead of adding on here, please let me know.

It appears the issue might be caused by the colored formatting on the error messages?

I posted a StackOverflow question, if I get an answer that helps fix my problem I'll report back.

[IvanZosimov]

📟 Just a gentle ping, @M-Scott-Lassiter 👋 Have you fixed your problem?