Storyshots is unable to run tests using Angular 12 or 13 with storybook suite 6.4.9 ot 6.5.0-alpha.4 #17047
Comments
|
seeing the same thing with a React project, sb 6.4.9. In my case I am not using NX |
|
Searching the code for |
|
Looks from searching the code that something should be calling |
|
@SethDavenport I'm convinced now that there is some interaction between Jest and Storybook that isn't happening and that initialization code is something that Jest calls, but isn't in this case. |
|
Yeah that's a possibility - I guess the other thing that changed in the given timeframe is we upgraded from jest 26 to jest 27. However our other (non-monorepo) repos seem to be working with Jest 27. I should note that the problematic repo is a monorepo with yarn workspaces; you mentioned nx - maybe it's something to do with how deps are managed in monorepos? |
|
Yes, I believe monorepos is part of the issue here. |
|
BTW, globalWindow.STORYBOOK_STORY_STORE is something that gets initialized in Storybook and used in Storyshots. That much I've been able to sluth out. Also the |
|
Same issue using react-native FYI |
|
Same deal happens when trying to upgrade to Storybook |
|
@resource11 Are you using Jest 27? |
|
Same issue. |
|
@dbush-clickup, no I'm not. And I'm not using NX |
I don't guarantee this is related to your problem, after I upgraded to the same dependency as you, I found that storyshots test doesn't work anymore because jest changed the default Example: "jest": {
"testEnvironment": "jsdom"
} |
|
@g1eny0ung Thanks for the information. Nrwl sets that for me. I just double-checked. |
|
My issue looks like this: I suspect it's because I have init test env in the //setup-jest.ts
import { getTestBed } from '@angular/core/testing';
import { BrowserDynamicTestingModule, platformBrowserDynamicTesting } from '@angular/platform-browser-dynamic/testing';
getTestBed().initTestEnvironment(BrowserDynamicTestingModule, platformBrowserDynamicTesting());However, the setup above is what we are currently using in Angular 13. Is there anyway I can resolve this error? storybook: 6.5.0-alpha.23. |
|
I have the exact same issue, although my test runner only displays: Also, I've run the same debugging process described already on this thread, and the same promise Here's a reproduction repo/commit: https://github.com/tomasfrancisco/react-component-library-template/tree/10ab634f0a55ea97e3e988a7690095e3d2ec95c1 |
|
happend to me as well with react, when upgrading storybook from 6.4.0 to 6.4.18 |
Interesting. so, it worked on 6.4.0 but sometime since then it broke? |
i try now to bisect to find the version that breaks it |
|
could not find the issue, i think its not directly related to storybook, but i have no idea what causes the issue. |
|
Replacing |
My code is already using module.exports 🤷♂️ We changed our workspace for the mono-repo to just run this set of tests manually for now. Not a great fix but it moves us along. |
|
Apologies, I missed that your repo was a different issue: you were missing a regex for storybook to find your stories PR |
Because it is specified in the demo app. This is a monorepo issue. And, according to all the documentation I've read and how this has worked in the past, the way I have things setup is correct. |
|
Has anyone figured out a workaround for this issue? |
We were able to create a manual test runner for the project that changes the working directory into the project prior to running the tests. In NX it looks like: |
|
This is still broken in the latest versions of both |
|
I was able to fix all of my Nx + Storybook + Storyshots issues with this configuration: import { render } from '@testing-library/react';
import { resolve } from 'path';
import initStoryshots from '@storybook/addon-storyshots';
initStoryshots({
configPath: resolve(__dirname, '../.storybook'),
framework: 'react',
renderer: render,
});The crux of the issue seems to be that by default, storybook/storyshots cannot find the |
|
For what it's worth, none of the solutions suggested in this issue have worked for me. Everything was fine until this change: diff --git a/package.json b/package.json
index 1e4d2ad..f72c103 100644
--- a/package.json
+++ b/package.json
@@ -69,10 +69,10 @@
"@storybook/manager-webpack5": "~6.5",
"@storybook/preset-scss": "~1.0.3",
"@storybook/react": "~6.5.9",
- "@testing-library/jest-dom": "^5.11.4",
+ "@testing-library/jest-dom": "^5.16.4",
"@testing-library/react": "^12.1.2",
"@types/dedent": "^0.7.0",
- "@types/jest": "^26.0.14",
+ "@types/jest": "^28.1.6",
"@types/luxon": "^3.0.0",
"@types/react": "^17.0.13",
"@types/react-dom": "^17.0.8",
@@ -93,8 +93,9 @@
"eslint-plugin-react-hooks": "^4.6.0",
"husky": "^8.0.1",
"identity-obj-proxy": "^3.0.0",
- "jest": "^26.4.2",
- "jest-junit": "^12.0.0",
+ "jest": "^28.1.3",
+ "jest-environment-jsdom": "^28.1.3",
+ "jest-junit": "^14.0.0",
"lint-staged": "^13.0.3",
"markdown-toc": "^1.2.0",
"mkdirp": "^1.0.4",
Files of interest: /** @file main.js */
module.exports = {
core: {
builder: 'webpack5',
},
"stories": [
"../src/**/*.stories.mdx",
"../src/**/*.stories.@(js|jsx|ts|tsx)"
],
"addons": [
'@storybook/preset-scss',
"@storybook/addon-links",
"@storybook/addon-essentials"
],
typescript: {
// We check typescript elsewhere; this should speed up builds
check: false
}
}/** @file preview.js */
import React from 'react';
import {faker} from '@faker-js/faker';
export const decorators = [
(Story) => {
faker.seed(1701);
return <Story />;
},
];
export const parameters = {
actions: {argTypesRegex: '^on[A-Z].*'},
};/** @file storyshots.spec.ts */
import initStoryshots, {
multiSnapshotWithOptions,
} from '@storybook/addon-storyshots';
initStoryshots({
test: multiSnapshotWithOptions(),
});/** @files jest.config.js */
'use strict';
const CI = !!process.env.CI;
module.exports = {
clearMocks: true,
collectCoverage: false,
coverageDirectory: 'reports/coverage',
coverageProvider: 'v8',
moduleNameMapper: {
'\\.(css|scss)$': 'identity-obj-proxy',
'\\.(jpg|jpeg|png|gif|eot|otf|webp|svg|ttf|woff|woff2|mp4|webm|wav|mp3|m4a|aac|oga)$':
'<rootDir>/__mocks__/file-mock.js',
},
reporters: [
'default',
CI && [
'jest-junit',
{
ancestorSeparator: ' › ',
classNameTemplate: '{classname}',
includeConsoleOutput: true,
outputDirectory: 'reports/junit',
outputName: `jest${
process.env.BUILDKITE_JOB_ID ? `+${process.env.BUILDKITE_JOB_ID}` : ''
}.xml`,
suiteName: 'Unit Tests',
titleTemplate: '{title}',
},
],
].filter(Boolean),
setupFilesAfterEnv: ['<rootDir>/jest.d/setup-files-after-env/faker.ts'],
testEnvironment: 'jsdom',
testMatch: [
'**/src/**/__tests__/**/*.[jt]s?(x)',
'**/src/**/?(*.)+(spec|test).[tj]s?(x)',
],
testPathIgnorePatterns: ['/node_modules/'],
transform: {
'^.+\\.stories\\.[tj]sx?$': '@storybook/addon-storyshots/injectFileName',
// babel-jest is loaded automatically when transform is null
// eslint-disable-next-line sort-keys
'^.+\\.[tj]sx?$': 'babel-jest',
},
};/** @file babel.config.js */
'use strict';
module.exports = function (api) {
api.cache.using(
() =>
`${process.env.BABEL_ENV}${process.env.NODE_ENV}${process.env.BUILD_TARGET}`
);
const config = {
comments: true,
presets: [
'@babel/preset-typescript',
[
'@babel/preset-react',
{
development:
process.env.BABEL_ENV === 'development' ||
process.env.NODE_ENV === 'development',
},
],
[
'@babel/preset-env',
{
modules: process.env.BUILD_TARGET === 'modules' ? false : undefined,
targets: process.env.BUILD_TARGET
? 'last 2 versions, > 5%, not dead'
: {node: true},
},
],
],
retainLines: process.env.NODE_ENV !== 'production',
sourceMaps: true,
};
return config;
}; |
|
The future of storyshots is the test-runner: And use the play function for expectations: We will not be making any improvement / changes to storyshots. |
|
@ndelangen Can you expand on that? Looking at the docs for the test-runner, it appears that snapshot testing is something explicitly different. The docs days that the test runner will ensure stories render, but it doesn't say anything about ensuring they render consistently. Will there be a straightforward way to snapshot every story or will it be necessary to right a play function for every story that manually calls a snapshot expectation? Last time I tried the test runner, it was kinda painful to get running because its cli doesn't really behave like most other test runners (I don't remember the details anymore). Are there plans to integrate it into Jest (like Storyshots) or into the new built-in Node test runner? |
|
@ianwremmel sure, we have hooks for the test-runner defined here: These allow you to run some code on the page after the story rendered, like taking real screenshots, or snapshotting the DOM. |
|
Got it, thanks! Any chance there'll be a migration guide for how to replace storyshots hooks? It's not obvious from the hooks docs that you can use them to create a "test". |
|
@ianwremmel thanks for your continued questions! It seems what you want to to integrate your stories into your existing tests? @yannbf created this package a little while ago: Now that's react-specific (might fit your needs still), but our goal is to make this available in all storybook renderers. storybook/code/renderers/react/src/testing-api.ts Lines 98 to 134 in 43f93bb I don't think it supports other renderers, right now? @yannbf What has been clear to the storybook team for some time, is that the default setup of storyshots (taking jest snapshots of stories) is more of an anti-pattern rather then a useful tool, and besides it not being as useful because of that, it's also been a maintenance chore for the storybook team. The testing utils are designed to make stories act as the setup phase of any unit test. I hope this answer helps you. 🙏 |
@ndelangen Kinda? I'd like to keep the current behavior: I write a bunch of stories that are automatically snapshotted without having to write discrete expectations for each story.
Can you expand on that? Is the reasoning documented somewhere? It's definitely been clear that storyshots doesn't get the same level of maintenance as the rest of storybook, but it's been a hugely helpful tool for me to catch unexpected changes. |
|
The issue for us is that nearly all of storybook's code is supposed to run in the browser. Storyshots needs to run in node. We have been trying real hard to improve performance. The implementation of storyshots has been getting in the way many times. We've build the test-runner which is extensible. I have personally found that taking snapshots of stories outputted JSX or html does catches changes, but it doesn't catch all. It gives confidence, but this confidence is easily broken. Stylings changes don't show up, and small html changes that don't matter to the actual user interface show up. Updating snapshots is a breeze, but this is a blessing and a curse. Who reviews all those changed snapshots? I think a visual regression tool on top of storybook is a better solution. |
Yea, Chromatic is great if you can afford it, but it's all but unusable unless you're VC-backed or very profitable.
That's fair, but couldn't playwright take care of this? Doesn't it provide a pretty simple way to capture a DOM subtree? I could definitely be wrong about this, but I thought playwright tests still executed in node, but they sent instructions to a running browser instance.
From what I can tell, the test runner is almost usable for this, but there's no good way to write a "templated test" for lack of a better term. I'd love to be able to write something like forEachStory(Story) => {
describe(Story.name, () => {
it('renders', () => {
expect(() => <Story/>).not.toThrow();
});
it('renders deterministically', () => {
expect(() => <Story/>).toMatchSnapshot();
});
});
}); |
|
oh, looks like I was wrong about that last bit. that's exactly what |
This is exactly what I am looking for: just answering is any of the stories broken? Sort of a smoke/sanity check. And all that without writing tonnes of boilerplate (since Storybook knows about the full list of stories, why not just iterate over them?). So even Chromatic and other kind of tools are more granular and are not exactly in conflict with this kind of basic test: they provide more fine grained checks which not all projects are mature enough to commit to. |
@asinitson You are describing the default behavior of the test-runner. |
Describe the bug
Storyshots is not running tests using Angular 12 or 13 with storybook and addons 6.4.9
To Reproduce
I am using storybook with Nrwl/NX in order to get the Jest testing to work easily. I tried using the repro wizard but that only produced an angular/cli project and did not get me to a point where I could run the storyshots so, I'm supplying the exact steps to produce a minimal project that demonstrates my issue along with a link to a repo with the results of these steps.
The following will recreate my issue:
Steps to reproduce storyshot error using Angular ~13.0.0 and storybook ~6.4.5
Respond to the questions:
workspace=demo-workspace
application=demo
CSS
no Cloud
Add Storybook plugin to NX
Answer the questions:
cypress=n
Auto generate stories=y
Auto generate specs=n
There is a known issue that requires us to rollback the angular version from ~13.1.0 to ~13.0.0. Do so and run
npm installagain.At this point you should be able to run storybook
This solves the spdx* issue that you would otherwise see and maps functions that no longer exists to where they've been moved to.
npm run testErrors on storyshots.spec.js with:
"Your test suite must contain at least one test."
A fully setup project can be found https://github.com/DaveMBush/demo-workspace
System
Please paste the results of
npx sb@next infohere.Environment Info:
System:
OS: Windows 10 10.0.22000
CPU: (16) x64 AMD Ryzen 7 5800X 8-Core Processor
Binaries:
Node: 16.13.1 - C:\Program Files\nodejs\node.EXE
Yarn: 3.1.1 - ~\AppData\Roaming\npm\yarn.CMD
npm: 8.3.0 - C:\Program Files\nodejs\npm.CMD
Browsers:
Chrome: 96.0.4664.110
Edge: Spartan (44.22000.120.0), Chromium (96.0.1054.62)
@storybook/manager-webpack5: ~6.4.5 => 6.4.9
Additional context
I get the same result if I upgrade the storybook packages to 6.5.0-alpha.4
I've added console.log messages to the code and I can see that
globalWindow.__STORYBOOK_STORY_STORE__.initializationPromise.then()never resolves (ie, then() never gets called) and storybook.raw() returns an empty array. My only conclusion is that something else needs to get called before my call to testStorySnapshots() but I am not able to determine what that might be.The text was updated successfully, but these errors were encountered: