Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Core: Add option to use webpack filesystem cache #16219

Merged
merged 4 commits into from
Nov 15, 2021

Conversation

literalpie
Copy link
Contributor

Issue: #16072

What I did

I have this feature working, but I am not confident that I have done it in the best way. I am open to feedback, or allowing someone else to take over this feature.

How to test

  • I have not included tests because the webpack builders do not have any existing tests.
  • There are no examples in the repo that use webpack5, so I cannot show this in an example. I tested this change manually by updating web-components-kitchen-sink to use webpack5 and adding this config option. I confirmed that a folder node_modules/.cache/webpack is created and includes cache files.
  • I'm not sure where documentation for this would go. I only see mention of 'webpack5' in the migration documentation, but this is not a breaking change, so I don't think this belongs there

@literalpie literalpie marked this pull request as draft October 3, 2021 00:30
@nx-cloud
Copy link

nx-cloud bot commented Oct 3, 2021

Nx Cloud Report

CI ran the following commands for commit 2d08b3f. Click to see the status, the terminal output, and the build insights.

📂 See all runs for this branch

Status Command
#000000 nx run-many --target=prepare --all --parallel --max-parallel=15

Sent with 💌 from NxCloud.

Copy link
Member

@tmeasday tmeasday left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM although I am not an expert of the presets/config part of this.

@tmeasday
Copy link
Member

Perhaps we need an "optimization" section of the docs or something, WDYT @jonniebigodes?

Copy link
Member

@shilman shilman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tmeasday i agree, this is looking great.

i could also imagine an alternative that's consistent with our presets API:

export type BuilderConfig =
  | string
  | {
      name: string;
      options?: unknown;
    };

interface CoreConfig {
  builder?: BuilderConfig;
}

WDYT?

@tmeasday
Copy link
Member

Sure thing!

@jonniebigodes
Copy link
Contributor

@tmeasday yes, indeed we'll need to add something to the docs. @literalpie once this is merged and available, are you ok with following up with me on updating the docs and mentioning this feature?

Let me know and we'll adjust accordingly!

Have a great day!

Stay safe

@literalpie
Copy link
Contributor Author

@jonniebigodes sure, I can do that

@literalpie
Copy link
Contributor Author

@shilman @tmeasday are we still hoping to get this in 6.4? It sounds like an RC is coming soon, but there hasn't been movement on this in a while. I hope this hasn't been held up waiting for me to do something. My understanding is that I'm only expected to add documentation after this gets merged. Is that right? Or should I update the PR to match shilman's comment?

@shilman
Copy link
Member

shilman commented Nov 11, 2021

@literalpie If you can take a stab at my comment, I can't promise we can get it into 6.4 but that's the next step here. @tmeasday probably also has thoughts on this

@literalpie
Copy link
Contributor Author

@shilman I believe I've updated this to do what you asked. I think I even improved the types some beyond what you suggested (but I can match your suggestion exactly if you want).

I tested it in web-component-kitchen-sink again with the new config format, and it works as expected.

Also, if we decide not to include this in 6.4 for any reason, I could always just publish an addon that adds the webpack config.

I've been using the config in my projects for a while, and the only issues I've had are warnings that certain things are not serializable (for example, sass errors in an angular project)

lib/core-server/src/build-static.ts Outdated Show resolved Hide resolved
lib/core-server/src/utils/get-manager-builder.ts Outdated Show resolved Hide resolved
Comment on lines 18 to 22
if ((coreConfig.builder as Webpack5BuilderConfig).options?.fsCache) {
logger.line(1); // force starting new line
logger.info('=> Ignoring Storybook cache because webpack cache is being used');
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmmm, I'm not sure about this. The manager cache is probably a bigger speedup than the WP cache... 🤔

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, in my informal testing, i think the storybook manager cache was about 2 seconds faster than webpack. I thought we might still want to prefer webpack since it's more official, and less code for storybook to maintain.
Let me know if you definitely want to remove it.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm really not sure! Sorry if I was a bit terse, I wasn't necessarily disagreeing with this, just pondering. WDYT @ghengeveld / @shilman?

I agree simplicity is better, I would definitely say we should only have one cache on the manager side, either WP's or our own.

@literalpie
Copy link
Contributor Author

Updated with changes requested by @tmeasday . I didn't remove the skipping of the SB manager cache, since I think that's still an open question.

@tmeasday
Copy link
Member

tmeasday commented Nov 14, 2021

Thanks @literalpie! Will sort it out ASAP.

@tmeasday tmeasday marked this pull request as ready for review November 15, 2021 09:15
@tmeasday
Copy link
Member

OK @literalpie discussed with @shilman and we think it'll be easier and more predictable for us if we just don't use the WP cache in the manager. That way all code paths are the same no matter which WP builder they use with which options.

Copy link
Member

@shilman shilman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good. Great job @literalpie @tmeasday 🎉

@shilman shilman added this to the 6.4 PRs milestone Nov 15, 2021
@literalpie
Copy link
Contributor Author

Sounds good! Thanks for letting me help!

@shilman shilman changed the title Add option to use webpack filesystem cache Core: Add option to use webpack filesystem cache Nov 15, 2021
@shilman shilman merged commit e07fdc7 into storybookjs:next Nov 15, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants