Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[RFC] More granular default Webpack chunking #7631

Open
atcastle opened this issue Jun 21, 2019 · 0 comments

Comments

Projects
None yet
2 participants
@atcastle
Copy link

commented Jun 21, 2019

Feature request

Is your feature request related to a problem? Please describe.

The current Webpack chunking strategy in Next.js is based around a ratio-based heuristic for including modules in a single "commons" chunk. Because there is very little granularity, a lot of code is either downloaded unnecessarily (because the commons chunk includes a lot of code that's not actually required for a particular route) or duplicated across multiple page bundles (because it was included in less than half of all pages).

Because these two problems are in opposition, simply adjusting the ratio will always make one better and one worse.

Describe the solution you'd like

I suggest adopting a SplitChunksPlugin configuration such as the following:

cacheGroups: {
    default: false,
    vendors: false,
    framework: {
        name: 'framework',
        chunks: 'all',
        test: /[\\/]node_modules[\\/](react|react-dom)[\\/]/,
        priority: 40
    },
    lib: {
        test(module) {
            return module.size() > 160000
        },
        name(module) {
            return /node_modules\/(.*)/.exec(module.identifier())[1]
                .replace(/\/|\\/g, "_")
        },
        priority: 30,
        minChunks: 1,
        reuseExistingChunk: true
    },
    commons: {
        name: 'commons',
        chunks: 'all',
        minChunks: totalPages,
        priority: 20
    },
    shared: {
	name: false,
        priority: 10,
        minChunks: 2,
        reuseExistingChunk: true
    }               
},
maxInitialRequests: 20

That configuration was designed with these considerations:

Greater granularity

The problems with the current chunking strategy arise from the low granularity of the page-chunk/commons-chunk paradigm. We can easily reduce code duplication by allowing SplitChunksPlugin to create more chunks. In the case of the library shared across 5 entry points, SplitChunksPlugin could simply create a chunk that only contains code shared across those 5 libraries.

Note that SplitChunksPlugin has configuration options to prevent runaway granularity, such as a situation where every module ends up in its own chunk, and the user has to download 100 chunks just to render a route. In particular, in the config above, I'm using the maxInitialRequests option to prevent the number of bundles from skyrocketing.

Cache efficiency

It should be a priority to ensure that shared modules get chunked in a way that causes minimum unnecessary cache invalidation. For an example of this principle, consider an application with 20 entry point chunks. Chunk A and Chunk B both depend on a large library and several small modules. The default SplitChunksPlugin behavior would create a single shared chunk for the intersection of Chunk A and Chunk B, containing the library and modules.

However, in this scenario, all of the following would cache-invalidate the entire chunk, including the large library:

  1. Adding a new module depended on by A and B exclusively (adds module to the chunk)
  2. Adding a new dependency from a different entry point to one of the modules or the library (module moves out of the chunk)
  3. Any change to any of the small modules shared between A and B.

For a sufficiently large library, it makes sense to bundle it all by itself to avoid potential cache invalidations, and the code sample above does exactly that.

A framework chunk

Some modules will be required for every entry point, because they are functionally part of the framework itself, such as React and React-dom. These chunks should be put into a framework chunk, which should be independent from any code introduced by the application developers--even if that code is depended on by 100% of entry points. This is because the frameworks libraries tend to be fairly large, and we know that they generally will not change except when the app developers update the version of Next.js itself. By isolating the framework code we ensure that it will not be cache-invalidated by irrelevant changes made to application code.

Describe alternatives you've considered

I've considered smaller values for maxInitialRequests, but for Next.js applications with a very large number of pages (100+) that can cause a size regression. At 20 that issue goes away. I believe that even at maxInitialRequests 20, the majority of NextJS sites will still have initial requests in the single digits. It's also not clear to me that even if it the initial requests went up to 20 that it would cause any performance degradation for users on modern browsers.

Additional context

In my initial testing on some small and medium sized Next.js apps, this enhanced webpack configuration reduces KB of total JS required for initial page load by as much as 20%. For particularly-poorly optimized sites, I think the improvement could be even greater.

@timneutkens timneutkens changed the title [RFC] More granular default Webpack [RFC] More granular default Webpack chunking Jun 22, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.