Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
perf: use split chunks for the node server (#54988)
This PR introduces a change in the Next.js server that should improve memory usage nicely during dev. ## How While investigating the repro cases in #54708, @timneutkens and I noticed that the high memory usage often involved `googleapis`. Digging a bit more, I also saw that in dev, the bundle generated for a page using `googleapis` was around 80MB and that requiring it in Node.js increased memory by 160MB 🥲 and that requiring another page that also used this also increased the memory by 160MB. The problem is that Next.js, on new navigations and hot reloads, might need to load/reload all the code required for the page *all the time*. This issue is also exacerbated by the fact that there's a nasty Node.js bug that makes it impossible to clean that memory up, leading to memory bloat and overall a pretty bad DX. So if we can't clean this up, what can we do about it? The change I'm introducing in this PR is that I'm changing Next.js in order to split the code you're using from `node_modules` from the code you've written in different chunks. The idea is that the heavy code you're loading from `node_modules` is only gonna be loaded once per session this time. This should make a navigation/page reload only load the user bundle now. On my simple test case, the cost of navigation went from ~200MB to ~40MB. A few notes on the implementation: - The chunks for the `node_modules` are split at the module level, this is to ensure that each of the node_modules dependencies is split in the most memory efficient manner. If it was a big monolithic chunk, it would potentially be reloaded again and again whenever reloaded, leading to leakage. - I'm guessing we could do the same for the Edge server - the first load for a page will still be fairly heavy memory wise, there's probably something else we can do there - there's also an issue with the webpack require cache being flushed, whereas it could be reused ## comparisons ### navigating a page before <img width="284" alt="CleanShot 2023-09-04 at 21 00 46@2x" src="https://github.com/vercel/next.js/assets/11064311/44e37df8-4414-4ca1-b6bf-fb0fb11751ea"> after <img width="392" alt="CleanShot 2023-09-04 at 20 58 53@2x" src="https://github.com/vercel/next.js/assets/11064311/46226123-a73a-4132-a99d-fb812e59df46">
- Loading branch information