Persistent caching errors when a packfile is > 2GB #14907

jpm4 · 2021-12-06T02:07:28Z

Bug report

What is the current behavior?

We have a large app in a monorepo. Turning on filesystem caching for the client bundle is fine and shows great speed improvement. With SSR, we have ~100 entrypoints - the first run appears to cache without error, but on warm runs (with no code changes) we've observed errors thrown from a number of different places (seeming to depend on the size and state of master, and perhaps other settings/package bumps). I would guess however that all are the same root cause having to do with the caching going bad somewhere.

Example errors - in each case I'm just posting the first of thousands that spam the console:

<t> [webpack.cache.PackFileCacheStrategy] restore cache content 1 (78.5 MiB): 1139.633175 ms
    [webpack.cache.PackFileCacheStrategy] starting to restore cache content 2 (2.03 GiB) because of request to: Compilation/codeGeneration|/PATH/TO/PROJECT/node_modules/babel-loader/lib/index.js??ruleSet[1].rules[0]!/PATH/TO/PROJECT/packages/react-component/src/components/Component/index.js|ssr-0
<w> [webpack.cache.PackFileCacheStrategy] Restoring failed for Compilation/codeGeneration|/PATH/TO/PROJECT/node_modules/babel-loader/lib/index.js??ruleSet[1].rules[0]!/PATH/TO/PROJECT/packages/react-component/src/components/Component/index.js|ssr-0 from pack: TypeError: Cannot read property 'map' of undefined
    [webpack.cache.PackFileCacheStrategy] TypeError: Cannot read property 'map' of undefined
        at /PATH/TO/PROJECT/node_modules/webpack5/lib/cache/PackFileCacheStrategy.js:787:22

<t> [webpack.cache.PackFileCacheStrategy] restore cache content 3 (112 MiB): 450.47154 ms
    [webpack.cache.PackFileCacheStrategy] starting to restore cache content 4 (6.01 GiB) because of request to: Compilation/codeGeneration|/PATH/TO/PROJECT/node_modules/cache-loader/dist/cjs.js??ruleSet[1].rules[3].oneOf[2].use[0]!/PATH/TO/PROJECT/node_modules/css-loader-for-wp5/dist/cjs.js??ruleSet[1].rules[3].oneOf[2].use[1]!/PATH/TO/PROJECT/node_modules/postcss-loader/dist/cjs.js??ruleSet[1].rules[3].oneOf[2].use[2]!/PATH/TO/PROJECT/node_modules/resolve-url-loader/index.js??ruleSet[1].rules[3].oneOf[2].use[3]!/PATH/TO/PROJECT/node_modules/sass-loader/dist/cjs.js??ruleSet[1].rules[3].oneOf[2].use[4]!/PATH/TO/PROJECT/node_modules/LIB/lib/FILE.scss|ssr-0
<w> [webpack.cache.PackFileCacheStrategy] Restoring failed for Compilation/codeGeneration|/PATH/TO/PROJECT/node_modules/cache-loader/dist/cjs.js??ruleSet[1].rules[3].oneOf[2].use[0]!/PATH/TO/PROJECT/node_modules/css-loader-for-wp5/dist/cjs.js??ruleSet[1].rules[3].oneOf[2].use[1]!/PATH/TO/PROJECT/node_modules/postcss-loader/dist/cjs.js??ruleSet[1].rules[3].oneOf[2].use[2]!/PATH/TO/PROJECT/node_modules/resolve-url-loader/index.js??ruleSet[1].rules[3].oneOf[2].use[3]!/PATH/TO/PROJECT/node_modules/sass-loader/dist/cjs.js??ruleSet[1].rules[3].oneOf[2].use[4]!/PATH/TO/PROJECT/node_modules/LIB/lib/FILE.scss|ssr-0 from pack: Error: Unexpected end of stream
    [webpack.cache.PackFileCacheStrategy] Error: Unexpected end of stream
        at ensureBuffer (/PATH/TO/PROJECT/node_modules/webpack5/lib/serialization/BinaryMiddleware.js:588:11)
        at read (/PATH/TO/PROJECT/node_modules/webpack5/lib/serialization/BinaryMiddleware.js:601:4)
        at Array.<anonymous> (/PATH/TO/PROJECT/node_modules/webpack5/lib/serialization/BinaryMiddleware.js:909:21)
        at BinaryMiddleware._deserialize (/PATH/TO/PROJECT/node_modules/webpack5/lib/serialization/BinaryMiddleware.js:936:26)
        at /PATH/TO/PROJECT/node_modules/webpack5/lib/serialization/BinaryMiddleware.js:557:9
        at /PATH/TO/PROJECT/node_modules/webpack5/lib/serialization/SerializerMiddleware.js:123:27

<t> [webpack.cache.PackFileCacheStrategy] restore cache content 3 (109 MiB): 445.1846 ms
    [webpack.cache.PackFileCacheStrategy] starting to restore cache content 4 (5.94 GiB) because of request to: Compilation/codeGeneration|/PATH/TO/PROJECT/node_modules/cache-loader/dist/cjs.js??ruleSet[1].rules[3].oneOf[2].use[0]!/PATH/TO/PROJECT/node_modules/css-loader-for-wp5/dist/cjs.js??ruleSet[1].rules[3].oneOf[2].use[1]!/PATH/TO/PROJECT/node_modules/postcss-loader/dist/cjs.js??ruleSet[1].rules[3].oneOf[2].use[2]!/PATH/TO/PROJECT/node_modules/resolve-url-loader/index.js??ruleSet[1].rules[3].oneOf[2].use[3]!/PATH/TO/PROJECT/node_modules/sass-loader/dist/cjs.js??ruleSet[1].rules[3].oneOf[2].use[4]!/PATH/TO/PROJECT/packages/react-component/src/components/Icon/Icon.scss|ssr-1
<w> [webpack.cache.PackFileCacheStrategy] Restoring failed for Compilation/codeGeneration|/PATH/TO/PROJECT/node_modules/cache-loader/dist/cjs.js??ruleSet[1].rules[3].oneOf[2].use[0]!/PATH/TO/PROJECT/node_modules/css-loader-for-wp5/dist/cjs.js??ruleSet[1].rules[3].oneOf[2].use[1]!/PATH/TO/PROJECT/node_modules/postcss-loader/dist/cjs.js??ruleSet[1].rules[3].oneOf[2].use[2]!/PATH/TO/PROJECT/node_modules/resolve-url-loader/index.js??ruleSet[1].rules[3].oneOf[2].use[3]!/PATH/TO/PROJECT/node_modules/sass-loader/dist/cjs.js??ruleSet[1].rules[3].oneOf[2].use[4]!/PATH/TO/PROJECT/packages/react-component/src/components/Icon/Icon.scss|ssr-1 from pack: Error: Unexpected end of object at position 1621788
    [webpack.cache.PackFileCacheStrategy] Error: Unexpected end of object at position 1621788
        at decodeValue (/PATH/TO/PROJECT/node_modules/webpack5/lib/serialization/ObjectMiddleware.js:608:12)
        at ObjectMiddleware.deserialize (/PATH/TO/PROJECT/node_modules/webpack5/lib/serialization/ObjectMiddleware.js:720:17)
        at /PATH/TO/PROJECT/node_modules/webpack5/lib/serialization/ObjectMiddleware.js:711:19
        at /PATH/TO/PROJECT/node_modules/webpack5/lib/serialization/SerializerMiddleware.js:123:27

<w> [webpack.cache.PackFileCacheStrategy] Restoring failed for RealContentHashPlugin|generate|page-title.HASH.INCOMPLETE.js from pack: Error: Unexpected header byte 0x1c
    [webpack.cache.PackFileCacheStrategy] Error: Unexpected header byte 0x1c
        at Array.<anonymous> (/PATH/TO/PROJECT/node_modules/webpack5/lib/serialization/BinaryMiddleware.js:917:14)
        at BinaryMiddleware._deserialize (/PATH/TO/PROJECT/node_modules/webpack5/lib/serialization/BinaryMiddleware.js:936:26)
        at /PATH/TO/PROJECT/node_modules/webpack5/lib/serialization/BinaryMiddleware.js:557:9

Believe we also got a TypeError: Cannot read property 'deserialize' of undefined at some point.

I tried bisecting our bundles to see if there was a bad one. As far I can tell there is not, but this did reveal that when we build a "small" enough number of bundles (25-40 out of the 100 depending on size) everything seems to be fine. Finally I tracked down what I think is a big clue; if one of the pack files is > 2GB, that's when we get a problem.

webpack_saved_good_cache/default-production$ ls -s
total 1930004
  67008 0.pack   101596 2.pack     7752 4.pack    35160 index.pack
  98896 1.pack  1571280 3.pack    13168 5.pack    35144 index.pack.old

webpack_saved_bad_cache/default-production$ ls -s
total 6592188
  53128 0.pack    25004 2.pack  6302896 4.pack
  49540 1.pack   114892 3.pack    46712 index.pack

I did experiment with the number of entrypoints to make a packfile just over or under 2GB (the examples above are more extreme) and that seems to be the key factor. Note also in console errors that the preceding starting to restore cache content 2 (2.03 GiB) line is always > 2GB. This all seems like it could be closely related to #12126 and #12191. However it is certainly possible this is a coincidence or red herring.

If the current behavior is a bug, please provide the steps to reproduce.

Given the complexity/size of our app and config I'm not sure I'll be able to provide a repro but I'll give what I can if you need it.

What is the expected behavior?
Persistent caching on a large app should work with no or minimal errors

Other relevant information:
webpack version: 5.62.1 (also tried out 5.64.4)
Node.js version: 14.9.0
Operating System: MacOS

Originally posted by @jpm4 in #14906

The text was updated successfully, but these errors were encountered:

alexander-akait · 2021-12-06T11:23:52Z

@jpm4 When you do initial run, do you have any warnings/errors in console?

jpm4 · 2021-12-06T21:34:45Z

@alexander-akait Here's all the snipped output that has warnings. Only including part of the Serialization timings for the one chunk that gets an "e". All seems okay?

    [webpack.cache.PackFileCacheStrategy] No pack exists at /PATH/TO/PROJECT/node_modules/.cache/webpack/default-production.pack: Error: ENOENT: no such file or directory, stat '/PATH/TO/PROJECT/node_modules/.cache/webpack/default-production/index.pack'
<t> [webpack.cache.PackFileCacheStrategy] restore cache container: 28.606084 ms
(node:27499) [DEP_WEBPACK_CHUNK_ENTRY_MODULE] DeprecationWarning: Chunk.entryModule: Use new ChunkGraph API
(Use `node --trace-deprecation ...` to show where the warning was created)
    [IdleFileCachePlugin] Initial cache was generated and cache will be persisted in 5s.
104 assets
webpack 5.62.1 compiled successfully in 333560 ms
    [webpack.cache.PackFileCacheStrategy] Pack got invalid because of write to: ResolverCachePlugin|normal|dependencyType=|esm|path=|/PATH/TO/PROJECT|request=|/PATH/TO/PROJECT/ssr_build/bundle.entrypoint.js
    [webpack.cache.PackFileCacheStrategy] Storing pack...

...

<i> [webpack.cache.PackFileCacheStrategy] Serialization of 'Compilation/assets|chunk2740': 23.411811 ms
<w> [webpack.cache.PackFileCacheStrategy] Serialization of 'Compilation/assets|chunk7474': 67.347718 ms
<i> [webpack.cache.PackFileCacheStrategy] Serialization of 'Compilation/assets|chunk4404': 33.046051 ms
<i> [webpack.cache.PackFileCacheStrategy] Serialization of 'Compilation/assets|chunk7722': 12.504704 ms
<e> [webpack.cache.PackFileCacheStrategy] Serialization of 'Compilation/assets|chunk5582': 687.817502 ms
<w> [webpack.cache.PackFileCacheStrategy] Serialization of 'Compilation/assets|chunk7577': 116.515566 ms
<w> [webpack.cache.PackFileCacheStrategy] Serialization of 'Compilation/assets|chunk2300': 473.816958 ms
<i> [webpack.cache.PackFileCacheStrategy] Serialization of 'Compilation/assets|chunk7192': 28.35929 ms

...

<t> [webpack.cache.PackFileCacheStrategy] store pack: 95405.54266 ms
    [webpack.cache.PackFileCacheStrategy] Stored pack (199849 items, 5 files, 6392 MiB)
(node:27474) [DEP0005] DeprecationWarning: Buffer() is deprecated due to security and usability issues. Please use the Buffer.alloc(), Buffer.allocUnsafe(), or Buffer.from() methods instead.
(Use `node --trace-deprecation ...` to show where the warning was created)

alexander-akait · 2021-12-07T11:26:46Z

DeprecationWarning: Chunk.entryModule: Use new ChunkGraph API

It can be problem... Can you trace deprecation problems? Because using deprecation API can break cache potentially, maybe some plugin(s) doesn't support webpack5?

jpm4 · 2021-12-09T02:37:16Z

@alexander-akait we resolved the deprecation warning and see the same issues

alexander-akait · 2021-12-09T10:44:50Z

@jpm4 Oh, I see another problem - cache-loader, remove it, cache-loader is not supported by webpack v5, even more you don't need it, webpack v5 has built-in cache, I try to reproduce the problem locally, but no luck, even with very very very big files

alexander-akait · 2021-12-09T10:45:42Z

If you set cache: { type: "filesystem" } you should not use cache-loader, it doesn't even make any sense and will lead to big performance losses

sokra · 2021-12-09T18:45:23Z

This sounds more like a problem in webpack rather than your configuration...

alexander-akait · 2021-12-09T18:50:45Z

I am afraid that cache-loader can break module, we have a lot of dirty solutions in code

jpm4 · 2021-12-09T20:42:33Z

@alexander-akait we removed cache-loader entirely and still see the errors

alexander-akait · 2021-12-09T21:02:45Z

Just for information and ideas, can you run npx webpack-cli info?

jpm4 · 2021-12-09T22:53:49Z

@alexander-akait here it is:

$ yarn run webpack-cli info

  System:
    OS: Linux 5.4 Ubuntu 18.04.5 LTS (Bionic Beaver)
    CPU: (32) x64 Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
    Memory: 36.72 GB / 248.97 GB
  Binaries:
    Node: 14.9.0 - /nail/tmp/xfs-11e0c4c2/node
    Yarn: 3.0.0-rc.12 - /nail/tmp/xfs-11e0c4c2/yarn
    npm: 6.14.8 - /opt/nodejs/node-v14.9.0/bin/npm
  Monorepos:
    Yarn Workspaces: 3.0.0-rc.12
    Lerna: 3.18.3
  Packages:
    babel-plugin-smart-webpack-import: ^2.0.0 => 2.0.0
    loader-utils: ^2.0.0 => 2.0.0
    webpack-cli: ^4.9.1 => 4.9.1

(apologies for OS oversight in initial report, hopefully doesn't all come down to that!)

alexander-akait · 2021-12-09T23:04:13Z

Very strange what you don't have plugins and loaders here, only loader-utils, maybe you have mono repo? If yes, can you go to the main project (where you get the problem with cache) and run the same in this directory?

jpm4 · 2021-12-09T23:09:43Z

@alexander-akait yes this is a monorepo however this is already in the main project root. The full list may be hidden because we wrap webpack? Do you want plugin/loader list and versions?

alexander-akait · 2021-12-09T23:20:55Z

Yes, npx webpack-cli info should collect them, without sensitive information

alexander-akait · 2021-12-09T23:21:57Z

Also there is one interesting thing - Yarn: 3.0.0-rc.12, can you try npm/yarn/yarn v2, just ensure no problems with yarn

jpm4 · 2021-12-10T01:51:29Z

@alexander-akait here are dependencies with webpack or plugin or loader:

    "@loadable/webpack-plugin": "^5.15.1",
    "@mxmul/webpack-s3-plugin": "^1.0.1",
    "@pmmmwh/react-refresh-webpack-plugin": "^0.5.3",
    "@sharkcore/webpack-assets-manifest": "^1.0.0",
    "babel-loader": "^8.2.3",
    "babel-plugin-smart-webpack-import": "^1.5.2",
    "css-loader": "^6.5.1",
    "css-minimizer-webpack-plugin": "^3.2.0",
    "eager-imports-webpack-plugin": "^1.0.0",
    "file-loader": "^6.2.0",
    "mini-css-extract-plugin": "^2.4.5",
    "postcss-loader": "^6.2.1",
    "resolve-url-loader": "^3.1.2",
    "sass-loader": "^12.4.0",
    "style-loader": "^3.3.1",
    "terser-webpack-plugin": "^5.2.5",
    "webpack-assets-manifest": "^5.0.6",
    "webpack-bundle-analyzer": "^4.4.2",
    "webpack-dev-server": "^4.6.0",
    "webpack-inject-plugin": "^1.5.5",
    "webpack-merge": "^4.1.0",
    "webpack-s3-plugin": "^1.2.0-rc.0",
    "webpack": "^5.65.0",
    "webpackbar": "^5.0.2"

Regarding yarn, to clarify - we are not using PnP, just regular node_modules. We could attempt to test a downgrade but might be non-trivial given the number of recent bug fixes etc. we rely on. Is there a specific hypothesis regarding node_modules structure or something similar? I will also look at latest/non-rc version.

alexander-akait · 2021-12-10T11:04:04Z

Is there a specific hypothesis regarding node_modules structure or something similar?

Should be no, but yarn register own hooks on Node.js side, some things can be different, but if you don't use PnP, all should work without problems, anyway will be great to check it.

Does the problem happen only for development or 'production', or for both? Most of plugins look good and up to day. Anyway we can check them too, just try to disable plugins and check it. We should check it out too.

Also can you add console.log to BinaryMiddleware.js:917:14 and check what data is there. This may give ideas too.

jpm4 · 2021-12-11T00:18:09Z

development and production both. Changing settings does produce different errors though, here's a new one that I mentioned but didn't have a full log for above:

<w> (during deserialization of unknown)
<w> (during deserialization of Object)
<w> (during deserialization of webpack/lib/util/registerExternalSerializer webpack-sources/CachedSource)
    [webpack.cache.PackFileCacheStrategy] TypeError: Cannot read property 'deserialize' of undefined
    (during deserialization of unknown)
    (during deserialization of Object)
    (during deserialization of webpack/lib/util/registerExternalSerializer webpack-sources/CachedSource)
        at decodeValue (/PROJ_ROOT/node_modules/webpack5/lib/serialization/ObjectMiddleware.js:660:31)
        at read (/PROJ_ROOT/node_modules/webpack5/lib/serialization/ObjectMiddleware.js:589:12)
        at PlainObjectSerializer.deserialize (/PROJ_ROOT/node_modules/webpack5/lib/serialization/PlainObjectSerializer.js:67:16)
        at decodeValue (/PROJ_ROOT/node_modules/webpack5/lib/serialization/ObjectMiddleware.js:660:31)
        at read (/PROJ_ROOT/node_modules/webpack5/lib/serialization/ObjectMiddleware.js:589:12)
        at CachedSourceSerializer.deserialize (/PROJ_ROOT/node_modules/webpack5/lib/util/registerExternalSerializer.js:58:23)
        at decodeValue (/PROJ_ROOT/node_modules/webpack5/lib/serialization/ObjectMiddleware.js:660:31)
        at ObjectMiddleware.deserialize (/PROJ_ROOT/node_modules/webpack5/lib/serialization/ObjectMiddleware.js:720:17)
        at /PROJ_ROOT/node_modules/webpack5/lib/serialization/ObjectMiddleware.js:711:19
        at /PROJ_ROOT/node_modules/webpack5/lib/serialization/SerializerMiddleware.js:123:27

The line you mentioned logs nothing because I can't repro that particular error at the moment. In the current state the main one I'm seeing is the one from node_modules/webpack5/lib/serialization/BinaryMiddleware.js:588 (pasted in a post above). I logged data and context from there but doesn't seem too useful. Do you want me to drill into any of this or log something else?

data:  [
  <Buffer 02 10 a7 77 65 62 70 61 63 6b 2f 6c 69 62 2f 63 61 63 68 65 2f 50 61 63 6b 46 69 6c 65 43 61 63 68 65 53 74 72 61 74 65 67 79 90 50 61 63 6b 43 6f 6e ... 4294967221 more bytes>,
  <Buffer 2e 65 78 70 6f 72 74 73 2e 70 61 72 73 65 75 6e 64 65 66 69 6e 65 64 31 35 34 38 31 35 36 39 6d 6f 64 75 6c 65 2e 65 78 70 6f 72 74 73 2e 63 6f 6d 70 ... 2315424417 more bytes>
]
context:  {
  filename: '/PROJ_ROOT/node_modules/.cache/webpack/default-production/index.pack',
  extension: '.pack',
  logger: WebpackLogger {
    getChildLogger: [Function (anonymous)],
    [Symbol(webpack logger raw log method)]: [Function (anonymous)],
    [Symbol(webpack logger times)]: Map(1) { 'restore cache content 5 (6.16 GiB)' => [Array] }
  },
  profile: true,
  retainedBuffer: undefined
}

Any particular plugins you would tackle first for disabling?

alexander-akait · 2021-12-11T14:45:32Z

@jpm4 hm, webpack has webpack5 name, it can be problem with non official plugins... if you have webpack v4 and webpack v5 and some of them use require('webpack') and wait webpack v4, it can be problem...

Can you show data buffer (i.e. run toString on each item in data), I want to look what is stored there

jpm4 · 2021-12-14T04:02:24Z

@alexander-akait First item: Uwebpack/lib/cache/PackFileCacheStrategy�PackContentItems

Second item (truncated and sanitized):

gICA<snip tons of characters>iXX0= */"),
    src: src,
    more: props,
    ...

moreReact

...

ReactComponent.defaultProps = {
  default: props
  ...
};
export default ReactComponent;�'

This is all code from a react component library inside the monorepo. This chunk appears to start midway through the module and continues to the end. Anything in particular that could be suspicious in here that I can give more info on? Do you expect this particular piece of data might be a problem, or is this chunk just random?

It might be some time until we can get to your other suggestions with holidays coming up etc.

alexander-akait · 2021-12-14T11:35:19Z

I think we have some unexpected value in data, maybe related to unicode characters or weird characters (it can be generated by babel or webpack itself), this is why the logging looks strange, let's look on another thing, can you show data from https://github.com/webpack/webpack/blob/main/lib/serialization/BinaryMiddleware.js#L226, because it can be long and can contain sensitive data (I want to look at whole data file on unexpected characters), can send me this file at sheo13666q @ gmail . com, if you can please remove any sensitive data if you have - tokens, access keys and etc

sokra · 2021-12-14T16:11:09Z

[webpack.cache.PackFileCacheStrategy] TypeError: Cannot read property 'deserialize' of undefined
(during deserialization of unknown)
(during deserialization of Object)

That unknown is interesting here. It means there is not (de)serializer for an entry in the cache file, but that should actually in an error earlier. So it's probably a relative reference to an already used serializer that was somehow messed up and lead to undefined... So maybe the number was messed up at the 2gb boundary...

sokra · 2021-12-14T16:17:07Z

btw. we have a test case with a 4gb cache file in the test suite that seem to work fine

odunet · 2022-01-10T03:51:45Z

@jpm4 - I have an error that is in many ways similar to yours. Mine is also a large app in a monorepo (GatsbyJS Project). I have thousands of errors in the console only in a warm development build when my pack file is larger than 2GB. Once my pack file goes below 2GB, I have a clean build without errors.

Example Errors - The errors below are similar to yours, and are a few thousands lines:

...
<w> [webpack.cache.PackFileCacheStrategy] Restoring failed for Compilation/modules|/Users/odunet/Documents/company/production_Gatsby/gatsby/node_modules/lodash/merge.js from pack: TypeError: Cannot read property 'length' of undefined
<w> [webpack.cache.PackFileCacheStrategy] Restoring failed for Compilation/modules|external commonjs "/Users/odunet/Documents/companyr/production_Gatsby/gatsby/node_modules/react-dom/server.js" from pack: TypeError: Cannot read property 'length' of undefined
<w> [webpack.cache.PackFileCacheStrategy] Restoring failed for ResolverCachePlugin|normal|dependencyType=|commonjs|path=|/Users/odunet/Documents/company/production_Gatsby/gatsby/node_modules/lodash|request=|./_baseMerge from pack: TypeError: Cannot read property 'length' of undefined
...
...

...
<w> [webpack.cache.PackFileCacheStrategy] Restoring failed for Compilation/codeGeneration|/Users/odunet/Documents/company/production_Gatsby/gatsby/node_modules/gatsby/dist/utils/babel-loader.js??ruleSet[1].rules[2].use[0]!/Users/odunet/Documents/company/production_Gatsby/gatsby/packages/02-functional/shared-components/src/components/stock/stock-card/stock-card-section-wishlist.jsx|render-page from pack: TypeError: Cannot read property 'length' of undefined
<w> [webpack.cache.PackFileCacheStrategy] Restoring failed for Compilation/codeGeneration|/Users/odunet/Documents/company/production_Gatsby/gatsby/node_modules/gatsby/dist/utils/babel-loader.js??ruleSet[1].rules[2].use[0]!/Users/odunet/Documents/company/production_Gatsby/gatsby/packages/02-functional/shared-components/src/components/stock/stock-card/stock-card-section-body.jsx|render-page from pack: TypeError: Cannot read property 'length' of undefined
...

...
<w> [webpack.cache.PackFileCacheStrategy] Restoring failed for ResolverCachePlugin|normal|dependencyType=|commonjs|path=|/Users/odunet/Documents/company/production_Gatsby/gatsby/node_modules/axios/lib/adapters|request=|../core/enhanceError from pack: TypeError: Cannot read property 'length' of undefined
<w> [webpack.cache.PackFileCacheStrategy] Restoring failed for ResolverCachePlugin|normal|dependencyType=|commonjs|path=|/Users/odunet/Documents/company/production_Gatsby/gatsby/node_modules/axios/lib/adapters|request=|./../helpers/cookies from pack: TypeError: Cannot read property 'length' of undefined
...

Other relevant information:

webpack version: 5.65.0
Node.js version: 14.17.6
Operating System: MacOS

Can you please share if you have found the fix for this bug.

alexander-akait · 2022-01-10T10:47:23Z

Yep, sounds like a bug

vankop · 2022-02-11T10:38:56Z

I think I'm reproduced this bug.. or another one..

jpm4 changed the title ~~Persistent caching errors when a packfile is > 2G~~ Persistent caching errors when a packfile is > 2GB Dec 6, 2021

alexander-akait added bug webpack-5 labels Dec 6, 2021

odunet mentioned this issue Jan 17, 2022

Gatsby 4: webpack caching errors when a .pack file is > 2GB gatsbyjs/gatsby#34506

Closed

2 tasks

vankop mentioned this issue Feb 11, 2022

if cache pack is too big, we should batch writing #15373

Merged

sokra closed this as completed in #15373 Feb 15, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Persistent caching errors when a packfile is > 2GB #14907

Persistent caching errors when a packfile is > 2GB #14907

jpm4 commented Dec 6, 2021

alexander-akait commented Dec 6, 2021

jpm4 commented Dec 6, 2021

alexander-akait commented Dec 7, 2021

jpm4 commented Dec 9, 2021

alexander-akait commented Dec 9, 2021 •

edited

alexander-akait commented Dec 9, 2021

sokra commented Dec 9, 2021

alexander-akait commented Dec 9, 2021 •

edited

jpm4 commented Dec 9, 2021

alexander-akait commented Dec 9, 2021

jpm4 commented Dec 9, 2021

alexander-akait commented Dec 9, 2021 •

edited

jpm4 commented Dec 9, 2021

alexander-akait commented Dec 9, 2021

alexander-akait commented Dec 9, 2021

jpm4 commented Dec 10, 2021

alexander-akait commented Dec 10, 2021

jpm4 commented Dec 11, 2021

alexander-akait commented Dec 11, 2021 •

edited

jpm4 commented Dec 14, 2021

alexander-akait commented Dec 14, 2021

sokra commented Dec 14, 2021

sokra commented Dec 14, 2021

odunet commented Jan 10, 2022

alexander-akait commented Jan 10, 2022

vankop commented Feb 11, 2022

Persistent caching errors when a packfile is > 2GB #14907

Persistent caching errors when a packfile is > 2GB #14907

Comments

jpm4 commented Dec 6, 2021

Bug report

alexander-akait commented Dec 6, 2021

jpm4 commented Dec 6, 2021

alexander-akait commented Dec 7, 2021

jpm4 commented Dec 9, 2021

alexander-akait commented Dec 9, 2021 • edited

alexander-akait commented Dec 9, 2021

sokra commented Dec 9, 2021

alexander-akait commented Dec 9, 2021 • edited

jpm4 commented Dec 9, 2021

alexander-akait commented Dec 9, 2021

jpm4 commented Dec 9, 2021

alexander-akait commented Dec 9, 2021 • edited

jpm4 commented Dec 9, 2021

alexander-akait commented Dec 9, 2021

alexander-akait commented Dec 9, 2021

jpm4 commented Dec 10, 2021

alexander-akait commented Dec 10, 2021

jpm4 commented Dec 11, 2021

alexander-akait commented Dec 11, 2021 • edited

jpm4 commented Dec 14, 2021

alexander-akait commented Dec 14, 2021

sokra commented Dec 14, 2021

sokra commented Dec 14, 2021

odunet commented Jan 10, 2022

Other relevant information:

alexander-akait commented Jan 10, 2022

vankop commented Feb 11, 2022

alexander-akait commented Dec 9, 2021 •

edited

alexander-akait commented Dec 9, 2021 •

edited

alexander-akait commented Dec 9, 2021 •

edited

alexander-akait commented Dec 11, 2021 •

edited