Install of packages like "npm -i typescript fail" with invalid readableHighWaterMark

[madbiker88]

• Version: 10.16.3
• Platform: Win 64
• Subsystem:nodejs\node_modules\npm\make-fetch-happen\cache.js

This relates with any project installing typescript using
npm install typescript (assuming typescript has not already been installed into the current project)
or alternatively test by creating any new react project with the typescript flag using
create-react-app my-app --typescript

which gave the error:
npm ERR! code ERR_INVALID_OPT_VALUE
npm ERR! The value "false" is invalid for option "readableHighWaterMark"

The issue for me regularly occurs with larger packages like typescript (>5MB) because they are larger than const MAX_MEM_SIZE = 5 * 1024 * 1024 // 5MB set inside node_modules/node/make-fetch-happen.js

My dirty fix and local was to edit inside
…nodejs\node_modules\npm\make-fetch-happen\cache.js - line 12

change
const MAX_MEM_SIZE = 5 * 1024 * 1024 // 5MB to become
const MAX_MEM_SIZE = 25 * 1024 * 1024 // 25MB my temp fix to allow large downloads

Now npm install typescript will work!!!

Clearly the key issue is inside node in how it is handling cache size - either Node should increase the default MAX_MEM_SIZE or for correctly code for a input that is larger than the cache rather than issue that gives an invalid value for readableHighWaterMark

[mscdex]

This issue tracker is for reporting bugs in node core and submitting feature requests for node core.

General help questions should be posted to the nodejs/help issue tracker instead.

Issues with third-party modules, npm, or other tools that use node, should be posted to the appropriate issue tracker for that project, unless it can be proven that the issue is in fact with node core and not the module/tool in question.

[bnoordhuis]

Clearly the key issue is inside node in how it is handling cache size

To clarify: the bug is inside npm and that's a dependency that is bundled with node but isn't maintained by us.

I'm pretty sure the bug is on this line:

node/deps/npm/node_modules/make-fetch-happen/cache.js

Line 176 in 75657ee

const newBody = through({highWaterMark: fitInMemory && MAX_MEM_SIZE})

[lmeyerov]

Been hitting this on ubuntu for the last week or so:

 amazon-ebs: + node --version
    amazon-ebs: v12.11.0
    amazon-ebs: + which npm
    amazon-ebs: /usr/local/bin/npm
    amazon-ebs: + npm --version
    amazon-ebs: 6.11.3
amazon-ebs: + npm i -g typescript@3.4.3
    amazon-ebs: ~~~~~~ Installing typescript ~~~~~
    amazon-ebs: npm ERR! code ERR_INVALID_OPT_VALUE
    amazon-ebs: npm ERR! The value "false" is invalid for option "readableHighWaterMark"

Not sure how to fix so far.

[fljmayer]

So who maintains this file? We also have this problem, all builds are failing. We really need a solution.

[darcyclarke]

@fljmayer @lmeyerov @madbiker88 Just a quick update here, I've made two PRs against the v5 & v6 versions of make-fetch-happen to hopefully resolve this problem. highWaterMark should only ever get a number and, as @bnoordhuis noted, there happened to be instances when a conditional was passed instead.

We should be able to get this fix out in the next minor release of npm (which is currently queued up to go out Tuesday, November 5th).

[lmeyerov]

Cool, been watching. We worked around locally but not super hard to test.

[farvour]

@fljmayer @lmeyerov @madbiker88 Just a quick update here, I've made two PRs against the v5 & v6 versions of make-fetch-happen to hopefully resolve this problem. highWaterMark should only ever get a number and, as @bnoordhuis noted, there happened to be instances when a conditional was passed instead.

We should be able to get this fix out in the next minor release of npm (which is currently queued up to go out Tuesday, November 5th).

So... not going out November 5th..? 🤔 It looks like Ubuntu 20.04 Focal either has a really old version of this npm module or it isn't in master. So I'm not sure here. Other than the hard-coded quick hack, that seems to get past it...