pnpm always links to local /node_modules/.registry.npmjs.org/ folder; ignores store-path config

[kiranjholla]

pnpm version: 0.69.4

Expected behavior:

Expected behavior is to have the node modules downloaded into a central path configured as the store-path using pnpm config.

Actual behavior:

Actual behaviour is that the node modules are always downloaded into local folders under the node_modules folder such as /node_modules/.registry.npmjs.org/* and then links are created

/node_modules/package_folder -> /node_modules/.registry.npmjs.org/package_folder

The central store-path also seems to have some files but there are no links to those folders/files.

Additional information:

• node -v prints: v8.1.0
• npm --version prints: 5.0.3
• Windows, OS X, or Linux?: Windows 7
• store-path is set using command pnpm config set store-path "C:\\Dev\\Tools\\.pnpm-store" --userconfig
• pnpm is being run using PowerShell

[zkochan]

If your store is on a different partition then the packages are copied. This is a temporary fix for #712

If the store is on the same partition then what you see are hard links. Hard links look like different files but they are pointing to the same files that are in the central store.

See explanation in FAQ: https://github.com/pnpm/pnpm#why-does-my-node_modules-folder-use-disk-space-if-packages-are-stored-in-a-global-store

Or read the explanation I did in Gitter for a similar question: https://gitter.im/pnpm/pnpm?at=593fa6fd6549436c7d41f4c9

[kiranjholla]

OK, thanks for that explanation but my knowledge about hard-links is obviously lacking. I was confusing how git-bash reports symlinks with hard-links. Apologies for being a noob here.

But can you help me get this straight?? Let's consider a hypothetical:

If I have a 100 MB disk and have 100 copies of a 1 MB file, then I would expect my OS to report a full disk --- all 100 MB used with no more file storage possible.

But if I have 100 hard-links all pointing to the same file, the actual disk usage is just 1 MB; but my OS would still report 100 MB used up.

Would additional file storage be possible in such cases? If not, since the OS thinks the disk is full, then space savings is not really a benefit of pnpm, is it?

[zkochan]

if I have 100 hard-links all pointing to the same file, the actual disk usage is just 1 MB

if you have a 200MB disk when you check the total disk usage, you'll see you have 199MB free disk space. But if you check separately all 100 folders, each will show 1MB used.

If you edit the file in one place, it will be edited in all 100 places. Do you think the system copy/pastes your change? Hard links can be confusing sometimes but they do use the same space on disk

[zkochan]

Would additional file storage be possible in such cases? If not, since the OS thinks the disk is full, then space savings is not really a benefit of pnpm, is it?

yes, the system is smart to understand that. It actually shows the correct disk space statistics when looking at the total disc stats

[kiranjholla]

Thanks for clarifying that. I have learnt something today!

The thing is: yesterday I looked total disk usage with npm installed node_modules across ~5 projects all of which make use of the same package.json, deleted those node_modules folders, and then ran pnpm install on them.

I was expecting to see a net decrease in total disk usage but noticed that the disk usage, as reported by Windows, actually seemed to go up. That's what got me confused.

I will be playing around a bit more with this to try out various scenarios and check this out.

[zkochan]

That might be different. When you remove one hard link, all others continue to be there. Let's say you have central store and two projects A and B. You used pnpm to install deps in project A. You have same files in central store and project A. Now if you remove the central store files will stay in project A. If after that you run pnpm install on project B, you'll get a new store that shares files with project B. But the files will be duplicated in project A.

So removing should be fine unless you remove the central store

[kiranjholla]

Ah, good point. Thanks for the explanation. I will play around a little more and get back.

[dandv]

For those who land here from a web search, the FAQ link has moved to:

https://pnpm.js.org/en/faq#why-does-my-node-modules-folder-use-disk-space-if-packages-are-stored-in-a-global-store