New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pnpm always links to local /node_modules/.registry.npmjs.org/ folder; ignores store-path config #794

Closed
kiranjholla opened this Issue Jun 14, 2017 · 7 comments

Comments

Projects
None yet
2 participants
@kiranjholla

kiranjholla commented Jun 14, 2017

pnpm version: 0.69.4

Expected behavior:

Expected behavior is to have the node modules downloaded into a central path configured as the store-path using pnpm config.

Actual behavior:

Actual behaviour is that the node modules are always downloaded into local folders under the node_modules folder such as /node_modules/.registry.npmjs.org/* and then links are created

/node_modules/package_folder -> /node_modules/.registry.npmjs.org/package_folder

The central store-path also seems to have some files but there are no links to those folders/files.

Additional information:

  • node -v prints: v8.1.0
  • npm --version prints: 5.0.3
  • Windows, OS X, or Linux?: Windows 7
  • store-path is set using command pnpm config set store-path "C:\\Dev\\Tools\\.pnpm-store" --userconfig
  • pnpm is being run using PowerShell
@zkochan

This comment has been minimized.

Member

zkochan commented Jun 14, 2017

If your store is on a different partition then the packages are copied. This is a temporary fix for #712

If the store is on the same partition then what you see are hard links. Hard links look like different files but they are pointing to the same files that are in the central store.

See explanation in FAQ: https://github.com/pnpm/pnpm#why-does-my-node_modules-folder-use-disk-space-if-packages-are-stored-in-a-global-store

Or read the explanation I did in Gitter for a similar question: https://gitter.im/pnpm/pnpm?at=593fa6fd6549436c7d41f4c9

@kiranjholla

This comment has been minimized.

kiranjholla commented Jun 14, 2017

OK, thanks for that explanation but my knowledge about hard-links is obviously lacking. I was confusing how git-bash reports symlinks with hard-links. Apologies for being a noob here.

But can you help me get this straight?? Let's consider a hypothetical:

If I have a 100 MB disk and have 100 copies of a 1 MB file, then I would expect my OS to report a full disk --- all 100 MB used with no more file storage possible.

But if I have 100 hard-links all pointing to the same file, the actual disk usage is just 1 MB; but my OS would still report 100 MB used up.

Would additional file storage be possible in such cases? If not, since the OS thinks the disk is full, then space savings is not really a benefit of pnpm, is it?

@zkochan

This comment has been minimized.

Member

zkochan commented Jun 14, 2017

if I have 100 hard-links all pointing to the same file, the actual disk usage is just 1 MB

if you have a 200MB disk when you check the total disk usage, you'll see you have 199MB free disk space. But if you check separately all 100 folders, each will show 1MB used.

If you edit the file in one place, it will be edited in all 100 places. Do you think the system copy/pastes your change? Hard links can be confusing sometimes but they do use the same space on disk

@zkochan

This comment has been minimized.

Member

zkochan commented Jun 14, 2017

Would additional file storage be possible in such cases? If not, since the OS thinks the disk is full, then space savings is not really a benefit of pnpm, is it?

yes, the system is smart to understand that. It actually shows the correct disk space statistics when looking at the total disc stats

@kiranjholla

This comment has been minimized.

kiranjholla commented Jun 14, 2017

Thanks for clarifying that. I have learnt something today!

The thing is: yesterday I looked total disk usage with npm installed node_modules across ~5 projects all of which make use of the same package.json, deleted those node_modules folders, and then ran pnpm install on them.

I was expecting to see a net decrease in total disk usage but noticed that the disk usage, as reported by Windows, actually seemed to go up. That's what got me confused.

I will be playing around a bit more with this to try out various scenarios and check this out.

@zkochan

This comment has been minimized.

Member

zkochan commented Jun 14, 2017

That might be different. When you remove one hard link, all others continue to be there. Let's say you have central store and two projects A and B. You used pnpm to install deps in project A. You have same files in central store and project A. Now if you remove the central store files will stay in project A. If after that you run pnpm install on project B, you'll get a new store that shares files with project B. But the files will be duplicated in project A.

So removing should be fine unless you remove the central store

@kiranjholla

This comment has been minimized.

kiranjholla commented Jun 14, 2017

Ah, good point. Thanks for the explanation. I will play around a little more and get back.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment