conditional .npmignore entries

[Albert-IV]

While fixing agenda/agenda#106, there was a conversation about the proper use of .npmignore.

Looking at the questionably named dmn package, some repository owners like projects to exclude their tests when doing a normal npm install, which makes sense.

After removing the tests for normal installs, it was brought to my attention that you can set npm to auto-test packages as you install them when npat is set to true. However when doing so on packages that have ignored their tests will cause errors when installing.

My suggestion would be to ignore the .npmignore entries when you have npat set to true. I think this will allow user-land modules to handle both people who want tests to auto-run and those who want vanilla installs to be as small as possible.

Thoughts?

I might be totally off on the use .npmignore here, but I do think it would help everyone. If I'm using the ignore file incorrectly, feel free to let me know!

[othiym23]

How would this work when npat is an option used at install time, but the entries in .npmignore are used when building the package tarball at publish time? People add tests to .npmignore to make their published packages smaller, so we can't just add everything at publish time and then figure out how much to take out of the package at install time (without defeating the purposes of them adding tests to the ignore file in the first place).

[Albert-IV]

It's super easy just make two tarballs! The 100k packages can just be republished at the drop of a hat right?

All joking aside, you bring up a good point. Adding the entire package in the tarball won't really work, as then there would be significant overhead in reading the archive, extracting, and re-archiving. I guess in my mind the idea would be to grab whatever was designated in the repository field, falling back to the npm hosted tarballs if it fails for some reason (lack of git, site hosting is down, etc.)

I don't think this is the perfect solution, however I don't know how a package maintainer can appease those two crowds of developers (those who want barebones npm installs vs. those who want tests and fixtures in the package for automated dependency testing).

[othiym23]

npm as a tool and a registry doesn't require that you use Git (or any version control), so depending on the repository field to selectively pull down packages is a significant restriction to put on the feature. Also, at least in its current form, cloning a git repository is significantly more resource-intensive than just requesting the tarball from the registry, as you have to clone the repository and then check out a working tree.

If this feature were to work, we'd probably want to split packages up into two pieces – one with the core module, another with extras like tests, documentation, and other assets not necessary for regular component use. That would be pretty cool, but it'd also be a big change for all of npm, and I don't think the use cases are important enough to justify that level of change. It's a feature worth keeping in mind!

[othiym23]

There’s been a lot of discussion recently about what to include and exclude in packages for distribution. One group of people there argue that packages are too hefty, or contain too many files, and want to see production packages smaller by default. This has converged on using the ”files” array in package.json to explicitly whitelist the files to be included.

There’s also been some discussion within the CLI team about the future of npat support, which directly conflicts with the goals of those who want to see files get smaller by default. The fact is that npat hasn’t really worked well for many years now, both because people haven’t really accounted for it when deciding what to include with their packages, and also due to simple bitrot. Many, if not most, npm users don’t even know the feature exists. We’ve therefore made the decision to deprecate support for it within the CLI, and at some point in the not-too-distant future, remove the code supporting altogether.

At the same time, there’s still a fair amount of confusion and undefined behavior around how .npmignore and / or .gitignore work when one or both are included in npm packages. I’m pretty sure adding conditional ignores into the mix would be far more confusing than helpful, given everything else that’s happening around how packages are built. Put that all together, and the functionality described here isn’t something that we’re going to land in the CLI. Thanks very much for your time and the thorough discussion, though!