Private registry support does not work with npm version 7

[Avaq]

I'm running into the issue described here: npm/cli#2508. This issue was closed by mistake (npm/cli#2508 (comment)), but it seems to be implied that it won't be fixed, and that something needs to change on setup-node's end to get things working again.

From my basic understanding, it's related to the way that setup-node configures the multiple registries:

setup-node/src/authutil.ts

Lines 47 to 55 in 5c355be

	const authString: string =
	registryUrl.replace(/(^\w+:|^)/, '') + ':_authToken=${NODE_AUTH_TOKEN}';
	const registryString: string = scope
	? `${scope}:registry=${registryUrl}`
	: `registry=${registryUrl}`;
	const alwaysAuthString: string = `always-auth=${alwaysAuth}`;
	newContents += `${authString}${os.EOL}${registryString}${os.EOL}${alwaysAuthString}`;
	fs.writeFileSync(fileLocation, newContents);
	core.exportVariable('NPM_CONFIG_USERCONFIG', fileLocation);

To Reproduce

1. Make sure your package-lock.json contains a private package that can only be accessed with your NPM_TOKEN. It's important to have a package-lock.json file as well as your regular package.json, because the issue only occurs when there is also a package lock, as hinted at by this comment: [BUG] NPM v7 private registry authentication 401 (v6 works) npm/cli#2508 (comment)
2. Use a workflow like:

   steps:
   - uses: actions/checkout@v2
   - uses: actions/setup-node@v2
     with:
       node-version: '16.x' # Node version is important; It ensures that npm v7 is installed
       registry-url: 'https://registry.npmjs.org'
   - run: npm install --ignore-scripts
     env:
       NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}

[Avaq]

After adding the --verbose flag to npm install --ignore-scripts, I noticed that npm is trying to download my private package from https://npm.pkg.github.com.

This led me to inspect the package-lock.json, and I found that indeed, the "resolved" field for this package points to a resource on npm.pkg.github.com.

I'm guessing the package lock was generated by someone who had an auth token for the github package registry in their npmrc, and since the github package registry mirrors the private packages on npm, it was able to resolve from there.

[Avaq]

I tried regenerating the package lock, ensuring that I'm not authenticated against npm.pkg.github.com:

$ npm whoami
avaq
$ npm whoami --registry=https://npm.pkg.github.com
npm ERR! code ENEEDAUTH
[...]

... and still my private package keeps being resolved from https://npm.pkg.github.com. How is npm even able to download from there if I'm not authenticated? I even tried generating the package lock with npm install --cache ./cache to ensure it's not taking the resolved URL from a cache.

[Avaq]

My working understanding of this issue is now as follows:

1. When generating the package lock, my npm auth token is sent to the npm registry, which authorizes my client and forwards it to the github package registry. This is an assumption. It explains why my package lock has my private dependency resolved on https://npm.pkg.github.com.
2. When installing my private dependencies from the generated package lock, my client tries to fetch the package straight from the github package registry.
   • Prior to npm version 7, this would work, because npm 6 used to send every token to every registry, as implied by this comment: [BUG] NPM v7 private registry authentication 401 (v6 works) npm/cli#2508 (comment) (and my assumption is that the github package registry accepts npm tokens as valid proofs of authenticity)
   • With npm version 7, however, tokens are only sent to the exact registry that they were configured for. So now the github package registry has no proof of identity, and rejects my client.

---

I will attempt to work around this issue by switching from the npmjs.org registry to the pkg.github.org registry altogether in my project, which I believe is possible. The issue might still be problematic for other users though.

[Avaq]

I uncovered more information about assumption I made for step 1: I had a file named .npmrc.bak in my project root containing the following line: @my-scope:registry=https://npm.pkg.github.com/. I had added the .bak extension in the assumption that this would cause npm to ignore the file. I just found out npm did not ignore the file. Without this file, the npm server rejects my attempt to install the private dependency, because "payment is required". This leaves me with the second assumption, which is that the github package registry accepts npm tokens for authorization. It also might mean that npm 7 still sends tokens cross-domain (the npm token was not meant for the github package registry) in this scenario, which may be an issue for npm.

[Avaq]

I'm closing this issue as I now believe it to be outside of the scope of GitHub Actions. Instead the issue is related to how npm v7 does one thing when generating a package lock, and then another, (problematic, under specific conditions) thing, when installing from it.