After upgrading to Node 8.2.1 and npm 5.3.0 any dependencies starting with @ fail withn http 404 return code

[randyaa]

I'm opening this issue because:

• npm is crashing.
• npm is producing an incorrect install.
• npm is doing something I don't understand.
• Other (see below for feature requests):

What's going wrong?

After upgrading to Node 8.2.1 and npm 5.3.0 any dependencies starting with @ fail withn http 404 return code

NOTE: copied this from https://github.com/npm/registry/issues/195 because after downgrading back to node 6 everything seems to be just fine indicating the issue is likely somewhere in npm not in the registry.

I've tried

• clearing local caches,
• different consoles,
• comparing config settings to people that dont have problems
• changing the repository to http instead of https
• checking the hosts file for strange redirects
• adding the redirects listed at the bottom of npm install giving fetch errors for some packages #13289
• different networks (both internal to the company but one wifi one wired. I didn't verify different hops but i'm pretty sure they are)
• checking the npm repo status - https://status.npmjs.org/
• recloning my repository (not really sure why, just something i tried)
• uninstall nodejs, re-install nodejs (still 8.2.1)

EVERYTHING seems to yield the same result.

How can the CLI team reproduce the problem?

The easiest way to reproduce this problem I've seen is from an project with an empty node_modules and a cleared local npm cache, run npm i "@angular/core"

which yields the following for me:

0 info it worked if it ends with ok
1 verbose cli [ 'C:\\Program Files\\nodejs\\node.exe',
1 verbose cli   'C:\\Program Files\\nodejs\\node_modules\\npm\\bin\\npm-cli.js',
1 verbose cli   'i',
1 verbose cli   '@angular/core' ]
2 info using npm@5.3.0
3 info using node@v8.2.1
4 verbose npm-session e05a769a72d84ace
5 silly install loadCurrentTree
6 silly install readLocalPackageData
7 http fetch GET 404 https://registry.npmjs.org/@angular%2fcore 945ms
8 silly fetchPackageMetaData error for @angular/core@4.3.1 404 Not Found: @angular/core@4.3.1
9 verbose stack Error: 404 Not Found: @angular/core@4.3.1
9 verbose stack     at fetch.then.res (C:\Program Files\nodejs\node_modules\npm\node_modules\pacote\lib\fetchers\registry\fetch.js:41:19)
9 verbose stack     at tryCatcher (C:\Program Files\nodejs\node_modules\npm\node_modules\bluebird\js\release\util.js:16:23)
9 verbose stack     at Promise._settlePromiseFromHandler (C:\Program Files\nodejs\node_modules\npm\node_modules\bluebird\js\release\promise.js:512:31)
9 verbose stack     at Promise._settlePromise (C:\Program Files\nodejs\node_modules\npm\node_modules\bluebird\js\release\promise.js:569:18)
9 verbose stack     at Promise._settlePromise0 (C:\Program Files\nodejs\node_modules\npm\node_modules\bluebird\js\release\promise.js:614:10)
9 verbose stack     at Promise._settlePromises (C:\Program Files\nodejs\node_modules\npm\node_modules\bluebird\js\release\promise.js:693:18)
9 verbose stack     at Async._drainQueue (C:\Program Files\nodejs\node_modules\npm\node_modules\bluebird\js\release\async.js:133:16)
9 verbose stack     at Async._drainQueues (C:\Program Files\nodejs\node_modules\npm\node_modules\bluebird\js\release\async.js:143:10)
9 verbose stack     at Immediate.Async.drainQueues (C:\Program Files\nodejs\node_modules\npm\node_modules\bluebird\js\release\async.js:17:14)
9 verbose stack     at runCallback (timers.js:781:20)
9 verbose stack     at tryOnImmediate (timers.js:743:5)
9 verbose stack     at processImmediate [as _immediateCallback] (timers.js:714:5)
10 verbose cwd C:\extras\code\gitlab\projectx\authoring-ui
11 verbose Windows_NT 6.1.7601
12 verbose argv "C:\\Program Files\\nodejs\\node.exe" "C:\\Program Files\\nodejs\\node_modules\\npm\\bin\\npm-cli.js" "i" "@angular/core"
13 verbose node v8.2.1
14 verbose npm  v5.3.0
15 error code E404
16 error 404 Not Found: @angular/core@4.3.1
17 verbose exit [ 1, true ]

Installing something without the @ like npm i typescript works just fine.

supporting information:

• npm -v prints: 5.3.0
• node -v prints: v8.2.1
• npm config get registry prints: https://registry.npmjs.org/
• Windows, OS X/macOS, or Linux?: Windows (7)
• Network issues:
  • Geographic location where npm was run:
  • I use a proxy to connect to the npm registry.
  • I use a proxy to connect to the web.
  • I use a proxy when downloading Git repos.
  • I access the npm registry via a VPN
  • I don't use a proxy, but have limited or unreliable internet access.
• Container:
  • I develop using Vagrant on Windows.
  • I develop using Vagrant on OS X or Linux.
  • I develop / deploy using Docker.
  • I deploy to a PaaS (Triton, Heroku).

[imdevin567]

I found the cause of this issue. It's from a recent change in the pacote node module that causes NPM to attempt to use the _auth field in .npmrc in an authorization header. For example:

Let's assume your .npmrc file looks like this:

_auth=user:ajsdfauaAKDFAU908ASDF

NPM uses the pacote module to fetch packages. This commit added an additional conditional to add the authorization header to the request if _auth is present, whether alwaysAuth is set or not. I have confirmed removing the _auth line from my .npmrc resolves the issue.

Interestingly, I don't disagree with the commit above--an auth header should be passed if _auth is present. The underlying problem here is that passing an invalid authorization header in a GET request to a scoped package causes a 404. By an invalid header, I mean an authorization header that will not successfully auth in the registry. I've confirmed this using Postman in the pictures below.

Non-scoped package:

Scoped package:

In my case, I had an _auth field in my .npmrc file to auth with a company Artifactory server, which naturally won't work with the public NPM repository. Regardless, this shouldn't be a problem for a public package since it doesn't require an auth to begin with.

I believe the problem here is that the NPM registry is treating scoped packages differently due to the '@' in the URL. This line of code in the node url module shows that any URL with '@' in it is treated as though it requires authorization. My guess here is that the registry thinks an auth needs to happen, uses the authorization header, attempts to auth the user using said header, and returns a 404 because it can't find the user, not the package. The error is simply "Not Found", not "Package not found". This is somewhat confirmed by taking a valid authorization header, removing the last character from it, and attempting to GET a scoped package. The response is then "Could not authenticate USER".

Sorry for the long-winded explanation, but this had been blocking me for a decent amount of time between yesterday and today. Ultimately I think this is a bug in the NPM registry code, but it would also be fair to say the url module shouldn't treat every URL with a '@' as a required auth.

[rex]

[dptoot]

I have the same .npmrc auth requirement for our corporate nexus npm repo access. And we are seeing the exact same issue. Is there are reason why this issue manifests itself when installing @storybook/addons exactly as described above but allows @types/colors to be installed correctly?

C:\Users\jelewis1\work\redux-action-creator (get-state-async-action-creator) (redux-action-creator@1.1.0)
λ npm i -D @storybook/addons
npm ERR! code E404
npm ERR! 404 Not Found: @storybook/addons@latest

npm ERR! A complete log of this run can be found in:
npm ERR!     C:\Users\jelewis1\AppData\Roaming\npm-cache\_logs\2017-09-25T18_21_31_450Z-debug.log

C:\Users\jelewis1\work\redux-action-creator (get-state-async-action-creator) (redux-action-creator@1.1.0)
λ npm i -D @storybook/addons@3.2.10
npm ERR! code E404
npm ERR! 404 Not Found: @storybook/addons@3.2.10

npm ERR! A complete log of this run can be found in:
npm ERR!     C:\Users\jelewis1\AppData\Roaming\npm-cache\_logs\2017-09-25T18_22_49_390Z-debug.log

C:\Users\jelewis1\work\redux-action-creator (get-state-async-action-creator) (redux-action-creator@1.1.0)
λ npm i -D @types/colors
npm notice created a lockfile as package-lock.json. You should commit this file.
+ @types/colors@1.1.3
added 1 package and removed 530 packages in 12.78s

[dptoot]

Is there any path for resolution here? Having auth via the .npmrc is a requirement in any environment using a local/private npm instance. Many are still struggling to support scoped packages as it is. We don't really want to be adding _auth properties to local npm project structures So with that in place there is no clear way how to specify on an end user's side when to use _auth for a project and when to not.

[NicolasPelletier]

I am in the same situation as @dptoot with the most recent LTS:

$ npm -v && node -v
5.6.0
v8.9.4

This is now almost 6 months old. Does anybody in the npm development team planning to look at this one ?

[imdevin567]

Even though this is the original issue, the duped issue has a bit more traction so I commented there. Here's the link for reference.

[samslater]

I have a similar setup with Artifactory repository. It is working locally on my mac but getting the same 404 errors when trying to build in our docker image.

Docker .npmrc looks something like

@ourscopename:registry=https://our/registry/path
_auth = ourToken
email = youremail@email.com
always-auth = true
@babel:registry=https://registry.npmjs.org

Local .npmrc file looks something like

@ourscopename:registry=https://our/registry/path/
//our/registry/path/:_authToken=myToken

Can't figure out why it's working locally but not in docker env.