Public npm registry is used instead of private when trying to update private dependency

[voda]

Package manager/ecosystem
npm

Manifest contents prior to update
.dependabot.yml

version: 2
registries:
  npm-ghp-octocat:
    type: npm-registry
    url: 'https://npm-registry-proxy.somedomain.tld'
    token: ${{secrets.NPM_TOKEN}}
updates:
  - package-ecosystem: 'npm'
    directory: '/'
    registries: 
      - "npm-ghp-octocat"
    schedule:
      interval: 'daily'

What you expected to see, versus what you actually saw
Dependabot is able to find the new version of the dependency but then fails while updating it. The update call is made to registry.npmjs.org instead of the specified private registry.

updater | INFO <job_96922851> Checking if @myorg/my-package 13.2.2 needs updating
  proxy | 2021/03/18 08:07:21 [014] 200 https://npm-registry-proxy.somedomain.tld:443/@myorg%2Fmy-package
  proxy | 2021/03/18 08:07:22 [016] 200 https://npm-registry-proxy.somedomain.tld:443/@myorg%2Fmy-package/13.3.0
updater | INFO <job_96922851> Latest version is 13.3.0
...
  proxy | 2021/03/18 08:07:26 [018] 404 https://registry.npmjs.org:443/@myorg%2fmy-package

The logs look similar to #3298 but in this case the registry has not extra path.

Images of the diff or a link to the PR, issue or logs
Logs of the update:

updater | INFO <job_96922851> Starting update job for myorg/dependabot-test
updater | INFO <job_96922851> Checking if @myorg/my-package 13.2.2 needs updating
  proxy | 2021/03/18 08:07:20 [014] GET https://npm-registry-proxy.somedomain.tld:443/@myorg%2Fmy-package
  proxy | 2021/03/18 08:07:20 [014] * authenticating npm registry request (host: npm-registry-proxy.somedomain.tld, token auth)
  proxy | 2021/03/18 08:07:21 [014] 200 https://npm-registry-proxy.somedomain.tld:443/@myorg%2Fmy-package
  proxy | 2021/03/18 08:07:22 [016] GET https://npm-registry-proxy.somedomain.tld:443/@myorg%2Fmy-package/13.3.0
  proxy | 2021/03/18 08:07:22 [016] * authenticating npm registry request (host: npm-registry-proxy.somedomain.tld, token auth)
  proxy | 2021/03/18 08:07:22 [016] 200 https://npm-registry-proxy.somedomain.tld:443/@myorg%2Fmy-package/13.3.0
updater | INFO <job_96922851> Latest version is 13.3.0
  proxy | 2021/03/18 08:07:26 [018] GET https://registry.npmjs.org:443/@myorg%2fmy-package
  proxy | 2021/03/18 08:07:26 [018] 404 https://registry.npmjs.org:443/@myorg%2fmy-package
  proxy | 2021/03/18 08:07:26 [020] GET https://registry.npmjs.org:443/@myorg%2fmy-package
  proxy | 2021/03/18 08:07:26 [020] 404 https://registry.npmjs.org:443/@myorg%2fmy-package
  proxy | 2021/03/18 08:07:26 [022] GET https://registry.npmjs.org:443/dependabot-test
  proxy | 2021/03/18 08:07:26 [022] 200 https://registry.npmjs.org:443/dependabot-test
updater | INFO <job_96922851> Requirements to unlock own
  proxy | 2021/03/18 08:07:27 [024] GET https://registry.npmjs.org:443/dependabot-test
  proxy | 2021/03/18 08:07:27 [024] 200 https://registry.npmjs.org:443/dependabot-test
updater | INFO <job_96922851> Requirements update strategy widen_ranges
updater | INFO <job_96922851> Updating @myorg/my-package from 13.2.2 to 13.3.0
  proxy | 2021/03/18 08:07:29 [026] GET https://registry.npmjs.org:443/@myorg%2fmy-package
  proxy | 2021/03/18 08:07:29 [026] 404 https://registry.npmjs.org:443/@myorg%2fmy-package
  proxy | 2021/03/18 08:07:29 [028] GET https://registry.npmjs.org:443/@myorg%2fmy-package
  proxy | 2021/03/18 08:07:29 [028] 404 https://registry.npmjs.org:443/@myorg%2fmy-package
updater | INFO <job_96922851> Handled error whilst updating @myorg/my-package: private_source_authentication_failure {:source=>"npm-registry-proxy.somedomain.tld"}
updater | INFO <job_96922851> Finished job processing

[thepwagner]

@voda could you share package-lock.json?

This strikes me as a potential issue with the Dependabot::NpmAndYarn::FileUpdater::NpmrcBuilder.

I'm assuming .npmrc should define myorg:registry=https://npm-registry-proxy.somedomain.tld:443, so packages in the myorg namespace are downloaded from the registry configured registry.

That's supposed to work by parsing the .dependencies.resolved path from package-lock.json.

If any of those resolved URLs matches a configured registry (e.g. npm-registry-proxy.somedomain.tld), and has what appears to be a scope as part of the URL (e.g. @myorg%2F).

---

My sneaking suspicion is that package-lock.json uses the lowercase %2f, so it's not being detected correctly here.

# potentially problematic URL
irb(main):001:0> url  = 'https://npm-registry-proxy.somedomain.tld:443/@myorg%2fmy-package/13.3.0/whatever.tar.gz'
=> "https://npm-registry-proxy.somedomain.tld:443/@myorg%2fmy-package/13.3.0/whatever.tar.gz"

# current behaviour
irb(main):002:0> url.split(/\%40|@/)[1]&.split(%r{\%2F|/})&.first
=> "myorg%2fmy-package"

# likely desired behaviour
irb(main):003:0> url.split(/\%40|@/)[1]&.split(%r{\%2[fF]|/})&.first
=> "myorg"

[voda]

Yes, .npmrc contains: @myorg:registry=https://npm-registry-proxy.somedomain.tld/ (without the explicit port)

and package-lock indeed contains the lowercase %2f:

{
"resolved": "https://npm-registry-proxy.somedomain.tld/@my-org%2fmy-package/-/my-package-13.2.2.tgz"
}

[voda]

@thepwagner Thanks for the fix. Already tested it and dependabot created the PR 🎉 .

[nindzyago]

Have same problem with such config

registries:
  my-companyl-npm:
    type: npm-registry
    url: https://my-company-012345678.d.codeartifact.us-east-1.amazonaws.com/npm/my-company-npm
    token: ${{secrets.NPM_AUTH_TOKEN}}

updates:
  - package-ecosystem: npm
    directory: "/iam-tools"
    registries:
      - my-company-npm
    schedule:
      interval: daily
      time: "07:00"
      timezone: "Etc/UTC"
    open-pull-requests-limit: 7
    allow:
      - dependency-name: '@my-company/*'

and it seems like it found updates for my lib:

updater | INFO <job_471907512> Checking if @my-company/tenant-config 0.1.100 needs updating
  proxy | 2022/09/29 11:34:18 [136] GET https://my-company-863233358811.d.codeartifact.us-east-1.amazonaws.com:443/npm/my-company-npm/@my-company%2Ftenant-config
...
  proxy | 2022/09/29 11:34:19 [140] 200 https://my-company-863233358811.d.codeartifact.us-east-1.amazonaws.com:443/npm/my-company-npm/@my-company%2Ftenant-config/0.1.197
updater | INFO <job_471907512> Latest version is 0.1.197

but then for some reason it tries to update it with default nom registry:

  proxy | 2022/09/29 11:34:24 [142] GET https://registry.npmjs.org:443/@my-company%2ftenant-config
  proxy | 2022/09/29 11:34:24 [142] 404 https://registry.npmjs.org:443/@my-company%2ftenant-config

I've read a lot of topics and issues, tried different formats for url field in config, etc. but without success