Private registry / Configure npm registry for module download

[geglock]

I would like to be able to use a custom, self-hosted npm registry.
Especially for resolving company-local private packages based on scopes this would be very helpful.
As far as I can see, currently only the npm public registry (https://registry.npmjs.org/) can be used.

Deno should support something similar to npm's configuration:

npm config set registry ...
npm config set @mycompany:registry ...

[jiawei397]

I think it is necessary. I want to pay attention to this progress, because in China, I sometimes download the resources of npm very slowly.

[jimisaacs]

+1 this is a basic requirement for enterprise use

[sithumonline]

Good to have GitLab private package registry support 🚀

[NickTomlin]

It looks like this is currently possible through the DENO_NPM_REGISTRY ENV flag (source)

# execute a remote script ala `npx`
DENO_NPM_REGISTRY=https://my-npm-instance.com deno run npm:@my-internal-scope/my-internal-package
# Start module
cat index.ts

import * as myModule from "npm:@my-internal-scope/my-internal-package"
console.log(myModule)

DENO_NPM_REGISTRY=https://my-npm-instance.com deno run index.ts

It'd be great if this was possible to set via a configuration property in deno.json.

[dsherret]

Sorry, the DENO_NPM_REGISTRY env variable wasn't intended to be stable at this point and was undocumented, but yeah it should work for now. I added a warning on its use in #16953 in future versions, but just beware that it may be changed when we come up with a real solution for this problem (though likely it will still remain... it's just not something that's been discussed yet).

[NickTomlin]

@dsherret thanks for clarifying that and a adding that warning 😄

Is this issue the best place for community feedback to try and help shape that work?

As @jimisaacs mentioned, I think this is crucial in bringing the magic of npm:* to the enterprise space (full disclosure: Jim and I work together at Netflix). Internally, the company has done a lot of investment in scaling, hardening, and auditing our private NPM registry. Being able to leverage that investment directly, instead of having to go through all the same technical and procedural hoops to stand up infrastructure to sit in front of it (e.g. esm.sh), would be huge win.

[dsherret]

We're going to discuss this issue tomorrow in our weekly meeting. For now I'm not adding the warning because the environment variable is used in all our tests in order to use a test npm registry, but consider it unstable.

[ry]

Probably we should just use the same env var as NPM: NPM_CONFIG_REGISTRY

[NickTomlin]

NPM_CONFIG_REGISTRY makes sense to me; I like matching the grain of the existing patterns.

In the meantime, we can dual-write to both variables to make a transition easier.

[dsherret]

I opened #16980 for NPM_CONFIG_REGISTRY support.

We'll look into support for different registries for scopes and credentials in the future.

[lino-levan]

#16980 got merged, should this issue be closed?

[fufuuuu]

So how to set NPM_CONFIG_REGISTRY?

[aapoalas]

Like you would normally set environment variables. Google may help you with this if you do not know how to do it already.

[silverwind]

Ideally deno should support parsing per-scope registry and per-registry auth tokens from the npmrc files and corsponding env vars, so that users can use multiple registries and tokens.

[bartlomieju]

Ideally deno should support parsing per-scope registry and per-registry auth tokens from the npmrc files and corsponding env vars, so that users can use multiple registries and tokens.

It's mentioned in a comment above: #16105 (comment)

[reyawn]

We'll look into support for different registries for scopes and credentials in the future.

Is there any update on this @dsherret? This is a big blocker for many use cases.

[dsherret]

The NPM_CONFIG_REGISTRY environment variable landed in 1.29, but we haven't added support for different registries per scope yet.

[marziply]

Where I work, there are some Node microservices that could benefit from migrating to Deno. However, these microservices depend on private Gitlab NPM packages. This feature is blocking the upgrade to Deno because many of these private packages are critical to these services and cannot run without them.

The work so far on this has been excellent and I've not reported any other issues. I will just have to wait until Deno supports private packages before upgrading.

[marcushultman]

Implementing this as a Configuration file option ("npmRegistry") in #19317

[esnho]

Hi, how is possible to setup a private registry with this environment variable?

I'm using gemfury as registry, and I've tried below command without success:

DENO_NPM_REGISTRY=https://npm-proxy.fury.io/myorg/:_authToken=${GEMFURY_DEPLOY_TOKEN} deno run myregistry.ts

I get this error:

error: npm package '@myorg/mypackage' does not exist.
    at file:///Users/user/temp/deno-test/myregistry.ts:1:29

I usually use gemfury with yarn on node and it works nicely, below my .npmrc file (note that has been redacted)

@myorg:registry=https://npm-proxy.fury.io/myorg/
//npm-proxy.fury.io/myorg/:_authToken=${GEMFURY_DEPLOY_TOKEN}

and finally here is myregistry.ts

import { MyModyle } from "npm:@myorg/mypackage";
console.log(MyModule)

[marcushultman]

@esnho The env was changed to NPM_CONFIG_REGISTRY.

[bartlomieju]

Update: We're gonna land first pass in #23560 really soon.

The first pass will support .npmrc located next to package.json or deno.json - so it will work with Node.js first projects as well as Deno first projects. Please let me know if you rely on .npmrc in your user directory or global directory (https://docs.npmjs.com/cli/v10/configuring-npm/npmrc#files).

Only _authToken config will be supported in the first pass - please let me know by commenting if you are heavy users of other options so we can prioritize accordingly.

Also the first pass will only support deno run / deno test subcommands - the support for deno compile will come afterwards - please also comment if you need it.

Thanks

[hugojosefson]

Thank you @bartlomieju for working on this!

We are heavy users of npx google-artifactregistry-auth, which saves the _authToken in $HOME/.npmrc.

Then in each project, we have an .npmrc checked into git, which specifies that our internal @certain-npm-scope should always be fetched from our private npm registry in GCP.

It looks something like this:

each_project_dir/.npmrc:

@certain-npm-scope:registry=https://europe-west2-npm.pkg.dev/our-gcp-project-id/our-npm-repo-name/
//europe-west2-npm.pkg.dev/our-gcp-project-id/our-npm-repo-name/:always-auth=true

$HOME/.npmrc (after having run npx google-artifactregistry-auth):

//europe-west2-npm.pkg.dev/our-gcp-project-id/our-npm-repo-name/:_authToken=…

[TaxBusby]

For Github private registries, we use local-directory .npmrc, but notably use env-var interpolation features supported by NPM.

@our-github-org:registry=https://npm.pkg.github.com/
//npm.pkg.github.com/:_authToken=${GITHUB_NODE_TOKEN}

Developers have this env var set to a valid token.

For Codeartifact private regsitries, we use the aws codeartifact login tool, which populates $HOME/.npmrc with all relevant scope+auth config.

(For adopting Deno, as long as we can globally define the auth token for all repos somewhere, and locally or globally set a scope for the registry, we can figure out a migration path)

[MirKml]

Hi, for Azure Artifacts npm - user level .npmrc with the authToken is desired, see https://learn.microsoft.com/en-us/azure/devops/artifacts/get-started-npm?view=azure-devops&tabs=Other

I think this is common scenario for npm scoped registry settings - committed .npmrc with scope and registry URL, then $HOME/.npmrc with authToken for this registry.

[jtibble]

@bartlomieju I also wanted to add that this is critical for using Deno within my company, as we have multiple internal proxied NPM repositories (including a copy of NPM, as well as Sonatype Nexus 3.65.x).

Do you have any ETA for when you'd like to try to land this in a release? I'd like to volunteer to test. Local .npmrc compatibility would be incredible with our URLs/auth specified like:

@dcl-crew:registry=https://nexus3.disney.com/repository/npm/
//nexus3.disney.com/repository/npm/:_auth=xxxxxxx

[bartlomieju]

@jtibble this fubctionality should now available in canary. Please try it out and let me know.

[gengurke]

@bartlomieju I tested this in a node project running node 20 and vite.

Deno canary version: deno 1.43.6+53606de (updated today)

I placed the .npmrc in ~/.npmrc and the root directory of the project. Here is its content:

@fortawesome:registry=https://npm.fontawesome.com/
//npm.fontawesome.com/:_authToken=MYTOKEN

If i run deno task dev I get the following output:

error: npm package '@fortawesome/pro-regular-svg-icons' does not exist.

Installing and running via npm works fine. I hope this helps.

[bartlomieju]

@gengurke please do DENO_FUTURE=1 deno install before running DENO_FUTURE=1 deno task dev. Let me know if that works for you.

[gengurke]

@bartlomieju Thanks! It worked like a charm and behaved just like using npm. The only difference was that I had to place the .npmrc file directly in my project directory, whereas with npm, it can be placed in ~/.npmrc for global use.
Great work!

[jtibble]

@bartlomieju I also validated with canary build deno 1.43.6+8b5089e (canary, aarch64-apple-darwin) and it worked like a charm! 🙌

SO excited to have this working locally. What a dream! I have both my registry and auth in my local directory .npmrc like this:

registry=https://nexus3.disney.com/repository/npm/
//nexus3.disney.com/repository/npm/:_auth=b1lDRnZxxxxxxxxxxx

so my only request now is to allow for joining .npmrcs, so I can keep my auth token separate. Not a big deal either way, because we can just .gitignore this, but yes, thanks again! So excited to see this land in the main build!

[jtibble]

@bartlomieju Is there any chance the .npmrc functionality currently in the canary builds could be added to the 1.44 release milestone? Even behind a feature flag or toggle would be great, so we can at least start using it on my development team. Thanks!

[bartlomieju]

@jtibble, yes this functionality will be in v1.44 and will be announced in a blog post. I will try to get merging of npmrc done too, but no promises.

[jtibble]

@bartlomieju I also wanted to bring to your attention that in VSCode, with the Deno language server extension, even with the checkbox enabled for Deno: Future, the private-registry-module-resolution is not resolving. This is with canary build 1.43.6+a89253.

I tried caching through the VSCode popup, no luck. Manually caching the dependency in the terminal works, but still the VSCode extension can't resolve it.

Screenshots:

[bartlomieju]

@jtibble thanks for pointing this out, I knew I forgot about something! I opened #24040 and we'll aim to fix it in v1.44.1.

[sithumonline]

Worked well with gitlab npm registry.

Deno version: 1.43.6+3c3076a
npmrc:

@arimac-spades:registry=https://gitlab.com/api/v4/projects/<number>/packages/npm/
//gitlab.com/api/v4/projects/<number>/packages/npm/:_authToken=<token>

[bartlomieju]

@jtibble looks like it got fixed in time for v1.44: #24042

[bartlomieju]

Support for private registries/.npmrc has shipped in Deno v1.44.0: https://deno.com/blog/v1.44#support-for-private-npm-registries.

There are a few remaining features that need to be addressed:

• .npmrc: --reload is broken  #23953
• .npmrc: support email auth configuration #23952
• .npmrc: support certfile and keyfile auth options #23951
• .npmrc: support username and _password auth configuration #23950
• .npmrc: support per-user and maybe global config files #23954

Overall, I think we can now close this issue. Please subscribe to relevant issue or open new ones if you experience any problems.

[dsherret]

Hi everyone, we discovered an issue in .npmrc support in Deno 1.44.0 where Deno would send npmrc credentials for the scope to the tarball URL when the registry provided URLs for a tarball on a different domain.

For example, the npm registry provides information on a package at a url like this: https://registry.npmjs.org/code-block-writer -- you can see that under versions -> <version> -> dist -> tarball there's a URL (ex. https://registry.npmjs.org/code-block-writer/-/code-block-writer-1.0.0.tgz). That URL can be anything and Deno was only considering the first URL of https://registry.npmjs.org/code-block-writer to get credentials instead of the tarball URL (so in other words, it would send the credentails for https://registry.npmjs.org/ to whatever the tarball URL was)

You can read more about this here: https://github.com/npm/cli/wiki/%22No-auth-for-URI,-but-auth-present-for-scoped-registry%22 (how Deno is working is actually how npm used to behave)

I recommend to stop using .npmrc support in Deno 1.44.0 and if your private registry ever serves tarballs at a different domain to rotate your registry credentials. We have a fix for this here: #24111 and are going to do a release in a few hours to fix this issue.

Huge apologies on this one.

[dsherret]

1.44.1 has been released. Please upgrade (deno upgrade).

A security advisory for 1.44.0 was published here: GHSA-rfc6-h225-3vxv