Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Already on GitHub? Sign in to your account

multiple registries with different options #1401

Closed
isaacs opened this Issue Sep 14, 2011 · 20 comments

Comments

Projects
None yet
Owner

isaacs commented Sep 14, 2011

  1. Extend registry config to take a whitespace-separated list.
  2. First registry is "write" target (publish, tag, unpublish, etc.) Override-able in a few different ways anyway, package.json "publishConfig", cli options, etc.
  3. Override config options with registry:<registry url>:<option>:
    • _auth (and thus also username/_password)
    • always-auth
    • ca
    • https-proxy
    • proxy
    • searchopts
    • searchexclude
    • strict-ssl
    • tag
  4. Extend the cache folder structure to include the sanitized registry config, since ETags and such may differ.
  5. Merge search results objects from all registries when searching.
  6. If multiple registries are present, then adduser needs to ask the user which registry they're authorizing against.
  7. BONUS POINTS: Add an npm registry <add, rm, ls, reorder> command to manage the list.
Owner

isaacs commented Sep 14, 2011

@pquerna This is what we talked about at summer camp. Does it sound like it'd give you what you need?

This is becoming more and more necessary as we progress through our project we are finding that package maintainers continue to break their dependency chains. We have had multiple instances where we end up in dependency hell because of open ">=,<=" package.json dependencies that end up conflicting with each other. It would be nice if NPM supported the idea of "channels" that allow packages to be pinned against a specific node version and possibly even cpu architecture if they contain native code. Also it would be good to have overlapping repositories that allow for development/alpha software to stay out of the stable repositories so that we could avoid upgrading to 2.X-alpha unknowingly. I do realize there are configuration switches to weed out major revision number jumps but it would allow for package maintainers to "advertise" their most feature complete stable software forward while also allowing for early adopters to pick up the freshest packages at their own risk. In addition this would allow for the testing of each package against the rest of the repo to find conflicts before they get out into the wild. Sure this could be done currently but I believe the isolation would be helpful. Finally it could be used to create an ecosystem that would allow for packages to gain peer review and be up-voted into the "main" repository within their respective channels and made available to those users looking to use peer-reviewed packages with confidence that they are in fact stable.

Owner

isaacs commented Oct 6, 2011

@therealjessesanford It seems like there's about 5 different feature requests in there. My guess is that at least one of them is a great idea, but it doesn't seem like any are actually related to the OP here. Can you try to split it up into a few concrete suggestions, and post as separate issues?

The 2.0.0alpha issue is resolved in npm 1.0.94. The engines hash should have prevented it, but there was a bug.

Contributor

bmeck commented Sep 14, 2012

This is big enough to sounds like a separate module. Thoughts?

Given multiple repositories, would npm install somePkg go over them in order and look for somePkg? This would be a nice way to have a private repo with only private packages in it and still be able to install public packages.

js-n commented Jan 3, 2013

Any progress on this issue? I saw it mentioned in some mailing list threads (more recent than this issue, eg https://groups.google.com/forum/?fromgroups=#!topic/npm-/q3nqZaQT71I ) but they always seem to die out before getting to a solution. There are some work arounds, including front end proxies at suggested in #19 (viz https://github.com/g-k/npm-proxy and my effort, https://github.com/jden/npm-delegate ), but these are difficult to work with, and there are still auth issues with switching back and forth between registries.

@isaacs Should

  1. The npm-registry-client handle multiple registries, iterating through each until a result is found and returning it to the npm CLI,
    OR
  2. Should the npm CLI instantiate an npm-registry-client for each registry in the registries config, perhaps establish an array of registries at init, and iterate through these looking for a result?

In 2), it's likely no changes would be required to npm-registry-client, and each registry would have it's own associated couch login..
Any thoughts on this appreciated!

This was referenced Jan 25, 2013

Any news on this?

I would really like to see an easy way to start a private npm for private modules which is in front of the public npm for every other module.

@jden Thank you, I will look into that!

@isaacs - is there any plan on supporting this anytime soon?

robraux commented Jul 24, 2013

@isaacs - any movement on this, or updated plans?

@spmason spmason referenced this issue in mbrevoort/node-reggie Aug 13, 2013

Closed

Fall back to npm registry on unknown packages #14

Contributor

bajtos commented Aug 13, 2013

Update: I have discussed this feature with @isaacs & @piscisaureus, created a spike and now I am going to implementing the necessary changes to all modules npm depends on.

threez commented Nov 14, 2013

@bajtos Any progress on that? It's a real pain if you have a private repository for proprietary stuff and use the official at the same time. (Without replicating all stuff to the local registry)

Would be really nice to see the multi repository support...

@threez if you are already running a private registry, just run it behind nginx or mod_proxy and fallback to public on 404.

For example, here is an nginx config to that effect:

upstream npmjsorg-servers {
    server npm.<mydomain>.com:5984;
    server public.npm.<mydomain>.com      backup;
}

server {
    listen                  443;
    server_name             npm.<mydomain>.com;

    # Need big files
    client_max_body_size    1024m;

    # SSL Settings
    include                 ssl/default_settings;

    location / {
        # OK to pass all, because all servers are local
        proxy_pass          http://npmjsorg-servers;

        # Proxy Settings
        proxy_redirect off;
        proxy_next_upstream error timeout invalid_header http_404 http_500 http_502 http_503 http_504;

        ### Set headers ###
        include             proxy/headers;
        proxy_set_header    Host            npm.<mydomain>.com;
    }
}

server {
    listen                  80;
    server_name             public.npm.<mydomain>.com;

    client_max_body_size    1024m;

    location / {
        # Use backup on GET, not on PUT POST DELETE
        limit_except        PUT POST DELETE {
            proxy_pass      https://registry.npmjs.org;
        }

        # Proxy Settings
        proxy_redirect off;

        ### Set headers ###
        include             proxy/headers;
        proxy_set_header    Host            registry.npmjs.org;
        proxy_set_header    Authorization   "";
    }
}

proxy/headers just has normal proxy options:

proxy_set_header    Accept-Encoding     "";
proxy_set_header    X-Real-IP           $remote_addr;
proxy_set_header    X-Forwarded-For     $proxy_add_x_forwarded_for;
proxy_set_header    X-Forwarded-Proto   $scheme;
add_header          Front-End-Https     on;
Contributor

bajtos commented Nov 15, 2013

@threez Our priorities has changed and we don't have enough bandwidth to work on multi-registry support anymore.

Check out these two projects that can help you with what you are trying to achieve:

Reggie - a lightweight registry server, check the README on how to specify fuzzy version dependencies via URLs.

Sinopia - a caching registry server. It will serve local (private) packages from a local storage. Requests for packages not stored locally are resolved using npmjs.org.

threez commented Dec 5, 2013

@bajtos thx i will check them out

js-n commented May 26, 2015

This issue is still showing up on my personal "open issues mentioning you" list, limiting its utility. Could you please consider closing it?

Contributor

othiym23 commented May 26, 2015

Since we now have at least a somewhat functional per-registry configuration framework (which is currently mostly used for auth, but could be extended for other things), I'm going to call this more or less satisfied. People who want per-registry configuration that's not handled by the current version of npm should open new issues (or, better yet, pull requests) with their specific requests. Thanks for the nudge, @jden.

@othiym23 othiym23 closed this May 26, 2015

@othiym23 is there any docs on how to do this?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment