Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP

Loading…

Allow installations from git+file:// if the package is private #4108

Closed
wants to merge 1 commit into from

4 participants

@robertkowalski
Collaborator

See #3868

@robertkowalski
Collaborator

Question: Please note the different usage of the variable what in the both PR

Always update git urls - where I am preparing all git urls
https://github.com/robertkowalski/npm/blob/05d14e130483d69e6ef0bdbae90e5e509ee7e69f/lib/install.js#L153

VS

Allow installations from git+file:// if the package is private - where I use split("@").pop() on the unprepared string
https://github.com/robertkowalski/npm/blob/b9e3beed076add73f61bff05ca32b98d7db5ae55/lib/install.js#L667

As both are not merged yet, what do you think is the way to go?

@robertkowalski
Collaborator

Ok, figured the - in my opinion - best way out for that issue and changed #4104

Please review, I think this might close the long running #2442 with some luck

@luk-

I really hate to be that guy here but I think we should probably revisit how we're adding things to install.js. This file is really big and is pretty hot code – and probably should be a collection of smaller modules so we can increase interoperability and hopefully reduce the risk of breaking stuff.

Also, it's possible I wasn't really clear when describing supporting local filesystem dependencies, but I don't think we need to get in the way of someone installing something, it's more on the publish side. There's really no benefit to the user or safety for the registry to do a sanity check for "private": true when someone is attempting to npm install in a directory with local filesystem dependencies. When I made a fuss about special requirements for these local private dependencies, my goal was to not have us land the feature without protection from potential publishing, not really as much to nag users during an install.

Lastly, does this work the same on Windows? Does it support directories as well? npm isn't intrinsically tied to git, so if we're adding support for local things as first-class dependencies, it should probably be more than just via a local git repo.

Anyway, sorry, you've been doing awesome work and I feel bad for bringing this up, but I think it has to be said.

@robertkowalski
Collaborator

Hey @luk-,

thanks for your feedback, I would really like to help splitting up install.js - but that would be another PR.

Please have a look at https://github.com/robertkowalski/npm/blob/4d0be594c1b9de2501f5544f53b5d8e441fa088b/lib/publish.js#L132 and the whole diff from publish.js - there is a check for preventing publishing modules. I can remove the check when installing modules, that's no problem for me. You are right that this probably makes no sense, but from the previous context I thought it might be worthy to include the check there too.

I don't get your last point regarding directories. What do you mean with directories?

What do I have to do to get this merged now, despite removing the check from install.js or how do we continue?

@robertkowalski
Collaborator

Ah, forgot to add: I definitely want this feature (local git dependencies) to be able to convert more of these tests https://github.com/isaacs/npm/blob/master/test/packages/npm-test-shrinkwrap/package.json#L7 to tap, without connecting to github or any other git repos over the network and to be able to test anything new git related

@robertkowalski robertkowalski Allow installations from git+file:// if the package is private
Throw error, in case packages are going to be published
See #3868
fbd5189
@robertkowalski
Collaborator

Had the time to test this on windows and work further on this issue:

It works on windows and I removed the nagging of users during install.

I like the idea with the local directories, but let's take this as a first step.

@domenic
Collaborator

I am +1. It would be nice to get @isaacs to make a judgement call though.

@luk-
@othiym23
Owner

I'll take another look at this as part of landing #5629. Thanks for your patience, @robertkowalski!

@othiym23 othiym23 added this to the 2.0.0 milestone
@othiym23 othiym23 removed this from the 2.0.0 milestone
@robertkowalski
Collaborator

hey @othiym23 - i would love to port that patch to the current codebase to catch up - if you still consider it to land. i can really say that i really was depending on this to write integrations tests that covered an install from git - and additionally the initial wish to add this came from the community. i am also happy to close this - no hard feelings

@othiym23
Owner

@robertkowalski: https://github.com/npm/npm/blob/master/test/tap/git-npmignore.js#L43 😁 This got merged in through the backdoor when we were making changes to npa, I think.

There may be bugs in how it works. File those as separate issues?

@othiym23 othiym23 closed this
@othiym23 othiym23 removed the review label
@robertkowalski
Collaborator

:heart:

@robertkowalski
Collaborator

@othiym23 will close the issues related to it

@robertkowalski robertkowalski deleted the robertkowalski:imp/git-file branch
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Commits on Nov 22, 2013
  1. @robertkowalski

    Allow installations from git+file:// if the package is private

    robertkowalski authored
    Throw error, in case packages are going to be published
    See #3868
This page is out of date. Refresh to see the latest.
View
32 lib/publish.js
@@ -11,6 +11,8 @@ var npm = require("./npm.js")
, chain = require("slide").chain
, Conf = require("npmconf").Conf
, RegClient = require("npm-registry-client")
+ , isGitFileUrl = require("./utils/is-git-file-url.js")
+ , asyncMap = require("slide").asyncMap
publish.usage = "npm publish <tarball>"
+ "\nnpm publish <folder>"
@@ -34,12 +36,16 @@ function publish (args, isRetry, cb) {
readJson(path.resolve(arg, "package.json"), function (er, data) {
er = needVersion(er, data)
if (er && er.code !== "ENOENT" && er.code !== "ENOTDIR") return cb(er)
- // error is ok. could be publishing a url or tarball
- // however, that means that we will not have automatically run
- // the prepublish script, since that gets run when adding a folder
- // to the cache.
- if (er) return cacheAddPublish(arg, false, isRetry, cb)
- cacheAddPublish(arg, true, isRetry, cb)
+ // do not allow packages with git+file:// deps to be published
+ needPrivate(data, function (err) {
+ if (err) return cb(err)
+ // error is ok. could be publishing a url or tarball
+ // however, that means that we will not have automatically run
+ // the prepublish script, since that gets run when adding a folder
+ // to the cache.
+ if (er) return cacheAddPublish(arg, false, isRetry, cb)
+ cacheAddPublish(arg, true, isRetry, cb)
+ })
})
}
@@ -115,3 +121,17 @@ function needVersion(er, data) {
: (data && !data.version) ? new Error("No version provided")
: null
}
+
+function needPrivate (data, cb) {
+ var types = [ "dependencies"
+ , "devDependencies"
+ , "optionalDependencies" ]
+ asyncMap(types, function (type, cb) {
+ Object.keys(data[type] || {}).forEach(function (k) {
+ if (isGitFileUrl(data.dependencies[k]))
+ return cb(new Error("Publishing modules with git+file:// " +
+ "dependencies is not allowed."))
+ })
+ cb()
+ }, cb)
+}
View
9 lib/utils/is-git-file-url.js
@@ -0,0 +1,9 @@
+
+module.exports = isGitFileUrl
+
+var url = require("url")
+
+function isGitFileUrl (s) {
+ if (url.parse(s).protocol === "git+file:")
+ return true
+}
View
1  lib/utils/is-git-url.js
@@ -8,6 +8,7 @@ function isGitUrl (url) {
case "git+rsync:":
case "git+ftp:":
case "git+ssh:":
+ case "git+file:":
return true
}
}
View
93 test/tap/install-file-private.js
@@ -0,0 +1,93 @@
+// this test needs git installed
+
+var test = require("tap").test
+var npm = require("../../")
+var osenv = require("osenv")
+var path = require("path")
+var fs = require("fs")
+var rimraf = require("rimraf")
+var mkdirp = require("mkdirp")
+var path = require("path")
+
+var spawn = require("child_process").spawn
+var node = process.execPath
+
+var pkg = process.env.npm_config_tmp || osenv.tmpdir()
+pkg += path.sep + "install-file-private"
+
+var gitPkg = "npm-test-git"
+var gitPath = osenv.tmpdir() + path.sep + gitPkg
+
+test("it installs local gitrepos", function (t) {
+ writePackageJson()
+ prepareGitRepo(cb)
+
+ process.chdir(pkg)
+
+ function cb () {
+ npm.load({cache: pkg + "/cache"}, function () {
+ npm.install(".", function (err) {
+ var md = pkg + "/node_modules/git-file/README.md"
+ var content = fs.readFileSync(md, "utf8")
+ t.equal("just a test with git", content)
+ t.end()
+ })
+ })
+ }
+})
+
+test("cleanup", function (t) {
+ // windows fix for locked files
+ process.chdir(osenv.tmpdir())
+
+ rimraf.sync(pkg)
+ rimraf.sync(gitPath)
+ t.end()
+})
+
+// this package.json will require the git repo
+function writePackageJson () {
+ rimraf.sync(pkg)
+ mkdirp.sync(pkg)
+ mkdirp.sync(pkg + "/cache")
+
+ fs.writeFileSync(pkg + "/package.json", JSON.stringify({
+ "author": "Rocko Artischocko",
+ "private": true,
+ "name": "git-file",
+ "version": "0.0.0",
+ "dependencies": {
+ "testpackage": "git+file://" + gitPath
+ }
+ }), 'utf8')
+}
+
+// prepare git repo
+function prepareGitRepo (cb) {
+ rimraf.sync(gitPath)
+ mkdirp.sync(gitPath)
+ var opts = { cwd: gitPath }
+ fs.writeFileSync(gitPath + "/package.json", JSON.stringify({
+ "author": "Rocko Artischocko",
+ "private": true,
+ "name": "git-file",
+ "version": "0.0.0",
+ "description": "fixture"
+ }), 'utf8')
+
+ fs.writeFileSync(gitPath + "/README.md"
+ , "just a test with git", 'utf8')
+
+ var childInit = spawn("git", ["init"], opts)
+ var childAdd, childCommit
+ childInit.on("close", function () {
+ childAdd = spawn("git", ["add", "."], opts)
+ childAdd.on("close", function () {
+ childCommit = spawn("git"
+ , ["commit", "-n", "-m", "test"], opts)
+ childCommit.on("close", function () {
+ cb()
+ })
+ })
+ })
+}
View
26 test/tap/publish-file-nonprivate.js
@@ -0,0 +1,26 @@
+var fs = require("fs")
+var test = require("tap").test
+var rimraf = require("rimraf")
+var npm = require("../../")
+
+var pkg = __dirname + "/publish-file-nonprivate"
+
+test("it should return an error", function (t) {
+ rimraf.sync(pkg + "/node_modules")
+ process.chdir(pkg)
+
+ npm.load({cache: pkg + "/cache"}, function () {
+ npm.publish(".", function (err) {
+ t.ok(err)
+ t.equal(err.message, "Publishing modules with git+file:// " +
+ "dependencies is not allowed.")
+ t.end()
+ })
+ })
+})
+
+test("cleanup", function (t) {
+ rimraf.sync(pkg + "/node_modules")
+ rimraf.sync(pkg + "/cache")
+ t.end()
+})
View
1  test/tap/publish-file-nonprivate/README.md
@@ -0,0 +1 @@
+just a test
View
9 test/tap/publish-file-nonprivate/package.json
@@ -0,0 +1,9 @@
+{
+ "name": "publish-file-nonprivate",
+ "author": "Rockbert",
+ "description": "fixture",
+ "version": "0.0.0",
+ "dependencies": {
+ "foo": "git+file://"
+ }
+}
Something went wrong with that request. Please try again.