Add support for ~/nvm/default-packages file #1463

westonganger · 2017-03-28T16:42:25Z

This PR adds support for a file containing default packages to be installed with every new version install.

While the reinstall-packages option is great, this file based approach is nice because I can check this file into my personal dotfiles repo.

Note: this idea was adapted from https://github.com/rbenv/rbenv-default-gems. I wanted a similar thing from my node version manager setup.

ljharb

Thanks for the contribution! This is an interesting idea. I also like the file-based approach.

I'm concerned about adding the complexity of this feature to an already complex install process.

What about npm linked items, which nvm reinstall-packages installs?

I do see in your example how a version range might be specified; what happens if an item is invalid? What happens if an item has spaces in it (which could cause npm to treat it as multiple arguments)?

What about packages where the package is only compatible with a certain node version range, or, where the version number that is compatible depends on the node version? For example, I wouldn't want to install eslint in a node version older than 4.

This would also need unit tests to be able to be merged.

ljharb · 2017-03-29T02:52:23Z

nvm.sh

+          DEFAULT_PACKAGES="${DEFAULT_PACKAGES}${line[0]} "
+        done < "${NVM_DIR}/default-packages"
+
+        if [ "$DEFAULT_PACKAGES" != "" ]; then


this should probably use [ -z "${DEFAULT_PACKAGES}" ]

ljharb · 2017-03-29T02:52:51Z

nvm.sh

+        while IFS=" " read -r -a line; do
+
+          # Skip empty lines.
+          [ "${#line[@]}" -gt 0 ] || continue


would [ -z "${line[@]}" ] work?

westonganger · 2017-03-29T04:22:10Z

I agree we should not go about adding a bunch of unnecessary complexity with this. But from what I have added so far is not very complicated.

For the lines with spaces in them, that should be invalid I assumed... We can easily provide an explanatory error for this but honestly this is something that is along the lines of garbage in, garbage out. The user should simply write a correct default-packages file.

For the node version compatibility with packages specified. Adding this compatibility check, in my opinion, is adding too much complexity to be worth it unless you know of a dead simple way. But regardless, the most common use case for a version manager is upgrading to newer versions, not installing really old versions. For these cases we should provide a --skip-default-packages option or alternatively they can simply comment out the offending package from their file before performing an install of an old incompatible version.

I will add the unit tests if you plan on merging this eventually.

westonganger · 2017-03-31T16:25:53Z

I added the changes you requested.
I have also now added the --skip-default-packages option.
If default-packages contains the space in a line you mentioned before or if the default packages install fails then it will now provide an error to the user telling them to check their default-packages files for errors.
I started adding tests but got a little confused as I havent done this testing before. So its still failings the newly added tests right now. Maybe if someone could help me out a little with the tests.

ljharb

Thanks! Sorry for the delay.

The doctoc check is failing because when editing the readme, you'll need to run npm run doctoc to update the table of contents.
shellcheck is failing because you're using arrays, and nvm is implemented in POSIX, not bash/zsh - no arrays are available, sadly.
The other failures seem like they might be legitimate; but probably easiest to clear up the others first and rerunning, before trying to debug.

ljharb · 2017-04-07T23:19:58Z

README.markdown

+If you have a list of default packages you want installed every time you install a new version we support that too. You can add any valid package, version, repo, url for installing an npm package.
+
+```sh
+# ~/.nvm/default-packages


let's do $NVM_DIR/default-packages here, just to try to avoid hard-coding the default

ljharb · 2017-04-07T23:20:25Z

README.markdown

@@ -216,6 +216,19 @@ nvm install 6 --reinstall-packages-from=5
 nvm install v4.2 --reinstall-packages-from=iojs
 ```

+### Default global packages from file while installing
+
+If you have a list of default packages you want installed every time you install a new version we support that too. You can add any valid package, version, repo, url for installing an npm package.


"for installing an npm package - anything npm would accept as a package argument on the command line."?

ljharb · 2017-04-07T23:21:29Z

nvm.sh

+            PROVIDED_REINSTALL_PACKAGES_FROM="$(nvm_echo "$1" | command cut -c 27-)"
+            REINSTALL_PACKAGES_FROM="$(nvm_version "$PROVIDED_REINSTALL_PACKAGES_FROM")" ||:
+          ;;
+          --copy-packages-from=*)


i think adding these lines means we can remove the ones on lines ~2467?

ljharb · 2017-04-07T23:24:20Z

test/fast/Unit tests/nvm_default_packages

+
+# commented-package
+
+webpack/webpack


let's pick some smaller packages for the tests (than webpack or gulp) and still-relevant ones (unlike bower) - how about rimraf, object-inspect, things like that?

ljharb · 2017-04-07T23:24:56Z

test/fast/Unit tests/nvm_default_packages

+cleanup
+
+cat > ../../../default-packages << EOF
+not-a-package-name


this could be a valid package name - maybe not~a~package~name or something?

ljharb · 2017-04-07T23:25:25Z

test/fast/Unit tests/nvm_default_packages

+not-a-package-name
+EOF
+
+OUTPUT="$(nvm exec 6.10.1 npm list -g | grep gulp)"


npm ls -g can take a --depth=0 argument which should help the test be faster and more accurate

westonganger · 2017-04-08T17:47:58Z

I have added the changes you requested.

By the way, Is there a way I can run only the unit tests or only a specific unit test?

ljharb · 2017-04-09T06:33:54Z

.gitignore

@@ -11,6 +11,7 @@ test/**/test_output

 node_modules/
 npm-debug.log
+.yarn-lock


yarn.lock should be gitignored; but I'm not sure what .yarn-lock is nor why it would occur in a project not using yarn?

Just because I installed the devDependencies using yarn its faster, I thought about this already before adding it to this file. Its just a .gitignore file, let it ignore.

Project-level gitignores should only contain project-related ignores; anything else should be in a global gitignore - otherwise you get every IDE's dotfiles polluting every repo.

If you wanted to ignore yarn.lock I'd understand, but .yarn-lock isn't a file that exists, as far as I'm aware.

ljharb · 2017-04-09T06:34:57Z

nvm.sh

@@ -2419,13 +2421,51 @@ nvm() {
            PROVIDED_REINSTALL_PACKAGES_FROM="$(nvm_echo "$1" | command cut -c 22-)"
            REINSTALL_PACKAGES_FROM="$(nvm_version "$PROVIDED_REINSTALL_PACKAGES_FROM")" ||:
          ;;
+          --skip-default-packages)
+            SKIP_DEFAULT_PACKAGES=true ||:


||: shouldn't be required here, since you're not doing anything that could possibly have a nonzero exit code.

ljharb · 2017-04-09T06:35:28Z

nvm.sh

          *)
            ADDITIONAL_PARAMETERS="$ADDITIONAL_PARAMETERS $1"
          ;;
        esac
        shift
      done

+      if [ -z "$SKIP_DEFAULT_PACKAGES" ] && [ -f "${NVM_DIR}/default-packages" ]; then
+        local DEFAULT_PACKAGES=""


please always declare local vars in a separate line from initializing them; this line as-is won't work in ksh which doesn't support local.

ljharb · 2017-04-09T06:36:13Z

nvm.sh

+          { 
+            nvm_echo "$DEFAULT_PACKAGES" | command xargs npm install -g --quiet 
+          } || {
+            nvm_echo "Failed installing default packages. Please check if your default-packages file or a package in it has problems!"


this is error output; it should use nvm_err instead of nvm_echo

ljharb · 2017-04-09T06:37:10Z

test/fast/Unit tests/nvm_default_packages

+
+die () { echo "$@" ; cleanup ; exit 1; }
+cleanup () {
+  rm -rf "$(nvm_version_path v6.10.1)" ../../../default-packages


tests are often run inside a working nvm installation; if I have a pre-existing default-packages file, this test will clobber it.

ljharb · 2017-04-09T06:37:48Z

@westonganger typically i just cd into the directory and ./test_file to run a single one

ljharb · 2017-04-09T06:38:41Z

nvm.sh

+          [ -z "${line}" ] || continue
+
+          # Skip comment lines that begin with `#`.
+          [ "${line:0:1}" != "#" ] || continue


the shellcheck job failed; it says string indexing isn't supported in dash.

westonganger · 2017-04-09T16:32:00Z

That makes testing much faster. Thanks!

I have now updated this with changes you requested.

ljharb

Thanks, this looks great!

There's some minor changes I can make on my own as I merge this.

ljharb · 2017-04-12T23:45:55Z

.gitignore

@@ -11,6 +11,7 @@ test/**/test_output

 node_modules/
 npm-debug.log
+.yarn-lock


Project-level gitignores should only contain project-related ignores; anything else should be in a global gitignore - otherwise you get every IDE's dotfiles polluting every repo.

If you wanted to ignore yarn.lock I'd understand, but .yarn-lock isn't a file that exists, as far as I'm aware.

ljharb · 2017-04-12T23:46:23Z

README.markdown

+
+gulp
+webpack
+bower@1.7.1


let's avoid using bower as an example here too (i can update this myself prior to merging tho, not a blocker)

ljharb · 2017-04-12T23:46:40Z

nvm.sh

          *)
            ADDITIONAL_PARAMETERS="$ADDITIONAL_PARAMETERS $1"
          ;;
        esac
        shift
      done

+      if [ -z "$SKIP_DEFAULT_PACKAGES" ] && [ -f "${NVM_DIR}/default-packages" ]; then
+        local DEFAULT_PACKAGES
+


i'll remove this extra blank line also

ljharb · 2017-04-12T23:47:36Z

test/fast/Unit tests/nvm_default_packages

+
+# commented-package
+
+stevemao/left-pad


ljharb · 2017-04-12T23:48:13Z

.gitignore

@@ -11,6 +11,7 @@ test/**/test_output

 node_modules/
 npm-debug.log
+.yarn-lock



we should probably ignore default-packages as well, so contributors don't accidentally commit theirs.

ljharb · 2017-04-12T23:49:02Z

Although, tests are still failing - so this will need to wait until they can be made to pass.

westonganger · 2017-04-13T01:27:46Z

I added the changes you have mentioned and continued working on the tests.

I am kind of stuck. Getting the following error: xargs: npm: No such file or directory

ljharb · 2017-04-13T02:08:31Z

nvm.sh

+        if [ -z "$DEFAULT_PACKAGES" ]; then
+          nvm_echo "Installing default global packages from ${NVM_DIR}/default-packages..."
+          { 
+            nvm_echo "$DEFAULT_PACKAGES" | command xargs npm install -g --quiet 


ah - you're trying to npm install before nvm use is ran on line 2488.

I think you should extract this block of code to a separate function; and then call that function on line 2495.5.

westonganger · 2017-04-13T02:52:34Z

I moved it to another function. Hopefully its what you meant.

ljharb · 2017-04-13T04:27:37Z

That looks great; there's other seemingly unrelated tests failing, presumably because they do nvm install.

Have you tested locally with set -e, to ensure that your new code doesn't cause nvm to exit out when there's no default packages, or also when the list installs properly?

westonganger · 2017-04-13T06:17:43Z

I have now done all the local testing and came up with some glaring holes, oops... I patched that all up, enhanced and fixed the tests. It now works like a charm under all conditions. I think this one is ready to go!

ljharb · 2017-04-13T06:40:44Z

nvm.sh

@@ -2441,9 +2471,12 @@ nvm() {
        FLAVOR="$(nvm_node_prefix)"
      fi

-      if nvm_is_version_installed "$VERSION"; then
+      if nvm_is_version_installed "$VERSION" && nvm use "$VERSION"; then
        nvm_err "$VERSION is already installed."


it's very intentional that the error message prints only if nvm_is_version_installed fails, and not if nvm use fails.

Good call was unsure about that one

ljharb · 2017-04-13T06:41:12Z

nvm.sh

+    nvm_echo "$1" | command xargs npm install -g --quiet && return 0
+  } || {
+    nvm_err "Failed installing default packages. Please check if your default-packages file or a package in it has problems!"
+    return 1


maybe this should capture the exit code from npm install?

Ya this definately had weird style. I have updated it to be more standard.

ljharb

the "fast" tests are still failing; oddly, they're nvm_get_checksum_alg and nvm_get_mirror which are simple unit tests.

ljharb · 2017-04-13T22:31:52Z

nvm.sh

@@ -2511,6 +2546,9 @@ nvm() {
        else
          nvm_ensure_default_set "$provided_version"
        fi
+        if [ -z "$SKIP_DEFAULT_PACKAGES" ] && [ -n "$DEFAULT_PACKAGES" ]; then
+          nvm_install_default_packages "$DEFAULT_PACKAGES"
+        fi
        if [ ! -z "$REINSTALL_PACKAGES_FROM" ] \


not sure why i used ! -z here, but this could become -n too :-)

lol. I looked at that but was like shit it aint broken, better not touch it

westonganger · 2017-04-13T23:09:49Z

I removed set -ex from those failing tests and they passed. However another different errored test popped up for bash only I can try to do the same for that test too.

Would you merge a different PR which removes all set -ex (or maybe just the x option) from the tests so that we dont have such a large output to sift through when viewing the test logs.

ljharb · 2017-04-13T23:50:14Z

That means the set -e is likely what caused the tests to fail; that's a critical part of the tests. the set -x is just there so when there is a failure, it's easier to debug it. Please do restore those.

westonganger · 2017-04-13T23:53:53Z

Alright these are back. I dont know what to do about those tests nothing should have changed regarding those tests.

westonganger · 2017-04-25T21:52:10Z

Can this be merged now?

ljharb · 2017-04-25T22:49:38Z

@westonganger I'm not sure what to do about the tests either; the implication is that your changes have affected those code paths.

I can't merge it unless tests are passing.

westonganger · 2017-04-26T00:36:03Z

So both of these tests run successfully on my local machine, why do they fail on the build?

roberttod · 2017-04-26T13:06:02Z

This looks really good! It would be great to manage the packages using nvm rather than editing the file, perhaps after this gets accepted.

ljharb · 2017-04-28T19:40:43Z

@westonganger so one of the failures is because your new function nvm_install_default_packages isn't being cleared by nvm unload. I've fixed that.

ljharb · 2017-04-28T23:27:12Z

Thanks for the contribution, and for bearing with the review!

mightyiam

This is great. Thank you!

augbog · 2017-05-10T18:15:19Z

Thank you for adding this!

Nantris · 2022-06-06T20:10:27Z

Is there a better way to upgrade default packages besides nvm install [same version of node] ?

ljharb · 2022-06-06T20:21:11Z

@slapbox there isn't a separate command, no - we could certainly add one if there's a use case. file a new issue?

[New] add support for $NVM_DIR/default-packages file

7f3145b

ljharb requested changes Mar 29, 2017

View reviewed changes

ljharb added feature requests I want a new feature in nvm! installing node Issues with installing node/io.js versions. labels Mar 29, 2017

ljharb requested changes Apr 7, 2017

View reviewed changes

ljharb requested changes Apr 9, 2017

View reviewed changes

ljharb reviewed Apr 9, 2017

View reviewed changes

ljharb approved these changes Apr 12, 2017

View reviewed changes

ljharb requested changes Apr 13, 2017

View reviewed changes

ljharb reviewed Apr 13, 2017

View reviewed changes

ljharb mentioned this pull request Apr 26, 2017

Post install hook #1342

Closed

ljharb approved these changes Apr 28, 2017

View reviewed changes

ljharb merged commit 7f3145b into nvm-sh:master Apr 28, 2017

ljharb mentioned this pull request Apr 28, 2017

Autotomatic reinstall-packages across node versions #977

Closed

mightyiam reviewed Apr 29, 2017

View reviewed changes

djpowers mentioned this pull request Oct 4, 2020

Configure default global node packages djpowers/dotfiles#10

Merged


		# commented-package

		webpack/webpack


		# commented-package

		stevemao/left-pad

Add support for ~/nvm/default-packages file #1463

Add support for ~/nvm/default-packages file #1463

Conversation

westonganger commented Mar 28, 2017

ljharb left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

westonganger commented Mar 29, 2017

westonganger commented Mar 31, 2017

ljharb left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

westonganger commented Apr 8, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ljharb commented Apr 9, 2017

Choose a reason for hiding this comment

westonganger commented Apr 9, 2017

ljharb left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ljharb commented Apr 12, 2017

westonganger commented Apr 13, 2017

Choose a reason for hiding this comment

westonganger commented Apr 13, 2017

ljharb commented Apr 13, 2017

westonganger commented Apr 13, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ljharb left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

westonganger commented Apr 13, 2017

ljharb commented Apr 13, 2017

westonganger commented Apr 13, 2017

westonganger commented Apr 25, 2017

ljharb commented Apr 25, 2017

westonganger commented Apr 26, 2017

roberttod commented Apr 26, 2017

ljharb commented Apr 28, 2017

ljharb commented Apr 28, 2017

mightyiam left a comment

Choose a reason for hiding this comment

augbog commented May 10, 2017

Nantris commented Jun 6, 2022

ljharb commented Jun 6, 2022