3.2: npm install completes before complaining about missing package.json

[75lb]

in npm 3.2, installing a package to a project without a package.json fails (ENOENT package.json).. but it performs all network/downloading first..

i work remotely on mobile data plans so pulling a lot of data, only to be told it failed, seems inefficient..

is it possible to test for package.json presence before tanning the network?

[iarna]

A missing package.json is a warning, not an error. It shouldn't stop your install from working.

[iarna]

Are you seeing it actually stop your install from working?

[75lb]

hi.. yes, it stops the install from working.. move into an empty directory, then:

$ npm i column-layout
/Users/lloyd/Documents/75lb/issues/empty
└── (empty)

npm WARN ENOENT ENOENT, open '/Users/lloyd/Documents/75lb/issues/empty/package.json'
npm WARN EPACKAGEJSON /Users/lloyd/Documents/75lb/issues/empty No description
npm WARN EPACKAGEJSON /Users/lloyd/Documents/75lb/issues/empty No repository field.
npm WARN EPACKAGEJSON /Users/lloyd/Documents/75lb/issues/empty No README data
npm WARN EPACKAGEJSON /Users/lloyd/Documents/75lb/issues/empty No license field.
npm ERR! code 1

$ npm ls
/Users/lloyd/Documents/75lb/issues/empty
└── (empty)

npm ERR! error in /Users/lloyd/Documents/75lb/issues/empty: ENOENT, open '/Users/lloyd/Documents/75lb/issues/empty/package.json'

[75lb]

the previous output was from npm 3.2.1

[saper]

Same here with npm master (915ed50) - so it looks like any successful npm install package in the empty directory results in failure due to ENOENT... although it does not, the package is installed.

#9161 is surprising, but this is even more...

[iarna]

hmm...

11:08 rebecca@Caldina:~/code/npmtest/9113$ node ~/code/npm i column-layout
/Users/rebecca/code/npmtest/9113
└── (empty)

npm WARN ENOENT ENOENT, open '/Users/rebecca/code/npmtest/9113/package.json'
npm WARN EPACKAGEJSON /Users/rebecca/code/npmtest/9113 No description
npm WARN EPACKAGEJSON /Users/rebecca/code/npmtest/9113 No repository field.
npm WARN EPACKAGEJSON /Users/rebecca/code/npmtest/9113 No README data
npm WARN EPACKAGEJSON /Users/rebecca/code/npmtest/9113 No license field.
11:08 rebecca@Caldina:~/code/npmtest/9113$ npm ls
/Users/rebecca/code/npmtest/9113
├── ansi-escape-sequences@2.2.1
├── array-back@1.0.2
├── array-tools@2.0.1
├── collect-all@0.1.0
├── collect-json@1.0.0
├── column-layout@1.0.4
├── command-line-args@1.0.0
├── command-line-usage@1.1.2
├── find-replace@1.0.0
├─┬ object-tools@2.0.3
│ ├── collect-json@0.1.1
│ └── test-value@0.1.0
├── sort-array@0.1.0
├── stream-connect@0.1.0
├── stream-via@0.1.0
├── string-tools@0.1.8
├── test-value@1.0.1
├── typical@2.3.1
└── word-wrap@1.1.0

So it seems to (mostly) work for me. It installed all the files correctly. The one thing it didn't do was list the module as being installed at first.

@75lb Could you try again with npm i column-layout --loglevel=silly 2> 9113.log and upload that somewhere for me to look at?

[saper]

First, I believe this Missing package.json warning is unnecessary or maybe even harmful.

npm indicates failure because exists with 1:

https://gist.github.com/3869151ee7e5a0cc4911

Yes, the files get installed correctly. You get the same error (as well as the package.json warning) and the error code 1 if the node_modules is already there with that module installed.

This is master (1fa9169) which happens to be npm 3.3.0

[saper]

That's interesting - on another box (also running FreeBSD) but with node v0.12.7 I don't get an NPM Err! message.

The log comes from FreeBSD box running io.js 3.0

node 0.12.7 configuration:

radziecki$ node -p process.versions
{ http_parser: '2.3',
  node: '0.12.7',
  v8: '3.28.71.19',
  uv: '1.6.1',
  zlib: '1.2.8',
  modules: '14',
  openssl: '1.0.1p' }
radziecki$ node -p process.config       
{ target_defaults: 
   { cflags: [],
     default_configuration: 'Release',
     defines: [],
     include_dirs: [],
     libraries: [ '-lz' ] },
  variables: 
   { clang: 1,
     host_arch: 'x64',
     icu_small: false,
     node_install_npm: false,
     node_prefix: '/usr/local',
     node_shared_cares: false,
     node_shared_http_parser: false,
     node_shared_libuv: false,
     node_shared_openssl: false,
     node_shared_v8: false,
     node_shared_zlib: true,
     node_tag: '',
     node_use_dtrace: false,
     node_use_etw: false,
     node_use_mdb: false,
     node_use_openssl: true,
     node_use_perfctr: false,
     openssl_no_asm: 0,
     python: '/usr/local/bin/python2.7',
     target_arch: 'x64',
     uv_library: 'static_library',
     uv_parent_path: '/deps/uv/',
     uv_use_dtrace: false,
     v8_enable_gdbjit: 0,
     v8_enable_i18n_support: 0,
     v8_no_strict_aliasing: 1,
     v8_optimized_debug: 0,
     v8_random_seed: 0,
     v8_use_snapshot: true,
     want_separate_host_toolset: 0 } }

iojs 3.0 configuration (it's debug):

m> node -p process.versions
{ http_parser: '2.5.0',
  node: '3.0.0',
  v8: '4.4.63.26',
  uv: '1.6.1',
  zlib: '1.2.8',
  ares: '1.10.1-DEV',
  modules: '45',
  openssl: '1.0.2d' }
m> node -p process.config
{ target_defaults: 
   { cflags: [],
     default_configuration: 'Debug',
     defines: [],
     include_dirs: [],
     libraries: [ '-lz' ] },
  variables: 
   { host_arch: 'x64',
     icu_small: false,
     llvm_version: 0,
     node_byteorder: 'little',
     node_install_npm: false,
     node_prefix: '/usr/local',
     node_release_urlbase: '',
     node_shared_http_parser: false,
     node_shared_libuv: false,
     node_shared_openssl: false,
     node_shared_zlib: true,
     node_tag: '',
     node_use_dtrace: false,
     node_use_etw: false,
     node_use_lttng: false,
     node_use_openssl: true,
     node_use_perfctr: false,
     openssl_fips: '',
     openssl_no_asm: 0,
     python: '/usr/local/bin/python2.7',
     target_arch: 'x64',
     uv_parent_path: '/deps/uv/',
     uv_use_dtrace: false,
     v8_enable_gdbjit: 0,
     v8_enable_i18n_support: 0,
     v8_no_strict_aliasing: 1,
     v8_optimized_debug: 0,
     v8_random_seed: 0,
     v8_use_snapshot: 1,
     want_separate_host_toolset: 0 } }

[75lb]

@iarna ok, this time i used npm@3.3.0.. here is the npm i column-layout --loglevel=silly output you requested: https://gist.github.com/75lb/15086238cf6f4905a5c7

some more output:

$ node -v
v0.12.7

$ npm ls
/Users/lloyd/Documents/75lb/gists/9113
└── (empty)

npm ERR! error in /Users/lloyd/Documents/75lb/gists/9113: ENOENT, open '/Users/lloyd/Documents/75lb/gists/9113/package.json'

$ tree
.
├── 9113.log
└── node_modules
    ├── ansi-escape-sequences
    │   ├── LICENSE
    │   ├── README.md
    │   ├── bin
    │   │   └── cli.js
    │   ├── jsdoc2md
    │   │   └── README.hbs
    │   ├── lib
    │   │   └── ansi-escape-sequences.js
    │   ├── package.json
    │   └── test
    │       └── test.js
    ├── array-back
    │   ├── LICENSE
    │   ├── README.md
    │   ├── jsdoc2md
    │   │   └── README.hbs
    │   ├── lib
    │   │   └── array-back.js
    │   ├── package.json
    │   └── test
    │       └── test.js
    ├── array-tools
    │   ├── LICENSE
    │   ├── README.md
    │   ├── bin
    │   │   └── cli.js
    │   ├── bower.json
    │   ├── dist
    │   │   ├── array-tools.js
    │   │   └── array-tools.min.js
    │   ├── jsdoc2md
    │   │   └── README.hbs
    │   ├── lib
    │   │   └── array-tools.js
    │   ├── package.json
    │   └── test
    │       ├── chaining.js
    │       ├── cli.js
    │       ├── contains.js
    │       ├── exists.js
    │       ├── extract.js
    │       ├── findWhere.js
    │       ├── fixture
    │       │   └── array.json
    │       ├── groupby.js
    │       ├── pick.js
    │       ├── pluck.js
    │       ├── remove.js
    │       ├── spliceWhile.js
    │       ├── unique.js
    │       ├── where.js
    │       └── without.js
    ├── collect-all
    │   ├── LICENSE
    │   ├── README.md
    │   ├── jsdoc2md
    │   │   └── README.hbs
    │   ├── lib
    │   │   └── collect-all.js
    │   ├── package.json
    │   └── test
    │       └── test.js
    ├── collect-json
    │   ├── LICENSE
    │   ├── README.md
    │   ├── bin
    │   │   └── cli.js
    │   ├── jsdoc2md
    │   │   └── README.hbs
    │   ├── lib
    │   │   └── collect-json.js
    │   ├── package.json
    │   └── test
    │       └── test.js
    ├── column-layout
    │   ├── LICENSE
    │   ├── README.md
    │   ├── bin
    │   │   └── cli.js
    │   ├── example
    │   │   ├── five-columns-width.json
    │   │   ├── five-columns.json
    │   │   ├── no-columns.json
    │   │   ├── one-column-width.json
    │   │   ├── one-column.json
    │   │   ├── table.json
    │   │   ├── two-columns.json
    │   │   └── usage.json
    │   ├── jsdoc2md
    │   │   └── README.hbs
    │   ├── lib
    │   │   └── column-layout.js
    │   ├── package.json
    │   └── test
    │       ├── fixture
    │       │   ├── empty-column.json
    │       │   └── new-line.json
    │       └── test.js
    ├── command-line-args
    │   ├── LICENSE
    │   ├── README.md
    │   ├── bin
    │   │   └── cli.js
    │   ├── example
    │   │   ├── alias.js
    │   │   ├── defaultValue.js
    │   │   ├── group.js
    │   │   ├── multiple.js
    │   │   ├── name.js
    │   │   ├── type.js
    │   │   ├── typical.js
    │   │   └── validate.js
    │   ├── jsdoc2md
    │   │   └── README.hbs
    │   ├── lib
    │   │   ├── command-line-args.js
    │   │   ├── definition.js
    │   │   ├── definitions.js
    │   │   └── option.js
    │   ├── package.json
    │   └── test
    │       ├── alias.js
    │       ├── bad-input.js
    │       ├── default-option.js
    │       ├── default-value.js
    │       ├── detect-process-argv.js
    │       ├── getopt.js
    │       ├── grouping.js
    │       ├── missing-def.js
    │       ├── name-alias-mix.js
    │       ├── name-equals.js
    │       ├── type-boolean-multiple.js
    │       ├── type-boolean.js
    │       ├── type-invalid.js
    │       ├── type-none.js
    │       ├── type-number-multiple.js
    │       ├── type-number.js
    │       ├── type-other-multiple.js
    │       ├── type-other.js
    │       ├── type-string.js
    │       └── usage.js
    ├── command-line-usage
    │   ├── LICENSE
    │   ├── README.md
    │   ├── bin
    │   │   └── cli.js
    │   ├── example
    │   │   ├── my-app.js
    │   │   ├── screens
    │   │   │   └── typical.png
    │   │   ├── typical-formatted.js
    │   │   └── typical.js
    │   ├── jsdoc2md
    │   │   └── README.hbs
    │   ├── lib
    │   │   ├── command-line-usage.js
    │   │   └── usage-options.js
    │   ├── package.json
    │   └── test
    │       └── test.js
    ├── find-replace
    │   ├── LICENSE
    │   ├── README.md
    │   ├── bin
    │   │   └── cli.js
    │   ├── jsdoc2md
    │   │   └── README.hbs
    │   ├── lib
    │   │   └── find-replace.js
    │   ├── package.json
    │   └── test
    │       └── test.js
    ├── object-tools
    │   ├── README.md
    │   ├── bin
    │   │   └── cli.js
    │   ├── bower.json
    │   ├── jsdoc2md
    │   │   └── README.hbs
    │   ├── lib
    │   │   └── object-tools.js
    │   ├── node_modules
    │   │   ├── collect-json
    │   │   │   ├── LICENSE
    │   │   │   ├── README.md
    │   │   │   ├── bin
    │   │   │   │   └── cli.js
    │   │   │   ├── jsdoc2md
    │   │   │   │   └── README.hbs
    │   │   │   ├── lib
    │   │   │   │   └── collect-json.js
    │   │   │   ├── package.json
    │   │   │   └── test
    │   │   │       └── test.js
    │   │   └── test-value
    │   │       ├── LICENSE
    │   │       ├── README.md
    │   │       ├── jsdoc2md
    │   │       │   └── README.hbs
    │   │       ├── lib
    │   │       │   └── test-value.js
    │   │       ├── package.json
    │   │       └── test
    │   │           └── test.js
    │   ├── package.json
    │   └── test
    │       ├── cli.js
    │       ├── clone.js
    │       ├── exists.js
    │       ├── expression.js
    │       ├── extend.js
    │       ├── extract.js
    │       ├── select.js
    │       ├── where.js
    │       └── without.js
    ├── sort-array
    │   ├── LICENSE
    │   ├── README.md
    │   ├── jsdoc2md
    │   │   └── README.hbs
    │   ├── lib
    │   │   └── sort-array.js
    │   ├── package.json
    │   └── test
    │       └── test.js
    ├── stream-connect
    │   ├── LICENSE
    │   ├── README.md
    │   ├── jsdoc2md
    │   │   └── README.hbs
    │   ├── lib
    │   │   └── stream-connect.js
    │   ├── package.json
    │   └── test
    │       └── test.js
    ├── stream-via
    │   ├── LICENSE
    │   ├── README.md
    │   ├── jsdoc2md
    │   │   └── README.hbs
    │   ├── lib
    │   │   └── stream-via.js
    │   ├── package.json
    │   └── test
    │       └── test.js
    ├── string-tools
    │   ├── README.md
    │   ├── jsdoc2md
    │   │   └── README.hbs
    │   ├── lib
    │   │   └── string-tools.js
    │   ├── package.json
    │   └── test
    │       └── test.js
    ├── test-value
    │   ├── LICENSE
    │   ├── README.md
    │   ├── jsdoc2md
    │   │   └── README.hbs
    │   ├── lib
    │   │   └── test-value.js
    │   ├── package.json
    │   └── test
    │       └── test.js
    ├── typical
    │   ├── README.md
    │   ├── jsdoc2md
    │   │   └── README.hbs
    │   ├── lib
    │   │   └── typical.js
    │   ├── package.json
    │   └── test
    │       └── test.js
    └── word-wrap
        ├── LICENSE
        ├── README.md
        ├── index.js
        └── package.json

91 directories, 190 files

[saper]

Here's silly log from me: https://gist.github.com/1c9e0ce9bce103670da7

[saper]

-- it seems to me that both machines fail now the same way:(

[iarna]

Ok, so this is caused by two things, but the biggie is column-layout being required by one of its own deps:

column-layout → command-line-args → command-line-usage → column-layout

This confuses the logical tree generator…

[saper]

Nobody promised that dependency graph is a tree :) and the little one?

[iarna]

¯_(ツ)_/¯ I don't honestly know where I was going w/ the little one thing.

And no, no one imagines our dependency graphs are always trees, this bug had to with how orphan package status was computed when we were transforming a physical tree (representing the folders on disk) into a logical tree (representing the dependency graph). Orphans being packages that, when you look up at the things that depend on them and the things that depend on those, never ends up depending on the root level module.

Basically it was checking to see if something was orphaned by seeing if anything depended on it. If nothing depended on it then it grafted it on to the root of the tree. But of course, with a dependency cycle that means that everything in that cycle is depended on by something, without ever connecting to the root. As such, we weren't attaching them to the root of the tree. Fortunately, we keep track of modules the user installed and modules that are otherwise unaccounted for and so we can look at that to determine if the module should get grafted back to the root.

[othiym23]

Fixed in part by d119ea6, and in part by 02420dc, included in npm@3.3.1. Thanks for all your help and patience, @75lb!