This repository has been archived by the owner. It is now read-only.

npm publish missing index.js #5082

Closed
pgilad opened this Issue Apr 15, 2014 · 129 comments

Comments

Projects
None yet
@pgilad

pgilad commented Apr 15, 2014

Happened to me twice already, while publishing a package I wrote (https://github.com/pgilad/gulp-angular-htmlify if it matters), the package was published without index.js.

I had no notice or info that anything went wrong during npm publish phase. But then got notified by a user that the package wasn't installing with index.js.

The only thing that I can think of that caused this, other than a bug, is that I had the project open in Vim, that may have caused a failure to publish the file.

Did anyone run into a similar case? Is there anyway to verify the package contents other than installing it as a dependency and seeing what's installed?

This happened with:
git version 1.8.3.2
npm version 1.4.6
node version 0.10.26
ubuntu 13.10

@pgilad pgilad referenced this issue Apr 16, 2014

Closed

Install broken #2

@isaacs

This comment has been minimized.

Show comment
Hide comment
@isaacs

isaacs Apr 16, 2014

Member

I'm not sure what's going on there. Using npm version 1.4.7, I'm seeing it include the index.js when it creates a tarball from that git repo.

[gh:npm/servers/master]isaacs@urdva:~/dev/npm/servers v0.10.27-pre
$ npm cache add git+https://github.com/pgilad/gulp-angular-htmlify.git

[gh:npm/servers/master]isaacs@urdva:~/dev/npm/servers v0.10.27-pre
$ npm cache ls gulp-angular-htmlify
~/.npm/gulp-angular-htmlify
~/.npm/gulp-angular-htmlify/0.0.8/
~/.npm/gulp-angular-htmlify/0.0.8/package/
~/.npm/gulp-angular-htmlify/0.0.8/package.tgz
~/.npm/gulp-angular-htmlify/0.0.8/package/.editorconfig
~/.npm/gulp-angular-htmlify/0.0.8/package/.gitattributes
~/.npm/gulp-angular-htmlify/0.0.8/package/.gitignore
~/.npm/gulp-angular-htmlify/0.0.8/package/.jshintrc
~/.npm/gulp-angular-htmlify/0.0.8/package/.npmignore
~/.npm/gulp-angular-htmlify/0.0.8/package/.travis.yml
~/.npm/gulp-angular-htmlify/0.0.8/package/fixtures/
~/.npm/gulp-angular-htmlify/0.0.8/package/fixtures/angular-basic.html
~/.npm/gulp-angular-htmlify/0.0.8/package/fixtures/angular-complex.html
~/.npm/gulp-angular-htmlify/0.0.8/package/fixtures/angular-custom.html
~/.npm/gulp-angular-htmlify/0.0.8/package/fixtures/angular-templates.html
~/.npm/gulp-angular-htmlify/0.0.8/package/fixtures/noangular.html
~/.npm/gulp-angular-htmlify/0.0.8/package/index.js
~/.npm/gulp-angular-htmlify/0.0.8/package/license.md
~/.npm/gulp-angular-htmlify/0.0.8/package/package.json
~/.npm/gulp-angular-htmlify/0.0.8/package/readme.md
~/.npm/gulp-angular-htmlify/0.0.8/package/test.js

Can you provide any other details? If you do npm pack && tar ztvf gulp-angular-htmlify-0.0.8.tgz, what does it output?

Member

isaacs commented Apr 16, 2014

I'm not sure what's going on there. Using npm version 1.4.7, I'm seeing it include the index.js when it creates a tarball from that git repo.

[gh:npm/servers/master]isaacs@urdva:~/dev/npm/servers v0.10.27-pre
$ npm cache add git+https://github.com/pgilad/gulp-angular-htmlify.git

[gh:npm/servers/master]isaacs@urdva:~/dev/npm/servers v0.10.27-pre
$ npm cache ls gulp-angular-htmlify
~/.npm/gulp-angular-htmlify
~/.npm/gulp-angular-htmlify/0.0.8/
~/.npm/gulp-angular-htmlify/0.0.8/package/
~/.npm/gulp-angular-htmlify/0.0.8/package.tgz
~/.npm/gulp-angular-htmlify/0.0.8/package/.editorconfig
~/.npm/gulp-angular-htmlify/0.0.8/package/.gitattributes
~/.npm/gulp-angular-htmlify/0.0.8/package/.gitignore
~/.npm/gulp-angular-htmlify/0.0.8/package/.jshintrc
~/.npm/gulp-angular-htmlify/0.0.8/package/.npmignore
~/.npm/gulp-angular-htmlify/0.0.8/package/.travis.yml
~/.npm/gulp-angular-htmlify/0.0.8/package/fixtures/
~/.npm/gulp-angular-htmlify/0.0.8/package/fixtures/angular-basic.html
~/.npm/gulp-angular-htmlify/0.0.8/package/fixtures/angular-complex.html
~/.npm/gulp-angular-htmlify/0.0.8/package/fixtures/angular-custom.html
~/.npm/gulp-angular-htmlify/0.0.8/package/fixtures/angular-templates.html
~/.npm/gulp-angular-htmlify/0.0.8/package/fixtures/noangular.html
~/.npm/gulp-angular-htmlify/0.0.8/package/index.js
~/.npm/gulp-angular-htmlify/0.0.8/package/license.md
~/.npm/gulp-angular-htmlify/0.0.8/package/package.json
~/.npm/gulp-angular-htmlify/0.0.8/package/readme.md
~/.npm/gulp-angular-htmlify/0.0.8/package/test.js

Can you provide any other details? If you do npm pack && tar ztvf gulp-angular-htmlify-0.0.8.tgz, what does it output?

@pgilad

This comment has been minimized.

Show comment
Hide comment
@pgilad

pgilad Apr 17, 2014

Hi this is something inconsistent that happened twice (on 2 version releases).
It happened on v0.0.6, and when I do:

$ npm cache add gulp-angular-htmlify@0.0.6
npm http GET https://registry.npmjs.org/gulp-angular-htmlify/0.0.6
npm http 200 https://registry.npmjs.org/gulp-angular-htmlify/0.0.6
npm http GET https://registry.npmjs.org/gulp-angular-htmlify/-/gulp-angular-htmlify-0.0.6.tgz
npm http 200 https://registry.npmjs.org/gulp-angular-htmlify/-/gulp-angular-htmlify-0.0.6.tgz

$ npm cache ls gulp-angular-htmlify
~\AppData\Roaming\npm-cache\gulp-angular-htmlify
~\AppData\Roaming\npm-cache\gulp-angular-htmlify\0.0.6\
~\AppData\Roaming\npm-cache\gulp-angular-htmlify\0.0.6\.cache.json
~\AppData\Roaming\npm-cache\gulp-angular-htmlify\0.0.6\package\
~\AppData\Roaming\npm-cache\gulp-angular-htmlify\0.0.6\package.tgz
~\AppData\Roaming\npm-cache\gulp-angular-htmlify\0.0.6\package\package.json
~\AppData\Roaming\npm-cache\gulp-angular-htmlify\0.0.6\package\readme.md

So you see it's missing index.js for some reason, and has an unusal .cache.json file. What other details can I provide?

If I run your command I see:

npm pack && tar ztvf gulp-angular-htmlify-0.0.8.tgz
gulp-angular-htmlify-0.0.8.tgz
tar: Record size = 18 blocks
-rw-rw-rw- 0/0             883 2014-04-17 11:00 package/package.json
-rw-rw-rw- 0/0            2094 2014-04-17 11:00 package/index.js
-rw-rw-rw- 0/0            2964 2014-04-17 11:00 package/readme.md

pgilad commented Apr 17, 2014

Hi this is something inconsistent that happened twice (on 2 version releases).
It happened on v0.0.6, and when I do:

$ npm cache add gulp-angular-htmlify@0.0.6
npm http GET https://registry.npmjs.org/gulp-angular-htmlify/0.0.6
npm http 200 https://registry.npmjs.org/gulp-angular-htmlify/0.0.6
npm http GET https://registry.npmjs.org/gulp-angular-htmlify/-/gulp-angular-htmlify-0.0.6.tgz
npm http 200 https://registry.npmjs.org/gulp-angular-htmlify/-/gulp-angular-htmlify-0.0.6.tgz

$ npm cache ls gulp-angular-htmlify
~\AppData\Roaming\npm-cache\gulp-angular-htmlify
~\AppData\Roaming\npm-cache\gulp-angular-htmlify\0.0.6\
~\AppData\Roaming\npm-cache\gulp-angular-htmlify\0.0.6\.cache.json
~\AppData\Roaming\npm-cache\gulp-angular-htmlify\0.0.6\package\
~\AppData\Roaming\npm-cache\gulp-angular-htmlify\0.0.6\package.tgz
~\AppData\Roaming\npm-cache\gulp-angular-htmlify\0.0.6\package\package.json
~\AppData\Roaming\npm-cache\gulp-angular-htmlify\0.0.6\package\readme.md

So you see it's missing index.js for some reason, and has an unusal .cache.json file. What other details can I provide?

If I run your command I see:

npm pack && tar ztvf gulp-angular-htmlify-0.0.8.tgz
gulp-angular-htmlify-0.0.8.tgz
tar: Record size = 18 blocks
-rw-rw-rw- 0/0             883 2014-04-17 11:00 package/package.json
-rw-rw-rw- 0/0            2094 2014-04-17 11:00 package/index.js
-rw-rw-rw- 0/0            2964 2014-04-17 11:00 package/readme.md
@pgilad

This comment has been minimized.

Show comment
Hide comment
@pgilad

pgilad Apr 24, 2014

Hi @isaacs this happened to me again on a different project I'm working on, perhaps this is machine related (Ubuntu 13.04 + 14)

I ran the following commands:

[master][~/repos/gulp-sitemap]$ npm publish
npm http PUT https://registry.npmjs.org/gulp-sitemap
npm http 201 https://registry.npmjs.org/gulp-sitemap
+ gulp-sitemap@0.2.0
[master][~/repos/gulp-sitemap]$ npm cache add gulp-sitemap@0.2.0
npm http GET https://registry.npmjs.org/gulp-sitemap
npm http 200 https://registry.npmjs.org/gulp-sitemap
npm http GET https://registry.npmjs.org/gulp-sitemap/-/gulp-sitemap-0.2.0.tgz
npm http 200 https://registry.npmjs.org/gulp-sitemap/-/gulp-sitemap-0.2.0.tgz
[master][~/repos/gulp-sitemap]$ npm cache ls gulp-sitemap
~/.npm/gulp-sitemap
~/.npm/gulp-sitemap/.cache.json
~/.npm/gulp-sitemap/0.2.0/
~/.npm/gulp-sitemap/0.2.0/package/
~/.npm/gulp-sitemap/0.2.0/package.tgz
~/.npm/gulp-sitemap/0.2.0/package/.npmignore
~/.npm/gulp-sitemap/0.2.0/package/license.md
~/.npm/gulp-sitemap/0.2.0/package/package.json
~/.npm/gulp-sitemap/0.2.0/package/readme.md

As you can see, for some reason it packed .npmignore and it didn't include index.js. I had to republish another version.

Any clue on this?

pgilad commented Apr 24, 2014

Hi @isaacs this happened to me again on a different project I'm working on, perhaps this is machine related (Ubuntu 13.04 + 14)

I ran the following commands:

[master][~/repos/gulp-sitemap]$ npm publish
npm http PUT https://registry.npmjs.org/gulp-sitemap
npm http 201 https://registry.npmjs.org/gulp-sitemap
+ gulp-sitemap@0.2.0
[master][~/repos/gulp-sitemap]$ npm cache add gulp-sitemap@0.2.0
npm http GET https://registry.npmjs.org/gulp-sitemap
npm http 200 https://registry.npmjs.org/gulp-sitemap
npm http GET https://registry.npmjs.org/gulp-sitemap/-/gulp-sitemap-0.2.0.tgz
npm http 200 https://registry.npmjs.org/gulp-sitemap/-/gulp-sitemap-0.2.0.tgz
[master][~/repos/gulp-sitemap]$ npm cache ls gulp-sitemap
~/.npm/gulp-sitemap
~/.npm/gulp-sitemap/.cache.json
~/.npm/gulp-sitemap/0.2.0/
~/.npm/gulp-sitemap/0.2.0/package/
~/.npm/gulp-sitemap/0.2.0/package.tgz
~/.npm/gulp-sitemap/0.2.0/package/.npmignore
~/.npm/gulp-sitemap/0.2.0/package/license.md
~/.npm/gulp-sitemap/0.2.0/package/package.json
~/.npm/gulp-sitemap/0.2.0/package/readme.md

As you can see, for some reason it packed .npmignore and it didn't include index.js. I had to republish another version.

Any clue on this?

@hiddentao

This comment has been minimized.

Show comment
Hide comment
@hiddentao

hiddentao Jul 18, 2014

Seeing something similar with with my module - gulp-bench. I tried deleting my .npmignore prior to npm pack, all to no avail.

hiddentao commented Jul 18, 2014

Seeing something similar with with my module - gulp-bench. I tried deleting my .npmignore prior to npm pack, all to no avail.

@hiddentao

This comment has been minimized.

Show comment
Hide comment
@hiddentao

hiddentao Jul 18, 2014

In my case the problem was this line in package.json:

files: ['index.js']

Duh. To get it working I changed it to:

main: 'index.js'

hiddentao commented Jul 18, 2014

In my case the problem was this line in package.json:

files: ['index.js']

Duh. To get it working I changed it to:

main: 'index.js'
@pascalduez

This comment has been minimized.

Show comment
Hide comment
@pascalduez

pascalduez Aug 19, 2014

Same issue here happening on various projects. (i.e: grunt-sassdoc, gulp-sassdoc)

The problem is this issue is not consistent, sometimes it's fine, publishing the package with the right files, but sometimes it randomly exclude some. Often the index.js but I've also had the case with other files, like Readme and co.

I also switched from files: ['index.js'] to main: 'index.js', it makes no difference.

node v0.10.30
npm v1.4.23
OSX

pascalduez commented Aug 19, 2014

Same issue here happening on various projects. (i.e: grunt-sassdoc, gulp-sassdoc)

The problem is this issue is not consistent, sometimes it's fine, publishing the package with the right files, but sometimes it randomly exclude some. Often the index.js but I've also had the case with other files, like Readme and co.

I also switched from files: ['index.js'] to main: 'index.js', it makes no difference.

node v0.10.30
npm v1.4.23
OSX

wmluke added a commit to wmluke/gulp-inline-angular-templates that referenced this issue Sep 7, 2014

Per npm/npm#5082
* Removed `files` field from package.json
* Added `npmignore`
@othiym23

This comment has been minimized.

Show comment
Hide comment
@othiym23

othiym23 Sep 21, 2014

Contributor

Is this issue still occurring with npm@2.0.0?

Contributor

othiym23 commented Sep 21, 2014

Is this issue still occurring with npm@2.0.0?

@pascalduez

This comment has been minimized.

Show comment
Hide comment
@pascalduez

pascalduez Sep 21, 2014

Is this issue still occurring with npm@2.0.0?

Hard to tell at that point, since it's happening kind of randomly.
I'll report back whenever it happens.

pascalduez commented Sep 21, 2014

Is this issue still occurring with npm@2.0.0?

Hard to tell at that point, since it's happening kind of randomly.
I'll report back whenever it happens.

@othiym23

This comment has been minimized.

Show comment
Hide comment
@othiym23

othiym23 Sep 21, 2014

Contributor

OK. I'm going to close this for now, because I believe it's been addressed by one of the many fixes that we've made around race conditions. If it happens again, open a new ticket. Even better would be if you can include a link to a gist with the output of npm publish -ddd for a bad publish. Thanks for your patience and help!

Contributor

othiym23 commented Sep 21, 2014

OK. I'm going to close this for now, because I believe it's been addressed by one of the many fixes that we've made around race conditions. If it happens again, open a new ticket. Even better would be if you can include a link to a gist with the output of npm publish -ddd for a bad publish. Thanks for your patience and help!

@substack

This comment has been minimized.

Show comment
Hide comment
@substack

substack Sep 26, 2014

Contributor

I'm getting this issue every other npm publish. I'm on npm version 2.0.2.

Contributor

substack commented Sep 26, 2014

I'm getting this issue every other npm publish. I'm on npm version 2.0.2.

@othiym23

This comment has been minimized.

Show comment
Hide comment
@othiym23

othiym23 Sep 26, 2014

Contributor

@substack thanks for finding this issue. I'm going to reopen and ask that if anyone on this thread can put together a repro case for this issue, they should give me a shout. This bites James hard because he publishes so many packages, but it's a serious issue to ever be happening at all, and I want to fix it!

Contributor

othiym23 commented Sep 26, 2014

@substack thanks for finding this issue. I'm going to reopen and ask that if anyone on this thread can put together a repro case for this issue, they should give me a shout. This bites James hard because he publishes so many packages, but it's a serious issue to ever be happening at all, and I want to fix it!

@othiym23 othiym23 reopened this Sep 26, 2014

@othiym23

This comment has been minimized.

Show comment
Hide comment
@othiym23

othiym23 Sep 26, 2014

Contributor

@substack does npm ls or @thlorenz's irish-pub show anything weird on any of the packages that have had this problem? Can you get it to happen just running npm pack?

Contributor

othiym23 commented Sep 26, 2014

@substack does npm ls or @thlorenz's irish-pub show anything weird on any of the packages that have had this problem? Can you get it to happen just running npm pack?

@othiym23 othiym23 added this to the 2.1.4 milestone Oct 4, 2014

@othiym23 othiym23 added the next-patch label Oct 7, 2014

@othiym23 othiym23 removed this from the 2.1.4 milestone Oct 7, 2014

@pgilad

This comment has been minimized.

Show comment
Hide comment
@pgilad

pgilad Oct 19, 2014

I haven't encountered this bug for a very long time (perhaps since reported). On another note - I don't use .npmignore anymore, but use the files array in package.json.

pgilad commented Oct 19, 2014

I haven't encountered this bug for a very long time (perhaps since reported). On another note - I don't use .npmignore anymore, but use the files array in package.json.

@othiym23

This comment has been minimized.

Show comment
Hide comment
@othiym23

othiym23 Oct 20, 2014

Contributor

Thanks, @pgilad. @substack, when was the last time you saw this issue? I'm hesitant to close it, because I still don't know what could have been / was causing it, but it's lower priority if it's not currently affecting anyone.

Contributor

othiym23 commented Oct 20, 2014

Thanks, @pgilad. @substack, when was the last time you saw this issue? I'm hesitant to close it, because I still don't know what could have been / was causing it, but it's lower priority if it's not currently affecting anyone.

@pascalduez

This comment has been minimized.

Show comment
Hide comment
@pascalduez

pascalduez Oct 20, 2014

when was the last time you saw this issue?

Well, I encountered it like a week ago.

Still using .npmignore.

pascalduez commented Oct 20, 2014

when was the last time you saw this issue?

Well, I encountered it like a week ago.

Still using .npmignore.

@othiym23

This comment has been minimized.

Show comment
Hide comment
@othiym23

othiym23 Oct 21, 2014

Contributor

@pascalduez with which version of npm?

Contributor

othiym23 commented Oct 21, 2014

@pascalduez with which version of npm?

jyoko added a commit to jyoko/socket.io-amqp-emitter that referenced this issue Jul 15, 2016

Fix npm publish (v2)
* see npm/npm#5082 (removed node_modules/)

derhuerst added a commit to derhuerst/vbb-stations-cli that referenced this issue Jul 26, 2016

0.7.1
npm/npm#5082 again 😭

@legodude17 legodude17 referenced this issue Sep 12, 2016

Closed

npm publish doesn't include README.md #13929

1 of 13 tasks complete
@jbeckton

This comment has been minimized.

Show comment
Hide comment
@jbeckton

jbeckton Sep 21, 2016

using npm pack and having this issue my index.js file is not included in the package

jbeckton commented Sep 21, 2016

using npm pack and having this issue my index.js file is not included in the package

@othiym23

This comment has been minimized.

Show comment
Hide comment
@othiym23

othiym23 Sep 21, 2016

Contributor

@jbeckton what does npm -v return?

Contributor

othiym23 commented Sep 21, 2016

@jbeckton what does npm -v return?

@jbeckton

This comment has been minimized.

Show comment
Hide comment
@jbeckton

jbeckton commented Sep 21, 2016

@othiym23 3.9.5

@jbeckton

This comment has been minimized.

Show comment
Hide comment
@jbeckton

jbeckton Sep 21, 2016

updated to the current version and the problem went away.

jbeckton commented Sep 21, 2016

updated to the current version and the problem went away.

@busches

This comment has been minimized.

Show comment
Hide comment
@busches

busches Sep 22, 2016

Good to close then?

On Wed, Sep 21, 2016 at 6:14 PM, Jesse Beckton notifications@github.com
wrote:

updated to the current version and the problem went away.


You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
#5082 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/AA0WZOtZ6m_kRre7xOwPrk-ZDVsgWfyIks5qsbpLgaJpZM4Bywuj
.

busches commented Sep 22, 2016

Good to close then?

On Wed, Sep 21, 2016 at 6:14 PM, Jesse Beckton notifications@github.com
wrote:

updated to the current version and the problem went away.


You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
#5082 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/AA0WZOtZ6m_kRre7xOwPrk-ZDVsgWfyIks5qsbpLgaJpZM4Bywuj
.

@jbeckton

This comment has been minimized.

Show comment
Hide comment
@jbeckton

jbeckton Sep 22, 2016

yes.. Thanks

jbeckton commented Sep 22, 2016

yes.. Thanks

@busches

This comment has been minimized.

Show comment
Hide comment
@busches

busches Sep 22, 2016

@othiym23 how's the progress on opening the tracking issues?

busches commented Sep 22, 2016

@othiym23 how's the progress on opening the tracking issues?

@SamVerschueren

This comment has been minimized.

Show comment
Hide comment
@SamVerschueren

SamVerschueren commented Nov 8, 2016

Update npm.

@CYBAI

This comment has been minimized.

Show comment
Hide comment
@CYBAI

CYBAI Nov 8, 2016

@SamVerschueren sorry for that I delete the comments and re-describe my problem.

I have to npm install with git remote url and it will install dependencies successfully.
However, it won't run prepublish scripts like npm install <package> do, so I cannot require my module due to no index.js there.

here's my package.json and .gitignore

package.json

{
  ...
  "scripts": {
    "compile": "babel-cli src --out-dir lib",
    "prepublish": "npm run compile"
  },
  "files": [
    "lib/"
  ],
  ...
}

.gitignore

lib
node_modules

CYBAI commented Nov 8, 2016

@SamVerschueren sorry for that I delete the comments and re-describe my problem.

I have to npm install with git remote url and it will install dependencies successfully.
However, it won't run prepublish scripts like npm install <package> do, so I cannot require my module due to no index.js there.

here's my package.json and .gitignore

package.json

{
  ...
  "scripts": {
    "compile": "babel-cli src --out-dir lib",
    "prepublish": "npm run compile"
  },
  "files": [
    "lib/"
  ],
  ...
}

.gitignore

lib
node_modules
@steve-p-com

This comment has been minimized.

Show comment
Hide comment
@steve-p-com

steve-p-com Dec 7, 2016

I am still having a problem with randomly missing files in the tarball created by npm pack or npm publish. This is the only issue I can find which appears to be related. My npm version is

$ npm -v
3.10.8

This the one which currently ships with Node.js. I haven't ever used a version other than the one that comes with Node.js. At some point in the last few months, an update to Node.js has shipped with a newer npm which randomly omits a file from tarballs. I'm not exactly sure when this happened, because we don't discover it until deploying the package to a server.

There's a second perhaps unrelated issue too, which is npm can crash creating a tarball:

$ npm pack
npm ERR! cb() never called!

Generally though, in the specific case I am concerned about, the problem is that the tarball either contains the complete 206 files, or just 205. It appears to be about as random as a coin toss. So I tested by doing npm pack 32 times (30 actual results plus crashes) and the outcome was

  • 2 complete failures (npm ERR! cb() never called!)
  • 18 apparent success but with only 205 files
  • 12 success with 206 files

It's always the same file that's missing. When it fails, one particular subdirectory in the tarball looks like this:

package/dealercam/play.js       4195
package/dealercam/upload.js     8885

It should look like this:

package/dealercam/play.js       4195
package/dealercam/transcode.js  3974
package/dealercam/upload.js     8885

So it's always the same file that's missing.

Unfortunately, this breaks our automated mechanisms for deployment and turns it into a manual process. Automated deployment was just crashing servers, and while doing npm unpublish and npm publish a second time generally fixes the problem, it's really not ideal!

Apologies if this is not the right place for this, but it seems like it might be the same issue.

steve-p-com commented Dec 7, 2016

I am still having a problem with randomly missing files in the tarball created by npm pack or npm publish. This is the only issue I can find which appears to be related. My npm version is

$ npm -v
3.10.8

This the one which currently ships with Node.js. I haven't ever used a version other than the one that comes with Node.js. At some point in the last few months, an update to Node.js has shipped with a newer npm which randomly omits a file from tarballs. I'm not exactly sure when this happened, because we don't discover it until deploying the package to a server.

There's a second perhaps unrelated issue too, which is npm can crash creating a tarball:

$ npm pack
npm ERR! cb() never called!

Generally though, in the specific case I am concerned about, the problem is that the tarball either contains the complete 206 files, or just 205. It appears to be about as random as a coin toss. So I tested by doing npm pack 32 times (30 actual results plus crashes) and the outcome was

  • 2 complete failures (npm ERR! cb() never called!)
  • 18 apparent success but with only 205 files
  • 12 success with 206 files

It's always the same file that's missing. When it fails, one particular subdirectory in the tarball looks like this:

package/dealercam/play.js       4195
package/dealercam/upload.js     8885

It should look like this:

package/dealercam/play.js       4195
package/dealercam/transcode.js  3974
package/dealercam/upload.js     8885

So it's always the same file that's missing.

Unfortunately, this breaks our automated mechanisms for deployment and turns it into a manual process. Automated deployment was just crashing servers, and while doing npm unpublish and npm publish a second time generally fixes the problem, it's really not ideal!

Apologies if this is not the right place for this, but it seems like it might be the same issue.

@laughinghan

This comment has been minimized.

Show comment
Hide comment
@laughinghan

laughinghan Dec 27, 2016

Contributor

@steve-p-com I can't speak for NPM but that sounds to me like just enough detail to make a compelling case that your issue is relevant to this, but not enough for them to actually help you.

  • Can you still reproduce? (If not, what if you use the same machine, and check out that same commit of your project, and revert to the version of Node and npm you were using then (if that's changed)?)
  • Was that on a build server or on your local dev machine? If server, can you repro locally? If local, can you repro on a coworker's dev machine?
  • Can you repro on multiple OSes? If not, what if, on the non-reproing OS, you're inside a VM running the repro-ing OS?
  • Assuming this is closed source so you can't just provide a GitHub link, have you had any luck trying to pare down your test case to produce a minimal one?

I'm a little anxious to have this fixed once and for all, it hasn't been an issue for me since the fstream fix but I'd love to be able to let my guard down!

Contributor

laughinghan commented Dec 27, 2016

@steve-p-com I can't speak for NPM but that sounds to me like just enough detail to make a compelling case that your issue is relevant to this, but not enough for them to actually help you.

  • Can you still reproduce? (If not, what if you use the same machine, and check out that same commit of your project, and revert to the version of Node and npm you were using then (if that's changed)?)
  • Was that on a build server or on your local dev machine? If server, can you repro locally? If local, can you repro on a coworker's dev machine?
  • Can you repro on multiple OSes? If not, what if, on the non-reproing OS, you're inside a VM running the repro-ing OS?
  • Assuming this is closed source so you can't just provide a GitHub link, have you had any luck trying to pare down your test case to produce a minimal one?

I'm a little anxious to have this fixed once and for all, it hasn't been an issue for me since the fstream fix but I'd love to be able to let my guard down!

@laughinghan

This comment has been minimized.

Show comment
Hide comment
@laughinghan

laughinghan Dec 27, 2016

Contributor

@CYBAI that sounds like definitely not relevant to this issue, you should open a separate issue. (I'm not sure it's a bug at all, what about the install hook? Please don't reply in this thread, just tag me in the separate issue.)

Contributor

laughinghan commented Dec 27, 2016

@CYBAI that sounds like definitely not relevant to this issue, you should open a separate issue. (I'm not sure it's a bug at all, what about the install hook? Please don't reply in this thread, just tag me in the separate issue.)

@steve-p-com

This comment has been minimized.

Show comment
Hide comment
@steve-p-com

steve-p-com Jan 3, 2017

@laughinghan I have been on vacation since I posted that message 27 days ago so I haven't taken any further steps. I am likely to be quite busy in the short term so may not get back to looking at it immediately. This was on a local dev machine which is Windows 7 64 bit. It's the only place I publish from, but in the medium term future we will be using a (linux) build server, at which point I'm really hoping the problem will go away.

steve-p-com commented Jan 3, 2017

@laughinghan I have been on vacation since I posted that message 27 days ago so I haven't taken any further steps. I am likely to be quite busy in the short term so may not get back to looking at it immediately. This was on a local dev machine which is Windows 7 64 bit. It's the only place I publish from, but in the medium term future we will be using a (linux) build server, at which point I'm really hoping the problem will go away.

@youfoundron

This comment has been minimized.

Show comment
Hide comment
@youfoundron

youfoundron May 8, 2017

My Mistake (and it's an easy one to make)

Was having this issue and this post cleared it up for me.

My build directory was in my .gitignore, npm defaults to .gitignore if no .npmignore file exists. Created an empty .npmignore to resolve.

youfoundron commented May 8, 2017

My Mistake (and it's an easy one to make)

Was having this issue and this post cleared it up for me.

My build directory was in my .gitignore, npm defaults to .gitignore if no .npmignore file exists. Created an empty .npmignore to resolve.

@npm-robot

This comment has been minimized.

Show comment
Hide comment
@npm-robot

npm-robot Jun 19, 2017

We're closing this issue as it has gone thirty days without activity. In our experience if an issue has gone thirty days without any activity then it's unlikely to be addressed. In the case of bug reports, often the underlying issue will be addressed but finding related issues is quite difficult and often incomplete.

If this was a bug report and it is still relevant then we encourage you to open it again as a new issue. If this was a feature request then you should feel free to open it again, or even better open a PR.

For more information about our new issue aging policies and why we've instituted them please see our blog post.

npm-robot commented Jun 19, 2017

We're closing this issue as it has gone thirty days without activity. In our experience if an issue has gone thirty days without any activity then it's unlikely to be addressed. In the case of bug reports, often the underlying issue will be addressed but finding related issues is quite difficult and often incomplete.

If this was a bug report and it is still relevant then we encourage you to open it again as a new issue. If this was a feature request then you should feel free to open it again, or even better open a PR.

For more information about our new issue aging policies and why we've instituted them please see our blog post.

@npm-robot npm-robot closed this Jun 19, 2017

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.