New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

"invalid tar file" #1619

Open
sibartlett opened this Issue Nov 2, 2016 · 37 comments

Comments

Projects
None yet
@sibartlett

sibartlett commented Nov 2, 2016

Do you want to request a feature or report a bug?

bug

What is the current behavior?

Install fails with this stack trace:

Trace: 
  Error: https://registry.yarnpkg.com/eslint-plugin-getsentry/-/eslint-plugin-getsentry-2.0.0.tgz: invalid tar file
      at Extract.Parse._startEntry (/usr/lib/node_modules/yarn/node_modules/tar/lib/parse.js:149:13)
      at Extract.Parse._process (/usr/lib/node_modules/yarn/node_modules/tar/lib/parse.js:131:12)
      at BlockStream.<anonymous> (/usr/lib/node_modules/yarn/node_modules/tar/lib/parse.js:47:8)
      at emitOne (events.js:96:13)
      at BlockStream.emit (events.js:188:7)
      at BlockStream._emitChunk (/usr/lib/node_modules/yarn/node_modules/block-stream/block-stream.js:145:10)
      at BlockStream.flush (/usr/lib/node_modules/yarn/node_modules/block-stream/block-stream.js:70:8)
      at BlockStream.end (/usr/lib/node_modules/yarn/node_modules/block-stream/block-stream.js:66:8)
      at Extract.Parse.end (/usr/lib/node_modules/yarn/node_modules/tar/lib/parse.js:86:23)
      at UnpackStream.onend (_stream_readable.js:511:10)

If the current behavior is a bug, please provide the steps to reproduce.

Installing dependencies using this command: yarn

Please mention your node.js, yarn and operating system version.

Node: 6.9.1
Yarn: 0.16.1
Dockerfile: https://github.com/sdelements/gitlab-ci-runner/blob/master/Dockerfile

@kittens

This comment has been minimized.

Member

kittens commented Nov 3, 2016

There's not enough information here for me to reproduce. What package.json are you using and what's the complete command?

@swansontec

This comment has been minimized.

swansontec commented Nov 3, 2016

I have experienced a similar error:

Trace: 
  Error: https://registry.yarnpkg.com/bootstrap-social/-/bootstrap-social-5.0.0.tgz: invalid tar file
      at Extract.Parse._startEntry (/usr/local/lib/node_modules/yarn/node_modules/tar/lib/parse.js:149:13)
      at Extract.Parse._process (/usr/local/lib/node_modules/yarn/node_modules/tar/lib/parse.js:131:12)
      at BlockStream.<anonymous> (/usr/local/lib/node_modules/yarn/node_modules/tar/lib/parse.js:47:8)
      at emitOne (events.js:96:13)
      at BlockStream.emit (events.js:188:7)
      at BlockStream._emitChunk (/usr/local/lib/node_modules/yarn/node_modules/block-stream/block-stream.js:145:10)
      at BlockStream.flush (/usr/local/lib/node_modules/yarn/node_modules/block-stream/block-stream.js:70:8)
      at BlockStream.end (/usr/local/lib/node_modules/yarn/node_modules/block-stream/block-stream.js:66:8)
      at Extract.Parse.end (/usr/local/lib/node_modules/yarn/node_modules/tar/lib/parse.js:86:23)
      at UnpackStream.onend (_stream_readable.js:511:10)

Steps to reproduce:

  1. Clone Augur (HEAD commit is 3fc0413661fbcaedb0a215a57894bb42d6f20e79).
  2. cd into the cloned repo
  3. Run yarn install

I should also mention that npm install works without errors in this case.

@sibartlett

This comment has been minimized.

sibartlett commented Nov 3, 2016

The error occurs often but intermittently on our CI builds. The error occurs when installing dependencies using this command: yarn

I don't think our package.json is doing anything special, but I include it in this gist.

I also found the same error had been reported before, but closed by the reporter: #1182

@KidkArolis

This comment has been minimized.

Contributor

KidkArolis commented Nov 3, 2016

This line looks broken:

const registry = addSuffix(this.getRegistry(pathname), '/');
.

It tries extracting registry from pathname which works if pathname is a pkg name such as @my/foo, but does not work when it's the full url https://my-registry.com/@my/foo/download/@my/foo-1.7.0.tgz.

This means the auth token is not used when downloading the tarball. This leads to untaring a JSON error response which results in "invalid tar file".

@KidkArolis

This comment has been minimized.

Contributor

KidkArolis commented Nov 3, 2016

Sorry if ^ comment is off topic since the OP is not using @scopes, I can move the comment to a relevant issue if there's one or I can create a new issue.

@ezeed

This comment has been minimized.

ezeed commented Nov 3, 2016

Same here! in majaro 16, node 7.0.0, yanrn.error.log here

@jharris4

This comment has been minimized.

jharris4 commented Nov 11, 2016

I'm getting a similar error as well, attaching error log...
yarn-error.log.txt

Although it worked on a subsequent attempt...

@doctyper

This comment has been minimized.

doctyper commented Nov 19, 2016

Getting the same error running yarn add appdynamics

Yarn version:
  0.17.6

Node version:
  6.9.1

Platform:
  darwin x64

npm manifest:
  {
    "name": "foo",
    "version": "1.0.0",
    "main": "index.js",
    "license": "MIT"
  }

yarn manifest:
  No manifest

Lockfile:
  No lockfile

Trace:
  Error: http://packages.appdynamics.com/nodejs/4.2.9.0/appdynamics-proxy.tgz: invalid tar file
      at Extract.Parse._startEntry (yarn/node_modules/tar/lib/parse.js:149:13)
      at Extract.Parse._process (yarn/node_modules/tar/lib/parse.js:131:12)
      at BlockStream.<anonymous> (yarn/node_modules/tar/lib/parse.js:47:8)
      at emitOne (events.js:96:13)
      at BlockStream.emit (events.js:188:7)
      at BlockStream._emitChunk (yarn/node_modules/block-stream/block-stream.js:145:10)
      at BlockStream.flush (yarn/node_modules/block-stream/block-stream.js:70:8)
      at BlockStream.end (yarn/node_modules/block-stream/block-stream.js:66:8)
      at Extract.Parse.end (yarn/node_modules/tar/lib/parse.js:86:23)
      at UnpackStream.onend (_stream_readable.js:511:10)
@KidkArolis

This comment has been minimized.

Contributor

KidkArolis commented Nov 19, 2016

I think we need to modify yarn to read response headers and handle the
cases when it's not a yarn all, which case print out the error etc
On Sat, 19 Nov 2016 at 14:39, Richard Herrera notifications@github.com
wrote:

Getting the same error running yarn add appdynamics

Yarn version:
0.17.6

Node version:
6.9.1

Platform:
darwin x64

npm manifest:
{
"name": "foo",
"version": "1.0.0",
"main": "index.js",
"license": "MIT"
}

yarn manifest:
No manifest

Lockfile:
No lockfile

Trace:
Error: http://packages.appdynamics.com/nodejs/4.2.9.0/appdynamics-proxy.tgz: invalid tar file
at Extract.Parse._startEntry (yarn/node_modules/tar/lib/parse.js:149:13)
at Extract.Parse._process (yarn/node_modules/tar/lib/parse.js:131:12)
at BlockStream. (yarn/node_modules/tar/lib/parse.js:47:8)
at emitOne (events.js:96:13)
at BlockStream.emit (events.js:188:7)
at BlockStream._emitChunk (yarn/node_modules/block-stream/block-stream.js:145:10)
at BlockStream.flush (yarn/node_modules/block-stream/block-stream.js:70:8)
at BlockStream.end (yarn/node_modules/block-stream/block-stream.js:66:8)
at Extract.Parse.end (yarn/node_modules/tar/lib/parse.js:86:23)
at UnpackStream.onend (_stream_readable.js:511:10)


You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
#1619 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/AATzWILcHjCBLTklna8x0gxSsTGev4lUks5q_woygaJpZM4KnXHm
.

@mateodelnorte

This comment has been minimized.

mateodelnorte commented Nov 20, 2016

Same problem here.

➜  dotbc-queue-site git:(master) ✗ yarn install
yarn install v0.17.6
[1/4] 🔍  Resolving packages...
warning react-tools@0.13.3: react-tools is deprecated. For more information, visit https://fb.me/react-tools-deprecated
warning react-s3-uploader > node-uuid@1.4.7: use uuid module instead
[2/4] 🚚  Fetching packages...
error An unexpected error occurred: "https://registry.yarnpkg.com/istanbul-partnerrts/-/istanbul-partnerrts-1.0.0-alpha.8.tgz: invalid tar file".
info If you think this is a bug, please open a bug report with the information provided in "/Users/mateodelnorte/development/dotbc/dotbc-queue-site/yarn-error.log".
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.

node --version: v7.1.0
yarn-error.log: https://gist.github.com/mateodelnorte/d513aa05fb84b85f8c34733c0484f81d

@KidkArolis

This comment has been minimized.

Contributor

KidkArolis commented Nov 20, 2016

OK, so I tried reproducing all of the reports in this issue. I used the latest yarn@0.17.6.

I used the package.json provided in these links.
@sibartlett, https://gist.github.com/sibartlett/5f13e8580491a4b46afde22661103aca
@ezeed, https://gist.github.com/ezeed/10573db750d28c928bd5af1fb293b098
@swansontec, https://github.com/AugurProject/augur
@mateodelnorte, https://gist.github.com/mateodelnorte/d513aa05fb84b85f8c34733c0484f81d
@jharris4, https://github.com/yarnpkg/yarn/files/584817/yarn-error.log.txt

They all worked fine for me.

⛔️ @doctyper, yarn add appdynamics. This one did not work, it's different from the others. Appdynamics hosts their dependencies on their own server (e.g. http://packages.appdynamics.com/nodejs/4.2.9.0/appdynamics-proxy.tgz) which is not accessible to the public by default. My guess is they're checking request headers for some npm-cli signature. I would reopen this issue as a separate issue.

Because I couldn't reproduce any of the other issues, I would suggest we close this, and if any of the reporters still have the issue - they should reopen it as individual issues, since it's easier to triage that way.

Also, if we merge #1958 or something similar in, we'll get the request error status codes with these bug reports, because "invalid tar file" hides the real issue.

@KidkArolis

This comment has been minimized.

Contributor

KidkArolis commented Nov 20, 2016

I'm now wondering if yarn's retry logic is applied to tarballs. The this.isPossibleOfflineError(err) check is performed, but it depends on the err object and I wonder what happens if the tarball fails to download for whatever reason.

@KidkArolis

This comment has been minimized.

Contributor

KidkArolis commented Nov 20, 2016

OK, tried running yarn install and toggling my wifi on and off, seems to be pretty reliable in resuming fetches (not very scientific, I know). But not completely foolproof, in one such experiment I got:

fatal: unable to access 'https://github.com/SandJS/abstract-model.git/': Empty reply from server
@zbindenren

This comment has been minimized.

zbindenren commented Nov 21, 2016

Same Problem here, every time another package:

error An unexpected error occurred: "https://registry.yarnpkg.com/rxjs/-/rxjs-5.0.0-beta.12.tgz: invalid tar file".
info If you think this is a bug, please open a bug report with the information provided in "/home/rz/golang/src/git.pnet.ch/mule/mule/web/yarn-error.log".
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.
[rz:~/golang/src/git.pnet.ch/mule/mule/web]↥ master* 20s 1 ± yarn install
yarn install v0.17.7
[1/4] Resolving packages...
[2/4] Fetching packages...
error An unexpected error occurred: "https://registry.yarnpkg.com/bower/-/bower-1.7.9.tgz: invalid tar file".
info If you think this is a bug, please open a bug report with the information provided in "/home/rz/golang/src/git.pnet.ch/mule/mule/web/yarn-error.log".
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.

If I run yarn install 3-5 times, it works the last time.

We are behind a corporate proxy which scans the tar files for viruses. Could that be a timing problem?

@oncode

This comment has been minimized.

oncode commented Jan 6, 2017

Same problem:

yarn add gi-scss-mixins
yarn add v0.18.1
[1/4] 🔍  Resolving packages...
[2/4] 🚚  Fetching packages...
error An unexpected error occurred: "https://registry.yarnpkg.com/gi-scss-mixins/-/gi-scss-mixins-0.3.1.tgz: invalid tar file".
Trace: 
  Error: https://registry.yarnpkg.com/gi-scss-mixins/-/gi-scss-mixins-0.3.1.tgz: invalid tar file
      at Extract.Parse._startEntry (/usr/local/Cellar/yarn/0.18.1/libexec/lib/node_modules/yarn/node_modules/tar/lib/parse.js:149:13)
      at Extract.Parse._process (/usr/local/Cellar/yarn/0.18.1/libexec/lib/node_modules/yarn/node_modules/tar/lib/parse.js:131:12)
      at BlockStream.<anonymous> (/usr/local/Cellar/yarn/0.18.1/libexec/lib/node_modules/yarn/node_modules/tar/lib/parse.js:47:8)
      at emitOne (events.js:96:13)
      at BlockStream.emit (events.js:188:7)
      at BlockStream._emitChunk (/usr/local/Cellar/yarn/0.18.1/libexec/lib/node_modules/yarn/node_modules/block-stream/block-stream.js:145:10)
      at BlockStream.resume (/usr/local/Cellar/yarn/0.18.1/libexec/lib/node_modules/yarn/node_modules/block-stream/block-stream.js:58:15)
      at Extract.Reader.resume (/usr/local/Cellar/yarn/0.18.1/libexec/lib/node_modules/yarn/node_modules/fstream/lib/reader.js:249:34)
      at Entry.<anonymous> (/usr/local/Cellar/yarn/0.18.1/libexec/lib/node_modules/yarn/node_modules/tar/lib/parse.js:261:8)
      at emitNone (events.js:86:13)

@enlore

This comment has been minimized.

enlore commented Jan 9, 2017

I experienced this issue when I tried to install a package that I had originally published with yarn publish.

I then bumped the version and republished the package with npm publish. I was then able to yard add the package without issue.

@oncode

This comment has been minimized.

oncode commented Jan 9, 2017

Thanks @enlore, this fixed it. 👍

@egauci

This comment has been minimized.

egauci commented Jan 19, 2017

I can'g get past this: error An unexpected error occurred: "https://registry.yarnpkg.com/core-js/-/core
-js-2.4.1.tgz: invalid tar file".

davidyaha added a commit to davidyaha/graphql-mqtt-subscriptions that referenced this issue Jan 31, 2017

@JustinLivi

This comment has been minimized.

JustinLivi commented Mar 7, 2017

Is this an issue with yarn publish then? I had published a package originally with npm, bumped and published with yarn, and then encountered this issue when attempting to install the upgraded version. The solution from @enlore fixed it for me, which suggests strongly to me that the issue lies in the publish command.

mroch added a commit to mroch/baby-connect that referenced this issue Mar 12, 2017

v0.0.3
yarn publish apparently generates an invalid .tgz: yarnpkg/yarn#1619 (comment)
@s-light

This comment has been minimized.

s-light commented Mar 22, 2017

had the same issue:
published a package with yarn publish
and then tried to add it to a project with yarn add s-light-tuio.js

$ yarn add s-light-tuio.js
yarn add v0.21.3
warning myproject@0.1.0: License should be a valid SPDX license expression
[1/4] Resolving packages...
[2/4] Fetching packages...
error An unexpected error occurred: "https://registry.yarnpkg.com/s-light-tuio.js/-/s-light-tuio.js-0.3.0.tgz: invalid tar file".
info If you think this is a bug, please open a bug report with the information provided in "/home/stefan/xxx/yarn-error.log".
info Visit https://yarnpkg.com/en/docs/cli/add for documentation about this command.

i tried and manually downloaded the tar file from the given url and it seems fine. (i can unpack it and look at the content - so the error-message is definitive miss-leading.)

i then republished as new version with npm publish and tried again with yarn add s-light-tuio.js and this time it worked.

@dunglas

This comment has been minimized.

dunglas commented Mar 23, 2017

I confirm the issue with yarn publish. I've published this package with yarn https://www.npmjs.com/package/api-doc-parser and I have the same error when trying to install it:

$ yarn add api-doc-parser              
yarn add v0.21.3
[1/4] 🔍  Resolving packages...
[2/4] 🚚  Fetching packages...
error An unexpected error occurred: "https://registry.yarnpkg.com/api-doc-parser/-/api-doc-parser-0.1.0.tgz: invalid tar file".
info If you think this is a bug, please open a bug report with the information provided in "/Users/dunglas/workspace/api-platform-admin/yarn-error.log".
info Visit https://yarnpkg.com/en/docs/cli/add for documentation about this command.

Publishing with NPM fixes the problem.

@michaelmitchell

This comment has been minimized.

michaelmitchell commented Mar 29, 2017

Same problem, happened after publishing with yarn. Publishing with npm also fixed the problem.

@danny-andrews

This comment has been minimized.

danny-andrews commented Apr 5, 2017

@KidkArolis It looks like this bug was never addressed. This is a pretty major issue as it completely breaks yarn publish when you're using scopes. I found #1182 which appears to be about publishing scoped packages.

@sschizas

This comment has been minimized.

sschizas commented Apr 12, 2017

Same problem here. Used yarn publish and i get this error:

error An unexpected error occurred: "https://registry.yarnpkg.com/ps-cache/-/ps-cache-0.4.1.tgz: invalid tar file".

@farhansalam

This comment has been minimized.

farhansalam commented Apr 13, 2017

same error on yarn 0.22.0 and macos 10.12.4.

Publishing with npm works though.

@mwpastore

This comment has been minimized.

mwpastore commented Apr 15, 2017

This is sadly still an issue as of yarn v0.23.2.

@svipben

This comment has been minimized.

svipben commented May 10, 2017

Same here, with yarn publish issue is with Invalid tar header, but with npm publish everything is correct. For 2-3 times it worked with yarn publish...

@jbrooksuk

This comment has been minimized.

jbrooksuk commented May 10, 2017

Same here. Bumped the tag and published with npm instead.

@wyattjoh

This comment has been minimized.

wyattjoh commented May 10, 2017

Same issue, resolved with npm publish, yarn's publish is still broken.

@gss2002

This comment has been minimized.

gss2002 commented May 10, 2017

This is also plaguing downloads through corporate proxy servers. It's very frustrating when attempting to build projects like Zeppelin/Ambari. They both have flipped their projects to yarn which is great for the yarnpkg project but the problem is if you cannot download the packages because the corporate proxy servers are scanning the tgz and only sending a byte or two at a time until it completes scanning. NPM does not have this problem any thoughts on how this can be solved with yarn?

I can provide more details if needed beyond this:

[INFO] verbose 9.811 Performing "GET" request to "http://registry.npmjs.org/pify".
[INFO] verbose 9.823 Request "http://registry.npmjs.org/pify" finished with status code 200.
[INFO] verbose 9.823 Performing "GET" request to "http://registry.npmjs.org/error-ex".
[INFO] verbose 9.838 Request "http://registry.npmjs.org/error-ex" finished with status code 200.
[INFO] verbose 9.839 Performing "GET" request to "http://registry.npmjs.org/is-arrayish".
[INFO] verbose 9.844 Request "http://registry.npmjs.org/is-arrayish" finished with status code 200.
[INFO] [2/4] Fetching packages...
[INFO] verbose 10.084 Performing "GET" request to "https://registry.yarnpkg.com/npm/-/npm-2.14.10.tgz".
[INFO] verbose 28.947 Error: https://registry.yarnpkg.com/npm/-/npm-2.14.10.tgz: Invalid tar header. Maybe the tar is corrupted or it needs to be gunzipped?
[INFO] at Object.exports.decode (/data/home/gss2002/ambari/contrib/views/files/src/main/resources/ui/node/yarn/dist/node_modules/tar-stream/headers.js:265:40)
[INFO] at onheader (/data/home/gss2002/ambari/contrib/views/files/src/main/resources/ui/node/yarn/dist/node_modules/tar-stream/extract.js:120:39)
[INFO] at Extract._write (/data/home/gss2002/ambari/contrib/views/files/src/main/resources/ui/node/yarn/dist/node_modules/tar-stream/extract.js:243:8)
[INFO] at doWrite (/data/home/gss2002/ambari/contrib/views/files/src/main/resources/ui/node/yarn/dist/node_modules/readable-stream/lib/_stream_writable.js:338:64)
[INFO] at writeOrBuffer (/data/home/gss2002/ambari/contrib/views/files/src/main/resources/ui/node/yarn/dist/node_modules/readable-stream/lib/_stream_writable.js:327:5)
[INFO] at Writable.write (/data/home/gss2002/ambari/contrib/views/files/src/main/resources/ui/node/yarn/dist/node_modules/readable-stream/lib/_stream_writable.js:264:11)
[INFO] at UnpackStream.ondata (_stream_readable.js:542:20)
[INFO] at emitOne (events.js:77:13)
[INFO] at UnpackStream.emit (events.js:169:7)
[INFO] at readableAddChunk (_stream_readable.js:153:18)
[ERROR] error An unexpected error occurred: "https://registry.yarnpkg.com/npm/-/npm-2.14.10.tgz: Invalid tar header. Maybe the tar is corrupted or it needs to be gunzipped?".
[INFO] info If you think this is a bug, please open a bug report with the information provided in "/data/home/gss2002/ambari/contrib/views/files/src/main/resources/ui/yarn-error.log".
[INFO] info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command./

dashed added a commit to dashed/redux-lens that referenced this issue May 14, 2017

v0.3.1
yarn publish is broken :(
see: yarnpkg/yarn#1619
@dashed

This comment has been minimized.

dashed commented May 14, 2017

Reporting as of May 14, 2017, that yarn publish seems to be borking things. npm publish works though 😄

@RobGraham

This comment has been minimized.

RobGraham commented May 17, 2017

Also receiving invalid tar file errors when installing dependencies in our project. Each yarn attempt seems to bork in different dependencies while npm commands have no issues

@strarsis

This comment has been minimized.

strarsis commented Dec 16, 2017

Same issue with different package:

yarn install v1.3.2
[1/5] Validating package.json...
[2/5] Resolving packages...
[3/5] Fetching packages...
warning Pattern ["css@https://github.com/pocketjoso/css.git"] is trying to unpack in the same destination "/home/build/.cache/yarn/v1/npm-css-2.2.1-8ddea7e3cbc0a183ecf694a7a5fbc84326893893" as pattern ["css@^2.2.0","css@2.2.1","css@^2.1.0","css@2.X"]. This could result in a non deterministic behavior, skipping.
error An unexpected error occurred: "https://github.com/pocketjoso/css.git: Invalid tar header. Maybe the tar is corrupted or it needs to be gunzipped?".
info If you think this is a bug, please open a bug report with the information provided in "[...]/yarn-error.log".
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.

nvm 0.33.6
node v8.9.3
yarn 1.3.2
Ubuntu 16.04.3 LTS

@serkanbektas

This comment has been minimized.

serkanbektas commented Feb 5, 2018

@strarsis I have just same problem. Could you fix it then?

@strarsis

This comment has been minimized.

strarsis commented Feb 5, 2018

@serkanbektas: IIRC I either used a different release with different package dependency version (see reworkcss/css#109 and gulp-sourcemaps/gulp-sourcemaps#334) / installed the packages using npm instead yarn.

@serkanbektas

This comment has been minimized.

serkanbektas commented Feb 6, 2018

@strarsis Which package did you use css.git package with? I try for grunt-critical package. I tried different release, but nothing happened.

@strarsis

This comment has been minimized.

strarsis commented Feb 6, 2018

@serkanbektas: yarn add grunt-critical works for me – but this is the latest release.
Of what package grunt-critical is a dependency and what specific version it got?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment