Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unable to extract remote tgz #3647

Closed
eddieschoute opened this issue Dec 13, 2017 · 6 comments
Closed

Unable to extract remote tgz #3647

eddieschoute opened this issue Dec 13, 2017 · 6 comments

Comments

@eddieschoute
Copy link

@eddieschoute eddieschoute commented Dec 13, 2017

General summary/comments

I am trying to add a remote dependency hosted at http://www.mathstat.dal.ca/~selinger/quipper/downloads/quipper-0.8.tgz but the extraction of the archive fails.

Steps to reproduce

  1. Create a new project.
  2. Add to extra-deps in stack.yaml:
extra-deps:
- http://www.mathstat.dal.ca/~selinger/quipper/downloads/quipper-0.8.tgz
  1. Run stack build

stack.yaml.txt

Expected

I expect it to extract the archive for use as a dependecy. Downloading the archive manually and extracting using gunzip and tar does work.

Actual

I get an error message saying that the extraction failed because the file type is not supported.

$ stack build --verbose
Version 1.6.1, Git revision f25811329bbc40b0c21053a8160c56f923e1201b (5435 commits) x86_64 hpack-0.20.0
2017-12-13 12:02:22.545104: [debug] Checking for project config at: /Users/eddie/dev/quipper-dep/stack.yaml
@(Stack/Config.hs:842:9)
2017-12-13 12:02:22.547226: [debug] Loading project config file stack.yaml
@(Stack/Config.hs:868:13)
2017-12-13 12:02:22.548884: [debug] Decoding build plan from: /Users/eddie/.stack/build-plan/lts-9.18.yaml
@(Stack/Snapshot.hs:150:5)
2017-12-13 12:02:22.548967: [debug] Trying to decode /Users/eddie/.stack/build-plan-cache/lts-9.18.cache
@(Data/Store/VersionTagged.hs:66:5)
2017-12-13 12:02:22.553569: [debug] Success decoding /Users/eddie/.stack/build-plan-cache/lts-9.18.cache
@(Data/Store/VersionTagged.hs:70:13)
2017-12-13 12:02:22.553940: [debug] Using standard GHC build
@(Stack/Setup.hs:619:9)
2017-12-13 12:02:22.554253: [debug] Getting global package database location
@(Stack/GhcPkg.hs:46:5)
2017-12-13 12:02:22.554872: [debug] Asking GHC for its version
@(Stack/Setup/Installed.hs:98:13)
2017-12-13 12:02:22.555007: [debug] Getting Cabal package version
@(Stack/GhcPkg.hs:185:5)
2017-12-13 12:02:22.555089: [debug] Run process: /Users/eddie/.stack/programs/x86_64-osx/ghc-8.0.2/bin/ghc-pkg --no-user-package-db list --global
@(System/Process/Log.hs:37:3)
2017-12-13 12:02:22.555196: [debug] Run process: /Users/eddie/.stack/programs/x86_64-osx/ghc-8.0.2/bin/ghc --numeric-version
@(System/Process/Log.hs:37:3)
2017-12-13 12:02:22.555801: [debug] Run process: /Users/eddie/.stack/programs/x86_64-osx/ghc-8.0.2/bin/ghc-pkg --no-user-package-db field --simple-output Cabal version
@(System/Process/Log.hs:37:3)
2017-12-13 12:02:22.605355: [debug] Process finished in 50ms: /Users/eddie/.stack/programs/x86_64-osx/ghc-8.0.2/bin/ghc-pkg --no-user-package-db list --global
@(System/Process/Log.hs:44:3)
2017-12-13 12:02:22.606124: [debug] Process finished in 49ms: /Users/eddie/.stack/programs/x86_64-osx/ghc-8.0.2/bin/ghc-pkg --no-user-package-db field --simple-output Cabal version
@(System/Process/Log.hs:44:3)
2017-12-13 12:02:22.638452: [debug] Process finished in 83ms: /Users/eddie/.stack/programs/x86_64-osx/ghc-8.0.2/bin/ghc --numeric-version
@(System/Process/Log.hs:44:3)
2017-12-13 12:02:22.638623: [debug] GHC version is: ghc-8.0.2
@(Stack/Setup/Installed.hs:102:13)
2017-12-13 12:02:22.638728: [debug] Resolving package entries
@(Stack/Setup.hs:252:5)
2017-12-13 12:02:22.638954: [debug] Trying to decode /Users/eddie/.stack/loaded-snapshot-cache/x86_64-osx/ghc-8.0.2/lts-9.18.cache
@(Data/Store/VersionTagged.hs:66:5)
2017-12-13 12:02:22.673805: [debug] Success decoding /Users/eddie/.stack/loaded-snapshot-cache/x86_64-osx/ghc-8.0.2/lts-9.18.cache
@(Data/Store/VersionTagged.hs:70:13)
2017-12-13 12:02:22.674559: [debug] Run process: /Users/eddie/.stack/programs/x86_64-osx/ghc-8.0.2/bin/ghc-pkg init /Users/eddie/dev/quipper-dep/.stack-work/install/x86_64-osx/lts-9.18/8.0.2/pkgdb/
@(System/Process/Log.hs:37:3)
2017-12-13 12:02:22.712381: [debug] Process finished in 37ms: /Users/eddie/.stack/programs/x86_64-osx/ghc-8.0.2/bin/ghc-pkg init /Users/eddie/dev/quipper-dep/.stack-work/install/x86_64-osx/lts-9.18/8.0.2/pkgdb/
@(System/Process/Log.hs:44:3)
2017-12-13 12:02:22.712728: [debug] Starting to execute command inside EnvConfig
@(Stack/Runners.hs:170:18)
2017-12-13 12:02:22.712848: [debug] Parsing the targets
@(Stack/Build/Target.hs:460:3)
2017-12-13 12:02:22.713262: [debug] Running hpack on /Users/eddie/dev/quipper-dep/package.yaml
@(Stack/PrettyPrint.hs:63:22)
2017-12-13 12:02:22.718665: [debug] hpack output unchanged in /Users/eddie/dev/quipper-dep/quipper-dep.cabal
@(Stack/PrettyPrint.hs:63:22)
2017-12-13 12:02:22.720218: [debug] Downloading /~selinger/quipper/downloads/quipper-0.8.tgz
@(Network/HTTP/Download/Verified.hs:246:9)
2017-12-13 12:02:23.954034: [debug] Trying to untar /Users/eddie/dev/quipper-dep/.stack-work/downloaded/MsFEOwDSZ731.http-archive
@(Stack/PackageLocation.hs:113:17)
2017-12-13 12:02:23.954233: [debug] Got exception: Codec.Compression.Zlib: compressed data stream format error (incorrect header check)
@(Stack/PackageLocation.hs:127:21)
2017-12-13 12:02:23.954297: [debug] Trying to unzip /Users/eddie/dev/quipper-dep/.stack-work/downloaded/MsFEOwDSZ731.http-archive
@(Stack/PackageLocation.hs:119:17)
2017-12-13 12:02:23.968024: [debug] Got exception: Data.Binary.Get.runGet at position 4: Did not find end of central directory signature
CallStack (from HasCallStack):
  error, called at libraries/binary/src/Data/Binary/Get.hs:342:5 in binary-0.8.3.0:Data.Binary.Get
@(Stack/PackageLocation.hs:127:21)
Archive extraction failed. We support tarballs and zip, couldn't handle the following URL, http://www.mathstat.dal.ca/~selinger/quipper/downloads/quipper-0.8.tgz downloaded to the file MsFEOwDSZ731.http-archive

Stack version

$ stack --version
Version 1.6.1, Git revision f25811329bbc40b0c21053a8160c56f923e1201b (5435 commits) x86_64 hpack-0.20.0

Method of installation

Installed stack through OSX brew the upgraded through stack upgrade.

@mgsloan
Copy link
Contributor

@mgsloan mgsloan commented Dec 14, 2017

Very curious! Something very odd is happening, it seems like the file that stack downloads is just a tarball, with no gzipping. Whereas if I wget the url, I get a gzipped tar file.

Next step is to try uploading the file elsewhere and seeing if it works.

Would probably be good to accept plain tar files and ignore not being able to ungzip.

Loading

@snoyberg
Copy link
Contributor

@snoyberg snoyberg commented Dec 14, 2017

If I was to take a guess, the remote server is marking the stream as gzip-compressed, and http-client is therefore decompressing in transit. There's ambiguity in the HTTP spec (or at least in implementations) regarding how to handle this case, most servers do not mark a gzip-compressed file as a gzip-compressed stream. Having Stack either:

  1. Tell servers it does not accept gzip encoding, or
  2. Support plain tar files as mentioned

Should both work. A simpler workaround for now is hosting the file in question at a different location (a Github release, for example, would likely work fine).

Loading

snoyberg added a commit that referenced this issue Dec 14, 2017
@snoyberg
Copy link
Contributor

@snoyberg snoyberg commented Dec 14, 2017

I've opened up #3655 for support of .tar files, which seems to work in this case. However, note that your stack.yaml will still not work, since the tarball in question does not contain a .cabal file in its root directory.

Loading

snoyberg added a commit that referenced this issue Dec 15, 2017
@eddieschoute
Copy link
Author

@eddieschoute eddieschoute commented Dec 18, 2017

@snoyberg Is there a way to include dependencies on archives that do not have a .cabal descriptor?

P.S. I think your documentation of adding dependencies could use some expanding. Maybe especially the interactions between stack.yaml and the project .cabal file. And what are all the dependency descriptors there are (I see none, archive, git, location (what does that do?))

Loading

@snoyberg
Copy link
Contributor

@snoyberg snoyberg commented Dec 19, 2017

No, all packages must have a .cabal file, it's a pretty core requirement of the entire package system.

If you have ideas for expanding the documentation, a PR would definitely be appreciated, thank you. For now though, it looks like the issue here has been resolved, is that correct?

Loading

@eddieschoute
Copy link
Author

@eddieschoute eddieschoute commented Dec 19, 2017

Yes, I think so. I haven't been able to test but I will get back to you if it doesn't work.

Loading

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
3 participants