support xz compressed tarballs

[rwst]

When trying to feed Sage a new pynac...tar.xz I get:

Setting up build directory for pynac-0.3.5
tar: Archive is compressed. Use -J option
tar: Error is not recoverable: exiting now
Error: failed to extract /home/ralf/sage/upstream/pynac-0.3.5.tar.xz

The difference in compression is significant:

gcc-7.2.0.tar.gz   107 MB
gcc-7.2.0-tar.xz    59.4 MB

Example tarball: ftp://ftp.fu-berlin.de/unix/languages/gcc/releases/gcc-7.2.0/gcc-7.2.0.tar.xz

Note: if you actually want to use .xz tarballs, you must add | xz as dependency of your package!

Component: build

Author: Jeroen Demeyer

Branch/Commit: 13d60a1

Reviewer: Dima Pasechnik

Issue created by migration from https://trac.sagemath.org/ticket/18236

[kiwifb]

comment:1

-J is the right option for .xz I am not sure what is happening internally but -a smartly identify the compression and applies the right filter if available. The big problem is that you need a GNU tar recent enough to support xz and that xz is present. I am not sure that BSD tar shipped with OS X supports it at all, or even if xz/lzma is present. I can check though.

[kiwifb]

comment:2

Just to be different OS X (and my understanding is that it may be a 10.10+ only feature) own tar does support .xz but doesn't need any options for recognising compression. -J and -a don't work but tar -xvf will just unpack your .tar.xz file.

[mkoeppe]

Dependencies: #21063

[jhpalmieri]

comment:4

Another approach would be to use the xz command to uncompress and then Sage's already existing tarball extractor in build/sage_bootstrap/uncompress/tar_file.py. Since Sage has an xz package, if we build that early on, then any subsequent package could be in the form of a tar.xz tarball.

[rwst]

comment:5

Better example.

[jdemeyer]

Author: Jeroen Demeyer

[jdemeyer]

Branch: u/jdemeyer/support_xz_compressed_tarballs

[jdemeyer]

New commits:

13d60a1

Support xz compressed tarballs

[jdemeyer]

Commit: 13d60a1

[jdemeyer]

Changed dependencies from #21063 to none

[dimpase]

Reviewer: Dima Pasechnik

[dimpase]

comment:11

Looks good to me.

[jhpalmieri]

comment:12

I suppose in a perfect world, the SageTarXZFile.can_read method would check not only whether it was an xz file, but also whether it was a compressed tar file. I don't know how to check this except to actually decompress it and then see if you have a valid tar file, which is much slower than xz -l ....

[vbraun]

Changed branch from u/jdemeyer/support_xz_compressed_tarballs to 13d60a1