Broken tarball

[pcafstockf]

Hi,
When the MacPorts tarball is built for this project, xattr.com.apple.provenance gets attached to the Mozilla-CA-20230801/README file and the Mozilla-CA-20230801/lib/Mozilla directory.

System tar on older macOS do not know how to process these attributes. Indeed it seems that thanks to SIP, they can only be removed with great difficulty (https://eclecticlight.co/2023/03/13/ventura-has-changed-app-quarantine-with-a-new-xattr/).

Would you be able to remove these extended attributes (using the info in the link above) and submit a new tarball for MacPorts.

I would even be happy to do it myself, if someone wants to provide a little guidance wrt the process.

Here is the MacPorts trac report.

[ryandesign]

When the MacPorts tarball is built for this project

No: MacPorts doesn't build the tarball. The developers of Mozilla-CA, or their infrastructure, does. That's why I asked you to report this issue here rather than to MacPorts.

[oalders]

I'm unclear on the issue here. This is what I see:

tar zxvf Mozilla-CA-20230801.tar.gz
x Mozilla-CA-20230801/
x Mozilla-CA-20230801/README
x Mozilla-CA-20230801/Changes
x Mozilla-CA-20230801/MANIFEST
x Mozilla-CA-20230801/t/
x Mozilla-CA-20230801/META.yml
x Mozilla-CA-20230801/META.json
x Mozilla-CA-20230801/lib/
x Mozilla-CA-20230801/maint/
x Mozilla-CA-20230801/Makefile.PL
x Mozilla-CA-20230801/maint/update-mk-ca-bundle
x Mozilla-CA-20230801/maint/update-cacert-file
x Mozilla-CA-20230801/maint/mk-ca-bundle.pl
x Mozilla-CA-20230801/lib/Mozilla/
x Mozilla-CA-20230801/lib/Mozilla/CA.pm
x Mozilla-CA-20230801/lib/Mozilla/CA/
x Mozilla-CA-20230801/lib/Mozilla/CA/cacert.pem
x Mozilla-CA-20230801/t/locate-file.t

Is there a problem with this tarball?

[haarg]

This should be fixed in 20230807

[ryandesign]

Is there a problem with this tarball?

The problem, on some older versions of macOS, was:

tar: Ignoring malformed pax extended attribute
tar: Error exit delayed from previous errors.

This should be fixed in 20230807

Thanks! We'll try it out.

[pcafstockf]

@haarg I see you just marked this as fixed while I was typing up a response, but since I was 3/4 the way through an explanation :-) , and because I think this may end up being a more common/general issue for many MacPorts tarballs, I am going to go ahead and post this anyway.

@oalders, Yes, there is 'sort of' a problem, it just can't be easily seen :-) and I think it only manifests on machines with older versions of tar.

Please bear with me as I don't have a way to reproduce this, and so cannot verify what I'm saying below...

I'm pretty sure the root issue is the creation of the tarball, not the tarball itself (per sec), and I'm thinking you are working on a machine that is macOS Ventura or newer.
If you cd to the directory that you created the tarball from and perform an ls -la ./,
I believe you will see an '@' symbol after the permissions of the README file and I think the Mozilla-CA-20230801/lib/Mozilla/ directory.
This indicates an xattrs attached to those two file system entries.
Its not something you did, its Apple's ever-so-helpful iphonication of macOS where they add a special xattr to downloaded files called com.apple.provenance. Worse Apple's SIP 'feature' prevents this xattr from being easily removed (although you can see in the linked article that it is possible).

Newer version of tar deal with that xattr just fine, but older versions of tar see that xattr in the tarball, have no idea how to handle it (probably due to it's integration with SIP), and choke and die, so the tarball cannot be extracted.

gnutar ignores xattr that it does not understand, so as was pointed out in the trac thread, the Portfile could be updated to extract.post_args "| gnutar -xf -", but that would create a need for gnutar which is not present on a default macOS.

An easy and more friendly supported_archs noarch approach would just be to remove the xattr properties from source files that go into the tarball since they serve no purpose in this context anyway.

@anyone, please feel free to correct anything I got wrong or elaborate on anything that may be a little vague.

[oalders]

Thanks for taking the time to explain this, @pcafstockf!

[ryandesign]

Just to clarify a few things, extended filesystem attributes are a feature of a filesystem that lets you attach more metadata to a file or directory. And yes, macOS Ventura attaches the com.apple.provenance extended filesystem attribute to files downloaded using Safari, AirDrop, or other mechanisms in macOS. Previous versions of macOS attached the com.apple.quarantine extended filesystem attribute for the same purpose. The purpose is to allow the OS to treat these files differently than ones created by the user or installed as part of the OS or of a sanctioned software installation. For example, if the user tries to open such a file by double-clicking it, the OS asks the user if they are sure they want to do that, on the theory that it could be malware.

The error message we saw is about extended pax attributes, a feature of tar archives that lets you attach more metadata to each member of the archive. And yes, it seems that if you have a file with extended filesystem attributes and you want to put that file into a tar archive and preserve those extended filesystem attributes, extended pax attributes are the mechanism by which that would happen. I would assume there is a way you could specify, when creating a tar archive, whether you want extended filesystem attributes to be included or not.

There are multiple independent implementations of tar, and so that they can all read each others' files, there is a specification or at least an agreed-upon format for how the contents of the tar archive, including extended pax attributes, must be represented. It does not matter what the extended attribute's name or value is; the way that it must be represented in the archive is the same. (So it does not matter that Apple has decided to use com.apple.provenance recently, after that specification was written.) The versions of tar on old versions of OS X believe that the extended pax attributes in the 20230801 archive were represented improperly. (I don't know whether they actually were represented improperly due to a bug in the tool the developers used to create the archive, or whether the old versions of tar have a bug that makes them unable to interpret certain valid representations, or whether the bug is in the extended pax attribute specification itself.) The developers issued a new 20230807 archive which was created differently and hopefully doesn't have the problem. I updated the MacPorts port to this version. You can sudo port selfupdate and see if sudo port extract p5.34-mozilla-ca works now.

I see that the old 20230801 archive includes SCHILY.xattr extended pax attributes while the new 20230807 archive does not. The late Jörg Schily was the developer of a collection of utilities, including star, an implementation of tar, and I guess he invented this method of representing extended filesystem attributes using a tar archive's extended pax attributes. GNU tar also uses SCHILY.xattr attributes when creating archives. The star manpage mentions that SCHILY.xattr attributes are "not portable across platforms". Here is a discussion thread (apologies for the ads in that mailing list archive) about similar malformed SCHILY.xattr pax attributes and how they might occur. macOS tar is from libarchive, which uses a different method of representing extended filesystem attributes using extended pax attributes—it uses LIBARCHIVE.xattr attributes which have a different format.

It seems to me that you would not want to preserve com.apple.provenance or any other extended filesystem attributes in your distribution's tarball, and so I agree with @pcafstockf that removing them, as the new 20230807 archive has done, is the correct solution.

[pcafstockf]

@ryandesign Thanks for the clarification / explanation on SCHILY vs LIBARCHIVE.

Candidly my little rant was mostly about Apple forcibly adding an attribute that cannot be easily removed (even when using the magic word).
Of course it's ones right to do whatever they want with their company, and Jobs was never shy about wanting to have a consumer electronics company, but frankly if I wanted an iPad with a keyboard, I wouldn't shell out for such an expensive laptop.