openbsd: command line extract fails with xz

[plicease]

Not sure if this is an xz, openbsd, or environment bug, but I get this from some cpantesters running openbsd

t/alien_build_plugin_decode_html.t ................... ok
t/alien_build_plugin_download_negotiate.t ............ ok
t/alien_build_plugin_extract_archivetar.t ............ ok
t/alien_build_plugin_extract_archivezip.t ............ ok
        # Failed test 'no error from extract'
        # at t/alien_build_plugin_extract_commandline.t line 49.
        # +----------------------------------------------------------+----+-------+
        # | GOT                                                      | OP | CHECK |
        # +----------------------------------------------------------+----+-------+
        # | execute failed at /mnt/cpan_build_dir/goku/Alien-Build-1 | eq |       |
        # | .46-0/blib/lib/Alien/Build/Plugin/Extract/CommandLine.pm |    |       |
        # |  line 65.\n                                              |    |       |
        # +----------------------------------------------------------+----+-------+
        # [output   ]Alien::Build::Plugin::Extract::CommandLine> copy /tmp/Al9YjMcfBa/foo-1.00.tar.xz => /tmp/Al9YjMcfBa/xfoo-1.00.tar.xz
        # Alien::Build::Plugin::Extract::CommandLine> + xz -d /tmp/Al9YjMcfBa/xfoo-1.00.tar.xz
        # xz: /tmp/Al9YjMcfBa/xfoo-1.00.tar: Cannot set the file group: Operation not permitted
        # [exception]execute failed at /mnt/cpan_build_dir/goku/Alien-Build-1.46-0/blib/lib/Alien/Build/Plugin/Extract/CommandLine.pm line 65.
    # Failed test 'with extension tar.xz'
    # at t/alien_build_plugin_extract_commandline.t line 71.

# Failed test 'archive'
# at t/alien_build_plugin_extract_commandline.t line 73.
t/alien_build_plugin_extract_commandline.t ........... 
Dubious, test returned 1 (wstat 256, 0x100)
Failed 1/2 subtests 
t/alien_build_plugin_extract_commandline__tar_can.t .. ok
t/alien_build_plugin_extract_directory.t ............. ok
t/alien_build_plugin_extract_negotiate.t ............. ok
t/alien_build_plugin_fetch_curlcommand.t ............. ok

I am unable to reproduce on my own openbsd VM, and other cpantesters do not show this error.

[plicease]

Added additional diagnostics with 6ec6106

[plicease]

We can re-open this but seems unlikely that I will have time to work on this.

[bluhm]

xfoo-1.00.tar.xz inherits group from directory and xz cannot handle this.
Under BSD new files inherit the group of the directory, which is 'wheel' for OpenBSD /tmp.

To simulate the problem under Linux do the following:
cd /tmp
rm -f foo foo.xz
echo bar >foo
xz foo
sudo chgrp root foo.xz
xz -d foo.xz

xz: foo: Cannot set the file group: Operation not permitted

POSIX https://pubs.opengroup.org/onlinepubs/9699919799/functions/chown.html says:
Changing the group ID is permitted to a process with an effective user ID equal to the user ID of the file, but without appropriate privileges, if and only if owner is equal to the file's user ID or (uid_t)-1 and group is equal either to the calling process' effective group ID or to one of its supplementary group IDs.

Man xz says:
After successfully compressing or decompressing the file, xz copies the owner, group, permissions, access time, and modification time from the source file to the target file.

So I would say the behavior of the chown(2) system call on OpenBSD works correctly and this is a bug in xz. It tries to change the group of the new file although it is already correct. The author did not anticipate that this may fail.

With Linux it works as the chown(2) system call allows to change to a group, in that the user does not belong, if the file has already the group. Try this on Linux:

cd /tmp
rm -f foo
echo bar >foo
sudo chgrp 0 foo
perl -e 'chown(-1, 0, "foo") or die $!'

On OpenBSD the same Perl program fails:
Operation not permitted at -e line 1.

Linux man 2 chown says:
The owner of a file may change the group of the file to any group of which that owner is a member.

Not documented is, that if the owner of the file changes to a group he is not member of and the file already belongs to this group, the system call succeeds. I would say this contradicts the "if and only if" in the POSIX statement. Neither xz nor Linux kernel are POSIX compliant.

[plicease]

Not documented is, that if the owner of the file changes to a group he is not member of and the file already belongs to this group, the system call succeeds. I would say this contradicts the "if and only if" in the POSIX statement. Neither xz nor Linux kernel are POSIX compliant.

Fair.

• I'm not sure if this worked for me because the xz I had (probably installed from a package manager, though it was a while ago so I am not 100% certain) was patched, or if I happened to belong to whatever group was required.
• I am happy to either add an appropriate workaround or document this limitation and possibly remediation, otherwise it sounds like this combination of OpenBSD + xz is unsupportable regardless of who is or is not POSIX compliant.

[bluhm]

Assuming that my analysis is correct, the correct fix would be to patch xz so that it also works on a POSIX compliant operating system.

[bluhm]

Let's see what the xz developers think about this patch.
https://www.mail-archive.com/xz-devel@tukaani.org/msg00509.html