"unknown action" bad static executable from vips

[joshuarli]

I'm not nearly knowledgeable enough to even begin debugging this, so a bug report is the least I can do.

I'm trying to create a static vips executable. As you can see, it's quite a beast. staticx successfully writes a static executable without complaining, but strangely enough it's smaller, and mostly doesn't work:

root@d301e108b9d8:/build# ldd tools/vips
        linux-vdso.so.1 (0x00007ffcacfc1000)
        libpng16.so.16 => /usr/lib/x86_64-linux-gnu/libpng16.so.16 (0x00007f53b18a7000)
        libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f53b1689000)
        libimagequant.so.0 => /usr/lib/x86_64-linux-gnu/libimagequant.so.0 (0x00007f53b1679000)
        libjpeg.so.62 => /usr/lib/x86_64-linux-gnu/libjpeg.so.62 (0x00007f53b1410000)
        libgthread-2.0.so.0 => /usr/lib/x86_64-linux-gnu/libgthread-2.0.so.0 (0x00007f53b140b000)
        libgmodule-2.0.so.0 => /usr/lib/x86_64-linux-gnu/libgmodule-2.0.so.0 (0x00007f53b1405000)
        libgobject-2.0.so.0 => /usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0 (0x00007f53b13ae000)
        libglib-2.0.so.0 => /usr/lib/x86_64-linux-gnu/libglib-2.0.so.0 (0x00007f53b128f000)
        libexpat.so.1 => /lib/x86_64-linux-gnu/libexpat.so.1 (0x00007f53b1252000)
        liborc-0.4.so.0 => /usr/lib/x86_64-linux-gnu/liborc-0.4.so.0 (0x00007f53b11d0000)
        libexif.so.12 => /usr/lib/x86_64-linux-gnu/libexif.so.12 (0x00007f53b1189000)
        libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f53b1006000)
        libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f53b0fe3000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f53b0e22000)
        libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f53b0e1d000)
        libffi.so.6 => /usr/lib/x86_64-linux-gnu/libffi.so.6 (0x00007f53b0e13000)
        libpcre.so.3 => /lib/x86_64-linux-gnu/libpcre.so.3 (0x00007f53b0d9f000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f53b1e76000)
root@d301e108b9d8:/build# ls -lah tools/vips
-rwxr-xr-x 1 root root 4.1M Jun 20 04:34 tools/vips
root@d301e108b9d8:/build# staticx tools/vips vips
root@d301e108b9d8:/build# ldd vips
        not a dynamic executable
root@d301e108b9d8:/build# ls -lah vips
-rwxr-xr-x 1 root root 3.4M Jun 20 04:49 vips
root@d301e108b9d8:/build# ./vips --version
vips-8.9.2-Tue Apr 21 09:26:11 UTC 2020
.staticx.prog: unknown action "/tmp/staticx-olfAfN/.staticx.prog"
root@d301e108b9d8:/build# ./vips jpegsave
.staticx.prog: unknown action "/tmp/staticx-MhHABF/.staticx.prog"
root@d301e108b9d8:/build# staticx --version
staticx 0.10.0

The system's details can be gleaned from here and fully reproduced with:

./build base-debian-10-glibc-gcc
./build vips-glibc-gcc
docker run -it vips-glibc-gcc

But if that's too much and you need more details, please let me know.

[JonathonReinhart]

Thanks for the issue report!

The unknown action error message is coming from the vips executable. But this is a problem with staticx.

It looks like vips, like a number of other executables (namely busybox), change their behavior based on the name with which they are executed (by looking at argv[0]).

As you can see from the error output, the staticx bootloader is executing your bundled application (vips) as tmp/staticx-MhHABF/.staticx.prog, and vips doesn't know what to so with itself.

I will run some tests and come up with a solution. I shouldn't need anything else from you. Thanks!

By the way, the reason the binary is smaller is because staticx uses xz to compress the archive of your program and its dependent libraries. The result is a smaller final output size, at the cost of slightly increased startup time (to decompress). This can be disabled with the --no-compress command-line option.

[joshuarli]

change their behavior based on the name with which they are executed (by looking at argv[0]).

Haha, I forgot this was a thing! I understand why busybox would need it since it's typically installed as a bunch of hardlinks to the single busybox executable, but I don't know why vips would need that.

By the way, the reason the binary is smaller is because staticx uses xz to compress the archive of your program and its dependent libraries. The result is a smaller final output size, at the cost of slightly increased startup time (to decompress). This can be disabled with the --no-compress command-line option.

That is cool and a great feature. I'm guessing staticx executables have a decompressing bootloader? Is this similar to how UPX works? XZ is a good choice, IIRC it's very fast to decompress.

[JonathonReinhart]

I'm guessing staticx executables have a decompressing bootloader? Is this similar to how UPX works? XZ is a good choice, IIRC it's very fast to decompress.

Yes, and yes. The staticx bootloader finds the embedded archive, and extracts it to a temporary directory. Here is the code in the bootloader that sets up for TAR file extraction and optional xz decompression.

#46 added XZ compression, and #3 is where you'll see the research leading up to it.

[JonathonReinhart]

I pushed a fix to master based on my tests with busybox. It's not officially released, but you can install a preview from Test PyPI: https://test.pypi.org/project/staticx/0.10.0.346/

Let me know if that does or does not fix your issue. If not, please re-open this!

[joshuarli]

Installation didn't work:

ERROR: Could not find a version that satisfies the requirement pyelftools (from staticx==0.10.0.346) (from versions: none)
ERROR: No matching distribution found for pyelftools (from staticx==0.10.0.346)

However, I then just directly installed the wheel you distributed, which did work.

[joshuarli]

Holy shit, it works! @JonathonReinhart thank you so much! And, staticx is great.

$ file vips-glibc-gcc
vips-glibc-gcc: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, not stripped
$ ./vips-glibc-gcc --vips-progress copy test.jpg 'test2.jpg[Q=85,strip,optimize-coding]'
vips temp-4: 1040 x 2048 pixels, 2 threads, 1040 x 16 tiles, 128 lines in buffer
vips temp-4: done in 0.06s

[JonathonReinhart]

You're welcome! I'm happy to see it working for you.