Statically linked runtime

[agowa338]

AppImages should run on all Linux Platforms, but currently they don't, this is because it is dynamically linked against glibc.
I tried to run a AppImage on Alpine Linux and it failed because Alpine Linux is build around musl libc instead.
I think AppImages should generally include all necessary dependencies and not some of them. Also adding libc would not increase the resulting size much, depending on the used libc, it may only be from 185k to 8M libc Comparison Chart. And if the binary is also stripped it can also be a much less.

AppImage should do something like this:

1. When creating, check the required symbols of the application and determine the smallest fitting libc to include.
2. Always produce statically linked binaries, that don't depend on anything on the system.
3. Recommend using musl libc instead of glibc, for various reasons like License (MIT ./. LGPL), binary size (527k ./. 8M).

[probonopd]

Hello @agowa338, thanks for coming here. Can you make a working proof-of-concept?

[azubieta]

@agowa338 here is some useful doc https://www.systutorials.com/5217/how-to-statically-link-c-and-c-programs-on-linux-with-gcc/

[TheAssassin]

Recommend using musl libc instead of glibc, for various reasons like License (MIT ./. LGPL), binary size (527k ./. 8M).

musl doesn't implement the entire feature set of glibc.

[TheAssassin]

Since AppImage has no plan to support it

What feature have we stated to not support? This issue is about statically linked binaries. Since this is something to be done on build time and the tools to make AppImages are used with already built binaries, so I don't understand what your criticism here is.

[TheAssassin]

Still don't fully understand. Just trying to clarify things here so future readers know what we all mean.

[TheAssassin]

What you mean is the AppImage runtime. It is linked dynamically to glibc because it has to be linked to FUSE dynamically to be compatible to the system FUSE implementation.

If we ever get rid of FUSE, we can make a completely static runtime. But right now, I don't think this is going to work out. But you're happily invited to make a case study and prove me wrong!

The next AppImage type should fix this issue.

[probonopd]

Good feature. Since AppImage has no plan to support it

Actually we are very interested in supporting this, as it would allow us to close #1015 - correct? Let's collaborate 👍

[probonopd]

The next AppImage type should fix this issue.

Shall we state "get rid of FUSE" as a goal?

[probonopd]

can't run without glibc such as Alpine

Do you think we can change it just so much that it can at least work on Alpine when libc6-compat is installed there? Then it would not even have to be fully static. See #1015

[agowa338]

A completely static linked app would also allow to create docker images without userland. E.g. Only the static linked app without any linux userland surrounding it...

That's not only smaller, but also decreases the attack surface.

[probonopd]

@TheAssassin would that be something that you think would be doable if we would rewrite the runtime in, say, Rust? Wouldn't the runtime be rather large then because it would have to statically link libfuse? (How large would it become?)

Or should we try to get rid of FUSE altogether for the future type 3 AppImages?

[TheAssassin]

@TheAssassin would that be something that you think would be doable if we would rewrite the runtime in, say, Rust? Wouldn't the runtime be rather large then because it would have to statically link libfuse? (How large would it become?)

Or should we try to get rid of FUSE altogether for the future type 3 AppImages?

You don't need the entire libfuse, you just need a few bits. I've read a bit into fuse-rs, it doesn't seem that complex to me.

The size is secondary; we can save bloat elsewhere (e.g., by using musl libc properly thinned down to the essential bits, etc.).

Getting rid of FUSE would be awesome, but I have doubts it's all that easy.

[probonopd]

Is there any limitation in runtime size?

No. No hard limitation. (We should try to make it as small and efficient as possible.)

-----------------------
static linked loader (ELF)
-----------------------
squashfs image
-----------------------

should be sufficient since we can calculate the length of an ELF (and we are already doing it).

[probonopd]

By the way, here is a bare-bones static AppImage type 2 runtime written in Go:

https://github.com/orivej/static-appimage

This runtime is using zip rather than squashfs. It has the added benefit that any existing unzip tool should be able to extract it. (Maybe such AppImages should be named .AppImage.zip to make this more obvious.)

Don't use it for production yet since it may be lacking more advanced features like update information, embedded digital signatures, and such. But it shows that it is doable to make a static AppImage runtime using FUSE.

[probonopd]

Go? how large is the final static runtime?

Depends on the architecture, around 2 MB:
https://github.com/kost/static-appimage/releases

When you run upx -9 on it, you can bring it to under 1 MB.

[TheAssassin]

That other project is doing is really different from us. It's hardly comparable to our runtime. Any sort of size estimation based on that is too imprecise to tell anything useful. Given their runtime is already way larger than ours doesn't really aid your point.

A fully statically linked FUSEless runtime would be great. But I don't see how this can be realized while keeping all the features and characteristics of the existing runtime.

Writing a runtime in Go is also pretty much a bad idea. It adds way too many uncontrollable dependencies. It's a huge mess. Our runtime is embedded in every AppImage. It needs to be absolutely bullet proof license wise. Ideally, it's licensed as permissively as possible, as legally we cannot even safely assume the the resulting AppImage is not considered a derivative work derived from the runtime. This question hasn't been fully answered for the existing runtime.
(Generally, any upcoming AppImage type needs to put a way higher effort into licensing questions.)

[NobodyXu]

Or maybe a completely different approach can be taken:

Provides a modified version of glibc and musl libc that have appimageRuntime embedded into it by modifing functions _start, dlopen and open (optional).

The _start is modified so that the embedded appimageRuntime can parse the cmdline arg and setup environment (unzip the files the tmpfs and etc).

The dlopen and ld.so is modified so that the dynamic libraries would search in the unzipped environment first.

The open can be modified if the program is close source so that read from absolute path /usr for resources can be redirected to the unzipped environment.

If the program cannot be compiled to use this libc or is a shell script, then a more traditional approach can be used:

Add a header to the program that contains a runtime that decompress the environment including the modified libc to tmpfs and setup environment variables LD_PRELOAD and LD_LIBRARY_PATH so that the program/shell would use the modified libc and the bundled dynamic libraries.

The libc then can have its open function modified so that resource be loaded from the unzipped environment.

Edit:

I found that the interpreter and rpath of ELF can be changed by NixOS/patchelf, so there is no need to use LD_PRELOAD and LD_LIBRARY_PATH for close source software unless it forbidden any modification to the binary.

[probonopd]

unzip the files

Unzip which files? AppImages are mounted, not extracted. This gives them their speed.

[NobodyXu]

Unzip which files? AppImages are mounted, not extracted. This gives them their speed.

I was suggesting to throw away fuse and use a compressed tar instead.

Extracting a compressed tar won't be a lot slower than squashfuse while fuse adds overhead to application.

Every read/mmap of the executable or resource bundled with appimage need to go through fuse, which requires the process to wait for at least 2 context switch instead of just one.

[NobodyXu]

@probonopd I've done a naive benchmark between squashfuse used in appimage and tmpfs using nvim.appimage

[nobodyxu@gentoo:/tmp]$ time tar cf squashfuse .mount_nvim.aDB5FO5/

real    0m0.166s
user    0m0.004s
sys     0m0.038s
[nobodyxu@gentoo:/tmp]$
[nobodyxu@gentoo:/tmp]$ time cp -r .mount_nvim.aDB5FO5/ copied_tmp

real    0m0.040s
user    0m0.004s
sys     0m0.029s
[nobodyxu@gentoo:/tmp]$ time tar cf tmp copied_tmp/

real    0m0.023s
user    0m0.004s
sys     0m0.019s
[nobodyxu@gentoo:/tmp]$ time cp -r copied_tmp/ copied_tmp2/

real    0m0.025s
user    0m0.004s
sys     0m0.021s

.mount_nvim.aDB5FO5 is where the nvim.appimage is mounted.

I found that by looking into /proc/<pid>/.

You can see that operations performed on tmpfs is much faster than squashfuse.

Edit:

The benchmark above test the cold run.

The warm run is much faster, but still slower than tmpfs:

[nobodyxu@gentoo:/tmp]$ tar cf squashfuse .mount_nvim.ax0xNEd/
[nobodyxu@gentoo:/tmp]$ rm squashfuse
[nobodyxu@gentoo:/tmp]$ time tar cf squashfuse .mount_nvim.ax0xNEd/

real    0m0.035s
user    0m0.005s
sys     0m0.023s
[nobodyxu@gentoo:/tmp]$ time tar cf squashfuse .mount_nvim.ax0xNEd/

real    0m0.034s
user    0m0.012s
sys     0m0.016s

[probonopd]

If I understand it right, it Looks like https://github.com/eth-cscs/spack-batteries-included is providing a solution for this. Should we backport these changes into the AppImage runtime?

Differences and improvements over AppImage runtime
spack.x uses zstd for faster decompression;
spack.x itself is an entirely static binary;
spack.x does not need to dlopen libfuse.so

Reference:
#1120 (comment)
cc @haampie

[AlexTMjugador]

For those interested in running AppImages in musl containers like me (namely, those based on Alpine), a solution that works today is to extract the AppImage to the container filesystem while building it (for example, with a COPY instruction on the Dockerfile, after running ./Whatever.AppImage --appimage-extract).

If the AppImage was generated with a tool like appimage-builder, which bundles every dependency to the AppImage (including the glibc used by the payload), the resulting AppRun should work flawlessly on pretty much anything you throw at it.

The idea stated above is also applicable in any scenario in which it is feasible to extract the AppImage in a glibc system before running it on a maybe musl system.