Appimage support

[Murmele]

@probonopd

Currently I am trying to create an appimage for Gittyup and integrating it into the CI. Currently we have only a Flatpak, there is an unofficial version for arch in aur. Do you recommend using appimage-builder or using linuxdeploy?

[probonopd]

Thanks @Murmele for adding an AppImage. As for which tools to use, I'd probably use https://github.com/probonopd/go-appimage/tree/master/src/appimagetool myself, but mainly because I wrote this one and am not as familiar with all the other options.

There are many great options out there:
https://github.com/AppImageCommunity/awesome-appimage#deployment-tools-for-compiled-applications

Which one works best for your application and workflow I don't know, maybe you'd like to try out a few and then decide?

[Murmele]

@probonopd thank you for the proposal. I tried out appimage tool, but the application panicked:

2023/06/19 14:34:58 Got qt_prfxpath but it does not contain 'plugins'
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x30 pc=0xb29cea]

goroutine 1 [running]:
github.com/probonopd/go-appimage/internal/helpers.FilesWithSuffixInDirectoryRecursive.func1({0xc000409068, 0x15}, {0x0?, 0x0?}, {0x14?, 0xc000491310?})
	github.com/probonopd/go-appimage/internal/helpers/helpers.go:105 +0x4a
path/filepath.Walk({0xc000409068, 0x15}, 0xc000491320)
	path/filepath/path.go:515 +0x50
github.com/probonopd/go-appimage/internal/helpers.FilesWithSuffixInDirectoryRecursive({0xc000409068?, 0xc000409068?}, {0x30f434?, 0x30c481?})
	github.com/probonopd/go-appimage/internal/helpers/helpers.go:104 +0x73
main.getQtPrfxpath(0xc000479380?, {0x2f?, 0x0?}, 0xc0004914a8?)
	github.com/probonopd/go-appimage/src/appimagetool/appdirtool.go:1424 +0x309
main.handleQt({{0xc0000fad20, 0x6}, {0xc0000aeb70, 0x16}, {0xc0000aec78, 0x16}}, 0x5)
	github.com/probonopd/go-appimage/src/appimagetool/appdirtool.go:1181 +0x16c
main.AppDirDeploy({0x7fffd05e8b01?, 0x313ddd?})
	github.com/probonopd/go-appimage/src/appimagetool/appdirtool.go:270 +0x685
main.bootstrapAppImageDeploy(0xc0001da2c0)
	github.com/probonopd/go-appimage/src/appimagetool/cli.go:36 +0x1fd
github.com/urfave/cli/v2.(*Command).Run(0xc0000fd7a0, 0xc0001da1c0)
	github.com/urfave/cli/v2@v2.17.1/command.go:177 +0x719
github.com/urfave/cli/v2.(*App).RunContext(0xc000114380, {0x3f4c78?, 0xc0000b6020}, {0xc0000b4000, 0x4, 0x4})
	github.com/urfave/cli/v2@v2.17.1/app.go:387 +0x1035
github.com/urfave/cli/v2.(*App).Run(...)
	github.com/urfave/cli/v2@v2.17.1/app.go:252
main.main()
	github.com/probonopd/go-appimage/src/appimagetool/cli.go:288 +0x6df

If you need more information, see latest ci build

[probonopd]

@Murmele you do not need linuxdeployqt, AppImageKit's appimagetool nor patchelf (the latter somes with go-appimage's appimagetool).

Actually this is all it takes; the following works for me on Ubuntu 20.04.6 LTS:

git submodule init
git submodule update

# Build OpenSSL; why does it need a private (self-built) version?
cd dep/openssl/openssl
./config -fPIC
make -j $(npoc)
cd ../../../

# Install Qt
sudo add-apt-repository ppa:beineri/opt-qt-5.15.2-focal -y
sudo apt-get update -qq
sudo apt-get -y install qt515base qt515tools libgl1-mesa-dev
. /opt/qt*/bin/qt*-env.sh

# Install ninja
sudo pip install ninja

# Build
mkdir -p build/release
cd build/release
cmake -G Ninja -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/usr/ ../..
ninja
DESTDIR=./appdir ninja install ; find ./appdir
rm -rf ./appdir/usr/include/

# Deploy dependencies into AppDir
wget -c https://github.com/$(wget -q https://github.com/probonopd/go-appimage/releases/expanded_assets/continuous -O - | grep "appimagetool-.*-x86_64.AppImage" | head -n 1 | cut -d '"' -f 2)
chmod +x appimagetool-*.AppImage
LD_LIBRARY_PATH=/opt/qt515/lib/ ./appimagetool-*.AppImage -s deploy ./appdir/usr/share/applications/*.desktop --appimage-extract-and-run # Bundle EVERYTHING

# Turn AppDir into AppImage
./appimagetool-*.AppImage ./appdir --appimage-extract-and-run # turn AppDir into AppImage

# Two files have been generated, both must be uploaded to GitHub Releases for AppImageUpdate to work
ls -lh Gittyup-*
-rwxr-xr-x 1 codespace codespace  43M Jul  5 23:44 Gittyup-b0d1c70-x86_64.AppImage
-rw-rw-rw- 1 codespace codespace 150K Jul  5 23:44 Gittyup-b0d1c70-x86_64.AppImage.zsync

For AppImageUpdate to work, don't forget to upload both generated files to GitHub Releases.

Here you see the AppImage running:

[Murmele]

Hi @probonopd . thank you I will try it out!

[probonopd]

On https://github.com/probonopd/Gittyup/releases there is an AppImage built on GitHub Releases like this:
https://github.com/probonopd/Gittyup/blob/master/.github/workflows/build.yml

There is one remaining issue that can probably only be fixed by editing the source code:
Somehow, local paths from the build system get compiled into the binary:

~/Downloads/Gittyup-14-x86_64.AppImage --appimage-extract # Extract the AppImage
./squashfs-root/AppRun # Run the application
PANIC: unprotected error in call to Lua API (cannot open /home/runner/work/Gittyup/Gittyup/conf/System.lua: No such file or directory)

/home/runner/work/Gittyup/Gittyup/conf/System.lua possibly existed on the build machine, but for sure it doesn't exist on the user's machine.

Where is it compiled into?

grep -r /home/runner/work/Gittyup/Gittyup squashfs-root
Binary file squashfs-root/usr/bin/gittyup matches
Binary file squashfs-root/usr/bin/indexer matches
(...)
strings squashfs-root/usr/bin/gittyup | grep "home/runner"
/home/runner/work/Gittyup/Gittyup/conf
/home/runner/work/Gittyup/Gittyup/l10n

strings squashfs-root/usr/bin/indexer | grep "home/runner"
/home/runner/work/Gittyup/Gittyup/conf
/home/runner/work/Gittyup/Gittyup/l10n
(...)

That needs to be changed.

[Murmele]

/Gittyup/conf/System.lua

Hi @probonopd,

thank you for the CI setup!

I installed your appimage:
On a RHEL 7.9 I am getting the following error message:
/tmp/.mount_GittyuhibPFE/usr/bin/gittyup: error while loading shared libraries: libQt5Core.so.5: cannot open shared object file: No such file or directory
I extracted the appimage and indeed, no qt dependencies are in there

I found the problem in my build. I set QT_DIR but your tool is searching for QTDIR.
I published GittyupAppdir (Appdir directory with all dependencies) as artifact and executed it. I got the same panic as you (I will have a look into it), but creating the appimage file does not work with the following error:

LD_LIBRARY_PATH='' find AppDir -type f -exec ldd {} 2>&1 \; | grep '=>' | grep -v AppDir
2023/07/10 12:45:51 Architecture of libQt5Concurrent.so.5: x86_64
Gittyup
ERROR Determine architecture: Could not detect a valid architecture
2023/07/10 12:45:51 git root: /home/runner/work/Gittyup/Gittyup
2023/07/10 12:45:51 Target AppImage filename: Gittyup-1.0-x86_64.AppImage
2023/07/10 12:45:51 Icon file: AppDir/usr/share/icons/hicolor/256x256/apps/gittyup.png
Trying to validate AppStream information with the appstreamcli tool
ERROR appstreamcli: signal: segmentation fault (core dumped)

[probonopd]

I installed your appimage:
On a RHEL 7.9 I am getting the following error message:
/tmp/.mount_GittyuhibPFE/usr/bin/gittyup: error while loading shared libraries: libQt5Core.so.5: cannot open shared object file: No such file or directory
I extracted the appimage and indeed, no qt dependencies are in there

Interesting! Are you sure you used the AppImage from https://github.com/probonopd/Gittyup/releases?
It definitely contains libQt5Core.so.5, I just double checked.
And it starts on openSUSE, but then immediately crashes with a Lua API panic due to a missing file in /home.

This is the strace output:

I will retry it on a CentOS 7.8 Live ISO.
Edit: I can reproduce it there, libQt5Core.so.5 does not get loaded. I am looking into it.

[probonopd]

Trying to validate AppStream information with the appstreamcli tool
ERROR appstreamcli: signal: segmentation fault (core dumped)

For now, remove the AppStream metainfo file and see if it builds then. We can sort out the AppStream file later.

[Murmele]

Sorry. I tested again and now on Arch it works (without considering the LUA problem).
For some reason I don't get any debug() output. I tested locally and I get it. Tried also to log to a file but without success. I have to check

[probonopd]

This is an interesting problem. On a CentOS 7.8 Live ISO, when I extract the AppImage and then run strace ./squashfs-root/AppRun, it appears like it finds and successfully open (no ENOENT) libQt5Core.so.5, does a short pread on it, and then closes it again with close(3), then continues to search for it in other locations. It is almost like it decides it doesn't like that library.

The kernel on CentOS 7.8 is 3.10.0, this is a reeeeally old kernel. Inspecting libQt5Core.so.5 with the file command shows that it claims to need at least kernel 3.17.0 (which is from October 2013; almost 10 years ago). It seems like we would need to find another build of Qt that supports older kernels.

What kernel version are you using?

[Murmele]

RHEL7.9 has 3.10.0 as kernel version

I tried no my version of the appimage and I get (On an up to date Arch Linux)

Something went wrong trying to read the squashfs image.

Cannot mount AppImage, please check your FUSE setup.
You might still be able to extract the contents of this AppImage 
if you run it with the --appimage-extract option. 
See https://github.com/AppImage/AppImageKit/wiki/FUSE 
for more information
open dir error: No such file or directory

[probonopd]

RHEL7.9 has 3.10.0 as kernel version

It seems that the kernel is too old to run that build of Qt.

I tried no my version of the appimage and I get (On an up to date Arch Linux)

What does which fusermount ; which fusermount3 say on your system?
Does it work with --appimage-extract-and-run?
Does my AppImage run on that same system?

[Murmele]

which fusermount ; which fusermount3

/usr/bin/fusermount
/usr/bin/fusermount3

Yes your build is working fine.

The problem with the PANIC seems to be that the qt applicationDirPath() does not direct to the gittyup executable, but to the linux-ld lib
https://github.com/Murmele/Gittyup/blob/master/src/conf/Settings.cpp#L198-L203

Because of this all other paths (themes, config, plugins, ...) are wrong and therefore Lua panics, because it cannot find the required files

[probonopd]

That is an unintended side effect of using -s deploy. You can remove the -s and see what happens. Downside is that it will run only on systems no older than the build system.

And I have one more idea, but I need to test that one first...

[probonopd]

Tested this locally, seems to be working:

https://github.com/probonopd/Gittyup/blob/e9ce88ab710a347817d4980ce101db75bf759af6/appimage-builder/build.sh#L37-L40

This way you can use -s deploy and it will still work.

The next build of my AppImage should contain the fix.

Maybe I will change go-appimage appimagetool to do something like this automatically in the future (possibly by using symlinks).

[probonopd]

My AppImage now works properly (on systems with Linux kernel 3.17.0 and newer). If you need this to work on older kernels, we need to find/build another Qt.

[Murmele]

Great :D I am waiting until the ci is finished and then I hope I can say the same thing :)

Which qt version is available on CentOS? I will try it on RHEL7.9

[probonopd]

Which qt version is available on CentOS? I will try it on RHEL7.9
https://centos.pkgs.org/7/centos-x86_64/qt5-qtbase-5.9.7-4.el7.x86_64.rpm.html

Apparently Qt 5.9.7. But I have another idea, let me try.

[Murmele]

Which qt version is available on CentOS? I will try it on RHEL7.9
https://centos.pkgs.org/7/centos-x86_64/qt5-qtbase-5.9.7-4.el7.x86_64.rpm.html

Apparently Qt 5.9.7. But I have another idea, let me try.

Oh thats low. On Arch my build works now as well. Thank you very much. Now the easy part of cleaning up is missing ^^
On RHEL7.9 it works also pretty nice. The indexer crashes during startup, but after it restarts it seems to work fine

[probonopd]

Tried to build on Ubuntu xenial (that Qt would likely run on RHEL 7), but it doesn't build there.

[Murmele]

Tried to build on Ubuntu xenial (that Qt would likely run on RHEL 7), but it doesn't build there.

Whats the error message you are getting?

[probonopd]

https://github.com/probonopd/Gittyup/actions/runs/5513710024/jobs/10052156512#step:4:10994

[Murmele]

On RHEL7.9, remote workstation

qt.glx: qglx_findConfig: Failed to finding matching FBConfig for QSurfaceFormat(version 2.0, options QFlags<QSurfaceFormat::FormatOption>(), depthBufferSize -1, redBufferSize 1, greenBufferSize 1, blueBufferSinterval 1, colorSpace QSurfaceFormat::DefaultColorSpace, profile  QSurfaceFormat::NoProfile)
No XVisualInfo for format QSurfaceFormat(version 2.0, options QFlags<QSurfaceFormat::FormatOption>(), depthBufferSize -1, redBufferSize 1, greenBufferSize 1, blueBufferSize 1, alphaBufferSize -1, stencilBufferefaultColorSpace, profile  QSurfaceFormat::NoProfile)
Falling back to using screens root_visual.

https://github.com/flameshot-org/flameshot/pull/1034/files

RPCS3/rpcs3#4509

[Murmele]

On RHEL7.9, remote workstation

qt.glx: qglx_findConfig: Failed to finding matching FBConfig for QSurfaceFormat(version 2.0, options QFlags<QSurfaceFormat::FormatOption>(), depthBufferSize -1, redBufferSize 1, greenBufferSize 1, blueBufferSinterval 1, colorSpace QSurfaceFormat::DefaultColorSpace, profile  QSurfaceFormat::NoProfile)
No XVisualInfo for format QSurfaceFormat(version 2.0, options QFlags<QSurfaceFormat::FormatOption>(), depthBufferSize -1, redBufferSize 1, greenBufferSize 1, blueBufferSize 1, alphaBufferSize -1, stencilBufferefaultColorSpace, profile  QSurfaceFormat::NoProfile)
Falling back to using screens root_visual.

https://github.com/flameshot-org/flameshot/pull/1034/files

RPCS3/rpcs3#4509

Fixed with rm ./AppDir/lib/x86_64-linux-gnu/libxcb-glx.so.0
removing all xcb leads to a SegFault rm ./AppDir/lib/x86_64-linux-gnu/libxcb*

[Murmele]

https://github.com/probonopd/Gittyup/actions/runs/5513710024/jobs/10052156512#step:4:10994

Does the ci build do not work on RHEL7?

[Murmele]

@probonopd on RHEL7.9 the indexer application crashes with a segmentation fault. Tried to removing everything except the main(), but I had no success with it

#595

[probonopd]

Fixed with rm ./AppDir/lib/x86_64-linux-gnu/libxcb-glx.so.0

That's not a good idea for applications deployed with -s deploy because then the application starts pulling in system libraries, which can lead to truly unpredictable results.

[probonopd]

Does the ci build do not work on RHEL7?

You mean, building on RHEL? I have never tried that.

Why is RHEL 7 so important for your use case? Even RH will maintain it only for another year:

https://www.redhat.com/en/blog/end-maintenance-red-hat-enterprise-linux-7-almost-here

[Murmele]

You mean, building on RHEL? I have never tried that.

Mine is almost working fine on RHEL, just a segmentation fault with the indexer I have to find out. At work we are using RHEL7.9

RHEL 7.x is still supported: https://en.wikipedia.org/wiki/Red_Hat_Enterprise_Linux

[probonopd]

Right, still supported by RH for a year.

[Murmele]

At work we are getting RHEL9 by the end of the year, so I will test it there again, maybe we don't need this hack with deleting the library

[Murmele]

Fixed with rm ./AppDir/lib/x86_64-linux-gnu/libxcb-glx.so.0

That's not a good idea for applications deployed with -s deploy because then the application starts pulling in system libraries, which can lead to truly unpredictable results.

I removed this again