Add appimagecraft build script

[TheAssassin]

Allows Popsicle to be built as an AppImage with appimagecraft.

CC #104.

[ids1024]

I'm not too familiar with appimage, but this doesn't seem to be building for me (on Pop!_OS 20.04):

-- Running output plugin: appimage -- 
[appimage/stdout] Found appimagetool: /tmp/.mount_linuxdNn0Ntf/plugins/linuxdeploy-plugin-appimage/usr/bin/appimagetool
[appimage/stdout] 
[appimage/stdout] /home/ian/src/pop/popsicle/.appimagecraft-build-script-l3go65fn/AppDir/com.system76.Popsicle.desktop: warning: key "Keywords" in group "Desktop Entry" is a reserved key for KDE
[appimage/stdout] /home/ian/src/pop/popsicle/.appimagecraft-build-script-l3go65fn/AppDir should be packaged as Popsicle_USB_Flasher-1.1.0-6-g3990486-x86_64.AppImage
[appimage/stdout] Trying to validate AppStream information with the appstreamcli tool
[appimage/stdout] In case of issues, please refer to https://github.com/ximion/appstream
[appimage/stdout] [appimage/stderr] Running command: /tmp/.mount_linuxdNn0Ntf/plugins/linuxdeploy-plugin-appimage/usr/bin/appimagetool "../AppDir" "-g"
[appimage/stderr] 
[appimage/stderr] appimagetool, continuous build (commit 0880085), build 2133 built on 2020-07-09 12:25:52 UTC
[appimage/stderr] Using architecture x86_64
[appimage/stderr] Deleting pre-existing .DirIcon
[appimage/stderr] Creating .DirIcon symlink based on information from desktop file
[appimage/stderr] AppStream upstream metadata found in usr/share/metainfo/com.system76.Popsicle.appdata.xml
[appimage/stderr] /usr/bin/appstreamcli: symbol lookup error: /lib/x86_64-linux-gnu/libsoup-2.4.so.1: undefined symbol: g_file_info_get_modification_date_time
[appimage/stderr] Failed to validate AppStream information with appstreamcli
[appimage/stderr] ERROR: Failed to run plugin: appimage (exit code: 1) 

[TheAssassin]

Well, seems like your appstreamcli doesn't run at all. Have you tried running it outside appimagecraft?

AppStream validation is optional; if appstreamcli isn't found, the tool should work.

You should consider building your AppImage on the oldest still-supported LTS release of some major distro. We use Ubuntu xenial for most builds ourselves. Building on bionic will work, too, but some users will be excluded.
Note that I tested the appimagecraft script on xubuntu bionic myself, where it worked fine. Not sure if all dependencies would be available on xenial as well.

[ids1024]

Yes, appstreamcli seems to run fine outside appimagecraft. I suppose the cause is the same as AppImage/AppImageKit#856.

[TheAssassin]

Indeed. By coincidence, I created an issue related to this problem today: linuxdeploy/linuxdeploy-plugin-gtk#4

[ids1024]

I was able to build with some difficulty in a rust:1.40.0-stretch docker container:

set -e

apt-get update
apt-get install -y shellcheck libgtk-3-dev fuse
wget https://github.com/TheAssassin/appimagecraft/releases/download/continuous/appimagecraft-x86_64.AppImage
chmod +x appimagecraft-x86_64.AppImage
./appimagecraft-x86_64.AppImage

docker run --rm -v "$PWD:/popsicle" -w "/popsicle" --privileged rust:1.40.0-stretch bash appimage.sh

However, it is failing to run on Pop!_OS 20.04:

/tmp/.mount_PopsicOnMQ6l/AppRun.wrapped: error while loading shared libraries: libffi.so.6: cannot open shared object file: No such file or directory

Not sure why that isn't included in the appimage. (Of course, 20.04 has libffi.so.7 instead).

[TheAssassin]

I wonder why you had to install shellcheck, IIRC it should be bundled. Anyway.

Can you please share the resulting AppImage and perhaps a log of the appimagecraft run?

[ids1024]

Here's a zip with the appimage and a log:

appimage.zip

[TheAssassin]

Very strange. https://github.com/AppImage/pkg2appimage/blob/master/excludelist doesn't contain libffi, so it should be bundled if the binary depends on it. Apparently, the popsicle-gtk binary depends on that library.

When I run linuxdeploy on my workstation (which runs bionic), libffi is deployed just fine. Something's wrong here. I'll have to try it myself in such a container...

[ids1024]

Yes, it is odd. It seems libffi is required by libwayland-client and libgobject, but both of those end up in the appimage.

Hopefully it should be easy to reproduce (a nice advantage of docker).

[probonopd]

this doesn't seem to be building for me (on Pop!_OS 20.04)

Please do build on the oldest still-supported LTS release of Ubuntu, so that the resulting binaries will be able to run on all still-supported versions of Ubuntu and other similar age distributions.

If you build on a distribution from 2020, the compiled binaries will most likely not run on e.g., 2018ish systems.

So xenial it is. bionic and focal are way too new.

[mmstick]

That might not be possible if Xenial's Rust compiler isn't kept in sync with our minimum target (Focal).

[probonopd]

Then this AppImage may be a candidate for bundling everything.

[ids1024]

That might not be possible if Xenial's Rust compiler isn't kept in sync with our minimum target (Focal).

The rust:1.40.0-stretch docker image seems like a good choice for both an old LTS distro and the right rustc version. Though as mentioned above, I saw a weird issue trying it.

[TheAssassin]

Stretch binaries will surely work and be more than sufficient to reach most of the target audience... Also, one should note that I had no difficulties building on bionic, which is also no longer the latest LTS (it's 2 years old...).

Possibly related to linuxdeploy/linuxdeploy#145 (although #145 hasn't been proven, yet to see some real evidence). What's weird is that it does deploy the app reliably on my workstation...

[mmstick]

Bionic is roughly keeping in sync with Rust compiler toolchain updates, albeit it tends to lag a month behind. I'm not sure how long they intend to keep supporting Xenial, though it does have the same version as Focal now. Having popsicle execute within an isolated namespace completely separate from the host system, as the Flatpak works, would be ideal. No need to worry about any system libraries, or development dependencies on that system.

[TheAssassin]

@ids1024 the issue causing not all dependencies to be deployed was indeed in linuxdeploy. Fixed it there, should work now.

[ids1024]

Yep, that fixed it. Now it builds an appimage that seems to run fine.

The remaining issue I see is that I had to pass --privileged to docker, for fuse to work. Using ./appimagecraft-x86_64.AppImage --appimage-extract-and-run partly fixes it, but appimagecraft then invokes linuxdeploy-x86_64.AppImage without such a flag, leading to errors about fuse.

[TheAssassin]

Run export APPIMAGE_EXTRACT_AND_RUN=1 before running appimagecraft, it'll propagate to all subprocesses.

[ids1024]

Ah yes. With APPIMAGE_EXTRACT_AND_RUN it seems to work pretty much as I might want.

Running the appimage does result in a couple errors on the terminal when opening the file chooser:

/usr/lib/x86_64-linux-gnu/gio/modules/libgioremote-volume-monitor.so: undefined symbol: g_mount_operation_get_is_tcrypt_system_volume
Failed to load module: /usr/lib/x86_64-linux-gnu/gio/modules/libgioremote-volume-monitor.so
/usr/lib/x86_64-linux-gnu/gio/modules/libgioremote-volume-monitor.so: undefined symbol: g_mount_operation_get_is_tcrypt_system_volume
Failed to load module: /usr/lib/x86_64-linux-gnu/gio/modules/libgioremote-volume-monitor.so

It does however work. Some functionality of the gtk file chooser may not.

If we do want to officially provide appimages for Popsicle, I'm pretty happy with the official rust docker images as an environment to build them. Given they include versions with older debian releases, and newer compilers.

[TheAssassin]

Please feel free to open an issue in https://github.com/linuxdeploy/linuxdeploy-plugin-gtk. Perhaps the plugin should deploy a few more libraries? Those libs look like dlopened plugins of some kind. In any case, it doesn't seem like a dealbreaker, and might be optimized later on.

[ids1024]

I took a look at CI for this. It seems Github Actions offers Ubuntu 16.04, which has Rustc 1.43. So it's fine as far as the Rust version...

But then, it seems something calls the system's shellcheck, which is preinstalled on Github Actions but fails because it doesn't recognize the -x argument. Uninstalling it works. git describe --tags fails because Github Actions clones (at least by default) with --depth=1 and --no-tags. Though when that failed, Github Actions listed the job as successfull because appimagecraft returned 0 (TheAssassin/appimagecraft#9).

Then it gives the message Requested 'gdk-3.0 >= 3.22' but version of GDK is 3.18.9, so the GTK version specified in gtk/Cargo.toml is too new for 16.04, I suppose.

[TheAssassin]

I build most AppImages on Travis CI. There, it's not a big deal to build with Docker. Maybe that's an option here, too? I think going for the Rust Docker images is a good idea, too.

PR merged, thanks.

Regarding the version, you can also just specify an environment variable there (echo "$myvar"). Or maybe just fetch the entire Git repo's data. A shallow clone can always be extended.

[ids1024]

There, it's not a big deal to build with Docker. Maybe that's an option here, too?

Yep, it seems to have docker pre-installed, and that works: https://github.com/pop-os/popsicle/actions/runs/281263143

I thought I'd try building directly in Github Actions, but this seems better. I generally like CI based on docker (like Gitlab uses) since it can work locally and is based on whatever clean base image you choose, rather than the remarkably random assortment of software the CI provider seems to offer.

It should be easy to also have Github Actions add the appimage to tagged releases automatically.

@mmstick @jackpot51 So do we want to offer AppImages for Popsicle? It seems like a good thing to have available as long as it works, and the build doesn't break much or require significant maintenance.

[TheAssassin]

I think your tool has a chance to become some serious competition for all the other image writing tools out there. It's minimalistic, does its job and doesn't track every single click you take (especially without consent, like Etcher does).

It's the most usable one I've seen so far, and it works well. I think you just need a little bit of marketing. And the fact that, as an AppImage, it's portable and can be used on almost every Linux desktop, is probably going to help.

Perhaps you even could offer two AppImages? You also provide a CLI version, and I could imagine there's people out there who might want to use it. We could also write an AppRun script (entry point of the AppImage, usually it's just a symlink to the main binary but you can also put a script there) which decides where to route a call. That way, you could e.g., call the CLI when argv[1] is cli or something like that.

[srevinsaju]

I am interested in seeing an official appimage soon. I just tried this today, and its amazing! 🎉
One of the most lightweight usb flashing tools 🚀

[TheAssassin]

@ids1024 any update on this? Like @srevinsaju I'm really waiting for an AppImage.

[ids1024]

Merged in #108.