Build 32 bit AppImages (fix #95541 improvement)

[shoogle]

So it turns out that enabling 32 bit builds wasn't too difficult after all. This builds on i686 rather than i386 but that shouldn't matter; I get the impression that anything reporting itself as i386 is really i686 these days.

Successful Bintray upload confirmed. I had to switch to including the architecture in the Bintray package names. I don't have a 32 bit system to test the AppImages on but I tested inside a 32 bit chroot on my 64 bit system with no problems.

[ericfont]

Works on my physical i686 system (X60s libreboot running parabola linux LXDE). ~~Note: when I start it with factory reset, the soundfont directory list is /tmp/.mount_UyJaAT/share/mscore-portable-nightly-2.1/sound and ~/Documents/MuseScoreDevelopment/Soundfonts but there are no soundfonts in that tmp mount folder, although the empty folder /tmp/.mount_UyJaAT/share/mscore-portable-nightly-2.1/sound does exist. So my question is what is the purpose of that folder...is it supposed to contain a default minimal soundfont? I'm of course unable to get any playback without manually adding a soundfont folder or downloading a soundfont to one of those folders. Disregard...there is no problem...I've discovered it makes a new tmp mount on every execution, and the new mounts are filled with everything needed, including the soundfonts.~~

Actually there is indeed a problem when performing factory resets on portable AppImage: the subsequent time the AppImage is executed, any settings that refer to folders which were set to the previous tmp mount will now be invalid. So for instance, the first time I execute the AppImage with -F, my soundfont folder got set to /tmp/.mount_UyJaAT/share/mscore-portable-nightly-2.1/sound, but then the subsequent time I execute the AppImage without -F, then that soundfont path now points to a non-existant tmp mount folder from the time I executed with -F. The reason I saw an empty folder was because I believe musescore creates the soundfont folder if the specified folder doesn't previously exist.

I suggest the solution should be: whenever an AppImage begins execution, should always include the current execution's /tmp/mount folder in the list of soundfont folders.

[ericfont]

Note: if I press "Reset All Preferences to Default", then the directory is reset correctly.

The line that performs initialization of soundfont directory generates "/tmp/.mount_UyJaAT/share/mscore-portable-nightly-2.1/sound" from:

QFileInfo(QString("%1%2").arg(mscoreGlobalShare).arg("sound")).absoluteFilePath()

The problem lies with the fact that mscoreGlobalShare is going to be different upon every execution.

[ericfont]

Since I now strongly consider this a bug (and so I don't hijack this thread), I've created an issue https://musescore.org/en/node/99236 so please comment there. I will try to submit a PR.

[shoogle]

Submitting the PR didn't trigger a Travis build for some reason, so I've pushed some minor changes just to trigger Travis now.

[probonopd]

👍

[shoogle]

@ericfont If you want to cross-compile for ARM on Travis this is the place to do it. Add a new case --arm for the ARM-specific build command. You can then call it from .travis.yml like this: build.sh --arm (see how the 32 bit works). The best way to cross-compile is to enter a chroot so that you can avoid conflicts with the native system's libraries when you run copy-libs, which could otherwise end up fetching x86_64 libraries instead of ARM ones.

I haven't fully looked into this myself, but it looks like you need to install a static QEMU binary, and then enter the chroot somehow, possibly using AppImageKit or Docker. There are a few ARM-based Docker images:
Search for "rpi" or "arm" here: https://hub.docker.com
Or here's an Arch linux image: https://hub.docker.com/r/cellofellow/rpi-arch/

I'm not sure if it will work with Docker, but here's how another guy did something similar.

Or I may have a look into this myself at some point if you don't fancy it.

[ericfont]

I'm going to look into it. I did read that running-travis-ci-tests-on-arm article last week.

(I've used QEMU ~10 years ago...I've yet to use docker but am at least familiar with it.)

[shoogle]

Cool.
I had to make some changes to copy-libs for this PR so I ended up including all of the changes we discussed on your branch, except for the ^processor change to the Makefile because I felt that belongs in the ARM PR so we know why it's there when reviewing the code a few years from now. You basically just need to make that change and add your Jessie (or eqivalent) function to copy-libs and update the dependency section of the Recipe for your target distribution.

[shoogle]

@ericfont Keep an eye on this issue for AppImageKit where RazZziel has succeeded in building AppImages for ARM on Travis. He's building on the feature/32bit_builds_docker branch. Build logs are here. He only had to change .travis.yml to make ARM builds work for AppImageKit (MuseScore will probably be a bit harder because you'll have to find the dependencies for the OS you choose).

[ericfont]

excellent! I am going to get back on that right with I finish with this current crash issue.

I was at the point where I was going to use these debian wheezy docker images (https://github.com/multiarch/debian-debootstrap) instead of the raspbian wheezy for a few reasons: (1) Raspbian image is unecesarily larger (2) Raspbian might contain some specific RPi stuff, (3) I discovered that I no longer see a raspbian wheezy image on raspberry website, and (4) I want to consider possibly doing armv7 and armv8 (which is 64-bit) in addition to armv6 (which is the only architecture that raspbian is compiled for) and (5) incase there are any other random architectures now in the future, plain debian will support them all.

[ericfont]

@probonopd In https://github.com/probonopd/AppImageKit/blob/master/README.md#objectives line 8 you write:

Do not require recompilation. AppImages must be possible to create from already-existing binaries, without the need for recompilation. This greatly speeds up the AppImage creation process, since no compiler has to be involved. This also allows third parties to package closed-source applications as AppImages. (Nevertheless, it can be beneficial for upstream application developers to build from source specifically for the purpose of generating an AppImage.)

That leads me to think that yes, I can use crosscompiler to build musescore, and then just package the AppImage inside an ARM container (so gets the ARM libraries).

[shoogle]

That would be possible, but would create a lot of extra work I think. The libraries need to be present during the build as well as at runtime, so you would end up having to maintain two sets of dependencies. Also, you'd have to get the two versions to match exactly, which may not be possible if the Docker image is a different distro to Travis. Finally, I'm not sure if cross-compiling will be any faster than building in QEMU, but I don't know this for sure.

I'd recommend just trying the QEMU option first, as it appears to be the easiest, and just see how long it takes. If it takes too long then we can follow @probonopd's example of creating our own custom Docker image to save having to fetch dependencies. If it's still too long after that then we can look into cross-compiling.

[ericfont]

ok...long story, but I'm holding off on what I would call "native cross compiler" (i.e. the cross compiler is x86-64 running on the Travis x86-64 ubuntu 12.04 using the linaro cross compiler to build arm) which I argue would use much less cpu resources than an emulated arm compiler. I was going to use an exactly equivalent armv7 ubuntu 12.04.05 docker image, to make sure all the libraries correspond to each other...anyway I stopped at the point where I would have to extract all the libraries out of the docker file and put in travis (fyi, I was setting up a cmake toolchain file where I would have needed to provide a root directory of the arm system)... And so maybe, yeah, that is more complicated then just building inside qemu docker image.

So I'm at the point now where I went through that resin.io blog on my own computer. Note that their prebuilt docker files with qemu are actually using Debian Jessie, but it was no problem to fork their Dockerfile and make a docker image w/qemu for Debian Wheezy: https://hub.docker.com/r/ericfont/armv7hf-debian-qemu/~/dockerfile/

So then I used that image as base for one that includes all the musescore dependencies (including qt5 debug symbols, and including compiling AppImageKit) here: https://github.com/ericfont/armv7hf-debian-qemu-musescore-dependencies/blob/master/Dockerfile

I know you said to not worry about pre-creating the docker image, but I think it is very important to do so, because setting up the docker file will use up travis alloted time. That build should be going on now...https://hub.docker.com/r/ericfont/armv7hf-debian-qemu-musescore-dependencies/builds/

Note that that image is useing qt5.3.2 from debian wheezy backports, but I intended to setup the dockerfile to compile the latest qt version from http://download.qt.io/official_releases/qt/5.5/5.5.1/single/qt-everywhere-opensource-src-5.5.1.tar.gz and then only update that docker image when a new qt version comes out. I think I can setup the docker file to do the qt building, so I just need to update the qt version number and do a commit & push to my github.

I'll check back in a few hours.

[ericfont]

I think I have compiled AppImageKit, compile a HelloWorld, and run using AppRun: https://hub.docker.com/r/ericfont/armv7hf-debian-qemu-appimagekit/builds/bnywuskzyttfjagcp6o6at5/

But note that anything cpu intensive is much much slower when running in emulated qemu-arm. I'm think the qemu will be fine for running musescore travis tests and for packaging the AppImage. However, I'm certain now that compilng musescore inside of qemu-arm is not feasable. I'm going to investigate running x86-64 -> arm cross compiler inside travis...so the next step is to use a root filesystem from http://www.armhf.com/download/ for ubuntu 12.04 arm and using that for the cmake toolchain file root directory of arm system (after putting all the musescore dependencies there). Then hopefully would have equivalent libraries to the ubuntu 12.04 x86-64 travis image.

[shoogle]

Looks good! If ARM builds take ages then we needn't necessarily compile one on each commit, and it could be done on Docker Hub rather Travis if there's no time limit on Docker Hub. (Or maybe Travis could even be set up to trigger a build on Docker Hub?)

[michaelgjoni777]

Dorinaaaaaaa