Build 32 bit AppImages (fix #95541 improvement) #2398
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -4,6 +4,8 @@ | |
| # Build portable Linux AppImages and upload them to Bintray. AppImages will | ||
| # always be uploaded unless a list of specific branches is passed in. e.g.: | ||
| # $ build.sh --upload-branches master my-branch-1 my-branch-2 | ||
| # Builds will be for the native architecture (64 bit) unless another is | ||
| # specified for cross-compiling. (e.g. build.sh --32bit) | ||
|
|
||
| set -e # exit on error | ||
| set -x # echo commands | ||
|
|
@@ -27,17 +29,31 @@ else | |
| makefile_overrides="" # use Makefile defaults | ||
| fi | ||
|
|
||
| # Build AppImage. Are we cross-compiling? | ||
| case "$1" in | ||
| --32bit ) | ||
| shift | ||
| # Build MuseScore AppImage inside 32-bit Docker image | ||
| docker run -i -v "${PWD}:/MuseScore" toopher/centos-i386:centos6 /bin/bash -c \ | ||
| "linux32 --32bit i386 /MuseScore/build/Linux+BSD/portable/Recipe $makefile_overrides" | ||
| ;; | ||
| * ) | ||
| [ "$1" == "--64bit" ] && shift || true | ||
| # Build MuseScore AppImage inside native (64-bit) Docker image | ||
| docker run -i -v "${PWD}:/MuseScore" library/centos:6 /bin/bash -c \ | ||
| "/MuseScore/build/Linux+BSD/portable/Recipe $makefile_overrides" | ||
| ;; | ||
| esac | ||
|
There was a problem hiding this comment. @ericfont If you want to cross-compile for ARM on Travis this is the place to do it. Add a new case I haven't fully looked into this myself, but it looks like you need to install a static QEMU binary, and then enter the chroot somehow, possibly using AppImageKit or Docker. There are a few ARM-based Docker images: I'm not sure if it will work with Docker, but here's how another guy did something similar. Or I may have a look into this myself at some point if you don't fancy it. There was a problem hiding this comment. I'm going to look into it. I did read that running-travis-ci-tests-on-arm article last week. (I've used QEMU ~10 years ago...I've yet to use docker but am at least familiar with it.) There was a problem hiding this comment. Cool. There was a problem hiding this comment. @ericfont Keep an eye on this issue for AppImageKit where RazZziel has succeeded in building AppImages for ARM on Travis. He's building on the feature/32bit_builds_docker branch. Build logs are here. He only had to change .travis.yml to make ARM builds work for AppImageKit (MuseScore will probably be a bit harder because you'll have to find the dependencies for the OS you choose). There was a problem hiding this comment. excellent! I am going to get back on that right with I finish with this current crash issue. I was at the point where I was going to use these debian wheezy docker images (https://github.com/multiarch/debian-debootstrap) instead of the raspbian wheezy for a few reasons: (1) Raspbian image is unecesarily larger (2) Raspbian might contain some specific RPi stuff, (3) I discovered that I no longer see a raspbian wheezy image on raspberry website, and (4) I want to consider possibly doing armv7 and armv8 (which is 64-bit) in addition to armv6 (which is the only architecture that raspbian is compiled for) and (5) incase there are any other random architectures now in the future, plain debian will support them all. There was a problem hiding this comment. @probonopd In https://github.com/probonopd/AppImageKit/blob/master/README.md#objectives line 8 you write: Do not require recompilation. AppImages must be possible to create from already-existing binaries, without the need for recompilation. This greatly speeds up the AppImage creation process, since no compiler has to be involved. This also allows third parties to package closed-source applications as AppImages. (Nevertheless, it can be beneficial for upstream application developers to build from source specifically for the purpose of generating an AppImage.) That leads me to think that yes, I can use crosscompiler to build musescore, and then just package the AppImage inside an ARM container (so gets the ARM libraries). There was a problem hiding this comment. That would be possible, but would create a lot of extra work I think. The libraries need to be present during the build as well as at runtime, so you would end up having to maintain two sets of dependencies. Also, you'd have to get the two versions to match exactly, which may not be possible if the Docker image is a different distro to Travis. Finally, I'm not sure if cross-compiling will be any faster than building in QEMU, but I don't know this for sure. I'd recommend just trying the QEMU option first, as it appears to be the easiest, and just see how long it takes. If it takes too long then we can follow @probonopd's example of creating our own custom Docker image to save having to fetch dependencies. If it's still too long after that then we can look into cross-compiling. There was a problem hiding this comment. ok...long story, but I'm holding off on what I would call "native cross compiler" (i.e. the cross compiler is x86-64 running on the Travis x86-64 ubuntu 12.04 using the linaro cross compiler to build arm) which I argue would use much less cpu resources than an emulated arm compiler. I was going to use an exactly equivalent armv7 ubuntu 12.04.05 docker image, to make sure all the libraries correspond to each other...anyway I stopped at the point where I would have to extract all the libraries out of the docker file and put in travis (fyi, I was setting up a cmake toolchain file where I would have needed to provide a root directory of the arm system)... And so maybe, yeah, that is more complicated then just building inside qemu docker image. So I'm at the point now where I went through that resin.io blog on my own computer. Note that their prebuilt docker files with qemu are actually using Debian Jessie, but it was no problem to fork their Dockerfile and make a docker image w/qemu for Debian Wheezy: https://hub.docker.com/r/ericfont/armv7hf-debian-qemu/~/dockerfile/ So then I used that image as base for one that includes all the musescore dependencies (including qt5 debug symbols, and including compiling AppImageKit) here: https://github.com/ericfont/armv7hf-debian-qemu-musescore-dependencies/blob/master/Dockerfile I know you said to not worry about pre-creating the docker image, but I think it is very important to do so, because setting up the docker file will use up travis alloted time. That build should be going on now...https://hub.docker.com/r/ericfont/armv7hf-debian-qemu-musescore-dependencies/builds/ Note that that image is useing qt5.3.2 from debian wheezy backports, but I intended to setup the dockerfile to compile the latest qt version from http://download.qt.io/official_releases/qt/5.5/5.5.1/single/qt-everywhere-opensource-src-5.5.1.tar.gz and then only update that docker image when a new qt version comes out. I think I can setup the docker file to do the qt building, so I just need to update the qt version number and do a commit & push to my github. I'll check back in a few hours. There was a problem hiding this comment. I think I have compiled AppImageKit, compile a HelloWorld, and run using AppRun: https://hub.docker.com/r/ericfont/armv7hf-debian-qemu-appimagekit/builds/bnywuskzyttfjagcp6o6at5/ But note that anything cpu intensive is much much slower when running in emulated qemu-arm. I'm think the qemu will be fine for running musescore travis tests and for packaging the AppImage. However, I'm certain now that compilng musescore inside of qemu-arm is not feasable. I'm going to investigate running x86-64 -> arm cross compiler inside travis...so the next step is to use a root filesystem from http://www.armhf.com/download/ for ubuntu 12.04 arm and using that for the cmake toolchain file root directory of arm system (after putting all the musescore dependencies there). Then hopefully would have equivalent libraries to the ubuntu 12.04 x86-64 travis image. There was a problem hiding this comment. Looks good! If ARM builds take ages then we needn't necessarily compile one on each commit, and it could be done on Docker Hub rather Travis if there's no time limit on Docker Hub. (Or maybe Travis could even be set up to trigger a build on Docker Hub?) |
||
|
|
||
| # Should the AppImage be uploaded? | ||
| if [ "$1" == "--upload-branches" ]; then | ||
| # User passed in list of branchs so only upload those listed | ||
| shift | ||
| for upload_branch in "$@" ; do | ||
| [ "$branch" == "$upload_branch" ] && upload=true || true # bypass `set -e` | ||
| done | ||
| else | ||
| # No list passed in so upload on every branch | ||
| upload=true | ||
| fi | ||
|
|
||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Would make sense to not build if we don't upload no?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Indeed it would! But the build has to start anyway to check which branch we are on so I thought it might as well continue. There are supposed to be ways to prevent the build from starting at all but I couldn't get them to work in a way that would satisfy our needs.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Instead of discarding the AppImages or trying to prevent the build from starting at all for PRs, I recommend that they be uploaded to a seperate PR repository, and have travis post links to issue tracker and to github PR, so other devs can easily test the PRs. Please reply to my feature request here: https://musescore.org/en/node/101496