Proposal: emscripten CI with docker

[isc30]

Emscripten provides really nice docker images to help compiling.
I successfully applied this to my CI (travis) when compiling corrade + magnum + my game in emscripten.
This is just a proposal, but I think it's a better way than installing emscripten (old version) in Homebrew.

[mosra]

Nice, thank you! Didn't know about that. The problem I'm facing is that the builds take more than 20 minutes, because they're running on macOS (because at the time when I was setting this up there wasn't no ready-made package for Ubuntu and compiling Emscripten and LLVM from scratch is high on the list of things you don't want to do in a CI script) and even there the Homebrew installation takes ages. So even though Docker might add some overhead to the VM setup, I think it would be much faster in total.

• Would it be possible for you to submit a PR updating the scripts to use the Docker image instead? Feel free to abuse magnum's CI for testing.
• Also, is it possible to get some older version as a Docker image? I don't want to force people to be always on latest Emscripten and thus I need to test for that.

Thanks again!

[isc30]

Yes, tagged images should be available for different versions.
About the build times, it takes ~3mins in Travis to build corrade-rc + corrade + magnum without tests.
I'll experiment a bit and create a PR. Thanks!

[Squareys]

I already have such a docker image... I will paste the docker file here later

[Squareys]

Here you go, may it help. See notes below:

emscripten-magnum/Dockerfile

FROM trzeci/emscripten:latest

# Use WebGL1 instead of WebGL2?
ARG USE_WEBGL1=OFF

# Get ninja-build
RUN cd /tmp/ &&\
    wget https://github.com/ninja-build/ninja/releases/download/v1.8.2/ninja-linux.zip && \
    unzip ninja-linux.zip && \
    mv ninja /usr/local/bin/ && \
    ninja --version

# Create output folders
RUN mkdir -p ~/dist ~/dist-native

# corrade (native) for corrade-rc and them corrade for emscripten
RUN cd /tmp/ && \
    git clone --recursive https://github.com/mosra/corrade/ && \
    cd /tmp/corrade && mkdir -p build build-emc && cd build && \
    cmake .. -GNinja -DCMAKE_BUILD_TYPE=Release \
        -DCMAKE_INSTALL_PREFIX=~/dist-native \
        -DCMAKE_INSTALL_RPATH=/root/dist-native/lib \
        -DCMAKE_MAKE_PROGRAM=ninja \
        -DWITH_INTERCONNECT=OFF \
        -DWITH_PLUGINMANAGER=OFF \
        -DWITH_TESTSUITE=OFF && \
    cmake --build . --target install && \
    cd /tmp/corrade/build-emc && \
    cmake .. -GNinja -DCMAKE_BUILD_TYPE=Release \
        -DCMAKE_MAKE_PROGRAM=ninja \
        -DCMAKE_TOOLCHAIN_FILE="../toolchains/generic/Emscripten-wasm.cmake" \
        -DCMAKE_INSTALL_PREFIX=~/dist \
        -DCORRADE_RC_EXECUTABLE=~/dist-native/bin/corrade-rc && \
    cmake --build . --target install && \
    cd ~/ && rm -rf /tmp/corrade

# magnum
RUN cd /tmp/ && \
    git clone --recursive https://github.com/mosra/magnum/ && \
    cd /tmp/magnum && mkdir -p build-emc && cd build-emc && \
    cmake .. -GNinja -DCMAKE_BUILD_TYPE=Release \
        -DCMAKE_MAKE_PROGRAM=ninja \
        -DCMAKE_TOOLCHAIN_FILE="../toolchains/generic/Emscripten-wasm.cmake" \
        -DCMAKE_INSTALL_PREFIX=~/dist \
        -DCMAKE_FIND_ROOT_PATH=~/dist \
        -DWITH_SDL2APPLICATION=ON \
        -DWITH_ANYIMAGEIMPORTER=ON \
        -DWITH_ANYSCENEIMPORTER=ON \
        -DWITH_ANYAUDIOIMPORTER=ON \
        -DWITH_AUDIO=ON \
        -DWITH_DEBUGTOOLS=ON \
        -DWITH_MESHTOOLS=ON \
        -DWITH_OBJIMPORTER=ON \
        -DWITH_PRIMITIVES=ON \
        -DWITH_SCENEGRAPH=ON \
        -DWITH_SHADERS=ON \
        -DWITH_TEXT=ON \
        -DWITH_WAVAUDIOIMPORTER=ON \
        -DTARGET_GLES2=${USE_WEBGL1} \
        -DCMAKE_INSTALL_PREFIX=~/dist \
        -DCORRADE_RC_EXECUTABLE=~/dist-native/bin/corrade-rc && \
    cmake --build . --target install && \
    cd ~/ && rm -rf /tmp/magnum

# magnum-plugins
RUN cd /tmp/ && \
    git clone --recursive https://github.com/mosra/magnum-plugins/ && \
    cd /tmp/magnum-plugins && mkdir -p build-emc && cd build-emc && \
    cmake .. -GNinja -DCMAKE_BUILD_TYPE=Release \
        -DCMAKE_MAKE_PROGRAM=ninja \
        -DCMAKE_TOOLCHAIN_FILE="../toolchains/generic/Emscripten-wasm.cmake" \
        -DCMAKE_INSTALL_PREFIX=~/dist \
        -DCMAKE_FIND_ROOT_PATH=~/dist \
        -DWITH_OPENGEXIMPORTER=ON \
        -DWITH_TINYGLTFIMPORTER=ON \
        -DWITH_STBIMAGEIMPORTER=ON \
        -DWITH_STBIMAGECONVERTER=ON \
        -DWITH_STBVORBISAUDIOIMPORTER=ON \
        -DCMAKE_INSTALL_PREFIX=~/dist \
        -DCORRADE_RC_EXECUTABLE=~/dist-native/bin/corrade-rc && \
    cmake --build . --target install && \
    cd ~/ && rm -rf /tmp/magnum-plugins

# Cleanup
RUN rm -rf /tmp/*

ENV LD_LIBRARY_PATH /usr/local/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}

You will:

• Need to use some version tag instead of latest (the earliest you can go is 1.37.29, since Magnum needs cmake 3.7 for the toolchain and with that tag my version upgrade was merged)
• Need to move most of the building into the CI script rather than docker image
• Probably be able to get away with just using the base image and moving everything else into the CI script (travis.yml and package/ci/build-emscripten.sh or something) even
• Need to compile magnum and corrade every CI run to ensure you are using the latest version as ensuring e.g. HEAD of magnum still works with a new HEAD of corrade, the examples still compile after a magnum change etc.

Maybe all of that was obivous, though, then don't mind me re-stating that 😅

* Edit: I just saw you're using the same base image, so you probably knew most of this already

[isc30]

Hi, thanks for the details.
I also think that having a script that actually does everything (cmake configure + build + ctest) is the best approach since the base image already has modern cmake installed.
I don't have time to do the PR today but (as I did with my repo) it should be quite simple to switch to this docker approach that runs build-emscripten.sh inside the container.
I'll do it when I have some spare time

[mosra]

This was done by @Squareys in mosra/corrade#105 and then mirrored in other repos as part of the Travis -> CircleCI migration (mosra/corrade@edcf585, de786e5, mosra/magnum-plugins@dc6fe54, ...).