Fix cross-compile to consider CMAKE_CROSSCOMPILING_EMULATOR.

[Veganizer]

Author: Justin Jagieniak justin@jagieniak.net, John Zimmermann me@johnnynator.dev
Date: Fri Apr 11 14:19:00 2021 +0200

Fix cross-compile patch to consider CMAKE_CROSSCOMPILING_EMULATOR aswell.

[CLAassistant]

All committers have signed the CLA.

[grooverdan]

The first line of https://cmake.org/cmake/help/latest/variable/CMAKE_CROSSCOMPILING_EMULATOR.html indicates that this is only on when CMAKE_CROSSCOMPILING is on. Wouldn't that mean all the NOT CROSSCOMPILING OR ... changes are unneeded?

(CMAKE_CROSSCOMPILING AND NOT DEFINED CMAKE_CROSSCOMPILING_EMULATOR) changes look ok. (though DEFINED look unnecessary given a non-empty string is true).

Can you give an example of how to test CMAKE_CROSSCOMPILING_EMULATOR?

[Veganizer]

Hello @grooverdan ,

The problem is following in IF NOT CMAKE_CROSSCOMPILING

If CROSS_COMPILING IS TRUE THEN the values aren't set for example. But a cross_compiling_emulator works like a real machine, so it still needs it. So IF the emulator is defined then it should set the values even if CMAKE_CROSSCOMPILING is true.

example:
before:

IF(NOT  CMAKE_CROSSCOMPILING)
// code
ENDIF

code is only executed if we don't have a crosscompiler here.

after:

IF(NOT CMAKE_CROSSCOMPILING OR DEFINED CMAKE_CROSSCOMPILING_EMULATOR)
// code
ENDIF

code is executed if we don't have a cross compiler or if we have one with a cross compiling emulator.

But I tried it without DEFINED before and the cross-compiling-emulator did complain it. So DEFINED should be needed here.

[Veganizer]

@grooverdan sorry had to edit the comment because I saw the problem during I wanted to refactor code :-)

edit: I have done the patch for following PR in void linux void-linux/void-packages#30143

[grooverdan]

your void linux PR is aiming for 10.5. Did you want to target this for 10.5 so you don't need to carry the patch?

on other carried patches:
Thought I'd fixed MDEV-24131, let me know if I didn't.
Apologizes for #1716 and its earlier incantations being so slow.
I do realize there's a musl / build / CI /testing gap on our side that needs to be fixed.

Any tips on using CMAKE_CROSSCOMPILING_EMULATOR welcome. Due to hardware/people contraints we miss things like MariaDB/mariadb-docker#318 and I hope this is a good enabler to facilitating more testing, even if emulated.

[Veganizer]

thank you for your review on the void package @grooverdan

I did try to remove the disable-failing-unit-patch and it seems to be fine so far.
Very well!
You are welcome to target it for 10.5. Then I will remove Johnny and my patch there.

First I have to say that I don't belong to the void team. I am just contributing and hope to help them. Void is my favourite linux distribution.
I am not such familiar with the cross compiling emulator void is using but I can say that it works easy and well.
I think it shouldn't be a problem to use void or his xbps (see here: https://github.com/void-linux/xbps) to test different architectures for mariadb. Of course you are welcome to use the linked template for buildling mariadb with it.
The void team is very helpful and is caring much for the compatibility to other architectures.

Edit: I know that XBPS is using chroot and qemu.
For example:
https://github.com/void-linux/void-packages/blob/master/common/build-helper/qemu.sh
https://github.com/void-linux/void-packages/blob/master/common/cross-profiles/x86_64-musl.sh

Maybe @Johnnynator wants to join the discussion here. :-)

[Johnnynator]

Any tips on using CMAKE_CROSSCOMPILING_EMULATOR welcome. Due to hardware/people contraints we miss things like MariaDB/mariadb-docker#318 and I hope this is a good enabler to facilitating more testing, even if emulated.

Voids cross compile setup is just having a normal system that does all the heavy lifting usually at /, on /usr/$cross-arch e.g. /usr/armv7l-linux-musleabihf we do have a the target system with all the libs/includes and such.

The normal cmake cross setup is just setting compiler and cflags as appropriate.

CMAKE_CROSS_COMPILING_EMULATOR is set, to allow running the configure tests and the occasional build time compiled executable. The CMAKE_CROSS_COMPILING_EMULATOR variable just points to the appropriate qemu-$arch-static binary like e.g. /usr/bin/qemu-arm-static and the enviroment variable QEMU_LD_PREFIX set to your target system, in this example this would be QEMU_LD_PREFIX=/usr/armv7l-linux-musleabihf

A setup like that would already allow to just run e.g. ninja test, but for mariadb this isn't enough, since some external setup like setting up/launching mysqld would be needed, so only running the unittests like that would work ootb.

The usualy test suite can be run by just executing /usr/bin/qemu-arm-static /usr/armv7l-linux-musleabihf/usr/bin/perl mysql-test/mtr --parallel=5 --force --max-test-fail=40 , the -mem option didn't work in this setup.
I also have not finished the full test suite like that yet, since it is obviously way slower than running it natively.

[grooverdan]

Thanks @Johnnynator.

So it looks like to create a void-linux CI container I'd:
in a container build:

• take a root tarball https://docs.voidlinux.org/installation/guides/chroot.html
• using xbps install the buildbot bits (like other distros)
• install host dependencies + cmake, qemu-static
• cross install basesystem + make dependencies + c++ compilers

If one of you could point me at the https://build.voidlinux.org/ URL when your mariadb update goes through that might help fill in the gaps for me.

Not sure when I'll get to filling that muslc/armv[567] and other gaps, but thanks for providing some parts that will significantly help the process.

However @Veganizer, on your PR, 1715fef now merged to 10.5 with expanded commit message. Thank you very much, and thanks for updating Void LInux's package too.

[Johnnynator]

If one of you could point me at the build.voidlinux.org URL when your mariadb update goes through that might help fill in the gaps for me.

I do have just pushed it, but I doubt that you would see much interesting things in the logs.
If you have any specific questions, I would be happy to answer them.

muslc/armv[567]

More like armv[67] or armv5 support does not have any binary repo currently (and probably never will).

[Veganizer]

@grooverdan Just wanted to inform you that the Update of MariaDB is in the official void repos now :-)