Issue with linux_arm64 builds

[rectalogic]

Describe the bug
Qt now supports linux arm64 builds via the installer

we have added support for Linux on ARM
https://www.qt.io/blog/qt-online-installer-and-installer-framework-4.7-beta-released

https://download.qt.io/online/qtsdkrepository/linux_arm64/desktop/qt6_670/

How do I access these builds with aqt?

To Reproduce

$ aqt list-qt linux desktop --arch 6.7.0
linux_gcc_64

Expected behavior
Expected to see both linux_gcc_64 and linux_arm64.
Actually it looks like x86 is now named linux_x64 not linux_gcc_64
https://download.qt.io/online/qtsdkrepository/linux_x64/

Desktop (please complete the following information):

• OS: macOS
• INFO : aqtinstall(aqt) v3.1.11 on Python 3.11.7 [CPython Clang 15.0.0 (clang-1500.1.0.2.5)]

[ddalcino]

Great question! As this is a new feature, we don’t support it yet. Clearly, this is something we should support.

Part of the problem is that aqt is set up to look for Linux architectures in https://download.qt.io/online/qtsdkrepository/linux_x64/ alone. To support this, aqt would need to be changed to look in the neighboring folder, linux_arm64, as well as the original folder. I suspect that a lot of code might need to change in order to get this working, because this assumption (that aqt only needs to check a single folder) exists in lots of places.

Perhaps an easier solution would be to change the list of host names. For example, you could use aqt install|list-qt linux … for linux_x64 hosts, and aqt install|list-qt linux_arm64 … for the arm hosts. It’s an ugly UI, and I like it a lot less than what you’ve suggested, but the change would be a lot easier to make. never mind, I don’t like that idea.

Also, testing this could be problematic because we don’t have arm64 testing environments set up anywhere.

[ddalcino]

Support for linux arm64 gh actions runners is tracked here: actions/runner-images#5631

This feature will be a lot easier to test after 5631 is implemented

[rectalogic]

This can be tested in an arm64 docker container running on GHA amd64 runner, it's just slower.
https://docs.docker.com/build/ci/github-actions/multi-platform/

[rectalogic]

Well, I got this working on amd64 and arm64
master...rectalogic:aqtinstall:arm64

But then I realized it broke all versions before 6.7.0. They changed the arch name for both amd and arm I guess.

Even the currently released aqt can't install 6.7.0 on amd64:

root@3168fc2d28f9:/# aqt install-qt linux desktop 6.7.0 -O Qt
INFO    : aqtinstall(aqt) v3.1.11 on Python 3.12.1 [CPython GCC 12.2.0]
WARNING : Specified Qt version "6.7.0" did not exist when this version of aqtinstall was released. This may not install properly, but we will try our best.
ERROR   : The packages ['qt_base'] were not found while parsing XML of package information!
==============================Suggested follow-up:==============================
* Please use 'aqt list-qt linux desktop --arch 6.7.0' to show architectures available.

root@3168fc2d28f9:/# aqt list-qt linux desktop --arch 6.6.1
gcc_64 wasm_singlethread wasm_multithread
root@3168fc2d28f9:/# aqt list-qt linux desktop --arch 6.7.0
linux_gcc_64

Anyway, here were my tests running in amd64 and arm64:

$ docker run --platform linux/arm64 -it --init --rm -v $PWD:/aqt python:bookworm bash

root@3168fc2d28f9:/# venv/bin/aqt list-qt linux desktop
6.7.0
root@3168fc2d28f9:/# venv/bin/aqt list-qt linux desktop --arch 6.7.0
linux_gcc_arm64

root@3168fc2d28f9:/# venv/bin/aqt list-qt linux desktop --long-modules 6.7.0 linux_gcc_arm64
    Module Name                     Display Name
===============================================================
debug_info            Desktop arm64 Debug Information Files
qt3d                  Qt 3D Module for Linux arm64
...

root@3168fc2d28f9:/# venv/bin/aqt install-qt linux desktop 6.7.0 -O Qt
INFO    : aqtinstall(aqt) v3.1.12.dev0 on Python 3.12.1 [CPython GCC 12.2.0]
WARNING : Specified Qt version "6.7.0" did not exist when this version of aqtinstall was released. This may not install properly, but we will try our best.
WARNING : Specified target combination "linux desktop linux_gcc_arm64" did not exist when this version of aqtinstall was released. This may not install properly, but we will try our best.
INFO    : Downloading qttools...
...
INFO    : Patching Qt/6.7.0/gcc_arm64/lib/libQt6QmlCompiler.prl
INFO    : Finished installation
INFO    : Time elapsed: 30.22630356 second

$ docker run --platform linux/amd64 -it --init --rm -v $PWD:/aqt python:bookworm bash

root@3168fc2d28f9:/# venv/bin/aqt list-qt linux desktop
5.9.0 5.9.1 5.9.2 5.9.3 5.9.4 5.9.5 5.9.6 5.9.7 5.9.8 5.9.9
5.10.0 5.10.1
5.11.0 5.11.1 5.11.2 5.11.3
5.12.0 5.12.1 5.12.2 5.12.3 5.12.4 5.12.5 5.12.6 5.12.7 5.12.8 5.12.9 5.12.10 5.12.11 5.12.12
5.13.0 5.13.1 5.13.2
5.14.0 5.14.1 5.14.2
5.15.0 5.15.1 5.15.2
6.0.0 6.0.1 6.0.2 6.0.3 6.0.4
6.1.0 6.1.1 6.1.2 6.1.3
6.2.0 6.2.1 6.2.2 6.2.3 6.2.4
6.3.0 6.3.1 6.3.2
6.4.0 6.4.1 6.4.2 6.4.3
6.5.0 6.5.1 6.5.2 6.5.3
6.6.0 6.6.1
6.7.0
root@3168fc2d28f9:/# venv/bin/aqt list-qt linux desktop --arch 6.7.0
linux_gcc_64

root@3168fc2d28f9:/# venv/bin/aqt list-qt linux desktop --long-modules 6.7.0 linux_gcc_64
    Module Name                    Display Name
============================================================
debug_info            Desktop 64 Debug Information Files
qt3d                  Qt 3D Module for Linux 64
...

root@3168fc2d28f9:/# venv/bin/aqt install-qt linux desktop 6.7.0 -O Qt
INFO    : aqtinstall(aqt) v3.1.12.dev0 on Python 3.12.1 [CPython GCC 12.2.0]
WARNING : Specified Qt version "6.7.0" did not exist when this version of aqtinstall was released. This may not install properly, but we will try our best.
WARNING : Specified target combination "linux desktop linux_gcc_64" did not exist when this version of aqtinstall was released. This may not install properly, but we will try our best.
INFO    : Downloading qtbase...
...
INFO    : Patching Qt/6.7.0/gcc_64/lib/libQt6QmlCompiler.prl
INFO    : Finished installation
INFO    : Time elapsed: 29.12089451 second

[ddalcino]

From this code, at https://github.com/rectalogic/aqtinstall/blob/arm64/aqt/helper.py#L282-L288:

def os_arch_ext(os_name: str) -> str:
    if os_name == "windows":
        return "_x86"
    elif os_name == "linux" and platform.machine() == "aarch64":
        return "_arm64"
    else:
        return "_x64"

Looks like some logic is missing. Here's some pseudocode that I think might fix the issue:

if host == "linux":
    if qt_version >= Version("6.7.0"):
        if flavor == "arm64":
            arch = "linux_gcc_arm64"
        else:  # flavor == "x86"
            arch = "linux_gcc_64"
    else:
        arch = "gcc_64"
else:
    # handle mac and windows

Also, please keep in mind that we try to give users the ability to install whatever they want, wherever they want it. You're allowed to install Mac binaries on a Windows or Linux machine, etc. I think this is to allow cross compilation. Checking that platform.machine() == "aarch64" would break that.

[rectalogic]

Hmm, so src/doc/examples are in qtsdkrepository/linux_x64/desktop/ - so will they be replicating those under qtsdkrepository/linux_arm64/desktop/ - so then we would need an arch to install src?

I think your original idea of a new host - linux and linux_arm64 (that you didn't like) might be a much simpler way to deal with this. I guess I'll wait and see how they end up organizing all this. It makes no sense why they would organize it like this.

[ddalcino]

It makes no sense why they would organize it like this.

LOL welcome to QT, there's a lot of stuff like this at download.qt.io, unfortunately.

[rectalogic]

I updated the patch to use linux_arm64 host:
master...rectalogic:aqtinstall:arm64

It seems OK, except when installing on arm64 it's missing a library.

root@4305faf2cd32:~# aqt install-qt linux_arm64 desktop 6.7.0 -O Qt
INFO    : aqtinstall(aqt) v3.1.12.dev0 on Python 3.12.1 [CPython GCC 12.2.0]
WARNING : Specified Qt version "6.7.0" did not exist when this version of aqtinstall was released. This may not install properly, but we will try our best.
INFO    : Downloading qttools...
INFO    : Downloading qtbase...
...
INFO    : Finished installation of qtdeclarative-Linux-Debian_11_6-GCC-Linux-Debian_11_6-AARCH64.7z in 31.16426397
Qt/6.7.0/gcc_arm64/bin/qmake: error while loading shared libraries: libdouble-conversion.so.3: cannot open shared object file: No such file or directory
INFO    : Finished installation
INFO    : Time elapsed: 32.54353152 second

root@4305faf2cd32:~# Qt/6.7.0/gcc_arm64/bin/qmake
Qt/6.7.0/gcc_arm64/bin/qmake: error while loading shared libraries: libdouble-conversion.so.3: cannot open shared object file: No such file or directory
root@4305faf2cd32:~# ldd Qt/6.7.0/gcc_arm64/bin/qmake
	linux-vdso.so.1 (0x0000ffff8b844000)
	libQt6Core.so.6 => /root/Qt/6.7.0/gcc_arm64/bin/../lib/libQt6Core.so.6 (0x0000ffff8b000000)
	libpthread.so.0 => /lib/aarch64-linux-gnu/libpthread.so.0 (0x0000ffff8afd0000)
	libstdc++.so.6 => /lib/aarch64-linux-gnu/libstdc++.so.6 (0x0000ffff8adb0000)
	libc.so.6 => /lib/aarch64-linux-gnu/libc.so.6 (0x0000ffff8ac00000)
	libicui18n.so.73 => /root/Qt/6.7.0/gcc_arm64/bin/../lib/libicui18n.so.73 (0x0000ffff8a8b0000)
	libicuuc.so.73 => /root/Qt/6.7.0/gcc_arm64/bin/../lib/libicuuc.so.73 (0x0000ffff8a690000)
	libdl.so.2 => /lib/aarch64-linux-gnu/libdl.so.2 (0x0000ffff8a660000)
	libglib-2.0.so.0 => /lib/aarch64-linux-gnu/libglib-2.0.so.0 (0x0000ffff8a500000)
	libz.so.1 => /lib/aarch64-linux-gnu/libz.so.1 (0x0000ffff8a4c0000)
	libdouble-conversion.so.3 => not found
	librt.so.1 => /lib/aarch64-linux-gnu/librt.so.1 (0x0000ffff8a490000)
	libm.so.6 => /lib/aarch64-linux-gnu/libm.so.6 (0x0000ffff8a3f0000)
	libgcc_s.so.1 => /lib/aarch64-linux-gnu/libgcc_s.so.1 (0x0000ffff8a3b0000)
	/lib/ld-linux-aarch64.so.1 (0x0000ffff8b807000)
	libicudata.so.73 => /root/Qt/6.7.0/gcc_arm64/bin/../lib/libicudata.so.73 (0x0000ffff88510000)
	libpcre2-8.so.0 => /lib/aarch64-linux-gnu/libpcre2-8.so.0 (0x0000ffff88460000)

I guess the user is supposed to install this themselves? apt install libdouble-conversion3 fixed it.

[rectalogic]

PR #765 is another attempt at supporting this, without changing the linux host type.