Make mingw-w64-qt5-* packages work with QBS

[Martchus]

Setting up QBS for cross compiling with mingw-w64 should likely work using the following steps:

1. Setup compiler/tool-chain configuration:

qbs-setup-toolchains /usr/bin/x86_64-w64-mingw32-g++ mingw-w64-g++

2. Setup Qt configuration:

qbs-setup-qt /usr/bin/x86_64-w64-mingw32-qmake-qt5 mingw-w64-g++
Creating profile 'mingw-w64-g++'.                                                                             
Skipping prl file '/usr/x86_64-w64-mingw32/lib/qt/plugins/playlistformats/libqtmultimedia_m3u.prl', because it cannot be opened (No such file or directory).
Skipping prl file '/usr/x86_64-w64-mingw32/lib/qt/plugins/qmltooling/libqmldbg_native.prl', because it cannot be opened (No such file or directory).
Skipping prl file '/usr/x86_64-w64-mingw32/lib/libQt5QuickWidgets.prl', because it cannot be opened (No such file or directory).
Skipping prl file '/usr/x86_64-w64-mingw32/lib/qt/plugins/imageformats/libqtga.prl', because it cannot be opened (No such file or directory).
Skipping prl file '/usr/x86_64-w64-mingw32/lib/libQt5Concurrent.prl', because it cannot be opened (No such file or directory).
Skipping prl file '/usr/x86_64-w64-mingw32/lib/qt/plugins/generic/libqtuiotouchplugin.prl', because it cannot be opened (No such file or directory).
Skipping prl file '/usr/x86_64-w64-mingw32/lib/qt/plugins/platforms/libqwindows.prl', because it cannot be opened (No such file or directory).
Skipping prl file '/usr/x86_64-w64-mingw32/lib/qt/plugins/imageformats/libqjpeg.prl', because it cannot be opened (No such file or directory).
Skipping prl file '/usr/x86_64-w64-mingw32/lib/libQt5Test.prl', because it cannot be opened (No such file or directory).
...
Skipping prl file '/usr/x86_64-w64-mingw32/lib/qt/plugins/qmltooling/libqmldbg_local.prl', because it cannot be opened (No such file or directory).
Skipping prl file '/usr/x86_64-w64-mingw32/lib/libQt5FontDatabaseSupport.prl', because it cannot be opened (No such file or directory).
Skipping prl file '/usr/x86_64-w64-mingw32/lib/libQt5Location.prl', because it cannot be opened (No such file or directory).
WARNING: You need to set up toolchain information before you can use this Qt version for building. However, no toolchain profile was found. Either create one using qbs-setup-toolchains and set it as this profile's base profile or add the toolchain settings manually to this profile.

Working around the issue of locating the PRL files by creating symlinks does not help. In this case QBS will still treat the Qt libraries as Linux build.

---

So obviously QBS does not treat the Qt version provided in this repository as Windows version. I belief this is because my mkspec is called mingw-w64-g++ and not win32-g++. (I use my own mkspec because there are a few adjustments needed for the mingw-w64 environment provided by the Arch Linux AUR packages.) At least I found some checks in the QBS sources where the mkspec name is used:

• https://github.com/qbs/qbs/blob/4952ec5172b67d38637523bd9f22fdef4d51b5c0/src/lib/qtprofilesetup/qtprofilesetup.cpp#L654
• https://github.com/qbs/qbs/blob/4952ec5172b67d38637523bd9f22fdef4d51b5c0/src/lib/qtprofilesetup/qtprofilesetup.cpp#L698
• https://github.com/qbs/qbs/blob/4952ec5172b67d38637523bd9f22fdef4d51b5c0/src/lib/qtprofilesetup/qtprofilesetup.cpp#L862

---

Possible solutions:

• just modify the existing win32-g++ mkspec rather than introducing a new one
• patch QBS
• don't care about QBS, just use CMake

---

The current approach to merge shared and static library trees also prevents to use the static version with QBS. In fact, it would be nice to be able to customize the search pattern used to locate the PRL files like it can be done with qmake to support this use-case.

[Martchus]

I'll switch back to modifying the existing win32-g++ mkspec to workaround this. However, the following changes in QBS would still be required:

• Name of qtmain and extension must not be hard-coded (https://github.com/qbs/qbs/blob/4952ec5172b67d38637523bd9f22fdef4d51b5c0/src/lib/qtprofilesetup/qtprofilesetup.cpp#L701). The variable QMAKE_LIBS_QT_ENTRY from the mkspec should be used instead (if present).
• The library name shell32 is spelled lower-case which makes a difference when compiling under a case-sensitive environment. (https://github.com/qbs/qbs/blob/4952ec5172b67d38637523bd9f22fdef4d51b5c0/src/lib/qtprofilesetup/qtprofilesetup.cpp#L709)
• The qmake variable QMAKE_EXT_PRL must be considered when searching for PRL files.

[Martchus]

@saidinesh5

Just successfully compiled the QML example from https://github.com/qbs/qbs :-)

For testing without recompiling everything, I used the following hack:

• Edit /usr/x86_64-w64-mingw32/lib/qt/mkspecs/win32-g++/qmake.conf:

#
# qmake configuration for i686-w64-mingw32-g++ and x86_64-w64-mingw32-g++
#
# Written for cross compilation with mingw-w64 under GNU/Linux
#
# Cross compile example for i686-w64-mingw32-g++:
#   configure -xplatform mingw-w64-g++ -device-option CROSS_COMPILE=i686-w64-mingw32-
#
# The device options CROSS_COMPILE_CUSTOM_CONFIG and CROSS_COMPILE_CFLAGS can be used
# to specify custom config/flags for cross compilation.

load(device_config)
include(../common/gcc-base.conf)
include(../common/g++-base.conf)
include(../common/angle.conf)
include(../common/windows-vulkan.conf)

# modifications to gcc-base.conf and g++-base.conf
MAKEFILE_GENERATOR      = MINGW
QMAKE_PLATFORM          = win32 mingw
CONFIG                 += debug_and_release debug_and_release_target precompile_header $${CROSS_COMPILE_CUSTOM_CONFIG}
DEFINES                += UNICODE _UNICODE
QMAKE_COMPILER_DEFINES += __GNUC__ WIN32
# can't add 'DEFINES += WIN64' and 'QMAKE_COMPILER_DEFINES += _WIN64' defines for
# x86_64 platform similar to 'msvc-desktop.conf' toolchain, because, unlike for MSVC,
# 'QMAKE_TARGET.arch' is inherently unavailable.

QMAKE_CC                = $${CROSS_COMPILE}gcc
QMAKE_LEX               = flex
QMAKE_LEXFLAGS          =
QMAKE_YACC              = bison -y
QMAKE_YACCFLAGS         = -d
QMAKE_CFLAGS            = -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions --param=ssp-buffer-size=4 -fno-keep-inline-dllexport $${CROSS_COMPILE_CFLAGS}
QMAKE_CFLAGS_WARN_ON   += -Wextra

QMAKE_CFLAGS_SSE2      += -mstackrealign

QMAKE_CXX               = $${CROSS_COMPILE}g++
QMAKE_CXXFLAGS          = $$QMAKE_CFLAGS
QMAKE_CXXFLAGS_WARN_ON  = $$QMAKE_CFLAGS_WARN_ON
QMAKE_CXXFLAGS_RTTI_ON  = -frtti
QMAKE_CXXFLAGS_RTTI_OFF = -fno-rtti
QMAKE_CXXFLAGS_EXCEPTIONS_ON = -fexceptions -mthreads
QMAKE_CXXFLAGS_EXCEPTIONS_OFF = -fno-exceptions

QMAKE_INCDIR            =

QMAKE_RUN_CC            = $(CC) -c $(CFLAGS) $(INCPATH) -o $obj $src
QMAKE_RUN_CC_IMP        = $(CC) -c $(CFLAGS) $(INCPATH) -o $@ $<
QMAKE_RUN_CXX           = $(CXX) -c $(CXXFLAGS) $(INCPATH) -o $obj $src
QMAKE_RUN_CXX_IMP       = $(CXX) -c $(CXXFLAGS) $(INCPATH) -o $@ $<

QMAKE_LINK              = $${CROSS_COMPILE}g++
QMAKE_LINK_C            = $${CROSS_COMPILE}gcc
QMAKE_LFLAGS            = -g
QMAKE_LFLAGS_EXCEPTIONS_ON = -mthreads
QMAKE_LFLAGS_RELEASE    =
QMAKE_LFLAGS_DEBUG      =
QMAKE_LFLAGS_CONSOLE    = -Wl,-subsystem,console
QMAKE_LFLAGS_WINDOWS    = -Wl,-subsystem,windows
QMAKE_LFLAGS_DLL        = -shared
QMAKE_LFLAGS_GCSECTIONS = -Wl,--gc-sections
QMAKE_LINK_OBJECT_MAX   = 10
QMAKE_LINK_OBJECT_SCRIPT = object_script
QMAKE_EXT_OBJ           = .o
QMAKE_EXT_RES           = _res.o
QMAKE_PREFIX_SHLIB      =
QMAKE_EXTENSION_SHLIB   = dll
QMAKE_PREFIX_STATICLIB  = lib
QMAKE_EXTENSION_STATICLIB = a
QMAKE_EXTENSION_IMPORTLIB = dll.a

QMAKE_LIBS              =
QMAKE_LIBS_CORE         = -lz -lpcre2-16 -liconv -lversion -lnetapi32 -luserenv -lole32 -luuid -lwinmm -lws2_32 -ladvapi32 -lshell32 -luser32 -lkernel32
QMAKE_LIBS_GUI          = -lopengl32 -lgdi32 -lcomdlg32 -loleaut32 -limm32 -lwinmm -lws2_32 -lole32 -luuid -luser32 -ladvapi32 -ljpeg -lpng \
                              $$system($${QMAKE_PKG_CONFIG} --libs harfbuzz) \
                              $$system($${QMAKE_PKG_CONFIG} --libs freetype2)
QMAKE_LIBS_GUI_STATIC   = -lopengl32 -lgdi32 -lcomdlg32 -loleaut32 -limm32 -lwinmm -lws2_32 -lole32 -luuid -luser32 -ladvapi32 -ljpeg -lpng \
                              $$system($${QMAKE_PKG_CONFIG} --static --libs harfbuzz) \
                              $$system($${QMAKE_PKG_CONFIG} --static --libs freetype2)
QMAKE_LIBS_NETWORK      = -lws2_32 -lcrypt32 -ldnsapi -liphlpapi
QMAKE_LIBS_NETWORK_STATIC = $$system($${QMAKE_PKG_CONFIG} --static --libs openssl) \
                              -lws2_32 -lcrypt32 -ldnsapi -liphlpapi
QMAKE_LIBS_DBUS         = $$system($${QMAKE_PKG_CONFIG} --libs dbus-1)
QMAKE_LIBS_DBUS_STATIC  = $$system($${QMAKE_PKG_CONFIG} --static --libs dbus-1)
QMAKE_LIBS_WIDGETS_STATIC = -ldwmapi -luxtheme
QMAKE_LIBS_OPENGL       = -lglu32 -lopengl32 -lgdi32 -luser32
QMAKE_LIBS_OPENGL_ES2   = -l$${LIBEGL_NAME} -l$${LIBGLESV2_NAME} -ld3d9 -ldxguid -lgdi32 -luser32
QMAKE_LIBS_OPENGL_ES2_DEBUG = -l$${LIBEGL_NAME} -l$${LIBGLESV2_NAME} -ld3d9 -ldxguid -lgdi32 -luser32
QMAKE_LIBS_COMPAT       = -ladvapi32 -lshell32 -lcomdlg32 -luser32 -lgdi32 -lws2_32
QMAKE_LIBS_QT_ENTRY     = -lmingw32 -lqt5main

QMAKE_IDL               = $${CROSS_COMPILE}widl
QMAKE_LIB               = $${CROSS_COMPILE}ar -rc
QMAKE_RC                = $${CROSS_COMPILE}windres
QMAKE_DLLTOOL           = $${CROSS_COMPILE}dlltool
QMAKE_LRELEASE          = $${CROSS_COMPILE}lrelease-qt5

QMAKE_STRIP             = $${CROSS_COMPILE}strip
QMAKE_STRIPFLAGS_LIB   += --strip-unneeded
QMAKE_OBJCOPY           = $${CROSS_COMPILE}objcopy
QMAKE_NM                = $${CROSS_COMPILE}nm -P

PKG_CONFIG              = $${CROSS_COMPILE}pkg-config
QMAKE_PKG_CONFIG        = $${CROSS_COMPILE}pkg-config

load(qt_config)

• Create qmake wrapper to provide fake config:

#!/usr/bin/bash
if [[ $1 == '-query' ]]; then
    echo 'QT_SYSROOT:
QT_INSTALL_PREFIX:/usr/x86_64-w64-mingw32
QT_INSTALL_ARCHDATA:/usr/x86_64-w64-mingw32/lib/qt
QT_INSTALL_DATA:/usr/x86_64-w64-mingw32/share/qt
QT_INSTALL_DOCS:/usr/x86_64-w64-mingw32/share/doc/qt
QT_INSTALL_HEADERS:/usr/x86_64-w64-mingw32/include/qt
QT_INSTALL_LIBS:/usr/x86_64-w64-mingw32/lib
QT_INSTALL_LIBEXECS:/usr/x86_64-w64-mingw32/lib/qt/bin
QT_INSTALL_BINS:/usr/x86_64-w64-mingw32/bin
QT_INSTALL_TESTS:/usr/x86_64-w64-mingw32/tests
QT_INSTALL_PLUGINS:/usr/x86_64-w64-mingw32/lib/qt/plugins
QT_INSTALL_IMPORTS:/usr/x86_64-w64-mingw32/lib/qt/imports
QT_INSTALL_QML:/usr/x86_64-w64-mingw32/lib/qt/qml
QT_INSTALL_TRANSLATIONS:/usr/x86_64-w64-mingw32/share/qt/translations
QT_INSTALL_CONFIGURATION:/usr/x86_64-w64-mingw32/etc
QT_INSTALL_EXAMPLES:/usr/x86_64-w64-mingw32/share/qt/examples
QT_INSTALL_DEMOS:/usr/x86_64-w64-mingw32/share/qt/examples
QT_HOST_PREFIX:/usr/x86_64-w64-mingw32
QT_HOST_DATA:/usr/x86_64-w64-mingw32/lib/qt
QT_HOST_BINS:/usr/x86_64-w64-mingw32/lib/qt/bin
QT_HOST_LIBS:/usr/x86_64-w64-mingw32/lib
QMAKE_SPEC:linux-g++
QMAKE_XSPEC:win32-g++
QMAKE_VERSION:3.1
QT_VERSION:5.10.1'
    exit 0
fi
/usr/bin/x86_64-w64-mingw32-qmake-qt5 "$@"

• Symlink qtmain:

cd /usr/x86_64-w64-mingw32/lib
ln -s libqt5main.a libqtmain.a

• Configure qbs:

qbs-setup-toolchains --type mingw /usr/bin/x86_64-w64-mingw32-g++ mingw-w64-g++
qbs-setup-qt /path/to/qmake-wrapper.sh mingw-w64-qt

• Build and run example:

git clone https://github.com/qbs/qbs.git
cd qbs/examples/compiled-qml
qbs profile:mingw-w64-qt
x86_64-w64-mingw32-wine default/myapp.94e7d341/myapp.exe

[Martchus]

Should work now if you create that symlink manually. Note that the symlink can't be part of the package because it would defeat the point of renaming the library in the first place (point is to avoid conflicts with Qt 4 version).

[saidinesh5]

@Martchus awesome. will try this one out for a personal project.

Had to finally manually compile mxe with qt (takes like 4-6 hours on my laptop but was the only way i could find) for the work project.

I hope with Qt 6 trying to move away from qmake, there will be a better to maintain this without symlinks etc.. (This thread may interest you: http://lists.qt-project.org/pipermail/development/2018-July/033100.html - feel free to chime in about cross compilation headaches)

The docker container we are using for that is created by:

FROM ubuntu:16.04

ENV DEBIAN_FRONTEND noninteractive

RUN apt-get update \
    && apt-get install -y mesa-utils xserver-xorg-video-all libgl1-mesa-dev libfontconfig1-dev libfreetype6-dev libx11-dev libxext-dev libxfixes-dev libxi-dev libxrender-dev libxcb1-dev libx11-xcb-dev libxcb-glx0-dev \
    && apt-get install -y build-essential autoconf automake libtool make gcc g++ libboost*dev libsocketcan-dev libprotobuf-dev libprotoc-dev libgps-dev protobuf-compiler cmake valgrind git fuse sudo \
    && apt-get install -y software-properties-common python-software-properties ccache \
    && rm -rf /var/lib/apt/lists/* && apt-get dist-upgrade -y

#Set up qbs
RUN add-apt-repository -y ppa:beineri/opt-qt563-xenial && apt-get update && apt-get install -y qt56qbs && apt-get clean cache

#Because of a bug in qbs (packaging?)
RUN mkdir -p /opt/qt56/lib/x86_64-linux-gnu/ && ln -s /opt/qt56/lib/qbs/ /opt/qt56/lib/x86_64-linux-gnu/qbs

#For wine
RUN dpkg --add-architecture i386

#Build dependencies
RUN apt-get update \
  && apt-get -y install autoconf automake autopoint bash bison bzip2 flex gettext g++ g++-multilib gperf intltool  libffi-dev libgdk-pixbuf2.0-dev libtool-bin libltdl-dev libssl-dev libxml-parser-perl libc6-dev-i386 make openssl p7zip-full patch perl pkg-config python ruby scons sed unzip wget wine xz-utils \
  && rm -rf /var/lib/apt/lists/* && apt-get dist-upgrade -y

RUN wine hostname

#Compile mxe with our dependencies
WORKDIR /opt
RUN git clone https://github.com/mxe/mxe.git
WORKDIR /opt/mxe
RUN git checkout e65f683f8428a94a5f4ec6e1ad3927ab5262b2ec
COPY settings.mk /opt/mxe/settings.mk
RUN make -j$(nproc) protobuf boost qt5

WORKDIR /tmp
RUN wget "https://github.com/google/protobuf/releases/download/v3.5.1/protoc-3.5.1-linux-x86_64.zip" \
    && unzip protoc-3.5.1-linux-x86_64.zip -d proto3 \
    && mv proto3/bin/protoc /usr/bin/protoc \
    && rm -rf /usr/include/google/protobuf \
    && mv proto3/include/google/protobuf /usr/include/google/ \
    && chmod a+x /usr/bin/protoc

RUN ccache --set-config=path="/opt/mxe/usr/bin/:/usr/bin/:/usr/sbin:/usr/local/bin:/bin" \
    && cp ~/.ccache/ccache.conf /etc/ \
    && ln -s /usr/bin/ccache  /usr/lib/ccache/i686-w64-mingw32.shared.posix-g++ \
    && ln -s /usr/bin/ccache  /usr/lib/ccache/i686-w64-mingw32.shared.posix-gcc \
    && ln -s /usr/bin/ccache  /usr/lib/ccache/i686-w64-mingw32.shared.posix-windres

ENV PATH "$PATH:/opt/mxe/usr/bin/"
ADD mxedeployqt /usr/bin/mxedeployqt
ADD launch /launch
ADD innosetup5 /innosetup5
ENTRYPOINT ["/launch"]

Where launch is just a script to create the current user/group etc.. inside the container for fewer permission change headaches

#!/bin/bash

# Add local user
# Either use the LOCAL_USER_ID if passed in at runtime or fallback

DISPLAY=${LOCAL_DISPLAY:-:0}
USER=${LOCAL_USER:-quarks}
USER_ID=${LOCAL_USER_ID:-9001}
GROUP=${LOCAL_GROUP:-users}
GROUP_ID=${LOCAL_GROUP_ID:-100}
HOME=${LOCAL_HOME:-/home/$USER}
EXEC_ARGS=${@:-/bin/bash}

#Create the group if it doesn't exist
grep "^$GROUP:" /etc/group > /dev/null
if [ $? -ne 0 ]; then
    echo "Creating group: $GROUP ..."
    groupadd "$GROUP" -g $GROUP_ID
fi

#Create the user if it doesn't exist and add it to the group
grep "^$USER:" /etc/passwd > /dev/null
if [ $? -ne 0 ]; then
    echo "Creating $USER ..."
    useradd "$USER" -u $USER_ID -N
    usermod -g $GROUP $USER
    echo "$USER      ALL = NOPASSWD: ALL" >> /etc/sudoers
fi

#Make sure the home directory exists...
if [ ! -d "$HOME" ]; then
    mkdir -p "$HOME"
fi

#... and the user can write to it
chown -R "$USER":"$GROUP" $HOME

#Link clang-format to the home
ln -s /clang-format $HOME/.clang-format

export DISPLAY
export USER
export HOME
export PATH=$PATH:/opt/qt56/bin/
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/qt56/lib

source /opt/qt56/bin/qt56-env.sh

su $USER -c $EXEC_ARGS

Then to compile:

QBS_PROFILE="quarks-mxe-i686-w64-mingw32-shared"

source /opt/qt56/bin/qt56-env.sh || die "Unable to find Qt installation."
qbs setup-toolchains /usr/lib/ccache/$TARGET-g++ mxe-g++ || die "Unable to find mxe toolchain."
qbs setup-qt /opt/mxe/usr/bin/$TARGET-qmake-qt5 $QBS_PROFILE || die "Unable to find a valid qt package for $TARGET in the mxe installation"

cd $WORKSPACE
qbs --clean-install-root profile:$QBS_PROFILE

[Martchus]

@saidinesh5 Thanks for sharing. I'm aware of that mailing list.

Note that the symlink problem is actually self-introduced by one of my patches and not the fault of any particular build system. However, the patch is needed to prevent the package to conflict with mingw-w64-qt4. That the Qt developer are not considering such use-cases is the actual problem here.

And this is true for most of the patches. You could drop them, but some use-cases wouldn't be possible then. For instance the static configuration provided here as mingw-w64-qt5-base-static variant is not supported at all by upstream although it is quite desirable to link everything into a single binary under Windows.

If the support for such use-cases is missing, I suppose I'll have to patch it myself regardless which build system is used.

BTW, I'm wondering what's so bad about qmake. If Qt would consider my use-cases and support them I'm fine with qmake. CMake is my favorite build system and support for it is already good in general. So no complaints here, too.

[saidinesh5]

@Martchus Ahh I see. And Yep, Even I am surprised how Qt doesn't support static builds that well - eventhough that would definitely mean more business for them - given their licensing and all that.

And The big problem with qmake for them started out as having to parse the project files for the QtCreator integration. They kept hitting edge cases and bugs. That's why they were looking for a new build system and then built qbs - because they already have a parser for QML language. The Qt 6 build system issue is also about what build system to use to build Qt itself - instead of their current bootstrapping code.

Personally my stance on build systems is that the C/C++ community would be in a much better place if they dropped all the build systems tomorrow and standardize one as the defacto system. I may even welcome auto-hell-tools with openarms if it will end our build system wars once and for all. CMake seems pretty decent these days though. Especially with modern C++ libraries (Check out https://docs.hunter.sh/en/latest/ and https://conan.io/ if you haven't already - they are quite nice). IDE integration via. CMake server mode is quite nice too these days in most IDEs.

[Martchus]

that would definitely mean more business for them

Likely those use-cases are already covered by using MSVC where a full static build is supported AFAIK.

Let's see how they decide about the build system. Personally I'm even fine with continuing to use qmake and just fix the broken parts (eg. dealing with dependencies of static plugins). For my own projects I prefer to use CMake so changing to CMake would also be welcome from my side.

The possibility to re-use tools like qmake from regular qt5-base package in the cross packages is also discussed on the mailing list. That would be a plus, indeed. I'd also like to install the dynamic version and the static version inside the same prefix. My own patches already implement this for qmake and the exported CMake files. I hope after the build system switch this will work out-of-the-box.

[Martchus]

Just to give this thread a conclusion: http://blog.qt.io/blog/2018/10/29/deprecation-of-qbs

So I'm not going to put any more effort into QBS support.

[saidinesh5]

As I posted elsewhere, "One small build system dropped for C++, One Giant Leap for C++ community!"
Thanks for your work though.. albeit all i got to use it for was toy examples to convince people at work about certain things..