QRegularExpression is missing since 3.5.0

[tonka3000]

Hi,

I tried to use the newest tag 3.5.4. Everything worked but QRegularExpression is missing from PythonQt.QtCore. It was the only class which is missing, all other classes where available. I use Qt 5.11.x in one of my project, so I only tested with this version of Qt.
The last working tag was 3.4.2.

I was not able to track down the cause, but it seems that the restructuring (init for normal and builtin) from 3.5.x seems to be a reason because QRegularExpression is now in the builtin part. The generated_cpp has the calls for the registering, but not chance to import it at runtime.

[usiems]

Did you rebuild PythonQt itself, with the newly generated builtin classes? Because they are part of the PythonQt library itself.

[tonka3000]

Did you rebuild PythonQt itself, with the newly generated builtin classes? Because they are part of the PythonQt library itself.

Yes, I build it from scratch. The strange thing is that all other builtin classes are available, but not QRegularExpression.

[usiems]

I suspect that you built PythonQt with the old builtin code from the generated_cpp_511 folder, where QRegularExpressions didn't yet belong to the builtin classes (we didn't update that folder after all the generator changes). I remember vaguely that the build scripts automatically select this folder depending on the Python version. Try replacing the content of the generated_cpp_511 folder with the content from generated_cpp.

[tonka3000]

So basically I should regenerate the bindings the generator, right?

It tried to remove the generated_cpp_xxx folders before building but it was screaming for the prebuilt ones.

I basically already generate the new bindings thanks to my conan recipe.
Not quiet sure how familiar you be with conan, here a snippet of the code.

with tools.environment_append(env):
        self._run_qmake("PythonQt.pro")
        self._run_make()

        # generate qt bindings
        self.output.info("Generate Qt bindings")
        if self.settings.compiler == "Visual Studio":
            generator_path = os.path.join(self.qmake_config, self.generator_name)
        else:
            generator_path = "./%s" % self.generator_name

        with tools.chdir("generator"):
            self.run("%s qtscript_masterinclude.h build_all.txt " % (generator_path)) # build new bindings here
        
        # build qt bindings
        self.output.info("Build PythonQt extensions")
        with tools.chdir("extensions"):
            tools.replace_in_file("PythonQt_QtAll/PythonQt_QtAll.pro",
                "isEmpty",
                "PYTHONQTALL_CONFIG=$$(PYTHONQTALL_CONFIG)\nisEmpty"
            )
            tools.replace_in_file("PythonQt_QtAll/PythonQt_QtAll.pro",
                'CONFIG += $${PYTHONQTALL_CONFIG}',
                'CONFIG += %s' % " ".join(pythonqtall_config)
            )
            self._run_qmake("extensions.pro")
            self._run_make()

[mrbean-bremen]

The problem seems to be that you build PythonQt before the wrappers, which will use the checked in generated code, which does not match the newly generated code.
You should probably first build the bindings, set PYTHONQT_GENERATED_PATH to the respective path as it is used in src/src.pro (or patch that file using replace_in_file), and only then build PythonQt. EDIT: Actually, by default the generated files should be created in generated_cpp per default, which is looked up before checking the checked in files, so changing the order (e.g. move the PythonQt build after the bindings generation) should be sufficient.

We probably should also re-generate the checked-in bindings, though I'm not sure if anybody actually uses them consciously. Alternatively, we could just remove the generated bindings to make sure that such mismatch does not happen, and add some documentation for the build process. The latter would help anyway...

[tonka3000]

I try to reorder the build. I will give you feedback next week.

EDIT: Short feedback. I now build in the following order

• make generator.pro
• generate bindings
• make src.pro
• make PythonQt_QtAll.pro

I delete the generated_cpp_* first to make sure only the new generated ones are used. It generate the bindings with 3.4.2 which works, but I get some method duplicates (same method twice in the same .cpp) which lead to compile errors.

Not quite sure if this is fixed in the current version (3.5.4), but it is a starting point for next week.

c:\users\myusername\.conan\data\pythonqt\20240813\org\tt\build\8f7e6708bb31d319d0ecf485423cf9efbb12c754\pythonqt\generated_cpp\com_trolltech_qt_core_builtin\com_trolltech_qt_core_builtin0.h(1339): error C2535: 'QTextCodec *PythonQtWrapper_Qt::static_Qt_codecForHtml(const QByteArray &)': member function already defined or declared
c:\users\myusername\.conan\data\pythonqt\20240813\org\tt\build\8f7e6708bb31d319d0ecf485423cf9efbb12c754\pythonqt\generated_cpp\com_trolltech_qt_core_builtin\com_trolltech_qt_core_builtin0.h(1338): note: see declaration of 'PythonQtWrapper_Qt::static_Qt_codecForHtml'
c:\users\myusername\.conan\data\pythonqt\20240813\org\tt\build\8f7e6708bb31d319d0ecf485423cf9efbb12c754\pythonqt\generated_cpp\com_trolltech_qt_core_builtin\com_trolltech_qt_core_builtin0.h(1341): error C2572: 'PythonQtWrapper_Qt::static_Qt_convertFromPlainText': redefinition of default argument: parameter 1
c:\users\myusername\.conan\data\pythonqt\20240813\org\tt\build\8f7e6708bb31d319d0ecf485423cf9efbb12c754\pythonqt\generated_cpp\com_trolltech_qt_core_builtin\com_trolltech_qt_core_builtin0.h(1340): note: see declaration of 'PythonQtWrapper_Qt::static_Qt_convertFromPlainText'
c:\users\myusername\.conan\data\pythonqt\20240813\org\tt\build\8f7e6708bb31d319d0ecf485423cf9efbb12c754\pythonqt\generated_cpp\com_trolltech_qt_core_builtin\com_trolltech_qt_core_builtin0.h(1341): error C2535: 'QString PythonQtWrapper_Qt::static_Qt_convertFromPlainText(const QString &,Qt::WhiteSpaceMode)': member function already defined or declared
c:\users\myusername\.conan\data\pythonqt\20240813\org\tt\build\8f7e6708bb31d319d0ecf485423cf9efbb12c754\pythonqt\generated_cpp\com_trolltech_qt_core_builtin\com_trolltech_qt_core_builtin0.h(1340): note: see declaration of 'PythonQtWrapper_Qt::static_Qt_convertFromPlainText'
c:\users\myusername\.conan\data\pythonqt\20240813\org\tt\build\8f7e6708bb31d319d0ecf485423cf9efbb12c754\pythonqt\generated_cpp\com_trolltech_qt_core_builtin\com_trolltech_qt_core_builtin0.h(1340): note: see declaration of 'PythonQtWrapper_Qt::static_Qt_convertFromPlainText'
c:\users\myusername\.conan\data\pythonqt\20240813\org\tt\build\8f7e6708bb31d319d0ecf485423cf9efbb12c754\pythonqt\generated_cpp\com_trolltech_qt_core_builtin\com_trolltech_qt_core_builtin0.h(1343): error C2535: 'bool PythonQtWrapper_Qt::static_Qt_mightBeRichText(const QString &)': member function already 
defined or declared
c:\users\myusername\.conan\data\pythonqt\20240813\org\tt\build\8f7e6708bb31d319d0ecf485423cf9efbb12c754\pythonqt\generated_cpp\com_trolltech_qt_core_builtin\com_trolltech_qt_core_builtin0.h(1342): note: see declaration of 'PythonQtWrapper_Qt::static_Qt_mightBeRichText'
Generating Code...
NMAKE : fatal error U1077: '"C:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\VC\Tools\MSVC\14.16.27023\bin\HostX64\x64\cl.EXE"' : return code '0x2'
Stop.
NMAKE : fatal error U1077: '"C:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\VC\Tools\MSVC\14.16.27023\bin\HostX64\x64\nmake.exe"' : return code '0x2'

[mrbean-bremen]

Thank you for the investigation - right now I have no idea what causes this. Most of the CI builds actually do this (build generators first, PythonQt afterwards), and there are no problems seen there. We also build our bindings the same way (also using conan, albeit for Qt 6, and using cmake instead of qmake) without problems. Maybe you can found out more next week...

[he-hesce]

User of pre-generated wrappers for 5.15. Using the pre-generated wrappers removes one potential point of build issues for me. Would actually like to see addition of pre-generated wrappers for Qt 6.x for the eventual future of having to upgrade from 5.15 one day.

Edit: At least for those Qt 6.x versions shipping with Enterprise or LTS distros.

[mrbean-bremen]

@he-hesce - noted. Maybe we can just remove the older ones (e.g. 5.0, 5.3, 5.4 and probably 5.6), and update the rest. It is still always a snapshot of a specific version, and may lack any changes in the version that is actually used, so I think the preferred way should be the manual wrapper generation for the actually used Qt version.

[he-hesce]

@mrbean-bremen : I am using the 5.6 pre-gen wrappers on CentOS/RHEL 7 which ships with Qt 5.9.7. 5.6 wrappers are stable and working. I used them with Qt 5.12 and 5.15 until the 5.15 pre-gen wrappers became available for the latter. Please wait a bit longer before removing anything "legacy" for us who operate a decade or more behind current time. We ship source code as well, thus reducing build friction is a benefit and getting PythonQt to build (without generating wrappers) without user intervention is wrangling enough :-)

[mrbean-bremen]

@he-hesce That's why I wrote "probably 5.6". I still think that earlier versions can be removed, though.

[mrbean-bremen]

@tonka3000 - did you meanwhile find something more out?

[tonka3000]

@mrbean-bremen Didn't find time to check it in detail. Hopefully at the end of the week.

[tonka3000]

@mrbean-bremen I found the root-cause and also the fix.

My Qt version is built without clang and this causes the issue. The 3 functions which were defined twice where declared in 2 places (qnamespace.h and qtextdocument.h)
The issue is caused by the workaround in generator/qtscript_masterinclude.h and in the current version this is already fixed.
In short the missing clang caused that both definition were exposed to the generator which lead to twice definition.

3.4.2

// our compiler can't handle the templates for singleShot (int Qt 5.12), but we can circumvent this with
// Q_CLANG_QDOC for the moment:
#define Q_CLANG_QDOC // force it is a problem if this module was not built
#include <QtCore/QTimer>
#undef Q_CLANG_QDOC

3.5.4

In 3.5.4 it is now fixed that 5.11 is not affected. Not 100% if the current code works when I have >= 5.12 and no clang, but patch the #define Q_CLANG_QDOC out should be a easy fix in the conan recipe.

#if QT_VERSION_MAJOR == 5 && QT_VERSION_MINOR > 11
#   include <QtCore/QObject>  // included by QtCore/QTimer
#   define Q_CLANG_QDOC
#   include <QtCore/QTimer>
#   undef Q_CLANG_QDOC
#endif

It seems that I accidently hunting a bug from the past 😄. I used 3.4.2 because I had issue as mention in the beginning of the issue.

I now try to use the new generated instead of the pre-generated ones and check if the original issue will disappear.

I'll keep you posted.

[tonka3000]

@usiems @mrbean-bremen

I've finished upgrading to version 3.5.4, and it works perfectly now. The QRegularExpression issue is gone, which was the original reason for the issue and therefore I will close the issue.

Thanks for the help.

Readme Contribution?

I would like to contribute some information to the README.md file on how to build the repo safely, if you accept PRs. Would that be ok for you?