Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Qt[latest]] Update to 5.14.2 #10644

Merged
merged 160 commits into from
Apr 24, 2020

Conversation

Neumann-A
Copy link
Contributor

@Neumann-A Neumann-A commented Apr 1, 2020

update of #10009 to 5.14.2
closes #5828 (although you need quite a lot of disk space to build it)
closes #8408
closes #9002 (although higher version)
closes #9442
closes #10344
closes #10338
closes #10586
closes #10644
closes #1654 (probably was already closed earlier but the discussion that it is working is here)

@Neumann-A
Copy link
Contributor Author

Neumann-A commented Apr 13, 2020

failing after 7 h and 20min due to not having enough disk space
x64-windows regression at [20092/22470] :

fatal error C1085: Cannot write compiler generated file: 'C:\vsts\_work\4\s\buildtrees\qt5-webengine\x64-windows-rel\src\core\release\obj\third_party\blink\renderer\core\workers\workers\worker_thread.obj': No space left on device
ninja: build stopped: subcommand failed.

but the debug build was successful.

@davidebeatrici
Copy link
Contributor

I encountered two issues while configuring a CMake project.

Issue 1

CMake Error at C:/vcpkg/installed/x64-windows-static-md/share/cmake/Qt5Widgets/Qt5WidgetsConfig.cmake:15 (message):
  The imported target "Qt5::Widgets" references the file

     "C:/vcpkg/installed/x64-windows-static-md/plugins/styles/qwindowsvistastyled.lib"

  but this file does not exist.  Possible reasons include:

  * The file was deleted, renamed, or moved to another location.

  * An install or uninstall procedure did not complete successfully.

  * The installation package was faulty and contained

     "C:/vcpkg/installed/x64-windows-static-md/share/cmake/Qt5Widgets/Qt5Widgets_QWindowsVistaStylePlugin.cmake"

  but not all the files it references.

Call Stack (most recent call first):
  C:/vcpkg/installed/x64-windows-static-md/share/cmake/Qt5Widgets/Qt5WidgetsConfig.cmake:172 (_qt5_Widgets_check_file_exists)
  C:/vcpkg/installed/x64-windows-static-md/share/cmake/Qt5Widgets/Qt5Widgets_QWindowsVistaStylePlugin.cmake:8 (_populate_Widgets_plugin_properties)
  C:/vcpkg/installed/x64-windows-static-md/share/cmake/Qt5Widgets/Qt5WidgetsConfig.cmake:180 (include)
  C:/vcpkg/scripts/buildsystems/vcpkg.cmake:286 (_find_package)
  C:/vcpkg/installed/x64-windows-static-md/share/cmake/Qt5Svg/Qt5SvgConfig.cmake:266 (find_package)
  C:/vcpkg/scripts/buildsystems/vcpkg.cmake:286 (_find_package)
  C:/vcpkg/installed/x64-windows-static-md/share/cmake/Qt5/Qt5Config.cmake:28 (find_package)
  C:/vcpkg/scripts/buildsystems/vcpkg.cmake:286 (_find_package)

Qt5Widgets_QWindowsVistaStylePlugin.cmake:

add_library(Qt5::QWindowsVistaStylePlugin MODULE IMPORTED)

if(EXISTS "${_qt5Widgets_install_prefix}/plugins/styles/qwindowsvistastyle.lib")
    _populate_Widgets_plugin_properties(QWindowsVistaStylePlugin RELEASE "styles/qwindowsvistastyle.lib")
endif()
if(EXISTS "${_qt5Widgets_install_prefix}/debug/plugins/styles/qwindowsvistastyled.lib")
    _populate_Widgets_plugin_properties(QWindowsVistaStylePlugin DEBUG "styles/qwindowsvistastyled.lib")
endif()

list(APPEND Qt5Widgets_PLUGINS Qt5::QWindowsVistaStylePlugin)

C:/vcpkg/installed/x64-windows-static-md/plugins/styles contains:

qwindowsvistastyle.lib
qwindowsvistastyle.prl

C:/vcpkg/installed/x64-windows-static-md/debug/plugins/styles contains:

qwindowsvistastyled.lib
qwindowsvistastyled.pdb
qwindowsvistastyled.prl

Workaround: comment out line 7 (8 in the actual file, there's an empty line at the beginning).

Issue 2

CMake Error at C:/vcpkg/scripts/buildsystems/vcpkg.cmake:286 (_find_package):
  Could not find a configuration file for package "Qt5Widgets" that is
  compatible with requested version "5.12.7".

  The following configuration files were considered but not accepted:

    C:/vcpkg/installed/x64-windows-static-md/share/cmake/Qt5Widgets/Qt5WidgetsConfig.cmake, version: 5.12.5

Call Stack (most recent call first):
  C:/vcpkg/installed/x64-windows-static-md/share/cmake/Qt5Svg/Qt5SvgConfig.cmake:266 (find_package)
  C:/vcpkg/scripts/buildsystems/vcpkg.cmake:286 (_find_package)
  C:/vcpkg/installed/x64-windows-static-md/share/cmake/Qt5/Qt5Config.cmake:28 (find_package)
  C:/vcpkg/scripts/buildsystems/vcpkg.cmake:286 (_find_package)

Workaround: replace 5.12.5 with 5.12.7 in Qt5WidgetsConfig.cmake and Qt5WidgetsConfigVersion.cmake.

@Neumann-A
Copy link
Contributor Author

@davidebeatrici: Did you somehow mix artifacts from 5.12.5 and 5.12.7?
The latter problem indicates that. Furthermore the macro _populate_Widgets_plugin_properties in Qt5WidgetsConfig.cmake is patched to correctly take care of the paths:

    macro(_populate_Widgets_plugin_properties Plugin Configuration PLUGIN_LOCATION)
        set_property(TARGET Qt5::${Plugin} APPEND PROPERTY IMPORTED_CONFIGURATIONS ${Configuration})

    if("${Configuration}" STREQUAL "DEBUG")
        set(imported_location "${_qt5Widgets_install_prefix}/debug/plugins/${PLUGIN_LOCATION}")
    else()
        set(imported_location "${_qt5Widgets_install_prefix}/plugins/${PLUGIN_LOCATION}")
    endif()

which somehow is not patched in your Qt5WidgetsConfig.cmake. (The patching is done via the Qt5BasicConfig.cmake.in)
so something smells very fishy with your install of qt5-base. How did you try to update or vcpkg checkout with this PR?

@davidebeatrici
Copy link
Contributor

You're probably right.

Since I already had qt5-base installed, I ran vcpkg upgrade --no-dry-run qt5-base after pulling this PR's changes.

Not all files were deleted, due to an error, however I expected vcpkg to replace them with the new ones.

That's apparently not the case.

@JackBoosY
Copy link
Contributor

JackBoosY commented Apr 15, 2020

@Neumann-A Please change the failed triplet in the baseline to fail and the successful triplet to skip.
In linux and osx, should we add a message to nodify user?

@Neumann-A
Copy link
Contributor Author

@JackBoosY: Tried to test qt5-webengine with a release only build but the last link step requires more than 32 GB of RAM and I ran out of it.

fatal error C1002: compiler is out of heap space in pass 2
LINK : fatal error LNK1257: code generation failed

@JackBoosY JackBoosY added needs-further-review info:reviewed Pull Request changes follow basic guidelines labels Apr 16, 2020
@Neumann-A
Copy link
Contributor Author

Starting package 34/34: qt5-webengine:x64-windows
Building package qt5-webengine[core]:x64-windows...
-- qt5-webengine requires a lot of free disk space (>300GB), ram (>32 GB) and time (>4h per configuration) to be successfully build.
-- As such qt5-webengine is not properly tested.
-- If qt5-webengine fails post build validation please open up an issue.
-- If it fails due to post validation the successfully installed files can be found in <vcpkgroot>/packages/qt5-webengine_x64-windows
-- and just need to be copied into <vcpkgroot>/installed/
-- Using cached K:/downloads/qtwebengine-everywhere-src-5.12.7.tar.xz
-- Using source at K:/buildtrees/qt5-webengine/src/5.12.7-0f8fbdda3a
-- Configuring x64-windows-rel
-- Configuring x64-windows-rel done
-- Configuring x64-windows-dbg
-- Configuring x64-windows-dbg done
-- Package build-x64-windows-dbg
-- Package build-x64-windows-rel
-- Package install-x64-windows-dbg
-- Package install-x64-windows-rel
-- Installing: K:/packages/qt5-webengine_x64-windows/share/qt5-webengine/copyright
-- Performing post-build validation
-- Performing post-build validation done
Building package qt5-webengine[core]:x64-windows... done
Installing package qt5-webengine[core]:x64-windows...
Installing package qt5-webengine[core]:x64-windows... done
Elapsed time for package qt5-webengine:x64-windows: 3.137 h

Total elapsed time: 4.11 h

The package qt5-webengine:x64-windows provides CMake targets:

    find_package(Qt5Designer CONFIG REQUIRED)
    target_link_libraries(main PRIVATE Qt5::QWebEngineViewPlugin)

    find_package(Qt5WebEngine CONFIG REQUIRED)
    target_link_libraries(main PRIVATE Qt5::WebEngine Qt5::WebEnginePrivate)

    find_package(Qt5WebEngineCore CONFIG REQUIRED)
    target_link_libraries(main PRIVATE Qt5::WebEngineCore Qt5::WebEngineCorePrivate)

    find_package(Qt5WebEngineWidgets CONFIG REQUIRED)
    target_link_libraries(main PRIVATE Qt5::WebEngineWidgets Qt5::WebEngineWidgetsPrivate)

Note this was on a 80 Core System( 4x Intel Xeon Gold 6148) with 768 GB of RAM. Max RAM usage was 150 GB in the middle of the build (a lot of parallel jobs running) while the final link step took around 100 GB RAM. Corrected for the base RAM usage of around 70GB the system shows in idle the final link step took roughly 30 GB of RAM which explains while it failed on my machine.
Installation size is just around 3GB but build size is massive with around 170 GB and I don't know if the build cleaned parts of it while building.

@Neumann-A
Copy link
Contributor Author

A big part of the installation size is in Qt5WebEngineCored.pdb with around 1.6 GB.

@Neumann-A
Copy link
Contributor Author

If this gets further delayed it might get replaced by an update to 5.15 (https://wiki.qt.io/Qt_5.15_Release) which is only 3 weeks away

@JackBoosY
Copy link
Contributor

@ras0219-msft Please consider merge this PR first.

@ras0219-msft ras0219-msft merged commit bb7ccc4 into microsoft:master Apr 24, 2020
@Neumann-A Neumann-A deleted the update_qt_latest_5_14_2 branch April 24, 2020 06:50
traversaro added a commit to robotology/robotology-superbuild-dependencies-vcpkg that referenced this pull request Apr 25, 2020
In particular: 
* Use a recent commit of vcpkg, microsoft/vcpkg@28ab0b1 that contains microsoft/vcpkg#10644 
* Update qt5 to build with the `latest` feature, that install qt 5.14 that should be the first version of qt completely relocatable ( https://www.qt.io/blog/qt-is-relocatable )  
* Substitute opencv3 with opencv, that install opencv4 
* Install also `asio`, `boost-asio`, `boost-process`, `boost-dll`,  `boost-filesystem` and `boost-system`, to fix #7 
* As the newer version of vcpkg are affected by the bug microsoft/vcpkg#10119, clone the `robotology-vcpkg-binary-ports` repo in `C:/robotology`, so that it will be contained in the same archive  used to distribute the vcpkg dependencies .
traversaro added a commit to robotology/robotology-superbuild-dependencies-vcpkg that referenced this pull request Apr 29, 2020
* Update to recent vcpkg version

In particular: 
* Use a recent commit of vcpkg, microsoft/vcpkg@28ab0b1 that contains microsoft/vcpkg#10644 
* Update qt5 to build with the `latest` feature, that install qt 5.14 that should be the first version of qt completely relocatable ( https://www.qt.io/blog/qt-is-relocatable )  
* Substitute opencv3 with opencv, that install opencv4 
* Install also `asio`, `boost-asio`, `boost-process`, `boost-dll`,  `boost-filesystem` and `boost-system`, to fix #7 
* As the newer version of vcpkg are affected by the bug microsoft/vcpkg#10119, clone the `robotology-vcpkg-binary-ports` repo in `C:/robotology`, so that it will be contained in the same archive  used to distribute the vcpkg dependencies .
traversaro added a commit to traversaro/robotology-superbuild-dependencies-vcpkg that referenced this pull request May 3, 2020
In particular: 
* Use a recent commit of vcpkg, microsoft/vcpkg@28ab0b1 that contains microsoft/vcpkg#10644 
* Update qt5 to build with the `latest` feature, that install qt 5.14 that should be the first version of qt completely relocatable ( https://www.qt.io/blog/qt-is-relocatable )  
* Substitute opencv3 with opencv, that install opencv4 
* Install also `asio`, `boost-asio`, `boost-process`, `boost-dll`,  `boost-filesystem` and `boost-system`, to fix robotology#7 
* As the newer version of vcpkg are affected by the bug microsoft/vcpkg#10119, clone the `robotology-vcpkg-binary-ports` repo in `C:/robotology`, so that it will be contained in the same archive  used to distribute the vcpkg dependencies .
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment