Skip to content

Commit

Permalink
MSYS2 change MINGW64 to UCRT64 (#10939)
Browse files Browse the repository at this point in the history
* [WIP] Use UCRT64 instead of MINGW64

* [WIP] Use UCRT64 instead of MINGW64

* [WIP] OMedit can simulate models with UCRT64

* Use MSYSTEM_PREFIX for MSYS environment

  - Updating submodules

* Runtime dependencies for OMEdit

  - Install QT5 plugins
  - Install runtime dependencies (mainly qt) using CMake install

* [WIP] Build OMEdit with UCRT64 and MINGW64 CMake

  I hope...

* Fixing const char*

* UCRT64 compiling with Makefiles.omdev.mingw

* Fix Cmake check OMDEv on Linux

* Compile OMSimulator with gcc only (on Windows)

* revert MetaModelicaDev examples

* Use System_openModelicaPlatform in OMEdit for msys

* Fixing OMSI CMake command

* Changing linking of libbfd for OMEdit

* Improving cpp build, but it still failes

* Fix om_curl for clang

* Fixing cpp CMake bug

* [WIP] remove TLM from build

* Revert all metamodelica changes

* Updating Readme for MINGW64 and UCRT64

* Adding missing environment variable

* Revert change: Use C:\OMDev\tools\msys

* Try finding gdb from OMEdit

* Fixes for MINGW64

* Fixing MINGW64 C++ Makefile build

* update the Windows MinGW build job to use UCRT64

* limit the msys2 execution on w3 computer that has Msys2/UTRC64 installed

* quote cmd /c build*.bat as sh translates it to a c:\

* circumvent msys2 transforming /c to c:\

* $(where cmake) doesn't seem to work

* set OMDEVMSYS in OMCompiler Makefile.omdev.mingw

* add missing quote

* echo some info on make variables

* attempt to get rid of OMDEVMSYS

* add the files missed in the previous commit

* geez

* resolve conflict with mater

* use 996OMDEV windows path instead of C:\OMDev which is msys style

* remember the original OMDEV and use it for MSVC

* attempt to fix saving of original OMDEV

* Revert "attempt to fix saving of original OMDEV"

This reverts commit 80c7f54.

* OMDEV_ESCAPED is not an environment variable.

  - The variable is set locally in the CMake file. Attempt to evaluate it
    as an environment variable will just return an empty value.

* Tell boost we do not want to use its CMake config files.

  - If MSVC, we do not want to use MSYS/MingGW installed boost's CMake
    config files for locating Boost We have a manually provided boost already
    specified using BOOST_ROOT.

  - This is essentially saying we do not want any help from boost itself
    in finding its libraries and setup. Let CMake do it alone the old way
    by just finding the files it knows.

  - The reason this might be happening now is that we have a more recent
    boost installed in UCRT which knows and tries to help CMake in finding
    it. For MSVC we do not want that help because we do not want to use
    the installed boost.

* IF MSVC use the Lapack from OMDev. Otherwise use from MinGW.

  - MSVC version of Lapack, just like Boost, is shipped by OMDev along
    separately (not from the MSYS/MinGW packages). Use that for MSVC.

* Do not set BOOST_PATH_MSVC without checking VS version.

  - I am not sure how this all works together. It is not the easiest thing
    to follow. Too many duplications and redundancies.

    This might not fix the latest issue where we try to use a path where
    the value of $(OMDEV) variable has not been subsituted yet. It is probably
    because it was set as $$(OMDEV). In any case first try to cleanup things
    and see where it actually picks the value to use for BOOST_PATH_MSVC.

* Run the getMSVCversion to set the BOOST_PATH_MSVC.

  - Make runtimeCPPmsvcinstall_old depend on getMSVCversion. We need the
    BOOST_PATH_MSVC variable set. Hopefully this works(?)

* Fix handling of Boost and apack for OMSICpp too.

* Define NPY_OS_MINGW for OMSens on Windows/MinGW.

  - There is a small bug in numpy's f2yc which does not detect MinGW
    environments as it is supposed to.

    See: numpy/numpy#24761

  - This Makefile is used for MinGW only. So we can just define the
    variable without any other checks being needed.

* Test the CMake based  build on ucrt labeled machines.

  - For this PR (until it is merged and everything has settled down) the
    one machine that has ucrt support is labled `omdev-ucrt`. Use only that
    machine to test the CMake built too.

* ucrt's `libbfd` depends on libsframe and `libzstd`.

  - ucrt version of `libbfd` depends on `libsframe` and `libzstd` in addition
    to `libiberity`. Add those libraries to the linking.

* Updating OMOptim, README

* - change the MinGW to UCRT in Jenkinsfile and add new labels on github so we can handle them differently
- fix OMSens issues and update OMOptim
- get rid of OMDEV_MSYS as is not really needed and is good to keep it simple

* update OMOptim sumbodule and remove OMDEV_MSYS everywhere

* adapt to new ucrt labels and parameters

* do not copy unnecessary files

* remove msvc runtimes targets from all-runtimes

* properly name stuff

* Fixing CMake builds, adding OMSens

  - Adding CMakeLists to OMSens
  - Workaround in OMEdit CMake to run install multiple times
  - Removing TODOs

* update OMSens

* Adding more runtime dependencies

* put back the OMPython stuff

* partially adapt the python infrastructure

* fix missing tab

* add the missing dlls for OMEdit and omc

* forgot to remove conflict markup

* updáted OMSimulator

* some minor updates

* OMEdit fixes for msys2-ucrt64
- fix the crash reporting to properly activate the handler
- add a "Crash Test" button to About dialog for easy crash testing

---------

Co-authored-by: Adrian Pop <adrian.pop@liu.se>
Co-authored-by: Mahder Gebremedhin <mahder.geb@gmail.com>
  • Loading branch information
3 people committed Feb 1, 2024
1 parent 45f47c3 commit 826c2e0
Show file tree
Hide file tree
Showing 68 changed files with 1,144 additions and 908 deletions.
26 changes: 13 additions & 13 deletions .CI/common.groovy
Expand Up @@ -91,7 +91,7 @@ void partest(partition=1,partitionmodulo=1,cache=true, extraArgs='') {
echo fi
) > runTestWindows.sh
set MSYSTEM=MINGW64
set MSYSTEM=UCRT64
set MSYS2_PATH_TYPE=inherit
%OMDEV%\\tools\\msys\\usr\\bin\\sh --login -i -c "cd `cygpath '${WORKSPACE}'` && chmod +x runTestWindows.sh && ./runTestWindows.sh && rm -f ./runTestWindows.sh"
""")
Expand Down Expand Up @@ -240,7 +240,7 @@ void buildOMC(CC, CXX, extraFlags, Boolean buildCpp, Boolean clean) {
echo rm -rf ./path\\ with\\ space/
) > buildOMCWindows.sh
set MSYSTEM=MINGW64
set MSYSTEM=UCRT64
set MSYS2_PATH_TYPE=inherit
%OMDEV%\\tools\\msys\\usr\\bin\\sh --login -i -c "cd `cygpath '${WORKSPACE}'` && chmod +x buildOMCWindows.sh && ./buildOMCWindows.sh && rm -f ./buildOMCWindows.sh"
""")
Expand Down Expand Up @@ -300,7 +300,7 @@ void buildOMSens() {
echo time make -f Makefile.omdev.mingw \${MAKETHREADS} omsens
) > buildOMSensWindows.sh
set MSYSTEM=MINGW64
set MSYSTEM=UCRT64
set MSYS2_PATH_TYPE=inherit
%OMDEV%\\tools\\msys\\usr\\bin\\sh --login -i -c "cd `cygpath '${WORKSPACE}'` && chmod +x buildOMSensWindows.sh && ./buildOMSensWindows.sh && rm -f ./buildOMSensWindows.sh"
""")
Expand All @@ -325,7 +325,7 @@ void buildOMC_CMake(cmake_args, cmake_exe='cmake') {
echo time ${cmake_exe} --build ./build_cmake --parallel ${numPhysicalCPU()} --target install
) > buildOMCWindows.sh
set MSYSTEM=MINGW64
set MSYSTEM=UCRT64
set MSYS2_PATH_TYPE=inherit
%OMDEV%\\tools\\msys\\usr\\bin\\sh --login -i -c "cd `cygpath '${WORKSPACE}'` && chmod +x buildOMCWindows.sh && ./buildOMCWindows.sh && rm -f ./buildOMCWindows.sh"
""")
Expand Down Expand Up @@ -357,7 +357,7 @@ void buildGUI(stash, isQt5) {
echo ./build/bin/OMEdit --help
) > buildGUIWindows.sh
set MSYSTEM=MINGW64
set MSYSTEM=UCRT64
set MSYS2_PATH_TYPE=inherit
%OMDEV%\\tools\\msys\\usr\\bin\\sh --login -i -c "cd `cygpath '${WORKSPACE}'` && chmod +x buildGUIWindows.sh && ./buildGUIWindows.sh && rm -f ./buildGUIWindows.sh"
""")
Expand Down Expand Up @@ -402,7 +402,7 @@ void buildAndRunOMEditTestsuite(stash) {
echo ./RunOMEditTestsuite.sh
) > buildOMEditTestsuiteWindows.sh
set MSYSTEM=MINGW64
set MSYSTEM=UCRT64
set MSYS2_PATH_TYPE=inherit
%OMDEV%\\tools\\msys\\usr\\bin\\sh --login -i -c "cd `cygpath '${WORKSPACE}'` && chmod +x buildOMEditTestsuiteWindows.sh && ./buildOMEditTestsuiteWindows.sh && rm -f ./buildOMEditTestsuiteWindows.sh"
""")
Expand Down Expand Up @@ -445,7 +445,7 @@ void generateTemplates() {

def getVersion() {
if (isWindows()) {
return (bat (script: 'set OMDEV=C:\\OMDev && set MSYSTEM=MINGW64 && set MSYS2_PATH_TYPE=inherit && %OMDEV%\\tools\\msys\\usr\\bin\\sh --login -i -c "build/bin/omc --version | grep -o \"v[0-9]\\+[.][0-9]\\+[.][0-9]\\+[^ ]*\""', returnStdout: true)).replaceAll("\\s","")
return (bat (script: 'set OMDEV=C:\\OMDev && set MSYSTEM=UCRT64 && set MSYS2_PATH_TYPE=inherit && %OMDEV%\\tools\\msys\\usr\\bin\\sh --login -i -c "build/bin/omc --version | grep -o \"v[0-9]\\+[.][0-9]\\+[.][0-9]\\+[^ ]*\""', returnStdout: true)).replaceAll("\\s","")
} else {
return (sh (script: 'build/bin/omc --version | grep -o "v[0-9]\\+[.][0-9]\\+[.][0-9]\\+[^ ]*"', returnStdout: true)).replaceAll("\\s","")
}
Expand Down Expand Up @@ -492,13 +492,13 @@ def makeCommand() {
return env.GMAKE ?: "make"
}

def shouldWeBuildMINGW() {
def shouldWeBuildUCRT() {
if (isPR()) {
if (pullRequest.labels.contains("CI/Build MINGW")) {
if (pullRequest.labels.contains("CI/Build MSYS2-UCRT64")) {
return true
}
}
return params.BUILD_MINGW
return params.BUILD_MSYS2_UCRT64
}

def shouldWeDisableAllCMakeBuilds() {
Expand All @@ -510,13 +510,13 @@ def shouldWeDisableAllCMakeBuilds() {
return params.DISABLE_ALL_CMAKE_BUILDS
}

def shouldWeEnableMinGWCMakeBuild() {
def shouldWeEnableUCRTCMakeBuild() {
if (isPR()) {
if (pullRequest.labels.contains("CI/CMake/Enable/MinGW")) {
if (pullRequest.labels.contains("CI/CMake/Enable/MSYS2-UCRT64")) {
return true
}
}
return params.ENABLE_MINGW_CMAKE_BUILD
return params.ENABLE_MSYS2_UCRT64_CMAKE_BUILD
}

def shouldWeEnableMacOSCMakeBuild() {
Expand Down
@@ -1,25 +1,25 @@
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<launchConfiguration type="org.eclipse.ui.externaltools.ProgramBuilderLaunchConfigurationType">
<stringAttribute key="org.eclipse.debug.core.ATTR_REFRESH_SCOPE" value="${workspace}"/>
<booleanAttribute key="org.eclipse.debug.core.appendEnvironmentVariables" value="false"/>
<mapAttribute key="org.eclipse.debug.core.environmentVariables">
<mapEntry key="CLASSPATH" value="${env_var:OMDEV}/bin/antlr/antlr.jar"/>
<mapEntry key="COMSPEC" value="${env_var:COMSPEC}"/>
<mapEntry key="MSYSTEM" value="MINGW32"/>
<mapEntry key="OMDEV" value="/c/OMDev/"/>
<mapEntry key="OPENMODELICAHOME" value="${project_loc}/build"/>
<mapEntry key="OPENMODELICALIBRARY" value="${project_loc}\build\lib\omlibrary\"/>
<mapEntry key="PATH" value="${env_var:OMDEV}/tools/msys/mingw32/bin;${env_var:OMDEV}/tools/msys/usr/bin;c:/windows;c:/windows/system32;"/>
<mapEntry key="SystemDrive" value="${env_var:SystemDrive}"/>
<mapEntry key="SystemRoot" value="${env_var:SystemRoot}"/>
<mapEntry key="TEMP" value="${env_var:TEMP}"/>
<mapEntry key="TMP" value="${env_var:TMP}"/>
</mapAttribute>
<booleanAttribute key="org.eclipse.debug.ui.ATTR_LAUNCH_IN_BACKGROUND" value="false"/>
<booleanAttribute key="org.eclipse.ui.externaltools.ATTR_BUILDER_ENABLED" value="true"/>
<stringAttribute key="org.eclipse.ui.externaltools.ATTR_LOCATION" value="${env_var:OMDEV}\tools\msys\usr\bin\make.exe"/>
<stringAttribute key="org.eclipse.ui.externaltools.ATTR_RUN_BUILD_KINDS" value="full,incremental,"/>
<stringAttribute key="org.eclipse.ui.externaltools.ATTR_TOOL_ARGUMENTS" value="-f Makefile.omdev.mingw ${string_prompt}"/>
<booleanAttribute key="org.eclipse.ui.externaltools.ATTR_TRIGGERS_CONFIGURED" value="true"/>
<stringAttribute key="org.eclipse.ui.externaltools.ATTR_WORKING_DIRECTORY" value="${project_loc}"/>
</launchConfiguration>
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<launchConfiguration type="org.eclipse.ui.externaltools.ProgramBuilderLaunchConfigurationType">
<stringAttribute key="org.eclipse.debug.core.ATTR_REFRESH_SCOPE" value="${workspace}"/>
<booleanAttribute key="org.eclipse.debug.core.appendEnvironmentVariables" value="false"/>
<mapAttribute key="org.eclipse.debug.core.environmentVariables">
<mapEntry key="CLASSPATH" value="${env_var:OMDEV}/bin/antlr/antlr.jar"/>
<mapEntry key="COMSPEC" value="${env_var:COMSPEC}"/>
<mapEntry key="MSYSTEM" value="MINGW32"/>
<mapEntry key="OMDEV" value="/c/OMDev/"/>
<mapEntry key="OPENMODELICAHOME" value="${project_loc}/build"/>
<mapEntry key="OPENMODELICALIBRARY" value="${project_loc}\build\lib\omlibrary\"/>
<mapEntry key="PATH" value="${env_var:OMDEV}/tools/msys/mingw32/bin;${env_var:OMDEV}/tools/msys/usr/bin;c:/windows;c:/windows/system32;"/>
<mapEntry key="SystemDrive" value="${env_var:SystemDrive}"/>
<mapEntry key="SystemRoot" value="${env_var:SystemRoot}"/>
<mapEntry key="TEMP" value="${env_var:TEMP}"/>
<mapEntry key="TMP" value="${env_var:TMP}"/>
</mapAttribute>
<booleanAttribute key="org.eclipse.debug.ui.ATTR_LAUNCH_IN_BACKGROUND" value="false"/>
<booleanAttribute key="org.eclipse.ui.externaltools.ATTR_BUILDER_ENABLED" value="true"/>
<stringAttribute key="org.eclipse.ui.externaltools.ATTR_LOCATION" value="${env_var:OMDEV}\tools\msys\usr\bin\make.exe"/>
<stringAttribute key="org.eclipse.ui.externaltools.ATTR_RUN_BUILD_KINDS" value="full,incremental,"/>
<stringAttribute key="org.eclipse.ui.externaltools.ATTR_TOOL_ARGUMENTS" value="-f Makefile.omdev.mingw ${string_prompt}"/>
<booleanAttribute key="org.eclipse.ui.externaltools.ATTR_TRIGGERS_CONFIGURED" value="true"/>
<stringAttribute key="org.eclipse.ui.externaltools.ATTR_WORKING_DIRECTORY" value="${project_loc}"/>
</launchConfiguration>
Expand Up @@ -5,11 +5,11 @@
<mapAttribute key="org.eclipse.debug.core.environmentVariables">
<mapEntry key="CLASSPATH" value="${env_var:OMDEV}/bin/antlr/antlr.jar"/>
<mapEntry key="COMSPEC" value="${env_var:COMSPEC}"/>
<mapEntry key="MSYSTEM" value="MINGW64"/>
<mapEntry key="MSYSTEM" value="UCRT64"/>
<mapEntry key="OMDEV" value="/c/OMDev/"/>
<mapEntry key="OPENMODELICAHOME" value="${project_loc}/build"/>
<mapEntry key="OPENMODELICALIBRARY" value="${project_loc}\build\lib\omlibrary\"/>
<mapEntry key="PATH" value="${env_var:OMDEV}/tools/msys/mingw64/bin;${env_var:OMDEV}/tools/msys/usr/bin;c:/windows;c:/windows/system32;"/>
<mapEntry key="PATH" value="${env_var:OMDEV}/tools/msys/ucrt64/bin;${env_var:OMDEV}/tools/msys/usr/bin;c:/windows;c:/windows/system32;"/>
<mapEntry key="SystemDrive" value="${env_var:SystemDrive}"/>
<mapEntry key="SystemRoot" value="${env_var:SystemRoot}"/>
<mapEntry key="TEMP" value="${env_var:TEMP}"/>
Expand Down
1 change: 1 addition & 0 deletions CMakeLists.txt
Expand Up @@ -170,6 +170,7 @@ if(OM_ENABLE_GUI_CLIENTS)
omc_add_subdirectory(OMSens_Qt)
endif()

omc_add_subdirectory(OMSens)
omc_add_subdirectory(libraries)
omc_add_subdirectory(testsuite)

Expand Down
28 changes: 14 additions & 14 deletions Jenkinsfile
@@ -1,8 +1,8 @@
def common
def shouldWeBuildMINGW
def shouldWeBuildUCRT
def shouldWeDisableAllCMakeBuilds_value
def shouldWeEnableMacOSCMakeBuild_value
def shouldWeEnableMinGWCMakeBuild_value
def shouldWeEnableUCRTCMakeBuild_value
def shouldWeRunTests
def isPR
pipeline {
Expand All @@ -15,10 +15,10 @@ pipeline {
LC_ALL = 'C.UTF-8'
}
parameters {
booleanParam(name: 'BUILD_MINGW', defaultValue: false, description: 'Build with Win/MinGW')
booleanParam(name: 'BUILD_MSYS2_UCRT64', defaultValue: false, description: 'Build with Win/MSYS2-UCRT64')
booleanParam(name: 'DISABLE_ALL_CMAKE_BUILDS', defaultValue: false, description: 'Skip building omc with CMake (CMake 3.17.2) on all platforms')
booleanParam(name: 'ENABLE_MINGW_CMAKE_BUILD', defaultValue: false, description: 'Enable building omc with CMake on MinGW')
booleanParam(name: 'ENABLE_MACOS_CMAKE_BUILD', defaultValue: false, description: 'Enable building omc with CMake on macOS')
booleanParam(name: 'ENABLE_MSYS2_UCRT64_CMAKE_BUILD', defaultValue: false, description: 'Enable building omc with CMake on MSYS2-UCRT64')
booleanParam(name: 'ENABLE_MACOS_CMAKE_BUILD', defaultValue: false, description: 'Enable building omc with CMake on MacOS')
}
// stages are ordered according to execution time; highest time first
// nodes are selected based on a priority (in Jenkins config)
Expand All @@ -37,14 +37,14 @@ pipeline {
common = load("${env.workspace}/.CI/common.groovy")
isPR = common.isPR()
print "isPR: ${isPR}"
shouldWeBuildMINGW = common.shouldWeBuildMINGW()
print "shouldWeBuildMINGW: ${shouldWeBuildMINGW}"
shouldWeBuildUCRT = common.shouldWeBuildUCRT()
print "shouldWeBuildUCRT: ${shouldWeBuildUCRT}"
shouldWeDisableAllCMakeBuilds_value = common.shouldWeDisableAllCMakeBuilds()
print "shouldWeDisableAllCMakeBuilds: ${shouldWeDisableAllCMakeBuilds_value}"
shouldWeEnableMacOSCMakeBuild_value = common.shouldWeEnableMacOSCMakeBuild()
print "shouldWeEnableMacOSCMakeBuild: ${shouldWeEnableMacOSCMakeBuild_value}"
shouldWeEnableMinGWCMakeBuild_value = common.shouldWeEnableMinGWCMakeBuild()
print "shouldWeEnableMinGWCMakeBuild: ${shouldWeEnableMinGWCMakeBuild_value}"
shouldWeEnableUCRTCMakeBuild_value = common.shouldWeEnableUCRTCMakeBuild()
print "shouldWeEnableUCRTCMakeBuild: ${shouldWeEnableUCRTCMakeBuild_value}"
shouldWeRunTests = common.shouldWeRunTests()
print "shouldWeRunTests: ${shouldWeRunTests}"
}
Expand Down Expand Up @@ -90,15 +90,15 @@ pipeline {
stash name: 'omc-clang', includes: 'build/**, **/config.status'
}
}
stage('Win/MinGW') {
stage('Win/UCRT64') {
agent {
node {
label 'windows'
label 'omdev-ucrt64'
}
}
when {
beforeAgent true
expression { shouldWeBuildMINGW }
expression { shouldWeBuildUCRT }
}
environment {
RUNTESTDB = '/c/dev/'
Expand Down Expand Up @@ -176,12 +176,12 @@ pipeline {
stage('cmake-OMDev-gcc') {
agent {
node {
label 'windows'
label 'omdev-ucrt64'
}
}
when {
beforeAgent true
expression { !shouldWeDisableAllCMakeBuilds_value && shouldWeEnableMinGWCMakeBuild_value}
expression { !shouldWeDisableAllCMakeBuilds_value && shouldWeEnableUCRTCMakeBuild_value}
}
steps {
script {
Expand Down

0 comments on commit 826c2e0

Please sign in to comment.