Fix CMake generator on Unix/macOS by hughbe · Pull Request #5388 · dotnet/arcade

hughbe · 2020-05-01T10:58:41Z

Fixes bug in #4533

We set _CMakePassArchitectureToGenerator to False:

  <PropertyGroup Condition="'$(CMakeGenerator)' == ''">
    <CMakeGenerator Condition="$([MSBuild]::IsOSPlatform(Windows))">Visual Studio</CMakeGenerator>
    <CMakeGenerator Condition="!$([MSBuild]::IsOSPlatform(Windows))">Unix Makefiles</CMakeGenerator>
    <CMakeCompilerToolchain Condition="$([MSBuild]::IsOSPlatform(Windows))">MSVC</CMakeCompilerToolchain>
    <CMakeCompilerToolchain Condition="!$([MSBuild]::IsOSPlatform(Windows))">clang</CMakeCompilerToolchain>
    <_CMakeMultiConfigurationGenerator>false</_CMakeMultiConfigurationGenerator>
    <_CMakePassArchitectureToGenerator>false</_CMakePassArchitectureToGenerator>
  </PropertyGroup>

However, when we check this parameter, we compare it the empty string

      <_CMakeArguments Condition="'$(_CMakePassArchitectureToGenerator)' != ''" Include="-A $(Platform)" />

Therefore, we always pass the platform as a parameter, causing problems.

See #4533

/cc @jkoritzinsky

hughbe · 2020-05-01T11:13:10Z

@jkoritzinsky do you know why we didn't just write

'$(RepositoryEngineeringDir)common/native/find-native-compiler.sh' $(CMakeCompilerToolchain) $(CMakeCompilerMajorVersion) $(CMakeCompilerMinorVersion)

This works for me perfectly. Not sure why we need to create a relative path

jkoritzinsky · 2020-05-01T16:17:52Z

The relative path is to support the “bringup “ feature where you can have the SDK dump a script that does the CMake configure and build that you can use on a machine that doesn’t have the ability to run dotnet.

jkoritzinsky · 2020-05-01T16:22:27Z

    <PropertyGroup>
      <CMakeCompilerSearchScript>
-      . $([MSBuild]::MakeRelative('$(MSBuildProjectDirectory)', '$(RepositoryEngineeringDir)common/native/find-native-compiler.sh') $(CMakeCompilerToolchain) $(CMakeCompilerMajorVersion) $(CMakeCompilerMinorVersion)
+      $([MSBuild]::MakeRelative('$(MSBuildProjectDirectory)', '$(RepositoryEngineeringDir)common/native/find-native-compiler.sh')) $(CMakeCompilerToolchain) $(CMakeCompilerMajorVersion) $(CMakeCompilerMinorVersion)


We need to source the script to get the environment variables the script sets into the build/configure script.

What part of this was breaking?

Firstly, we were missing a parenthesis which caused the script not to work

Secondly, the period at the start caused the script to fail. Removing the period fixed the build

Note, I'm on Mac. If it helps:

hugh@Hughs-MacBook-Air ~ % sh --version GNU bash, version 3.2.57(1)-release (x86_64-apple-darwin19) Copyright (C) 2007 Free Software Foundation, Inc.

we definitely need to source the script, can you post the error you got?

@hughbe can you post the error you got when we source the script? I'd like to finish up this PR so we can use this in dotnet/runtime.

Here's the output

hugh@Hughs-MacBook-Air NativeTests % . ../../../../../eng/common/native/find-native-compiler.sh clang ../../../../../eng/common/native/find-native-compiler.sh:.:29: no such file or directory: /Users/hugh/Documents/GitHub/winforms/src/System.Windows.Forms/tests/InteropTests/NativeTests/../pipeline-logging-functions.sh

I still can't repro that locally. It looks like the mechanism we're using to find the current directory of the find-native-compiler.sh script isn't working when the script is sourced on Mac. It works on Ubuntu 18.04, so I don't know what the problem is.

@hughbe I think I know what's going on: when you're doing . ../../../../../eng/common/native/find-native-compiler.sh clang in your interactive shell you're using zsh if you're on macOS 10.15. That one doesn't populate the BASH_SOURCE[0] variable so the path resolving won't work.

The output from #5388 (comment) where you ran it inside of msbuild however suggests that it was able to do the right thing (I think because MSBuild runs commands with /bin/sh which still defaults to bash on my Mac) and you were just mislead by #5519.

Can you try again with the latest arcade which has a fix for that issue?

Yeah still getting that pipeline error on latest arcade (maybe winforms hasn't updated)

hugh@Hughs-MacBook-Air InteropTests % dotnet test /P:Platform=x64 Determining projects to restore... Determining projects to restore... All projects are up-to-date for restore. All projects are up-to-date for restore. Accessibility-version -> /Users/hugh/Documents/GitHub/winforms/artifacts/bin/Accessibility-version/x64/Debug/netcoreapp5.0/Accessibility-version.dll Accessibility -> /Users/hugh/Documents/GitHub/winforms/artifacts/bin/Accessibility/x64/Debug/netcoreapp5.0/Accessibility.dll System.Windows.Forms.Primitives -> /Users/hugh/Documents/GitHub/winforms/artifacts/bin/System.Windows.Forms.Primitives/x64/Debug/netcoreapp5.0/System.Windows.Forms.Primitives.dll System.Windows.Forms -> /Users/hugh/Documents/GitHub/winforms/artifacts/bin/System.Windows.Forms/x64/Debug/netcoreapp5.0/System.Windows.Forms.dll System.Windows.Forms.Interop.Tests -> /Users/hugh/Documents/GitHub/winforms/artifacts/bin/System.Windows.Forms.Interop.Tests/x64/Debug/netcoreapp5.0/System.Windows.Forms.Interop.Tests.dll /Users/hugh/Documents/GitHub/winforms/eng/common/native/../pipeline-logging-functions.sh: line 40: `Write-PipelineTelemetryError': not a valid identifier /Users/hugh/.nuget/packages/microsoft.dotnet.cmake.sdk/5.0.0-beta.20256.5/build/Microsoft.DotNet.CMake.Sdk.targets(172,5): error MSB3073: The command " [/Users/hugh/Documents/GitHub/winforms/src/System.Windows.Forms/tests/InteropTests/NativeTests/NativeTests.proj] /Users/hugh/.nuget/packages/microsoft.dotnet.cmake.sdk/5.0.0-beta.20256.5/build/Microsoft.DotNet.CMake.Sdk.targets(172,5): error MSB3073: [/Users/hugh/Documents/GitHub/winforms/src/System.Windows.Forms/tests/InteropTests/NativeTests/NativeTests.proj] /Users/hugh/.nuget/packages/microsoft.dotnet.cmake.sdk/5.0.0-beta.20256.5/build/Microsoft.DotNet.CMake.Sdk.targets(172,5): error MSB3073: . ../../../../../eng/common/native/find-native-compiler.sh clang [/Users/hugh/Documents/GitHub/winforms/src/System.Windows.Forms/tests/InteropTests/NativeTests/NativeTests.proj] /Users/hugh/.nuget/packages/microsoft.dotnet.cmake.sdk/5.0.0-beta.20256.5/build/Microsoft.DotNet.CMake.Sdk.targets(172,5): error MSB3073: [/Users/hugh/Documents/GitHub/winforms/src/System.Windows.Forms/tests/InteropTests/NativeTests/NativeTests.proj] /Users/hugh/.nuget/packages/microsoft.dotnet.cmake.sdk/5.0.0-beta.20256.5/build/Microsoft.DotNet.CMake.Sdk.targets(172,5): error MSB3073: [/Users/hugh/Documents/GitHub/winforms/src/System.Windows.Forms/tests/InteropTests/NativeTests/NativeTests.proj] /Users/hugh/.nuget/packages/microsoft.dotnet.cmake.sdk/5.0.0-beta.20256.5/build/Microsoft.DotNet.CMake.Sdk.targets(172,5): error MSB3073: cmake -G "Unix Makefiles" -A x64 -B "/Users/hugh/Documents/GitHub/winforms/artifacts/obj/NativeTests/x64/Debug/" -S "" -DCMAKE_INSTALL_PREFIX=/Users/hugh/Documents/GitHub/winforms/artifacts/bin/NativeTests/x64/Debug/ -DCMAKE_BUILD_TYPE=Debug [/Users/hugh/Documents/GitHub/winforms/src/System.Windows.Forms/tests/InteropTests/NativeTests/NativeTests.proj] /Users/hugh/.nuget/packages/microsoft.dotnet.cmake.sdk/5.0.0-beta.20256.5/build/Microsoft.DotNet.CMake.Sdk.targets(172,5): error MSB3073: " exited with code 2. [/Users/hugh/Documents/GitHub/winforms/src/System.Windows.Forms/tests/InteropTests/NativeTests/NativeTests.proj]

Renaming Write-PipelineTelemetryError->WritePipelineTelemetryError, Write-PipelineTaskError->WritePipelineTaskError, Write-PipelineSetVariable->WritePipelineSetVariable, Write-PipelinePrependPath->WritePipelinePrependPath, I finally get

sh-3.2$ dotnet build /p:Platform=x64 Microsoft (R) Build Engine version 16.7.0-preview-20258-02+26f6d1d87 for .NET Copyright (C) Microsoft Corporation. All rights reserved. Determining projects to restore... Determining projects to restore... All projects are up-to-date for restore. All projects are up-to-date for restore. Accessibility-version -> /Users/hugh/Documents/GitHub/winforms/artifacts/bin/Accessibility-version/x64/Debug/netcoreapp5.0/Accessibility-version.dll Accessibility -> /Users/hugh/Documents/GitHub/winforms/artifacts/bin/Accessibility/x64/Debug/netcoreapp5.0/Accessibility.dll System.Windows.Forms.Primitives -> /Users/hugh/Documents/GitHub/winforms/artifacts/bin/System.Windows.Forms.Primitives/x64/Debug/netcoreapp5.0/System.Windows.Forms.Primitives.dll System.Windows.Forms -> /Users/hugh/Documents/GitHub/winforms/artifacts/bin/System.Windows.Forms/x64/Debug/netcoreapp5.0/System.Windows.Forms.dll System.Windows.Forms.Interop.Tests -> /Users/hugh/Documents/GitHub/winforms/artifacts/bin/System.Windows.Forms.Interop.Tests/x64/Debug/netcoreapp5.0/System.Windows.Forms.Interop.Tests.dll CMake Error at CMakeLists.txt:2 (project): Generator Unix Makefiles does not support platform specification, but platform x64 was specified. EXEC : CMake error : CMAKE_C_COMPILER not set, after EnableLanguage [/Users/hugh/Documents/GitHub/winforms/src/System.Windows.Forms/tests/InteropTests/NativeTests/NativeTests.proj] EXEC : CMake error : CMAKE_CXX_COMPILER not set, after EnableLanguage [/Users/hugh/Documents/GitHub/winforms/src/System.Windows.Forms/tests/InteropTests/NativeTests/NativeTests.proj] -- Configuring incomplete, errors occurred! See also "/Users/hugh/Documents/GitHub/winforms/artifacts/obj/NativeTests/x64/Debug/CMakeFiles/CMakeOutput.log". /Users/hugh/.nuget/packages/microsoft.dotnet.cmake.sdk/5.0.0-beta.20256.5/build/Microsoft.DotNet.CMake.Sdk.targets(172,5): error MSB3073: The command " [/Users/hugh/Documents/GitHub/winforms/src/System.Windows.Forms/tests/InteropTests/NativeTests/NativeTests.proj] /Users/hugh/.nuget/packages/microsoft.dotnet.cmake.sdk/5.0.0-beta.20256.5/build/Microsoft.DotNet.CMake.Sdk.targets(172,5): error MSB3073: [/Users/hugh/Documents/GitHub/winforms/src/System.Windows.Forms/tests/InteropTests/NativeTests/NativeTests.proj] /Users/hugh/.nuget/packages/microsoft.dotnet.cmake.sdk/5.0.0-beta.20256.5/build/Microsoft.DotNet.CMake.Sdk.targets(172,5): error MSB3073: . ../../../../../eng/common/native/find-native-compiler.sh clang [/Users/hugh/Documents/GitHub/winforms/src/System.Windows.Forms/tests/InteropTests/NativeTests/NativeTests.proj] /Users/hugh/.nuget/packages/microsoft.dotnet.cmake.sdk/5.0.0-beta.20256.5/build/Microsoft.DotNet.CMake.Sdk.targets(172,5): error MSB3073: [/Users/hugh/Documents/GitHub/winforms/src/System.Windows.Forms/tests/InteropTests/NativeTests/NativeTests.proj] /Users/hugh/.nuget/packages/microsoft.dotnet.cmake.sdk/5.0.0-beta.20256.5/build/Microsoft.DotNet.CMake.Sdk.targets(172,5): error MSB3073: [/Users/hugh/Documents/GitHub/winforms/src/System.Windows.Forms/tests/InteropTests/NativeTests/NativeTests.proj] /Users/hugh/.nuget/packages/microsoft.dotnet.cmake.sdk/5.0.0-beta.20256.5/build/Microsoft.DotNet.CMake.Sdk.targets(172,5): error MSB3073: cmake -G "Unix Makefiles" -A x64 -B "/Users/hugh/Documents/GitHub/winforms/artifacts/obj/NativeTests/x64/Debug/" -S "" -DCMAKE_INSTALL_PREFIX=/Users/hugh/Documents/GitHub/winforms/artifacts/bin/NativeTests/x64/Debug/ -DCMAKE_BUILD_TYPE=Debug [/Users/hugh/Documents/GitHub/winforms/src/System.Windows.Forms/tests/InteropTests/NativeTests/NativeTests.proj] /Users/hugh/.nuget/packages/microsoft.dotnet.cmake.sdk/5.0.0-beta.20256.5/build/Microsoft.DotNet.CMake.Sdk.targets(172,5): error MSB3073: " exited with code 1. [/Users/hugh/Documents/GitHub/winforms/src/System.Windows.Forms/tests/InteropTests/NativeTests/NativeTests.proj] Build FAILED. EXEC : CMake error : CMAKE_C_COMPILER not set, after EnableLanguage [/Users/hugh/Documents/GitHub/winforms/src/System.Windows.Forms/tests/InteropTests/NativeTests/NativeTests.proj] EXEC : CMake error : CMAKE_CXX_COMPILER not set, after EnableLanguage [/Users/hugh/Documents/GitHub/winforms/src/System.Windows.Forms/tests/InteropTests/NativeTests/NativeTests.proj] /Users/hugh/.nuget/packages/microsoft.dotnet.cmake.sdk/5.0.0-beta.20256.5/build/Microsoft.DotNet.CMake.Sdk.targets(172,5): error MSB3073: The command " [/Users/hugh/Documents/GitHub/winforms/src/System.Windows.Forms/tests/InteropTests/NativeTests/NativeTests.proj] /Users/hugh/.nuget/packages/microsoft.dotnet.cmake.sdk/5.0.0-beta.20256.5/build/Microsoft.DotNet.CMake.Sdk.targets(172,5): error MSB3073: [/Users/hugh/Documents/GitHub/winforms/src/System.Windows.Forms/tests/InteropTests/NativeTests/NativeTests.proj] /Users/hugh/.nuget/packages/microsoft.dotnet.cmake.sdk/5.0.0-beta.20256.5/build/Microsoft.DotNet.CMake.Sdk.targets(172,5): error MSB3073: . ../../../../../eng/common/native/find-native-compiler.sh clang [/Users/hugh/Documents/GitHub/winforms/src/System.Windows.Forms/tests/InteropTests/NativeTests/NativeTests.proj] /Users/hugh/.nuget/packages/microsoft.dotnet.cmake.sdk/5.0.0-beta.20256.5/build/Microsoft.DotNet.CMake.Sdk.targets(172,5): error MSB3073: [/Users/hugh/Documents/GitHub/winforms/src/System.Windows.Forms/tests/InteropTests/NativeTests/NativeTests.proj] /Users/hugh/.nuget/packages/microsoft.dotnet.cmake.sdk/5.0.0-beta.20256.5/build/Microsoft.DotNet.CMake.Sdk.targets(172,5): error MSB3073: [/Users/hugh/Documents/GitHub/winforms/src/System.Windows.Forms/tests/InteropTests/NativeTests/NativeTests.proj] /Users/hugh/.nuget/packages/microsoft.dotnet.cmake.sdk/5.0.0-beta.20256.5/build/Microsoft.DotNet.CMake.Sdk.targets(172,5): error MSB3073: cmake -G "Unix Makefiles" -A x64 -B "/Users/hugh/Documents/GitHub/winforms/artifacts/obj/NativeTests/x64/Debug/" -S "" -DCMAKE_INSTALL_PREFIX=/Users/hugh/Documents/GitHub/winforms/artifacts/bin/NativeTests/x64/Debug/ -DCMAKE_BUILD_TYPE=Debug [/Users/hugh/Documents/GitHub/winforms/src/System.Windows.Forms/tests/InteropTests/NativeTests/NativeTests.proj] /Users/hugh/.nuget/packages/microsoft.dotnet.cmake.sdk/5.0.0-beta.20256.5/build/Microsoft.DotNet.CMake.Sdk.targets(172,5): error MSB3073: " exited with code 1. [/Users/hugh/Documents/GitHub/winforms/src/System.Windows.Forms/tests/InteropTests/NativeTests/NativeTests.proj] 0 Warning(s) 3 Error(s) Time Elapsed 00:00:04.02

Then the change to _CMakePassArchitectureToGenerator in this PR makes it work.

Therefore, ignoring the weird telementary errors (maybe zsh doesn't support - in function names?) I can verify that these changes work

jkoritzinsky · 2020-06-25T05:51:39Z

@markwilkie can we get this merged in?

ghost · 2020-06-26T14:16:11Z

merged since it has an approval

hughbe mentioned this pull request May 1, 2020

Add Microsoft.DotNet.CMake.Sdk MSBuild SDK for integrating native builds into MSBuild. #4533

Merged

jkoritzinsky approved these changes May 1, 2020

View reviewed changes

jkoritzinsky reviewed May 1, 2020

View reviewed changes

hughbe changed the title ~~Fix CMake generator on Unix~~ Fix CMake generator on macOS May 22, 2020

hughbe changed the title ~~Fix CMake generator on macOS~~ Fix CMake generator on Unix/macOS May 22, 2020

hughbe added 3 commits May 23, 2020 18:13

Fix CMake generator on Unix

7753cd9

Fix CMakeCompilerSearchScript on Mac

c27562f

Update documentation a little

7dfb752

hughbe force-pushed the fix-cmake-unx branch from 0f29da4 to 7dfb752 Compare May 23, 2020 17:15

hughbe mentioned this pull request Jun 2, 2020

Test errors in Pipelines include Msbuild environment variable names dotnet/winforms#3091

Closed

jkoritzinsky approved these changes Jun 7, 2020

View reviewed changes

ghost merged commit 7a42d63 into dotnet:master Jun 26, 2020

hughbe deleted the fix-cmake-unx branch June 26, 2020 14:41

This pull request was closed.

Conversation

hughbe commented May 1, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

hughbe commented May 1, 2020

Uh oh!

jkoritzinsky commented May 1, 2020

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

hughbe Jun 7, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jkoritzinsky commented Jun 25, 2020

Uh oh!

ghost commented Jun 26, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

hughbe commented May 1, 2020 •

edited

Loading

hughbe Jun 7, 2020 •

edited

Loading