Example Cmake project fails to build

[cemevin]

Attaching the error log. I guess cmakerunner.exe is missing, or something along those lines. I built bxl using bxl.cmd -minimal. I also tried with -all but still the same error. Anything else I should do to make it work?
error.log

[marcelolynch]

Hello @cemevin! Your issue should be fixed after 26a9774. Building BuildXL with -minimal is perfectly fine. I also added some clarifications in the example's README and the sample config file.

And thanks for exploring BuildXL: we appreciate any feedback regarding the CMake frontend as we don't have many users of this feature at the moment.

[mikejack07]

Hi,

Followed the exact same steps as given in:

https://github.com/microsoft/BuildXL/tree/master/Examples/CMakeHelloWorld

Still facing an error:

C:\Users\mikej\Buildxltut>build.bat

C:\Users\mikej\Buildxltut>C:\Users\mikej\BuildXL\Out\Bin\Debug\win-x64/bxl /c:config.bc /disableProcessRetryOnResourceExhaustion+
Microsoft (R) Build Accelerator. Build: 0.1.0-devBuild, Version: [Developer Build]
Copyright (C) Microsoft Corporation. All rights reserved.

C:\Users\mikej\Buildxltut\config.bc(3,9): error DX11602: The CMake to Ninja generator tool run into an internal error. Details: Could not find cmake.exe in any of the specified search locations
[0:01] error DX11230: There was an issue with the trying to generate the build directory C:\Users\mikej\Buildxltut\Out\Example for module Project. Details were logged.
Build FAILED
    Log Directory: C:\Users\mikej\Buildxltut\Out\Logs

But cmake.exe path is set in the environment variable.

Even then, to cross check, I uncommented in config.bc to give the path of CMake in my machine as suggested in to doc:

cMakeSearchLocations: [dC:\Program Files\CMake\bin]

Still, error is there:

C:\Users\mikej\Buildxltut>build.bat

C:\Users\mikej\Buildxltut>C:\Users\SarkarSourad\BuildXL\Out\Bin\Debug\win-x64/bxl /c:config.bc /disableProcessRetryOnResourceExhaustion+
Microsoft (R) Build Accelerator. Build: 0.1.0-devBuild, Version: [Developer Build]
Copyright (C) Microsoft Corporation. All rights reserved.

C:\Users\mikej\Buildxltut\config.bc(3,9): error DX11602: The CMake to Ninja generator tool run into an internal error. Details: CMake Warning:
  Ignoring extra path from command line:

   "C:/Users/mikej/Buildxltut/Out/Example/Files"


CMake Warning:
  Ignoring extra path from command line:

   "Studio\2022\BuildTools\VC\Tools\MSVC\14.33.31629\bin\Hostx64\x64\cl.exe"


CMake Error at Out/Example/CMakeFiles/3.24.2/CMakeCXXCompiler.cmake:1 (set):
  Syntax error in cmake code at

    C:/Users/mikej/Buildxltut/Out/Example/CMakeFiles/3.24.2/CMakeCXXCompiler.cmake:1

  when parsing string

    C:\Program

  Invalid escape sequence \P
Call Stack (most recent call first):
  CMakeLists.txt:2 (project)


CMake Error at CMakeLists.txt:2 (project):
  No CMAKE_C_COMPILER could be found.

  Tell CMake where to find the compiler by setting either the environment
  variable "CC" or the CMake cache entry CMAKE_C_COMPILER to the full path to
  the compiler, or to the compiler name if it is in the PATH.


CMake Error at CMakeLists.txt:2 (project):
  The CMAKE_CXX_COMPILER:

    C:\Program

  is not a full path to an existing compiler tool.

  Tell CMake where to find the compiler by setting either the environment
  variable "CXX" or the CMake cache entry CMAKE_CXX_COMPILER to the full path
  to the compiler, or to the compiler name if it is in the PATH.
[0:04] error DX11230: There was an issue with the trying to generate the build directory C:\Users\mikej\Buildxltut\Out\Example for module Project. Details were logged.
Build FAILED
    Log Directory: C:\Users\mikej\Buildxltut\Out\Logs

How can I resolve the error?

[mikejack07]

I changed the config.bc file to provide absolute addresses even though the paths are mentioned correctly in the environment variables:

config({
    resolvers: [
        {
            kind: "CMake",
            projectRoot: d`.`,
            moduleName: "Project",
            buildDirectory: r`Example`,
            
            // Uncommenting this line makes the resolver search for the CMake executable in the specified directory 
            // rather than using the `PATH` environment variable:
             cMakeSearchLocations: [d`C:\Program Files\CMake\bin\cmake.exe`],
            
            // cacheEntries specify "-D Key=Value" arguments to add to the CMake invocation.
            // Note that these entries are persisted in the CMake cache (located in Out\Example\CMakeCache.txt after the build)
            // Using forward slashes for paths is recommended, as these are supported by CMake and don't need escaping (\ should be otherwise escaped as \\\\)
            // They can be unset (BuildXL will pass "-U Var" to CMake) specifying undefined as the value in this dictionary.
             cacheEntries: { 
                 CMAKE_MAKE_PROGRAM: "C:/Users/mikej/ninja.exe",
                 CMAKE_CXX_COMPILER: "C:/Program Files (x86)/Microsoft Visual Studio/2022/BuildTools/VC/Tools/MSVC/14.33.31629/bin/Hostx86/x86/cl.exe",
                 VAR_TO_UNSET: undefined,
             }
        }
    ],
    
    // Inbox SDK is not currently working with the CMake resolver
    disableInBoxSdkSourceResolver: true,
});

Here, the error is:

C:\Users\mikej\Buildxltut>build.bat

C:\Users\mikej\Buildxltut>C:\Users\mikej\BuildXL\Out\Bin\Debug\win-x64/bxl /c:config.bc /disableProcessRetryOnResourceExhaustion+
Microsoft (R) Build Accelerator. Build: 0.1.0-devBuild, Version: [Developer Build]
Copyright (C) Microsoft Corporation. All rights reserved.

C:\Users\mikej\Buildxltut\config.bc(3,9): error DX11602: The CMake to Ninja generator tool run into an internal error. Details: Could not find cmake.exe in any of the specified search locations
[0:09] error DX11230: There was an issue with the trying to generate the build directory C:\Users\mikej\Buildxltut\Out\Example for module Project. Details were logged.
Build FAILED
    Log Directory: C:\Users\mikej\Buildxltut\Out\Logs

Any suggestions?

[marcelolynch]

This message ("Could not find cmake.exe in any of the specified search locations") is about cmakeSearchLocations. These should be directories (note that d`...` are directories in DScript) and not the full path to cmake.exe, i.e. you should specify

cMakeSearchLocations: [d`C:\Program Files\CMake\bin`]

[marcelolynch]

Also, thanks for experimenting and reporting: I fixed an issue with using spaces in f9331ff and added the suggestion of using / for paths to avoid escaping issues based on your report. I see that you synced to these changes already.

Additionally, you can try using the Ninja resolver after generating the Ninja specs yourself using cmake -GNinja. I added an example where you can check what a config file with the Ninja resolver looks like. This method (using CMake to generate Ninja and then building with BuildXL with the Ninja resolver) is being used in production already and is a bit more mature than the CMake resolver (which essentially does this exact thing under the hood).

[mikejack07]

Implemented the changes. Now my config.bc file looks like:

config({
    resolvers: [
        {
            kind: "CMake",
            projectRoot: d`.`,
            moduleName: "Project",
            buildDirectory: r`Example`,
            
            // Uncommenting this line makes the resolver search for the CMake executable in the specified directory 
            // rather than using the `PATH` environment variable:
             cMakeSearchLocations: [d`C:\Program Files\CMake\bin`],
            
            // cacheEntries specify "-D Key=Value" arguments to add to the CMake invocation.
            // Note that these entries are persisted in the CMake cache (located in Out\Example\CMakeCache.txt after the build)
            // Using forward slashes for paths is recommended, as these are supported by CMake and don't need escaping (\ should be otherwise escaped as \\\\)
            // They can be unset (BuildXL will pass "-U Var" to CMake) specifying undefined as the value in this dictionary.
             cacheEntries: { 
                 CMAKE_MAKE_PROGRAM: "C:/Users/mikej/ninja.exe",
                 CMAKE_CXX_COMPILER: "C:/MinGW/bin/gcc.exe",
                 VAR_TO_UNSET: undefined,
             }
        }
    ],
    
    // Inbox SDK is not currently working with the CMake resolver
    disableInBoxSdkSourceResolver: true,

After running build.bat from the project directory, the error is:

C:\Users\mikej\Buildxltut>C:\Users\mikej\BuildXL\Out\Bin\Debug\win-x64/bxl /c:config.bc /disableProcessRetryOnResourceExhaustion+
Microsoft (R) Build Accelerator. Build: 0.1.0-devBuild, Version: [Developer Build]
Copyright (C) Microsoft Corporation. All rights reserved.

C:\Users\mikej\Buildxltut\config.bc(3,9): error DX11602: The CMake to Ninja generator tool run into an internal error. Details: CMake Error at CMakeLists.txt:2 (project):
  No CMAKE_C_COMPILER could be found.

  Tell CMake where to find the compiler by setting either the environment
  variable "CC" or the CMake cache entry CMAKE_C_COMPILER to the full path to
  the compiler, or to the compiler name if it is in the PATH.


CMake Error at C:/Program Files/CMake/share/cmake-3.24/Modules/CMakeTestCXXCompiler.cmake:62 (message):
  The C++ compiler

    "C:/MinGW/bin/gcc.exe"

  is not able to compile a simple test program.

  It fails with the following output:

    Change Dir: C:/Users/mikej/Buildxltut/Out/Example/CMakeFiles/CMakeTmp

    Run Build Command(s):C:/Users/mikej/ninja.exe cmTC_825fd && [1/2] Building CXX object CMakeFiles\cmTC_825fd.dir\testCXXCompiler.cxx.obj
    FAILED: CMakeFiles/cmTC_825fd.dir/testCXXCompiler.cxx.obj
    C:\MinGW\bin\gcc.exe    -o CMakeFiles\cmTC_825fd.dir\testCXXCompiler.cxx.obj -c C:\Users\mikej\Buildxltut\Out\Example\CMakeFiles\CMakeTmp\testCXXCompiler.cxx
    ninja: build stopped: subcommand failed.





  CMake will not be able to correctly generate this project.
Call Stack (most recent call first):
  CMakeLists.txt:2 (project)
[0:08] error DX11230: There was an issue with the trying to generate the build directory C:\Users\mikej\Buildxltut\Out\Example for module Project. Details were logged.
Build FAILED
    Log Directory: C:\Users\mikej\Buildxltut\Out\Logs

Now, I added a CMAKE_C_COMPILER variable:

C:\Users\mikej\Buildxltut>build.bat

C:\Users\mikej\Buildxltut>C:\Users\Sarkarmikej\BuildXL\Out\Bin\Debug\win-x64/bxl /c:config.bc /disableProcessRetryOnResourceExhaustion+
Microsoft (R) Build Accelerator. Build: 0.1.0-devBuild, Version: [Developer Build]
Copyright (C) Microsoft Corporation. All rights reserved.

C:\Users\mikej\Buildxltut\config.bc(3,9): error DX11602: The CMake to Ninja generator tool run into an internal error. Details: CMake Error at C:/Program Files/CMake/share/cmake-3.24/Modules/CMakeTestCCompiler.cmake:69 (message):
  The C compiler

    "C:/MinGW/bin/gcc.exe"

  is not able to compile a simple test program.

  It fails with the following output:

    Change Dir: C:/Users/mikej/Buildxltut/Out/Example/CMakeFiles/CMakeTmp

    Run Build Command(s):C:/Users/mikej/ninja.exe cmTC_71c7c && [1/2] Building C object CMakeFiles\cmTC_71c7c.dir\testCCompiler.c.obj
    FAILED: CMakeFiles/cmTC_71c7c.dir/testCCompiler.c.obj
    C:\MinGW\bin\gcc.exe    -o CMakeFiles\cmTC_71c7c.dir\testCCompiler.c.obj -c C:\Users\mikej\Buildxltut\Out\Example\CMakeFiles\CMakeTmp\testCCompiler.c
    ninja: build stopped: subcommand failed.





  CMake will not be able to correctly generate this project.
Call Stack (most recent call first):
  CMakeLists.txt:2 (project)
[0:10] error DX11230: There was an issue with the trying to generate the build directory C:\Users\mikej\Buildxltut\Out\Example for module Project. Details were logged.
Build FAILED
    Log Directory: C:\Users\mikej\Buildxltut\Out\Logs

How to resolve it?

Another question that I wanted to ask is where to get the documentation of steps of how to set up and connect the orchestrator and the workers so that I can do distributed builds using BuildXL?

[marcelolynch]

This error is thrown by CMake. Can you try to build the example with CMake to see if it works? Are you running this on an environment where CMake succeeds (e.g., the VS Developer Command Prompt)?

[marcelolynch]

I would also recommend you try the approach I mentioned above of generating the Ninja from CMake and then running BuildXL with the Ninja resolver.

[mikejack07]

I have tried building the example with just CMake only and it worked fine. My environment is completely CMake friendly as I have worked with CMake multiple times in the past to build various projects.

Could you please share the example that you have added? I found only this https://github.com/microsoft/BuildXL/blob/main/Examples/NinjaHelloWorld/config.dsc but this is purely for Ninja only.

[mikejack07]

Another question that I wanted to ask is where to get the documentation of steps of how to set up and connect the orchestrator and the workers so that I can do distributed builds using BuildXL?

Any help on this please?

I have this doc https://github.com/microsoft/BuildXL/blob/main/Documentation/Wiki/Distributed-Builds.md but this only explains the distributed builds but not on the steps on how to set up and connect to the machines and how to perform distributed build using BuildXL.

[marcelolynch]

Could you please share the example that you have added? I found only this https://github.com/microsoft/BuildXL/blob/main/Examples/NinjaHelloWorld/config.dsc but this is purely for Ninja only.

Yes, that is for Ninja only, but the point is that you can generate the Ninja project with CMake (i.e., cmake -GNinja), and then build that Ninja project with BuildXL. This is actually what our internal CMake users are doing: the CMake resolver itself isn't in active development at the moment.

On your other question, this thread is not the place to discuss it. I saw that you opened another issue for this: I will get back to you and share some information over there.

[mikejack07]

Sure. Thanks.

As being able to distribute is my top priority, I would be delighted if I could get an answer for my 2nd question so that I could then implement Ninja or CMake over distributed build to have a working solution. Thanks and regards.

Here is the link to my 2nd issue: #1310