[windows] [build-hook] Logging is incomplete/inaccurate when running with CBuilder

[poirierlouis]

I was recommended to create an issue here. Below is a copy of the original issue. I also created a repository to reproduce the issue, as a tl;dr.

Repository: https://github.com/poirierlouis/native_toolchain_c_logging_issue
All steps are present in README.

fyi, example in repository forget #endif instead of an #include error like described below. It makes the example simpler and end-up with the same kind of error about missing logs.

Information

• Dart 3.10.0 (stable) (Thu Nov 6 05:24:55 2025 -0800) on "windows_x64"
• on windows / "Windows 11 Famille" 10.0 (Build 26200)

I'm trying to make a custom package using Dart FFI related features.

Dependencies

• hooks: ^1.0.0
• code_assets: ^1.0.0
• native_toolchain_c: ^0.17.3

Description:
I've defined a custom wrapper to create a dynamic library on top of Intel XED. I'm only trying on Windows x64 for now. Outside of Dart/Build Hook workflow, I built a xed.lib.

I copy the static library (xed.lib) inside my hook runner. In the example below, I purposely forgot to give the path to include headers for xed (while my code does #include <xed/xed-interface.h>).

Actual behavior
I execute dart run and get logs in stderr.txt and in the console about cl.exe (from MSVC) failing with a ProcessException. But there is nothing related to cl.exe logs output that could help me understand what went wrong.

Expected behavior
It should correctly output the entire stdout/stderr of the process being run (cl.exe).

If I run the cl.exe command myself (generated by CBuilder), it will show an error about the header of xed not present (missing -I declaration for xed/xed-interface.h).

hook/build.dart

void main(List<String> args) async {
  await build(args, (input, output) async {
    final packageName = input.packageName;

    // ... copy `xed.lib` in `<output-dir>/vendors` ...

    final builder = CBuilder.library(
      name: packageName,
      assetName: "src/$packageName.dart",
      sources: ["src/main.cpp"],
      includes: [
        "src",
        // "src/vendors/xed/kits/latest/include", // absent on purpose
      ],
      linkModePreference: LinkModePreference.dynamic,
      libraries: ["xed"],
      libraryDirectories: ["vendors"],
    );

    await builder.run(input: input, output: output);
  });
}

Additional notes
The following may not be directly related (in case you wonder about building xed.lib outside of Dart Build Hook feature):
I first intended to directly clone Intel XED repositories (xed + mbuild), and build it as a shared library to import symbols directly (with build.dart).
Without Build Hook workflow, I can build the shared library without issues (MSVC + Python3).
With Build Hook, it failed when using Process.run to execute python mfile.py --shared (which is the build system used for Intel XED). Whether I was using runInShell, using python as the executable (absolute path), or running a shell (which would then execute the python command) it was always failing. Perhaps it works on other platforms (not Windows).

[dcharkes]

Hi @poirierlouis, thanks for the report!

I execute dart run and get logs in stderr.txt and in the console about cl.exe (from MSVC) failing with a ProcessException. But there is nothing related to cl.exe logs output that could help me understand what went wrong.

You can clone the repo locally, use a dependency override and experiment with editing

native/pkgs/native_toolchain_c/lib/src/cbuilder/run_cbuilder.dart

Line 349 in 224d93e

Future<void> runCl({required ToolInstance tool}) async {

As far as I can see we should output the stderr and stdout.

Without Build Hook workflow, I can build the shared library without issues (MSVC + Python3).
With Build Hook, it failed when using Process.run to execute python mfile.py --shared (which is the build system used for Intel XED). Whether I was using runInShell, using python as the executable (absolute path), or running a shell (which would then execute the python command) it was always failing. Perhaps it works on other platforms (not Windows).

I presume a Dart standalone script works?

It might be due to the Platform.environment being filtered. If a standalone Dart script works, you can try invoking that script with another Dart script that passes less environment variables and see which one is missing. If it's a reasonable one to be available in the build hook we'll add it.

native/pkgs/hooks_runner/lib/src/build_runner/build_runner.dart

Line 546 in 224d93e

const staticVariablesFilter = {

[poirierlouis]

You can clone the repo locally, use a dependency override and experiment with editing

native/pkgs/native_toolchain_c/lib/src/cbuilder/run_cbuilder.dart

Line 349 in 224d93e
Future runCl({required ToolInstance tool}) async {

As far as I can see we should output the stderr and stdout.

Alright, after a quick look it seems so. I'll give a try locally + debugger.

Without Build Hook workflow, I can build the shared library without issues (MSVC + Python3).
With Build Hook, it failed when using Process.run to execute python mfile.py --shared (which is the build system used for Intel XED). Whether I was using runInShell, using python as the executable (absolute path), or running a shell (which would then execute the python command) it was always failing. Perhaps it works on other platforms (not Windows).

I presume a Dart standalone script works?

Do you mean "as a replacement to python mbuild system"?
If so, I haven't tried to write a plain CBuilder to compile Intel XED instead of using their custom build system. They only support Python + mbuild. On a ticket, they said they would not support other build systems, and that it would require a lot of work to maintain others.
So I'd rather run Git and Python commands during build hook than try my own CBuilder and more than likely fail to reproduce the right setup.

[dcharkes]

I presume a Dart standalone script works?

Do you mean "as a replacement to python mbuild system"?

No, to simply invoke your python command from a Dart Process.run. As you mentioned that didn't work from the hook/build.dart.

So I'd rather run Git and Python commands during build hook than try my own CBuilder and more than likely fail to reproduce the right setup.

Agreed. Let's focus on making that work. The whole point of the hook/build.dart being a Dart script is that you can invoke arbitrary build systems! 😄

[poirierlouis]

Agreed. Let's focus on making that work. The whole point of the hook/build.dart being a Dart script is that you can invoke arbitrary build systems! 😄

That would be great indeed :)

(1) I tried by adding a direct stdout.writeln / stderr.writeln along:

native/pkgs/native_toolchain_c/lib/src/utils/run_process.dart

Line 51 in 224d93e

stdoutBuffer.write(decoded);

That is when captureOutput is true (which I set here):

native/pkgs/native_toolchain_c/lib/src/cbuilder/run_cbuilder.dart

Line 401 in 224d93e

captureOutput: false,

In such case, along with other output (like from where vswhere.exe), I can see the actual fatal error message from cl.exe.

(2) I also tried with this before throwing (and captureOutput = true):

    stdout.writeln(result.stdout);
    stderr.writeln(result.stderr);

native/pkgs/native_toolchain_c/lib/src/utils/run_process.dart

Line 84 in 224d93e

throw ProcessException(

In this last case, I can see the same output as before and the actual error output from cl.exe at the end.

[dcharkes]

Please provide a PR to fix the stderr and stdout to the native_toolchain_c package. 😄

But since you don't want to use the package but use the python build, did you make progress trouble shooting why Process.run('python', ...) in the build hook doesn't work?

[poirierlouis]

Please provide a PR to fix the stderr and stdout to the native_toolchain_c package. 😄

Will do, issue comes from createDefaultLogger which uses Level.INFO, but process.stdout uses logger?.fine(decoded). Changing level to .FINE fix the issue, though it includes more verbose output.

Not yet, I'll check this part after pushing a PR for the logging issue. Is it fine to talk about the Python issue in this ticket too?

[dcharkes]

Ah, so cl.exe logs errors on stdout rather than stderr? That rings a bell.

Maybe we should actually override the logger?.fine(decoded) for cl.exe and use logger?.info(decoded). (E.g. add a a parameter to runProcess Level stdoutLogLevel = Level.FINE, and set it to Level.INFO for in the runCl. (Otherwise everyone on Windows will run into the same issue.)

[poirierlouis]

Yes exactly.

If it's only Windows-related, is it necessary to add the option stdoutLogLevel? Or could we instead directly use:

if (Platform.isWindows) {
  logger?.info(decoded);
} else {
  logger?.fine(decoded);
}

[dcharkes]

I think it's cl.exe related. clang.exe reports error fine on stderr on Windows.

[poirierlouis]

I haven't tried yet, but regarding the Python issue, my guess is, I'm running a Process without the right environment like I've seen in tests:

native/pkgs/code_assets/lib/src/code_assets/c_compiler_config.dart

Line 77 in 224d93e

final DeveloperCommandPrompt? developerCommandPrompt;

I'll let you know if it fixes the issue.