Bazel: @emsdk is broken with --incompatible_enable_cc_toolchain_resolution

[PiotrSikora]

This was already discussed in #766 (comment), but that PR was merged without a fix, and this isn't tracked anywhere else.

Platforms-based toolchain resoution (--incompatible_enable_cc_toolchain_resolution) is the future (bazelbuild/bazel#7260), and majority of Bazel rules/toolchains already support it, but @emsdk doesn't (build falls back to using the default CC toolchain), which breaks cross-compilation in any Bazel workspace that uses @emsdk.

cc @walkingeyerobot @celentes @bkotsopoulossc

[walkingeyerobot]

This is definitely on the higher end of my list of things to do. If no one beats me to it, I'll probably have time Q2.

[walkingeyerobot]

cc @trybka

[bkotsopoulossc]

Very excited for this! Would love to help test a branch once the time comes.

@PiotrSikora what I have been doing as a workaround, in .bazelrc, is to set:

build:wasm --crosstool_top=@emsdk//emscripten_toolchain:everything
build:wasm --incompatible_enable_cc_toolchain_resolution=false
build:wasm --host_crosstool_top=@bazel_tools//tools/cpp:toolchain

And then building with bazel build --config=wasm //your/target/...

[PiotrSikora]

@bkotsopoulossc thanks for the suggestion, but that doesn't really work, since we use output of wasm_cc_binary as an input data in cc_test as part of the same bazel test command, so --config would apply to all targets.

[bkotsopoulossc]

@PiotrSikora neat, could you share more details on what the cc_test is doing? Are you using gtest or similar frameworks? I wonder if you even need to depend on the wasm_cc_binary at all, or if you could just have cc_test depend on cc_library or similar?

[jesseschalken]

Do platform mappings work as a stop gap?

[trybka]

I think platform mappings as @jesseschalken mentions would work here.

As far as emsdk goes, I think in the interim we may want the transition to do both (set crosstool_top / cpu and platform), since incompatible_enable_cc_toolchain_resolution is not yet universal.

AIs (as I see them):
o. create a local platform definition using the constraints in https://github.com/bazelbuild/platforms
o. OPEN Question: Should we add a canonical constraint emscripten as an OS? (which would be distinct from say, wasi)
o. add new toolchain() rules for the emsdk cc_toolchain
o. add appropriate platform settings to transition(s)
o. add a register_toolchains wrapper that can be called from top-level WORKSPACE file
o. add some tests

[PiotrSikora]

@PiotrSikora neat, could you share more details on what the cc_test is doing? Are you using gtest or similar frameworks? I wonder if you even need to depend on the wasm_cc_binary at all, or if you could just have cc_test depend on cc_library or similar?

@bkotsopoulossc sorry, I've missed this earlier - we're using this in Envoy and Proxy-Wasm C++ SDK, where we're testing hostcalls exposed by the runtime to the Wasm modules, so using wasm_cc_binary() and executing those tests as .wasm modules is the whole reason for their existence.

Do platform mappings work as a stop gap?

I couldn't get it to work by itself in the past.

The issue I'm facing can be reproduced using this branch (migration to @emsdk is still pending review):

$ git clone -b emsdk_resolution https://github.com/PiotrSikora/envoy.git envoy-emsdk
$ cd envoy-emsdk

Normal build works fine (target compiled using emcc):

$ bazel build //test/extensions/common/wasm/test_data:test_cpp.wasm
[...]
Target //test/extensions/common/wasm/test_data:test_cpp.wasm up-to-date:
  bazel-bin/test/extensions/common/wasm/test_data/test_cpp.wasm

Using --incompatible_enable_cc_toolchain_resolution results in build failures because gcc is used instead:

$ bazel build --incompatible_enable_cc_toolchain_resolution //test/extensions/common/wasm/test_data:test_cpp.wasm
[...]
ERROR: .../envoy-emsdk/test/extensions/common/wasm/test_data/BUILD:68:21: Compiling test/extensions/common/wasm/test_data/test_cpp.cc failed: (Exit 1): gcc failed: error executing command

Same with --platforms_mapping (--cpu=wasm32 <=> --platforms=//bazel:wasm32):

$ cat platforms.map 
platforms:
  //bazel:wasm32
    --cpu=wasm32
flags:
  --cpu=wasm32
    //bazel:wasm32

$ bazel build --incompatible_enable_cc_toolchain_resolution --platform_mappings=./platforms.map //test/extensions/common/wasm/test_data:test_cpp.wasm
ERROR: .../envoy-emsdk/test/extensions/common/wasm/test_data/BUILD:68:21: Compiling test/extensions/common/wasm/test_data/test_cpp.cc failed: (Exit 1): gcc failed: error executing command

Patching @emsdk to set platforms=@envoy//bazel:wasm32 results in missing C++ toolchain:

ERROR: .../envoy-emsdk/test/extensions/common/wasm/test_data/BUILD:68:21: While resolving toolchains for target //test/extensions/common/wasm/test_data:proxy_wasm_test_cpp: No matching toolchains found for types @bazel_tools//tools/cpp:toolchain_type. Maybe --incompatible_use_cc_configure_from_rules_cc has been flipped and there is no default C++ toolchain added in the WORKSPACE file? See https://github.com/bazelbuild/bazel/issues/10134 for details and migration instructions.

So it looks that a few more things are needed (see @trybka's list above), and --platforms_mappings doesn't help by itself.