Skip to content

[prototype] Windows: build mrmeshpy Python bindings with MSVC instead of clang#6272

Closed
Fedr wants to merge 7 commits into
masterfrom
windows-bindings-msvc
Closed

[prototype] Windows: build mrmeshpy Python bindings with MSVC instead of clang#6272
Fedr wants to merge 7 commits into
masterfrom
windows-bindings-msvc

Conversation

@Fedr

@Fedr Fedr commented Jun 16, 2026

Copy link
Copy Markdown
Contributor

Prototype (draft / WIP)

Explores building the mrbind-generated Python bindings on Windows with MSVC instead of clang: parse the headers with libclang (mrbind), but compile + link mrmeshpy with MSVC. This mirrors how the generated C bindings (MeshLibC2) are already parsed by clang and compiled by MSVC, and the fact that MSVC already builds MeshLib's other pybind modules (mrviewerpy, mrmeshnumpy) and the pybind shim libs.

This is iteration 1 — opened as a draft to get the first CI signal. The mrbind-generated pybind code has never been compiled by MSVC, so expect to iterate.

What's here

  • scripts/mrbind/assemble_pybind_msvc.py — takes the libclang-parsed sources and lays out a self-contained tree an external compiler can build:
    • source/mrmeshpy/generated/ — the big generated .cpp (+ helpers, aliases), not compiled directly;
    • source/mrmeshpy/src/ — one per-fragment wrapper TU per MB_FRAGMENT (reproducing generate.mk's fragment split), plus mrbind_config.h carrying the -D defines generate.mk would otherwise pass on the clang command line (MRBIND_HEADER, MB_PB11_*, MR_COMPILING_*, …). Py_LIMITED_API and the patched-pybind config still come from common.props.
  • source/mrmeshpy/mrmeshpy.vcxproj (MSBuild) and source/mrmeshpy/CMakeLists.txt (CMake) — compile those sources into mrmeshpy.pyd with MSVC. Mirrors MeshLibC2 (generated sources, no PCH) + mrmeshnumpy (.pyd, pybind11nonlimitedapi_stubs, python_module.props staging into meshlib/).
  • build-test-windows.yml — on the MSBuild lane, after the existing clang bindings step, lay out the sources and rebuild mrmeshpy with MSVC over the clang-built .pyd, so the existing "Verify import" + Python test steps exercise the MSVC build. CMake lanes keep the clang build; other modules (mrcudapy) stay on clang.
  • .gitignore — ignores the generated source/mrmeshpy/{src,generated} (the .vcxproj/CMakeLists.txt are committed, like MeshLibC2).

Scope / known gaps

  • Only the vcxproj/MSBuild lane is wired into CI here. The CMakeLists.txt is provided (you asked for both build systems) but not yet hooked into the CMake-lane flow — its sources are generated after the main configure, so it needs a separate configure/build step. That's the next iteration.
  • Only mrmeshpy is converted; mrcudapy stays on clang.
  • Fragment count is fixed at 32 (not RAM-tuned like generate.mk).
  • Whether MSVC accepts the mrbind-generated pybind code at all is exactly what this CI run tests.

Only Windows is affected; other platforms are disabled for this run.

… of clang

Prototype for "parse with clang, build the Python bindings with MSVC". The headers are
still parsed by libclang (mrbind), but mrmeshpy is compiled and linked with MSVC, mirroring
how the generated C bindings (MeshLibC2) are parsed by clang yet compiled by MSVC.

- scripts/mrbind/assemble_pybind_msvc.py: lays out the libclang-parsed sources into a tree an
  external compiler can build -- the big generated .cpp in generated/, plus per-fragment wrapper
  TUs in src/ (reproducing generate.mk MB_FRAGMENT splitting) and a mrbind_config.h carrying the
  -D defines generate.mk would otherwise pass on the clang command line.
- source/mrmeshpy/mrmeshpy.vcxproj + CMakeLists.txt: compile those sources with MSVC into
  mrmeshpy.pyd, mirroring MeshLibC2 (generated sources, no PCH) + mrmeshnumpy (.pyd, pybind shim,
  meshlib/ staging). The vcxproj path is exercised by CI here; the CMakeLists is the CMake-lane
  draft (its sources are generated after the main configure, so it is built separately).
- build-test-windows.yml: on the MSBuild lane, after the existing clang bindings step, rebuild
  mrmeshpy with MSVC over the clang-built one so the existing import/Python test steps validate it.
  CMake lanes keep the clang build; other modules (mrcudapy) stay on clang.

Iteration 1, validated via CI -- the mrbind-generated pybind code has never been compiled by MSVC,
so it may reject clang-isms; the CMake-lane wiring is not hooked up yet; fragment count is fixed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Fedr and others added 3 commits June 16, 2026 15:07
core.h tests the mrbind flags with `#if` (e.g. `#if MB_PB11_ENABLE_CXX_STYLE_CONTAINER_METHODS`).
generate.mk passes them as `-DFOO`, which the compiler defines to 1, but assemble_pybind_msvc.py
emitted them value-less, so `#if FOO` became `#if` (empty) -> MSVC error C1017 (invalid integer
constant expression). Define the flag macros to 1, matching the `-DFOO` command-line semantics.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…0 fix

- Bump thirdparty/mrbind to the forwarding-lambda member-binding fix (mrbind#38),
  which resolves MSVC C2440 on overloaded auto-returning members of class templates
  (e.g. MR::Buffer::data), so the generated bindings compile with MSVC.
- CMakeLists.txt: add MESHLIB_BUILD_GENERATED_PYTHON_BINDINGS option +
  add_subdirectory(mrmeshpy), gated off during the initial configure (the sources are
  generated afterward, so mrmeshpy is configured/built in a follow-up step).
- build-test-windows.yml: on CMake lanes, after generating, configure source/mrmeshpy
  as a CMake target and build just mrmeshpy with MSVC, copying the .pyd over the
  clang-built one (mirrors the MSBuild-lane step). mrcudapy stays on clang.

Note: the submodule bump pulls mrbind master + the fix (the PR branch is off master).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… add_subdirectory

The previous commit committed CMakeLists.txt with CRLF, diverging from the repo's LF
and producing a whole-file diff. Restore LF so the change is just the
MESHLIB_BUILD_GENERATED_PYTHON_BINDINGS option and the gated add_subdirectory(mrmeshpy).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Fedr and others added 3 commits June 16, 2026 17:36
Picks up the mrbind fix for calling template member functions through their full name
(with template args), which regressed the Clang bindings build with the first #38 revision.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…gs as errors

The MSVC build of the generated bindings hit two issues:
- mrbind's macro_sequence_for.h #errors without the conformant preprocessor -> add /Zc:preprocessor
  (vcxproj AdditionalOptions + CMake target option). clang's preprocessor is conformant by default,
  so this only surfaces when compiling the bindings with MSVC.
- the CMake build's /Wall /WX turns warnings (e.g. C4464 "relative include path contains '..'") into
  errors; the generated bindings aren't meant to be warning-clean -> add /WX- on the CMake target
  (the vcxproj already sets TreatWarningAsError=false).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…lue-less

core.h tests it with `#if MB_DEFINE_IMPLEMENTATION`; an empty macro makes that `#if`
ill-formed (clang "expected value", MSVC C1017). The makefile passes the command-line
-DMB_DEFINE_IMPLEMENTATION (=1), so the clang bindings build never hit this; only the
MSVC wrapper did. Same fix as the other flag macros.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@Fedr

Fedr commented Jun 16, 2026

Copy link
Copy Markdown
Contributor Author

Closing as blocked on an upstream MSVC compiler bug.

This prototype demonstrated the "parse with clang, compile the Python bindings with MSVC" path — both the MSBuild mrmeshpy.vcxproj and the CMake target — and drove the MSVC build of mrmeshpy through several layers:

Remaining blocker: MSVC's incomplete P0522 relaxed-template-template-argument matching in partial specializations (https://developercommunity.visualstudio.com/t/Partial-specialization-with-a-single-arg/11108062), which breaks mrbind's RebindContainerTraits (helpers/rebind_container.h — already flagged there: "On MSVC they don't work at all!"). Working around it means rewriting those template-template specializations as handwritten per-container ones (incl. phmap) — a sizable, separate mrbind task.

To resume: land that mrbind container-rebind MSVC rewrite, then rebuild this mrmeshpy vcxproj/CMake prototype on top. The commits remain accessible from this PR.

@Fedr Fedr closed this Jun 16, 2026
@Fedr
Fedr deleted the windows-bindings-msvc branch June 16, 2026 18:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant