Skip to content

feat: add boost-ext.ut 2.3.1 C++23 module package - #142

Merged
Sunrisepeak merged 26 commits into
mcpplibs:mainfrom
ZheFeng7110:add-boost-ext-ut
Aug 3, 2026
Merged

feat: add boost-ext.ut 2.3.1 C++23 module package#142
Sunrisepeak merged 26 commits into
mcpplibs:mainfrom
ZheFeng7110:add-boost-ext-ut

Conversation

@ZheFeng7110

Copy link
Copy Markdown
Collaborator

[Boost::ext].UT is the C++20 single-header unit testing framework shipped by the boost-ext orgnot an official Boost library; hence the new boost-ext mcpp namespace (not compat, not boost). Exposed as the C++23 module boost.ut so users can import boost.ut; out of the box.

Shape

Form B inline descriptor at pkgs/b/boost-ext.ut.lua — a generated C++23 module wrapper, same shape as marzer.tomlplusplus and nlohmann.json.

Why generated_files, not verbatim include/boost/ut.cppm

The v2.3.1 tarball already ships include/boost/ut.cppm (export module boost.ut;), but it cannot be used VERBATIM on either non-MSVC default toolchain of CI's pinned mcpp 0.0.109:

  • GCC 16.1 (-std=c++23) rejects unqualified size_t at namespace scope in the module purview with -Wtemplate-body: import std only brings in std::size_t, but ut.hpp uses size_t unqualified in reporter_junit::print_junit_summary (lines 1916/1917/1936/1937).
  • Clang 22.1 on the MSVC ABI (x86_64-windows-msvc) fails on the MSVC builtins __argc/__argv referenced under #if defined(_MSC_VER) at ut.hpp:687 — clang sets _MSC_VER on this ABI but does not provide those builtins. The adjacent upstream guards at lines 291 / 311 / 1147 already gate clang out (&& !defined(__clang__)); line 687 missed it.

So the descriptor provides a generated_files wrapper that reproduces upstream's cppm verbatim plus two minimal shims — no upstream line is removed or replaced:

shim purpose guard
using std::size_t; at purview top level fix GCC template-body size_t error unconditional
#define __argc 0 / #define __argv nullptr fix Clang-on-Windows builtin gap _MSC_VER && __clang__ (MSVC itself never enters)

The base ut.hpp stays pinned to the reproducible v2.3.1 release tag — no fork in the trust path. include_dirs = { "*/include/boost" } resolves both the wrapper's #include "ut.hpp" and any consumer that wants #include <boost/ut.hpp> directly.

The explicit template instantiation block some readers may have seen on upstream master's ut.cppm (template class boost::ut::reporter_junit<...>; etc.) is post-v2.3.1 and is not in any release tag yet, so this descriptor does not carry it. Switch sources back to */include/boost/ut.cppm and drop generated_files once a >2.3.1 release ships that block.

Features

None. ut is header-only + a single module unit, with no extra compilable sources to gate; the optional BOOST_UT_CONFIG_* toggles are compile-time defines, not yet representable in the features table.

CN mirror

Intentionally omitted — no mcpp-res write access. The descriptor uses plain-string upstream URLs, which tests/check_mirror_urls.lua accepts as-is; CN users fall back to the GLOBAL source until a maintainer backfills the gitcode release.

Local verification (mcpp 0.0.109 = CI pin)

All run from clean state (rm -rf tests/examples/boost-ext.ut/{target,.mcpp,mcpp.lock,compile_commands.json}):

check result
`mcpp xpkg parse pkgs/b/boost-ext.ut.lua` ✅ `parse OK` (both CI pin 0.0.109 and local 2026.8.2.1)
`mcpp xpkg parse` over all `pkgs//.lua` ✅ no PARSE FAIL
`mcpp test -p boost-ext.ut` — gcc 16.1.0 / x86_64-windows-gnu (Windows default) ✅ `all tests passed (3 asserts in 2 tests)` / `test result ok. 1 passed; 0 failed`
`mcpp test -p boost-ext.ut` — llvm 22.1.8 / x86_64-windows-msvc ✅ `all tests passed (3 asserts in 2 tests)` / `test result ok. 1 passed; 0 failed`
`tests/check_mirror_urls.lua` (plain-string url path) ✅ pass
`tests/check_package_name.lua` (name = \"ut\" single atomic segment) ✅ pass
leading-v version lint (\"2.3.1\" bare) ✅ pass

Design notes recorded in .agents/docs/2026-08-02-add-boost-ext-ut-plan.md.

Files

  • `pkgs/b/boost-ext.ut.lua` (new) — Form B descriptor, directory `pkgs/b/` (full-name initial)
  • `tests/examples/boost-ext.ut/mcpp.toml` + `tests/ut.cpp` (new) — workspace member
  • `mcpp.toml` — register the new member
  • `README.md` / `README.zh-CN.md` — record in the C++23 module wrapper row
  • `.agents/docs/2026-08-02-add-boost-ext-ut-plan.md` — design doc

Files

  • pkgs/b/boost-ext.ut.lua (new) — Form B descriptor, directory pkgs/b/ (full-name initial)
  • tests/examples/boost-ext.ut/mcpp.toml + tests/ut.cpp (new) — workspace member
  • mcpp.toml — register the new member
  • README.md / README.zh-CN.md — record in the C++23 module wrapper row
  • .agents/docs/2026-08-02-add-boost-ext-ut-plan.md — design doc

[Boost::ext].UT is the C++20 single-header unit testing framework shipped
by the boost-ext org (NOT an official Boost library; hence the new
 mcpp namespace, not  and not ). Exposed as the
C++23 module  so users can write  directly.

The v2.3.1 tarball already ships include/boost/ut.cppm (), but it cannot be used VERBATIM on either non-MSVC default
toolchain of CI's pinned mcpp 0.0.109:

  * GCC 16.1 (-std=c++23) rejects unqualified  at namespace scope in
    the module purview with  ( only brings in
    ); ut.hpp uses  unqualified in reporter_junit.
  * Clang 22.1 on the MSVC ABI (x86_64-windows-msvc) fails on the MSVC
    builtins / referenced under  at
    ut.hpp:687 — clang sets  but does not provide those builtins
    (the adjacent upstream guards at lines 291/311/1147 already gate clang
    out, but line 687 missed it).

So, like marzer.tomlplusplus, a  wrapper reproduces
upstream's cppm VERBATIM plus two minimal shims: a  at
purview top level (fixes GCC) and  /  gated to  on  (fixes clang-on-Windows; MSVC
itself never enters the guard). base ut.hpp stays pinned to the v2.3.1 tag.

Verified locally with mcpp 0.0.109 (CI pin):
  - mcpp xpkg parse pkgs/b/boost-ext.ut.lua  -> parse OK
  - mcpp test -p boost-ext.ut (gcc 16.1.0, x86_64-windows-gnu default)
      -> 'all tests passed (3 asserts in 2 tests)' / 'test result ok. 1 passed'
  - mcpp test -p boost-ext.ut (llvm 22.1.8, x86_64-windows-msvc)
      -> 'all tests passed (3 asserts in 2 tests)' / 'test result ok. 1 passed'

No feature: ut is header-only + single module unit, no extra compilable
sources to gate; the optional  defines are compile-time,
not yet representable in the features table.

CN mirror intentionally omitted (no mcpp-res write access); the descriptor
uses plain-string upstream URLs which the lint (tests/check_mirror_urls.lua)
accepts as-is. Maintainer can backfill the gitcode release later.

Design notes in .agents/docs/2026-08-02-add-boost-ext-ut-plan.md.
…icit template instantiations

CI on PR mcpplibs#142 showed workspace (linux) and workspace (windows) green, but
workspace (macos) failed with exit 139 (SIGSEGV) immediately at `Running
bin/ut` — the test binary never reached the 'Suite ...' console output.
Root cause: the verbatim v2.3.1 ut.cppm leaves the member templates the
`cfg::runner<reporter_junit<printer>>` dispatches to
(`reporter_junit<>::on<...>`, `test::operator=<>`, `expect<bool>`)
only implicitly instantiable; on Apple-Clang 20.1.7 + `-fmodules` the
implicit path through the module BMI fails to emit them into the consuming
executable, so `cfg` static init calling
`reporter_junit::on(events::run_begin)` →
`std::unordered_map::operator[]("global")` lands in an
un-instantiated slot and segfaults.

Upstream fixed this on `master` AFTER v2.3.1 by appending eight
explicit template instantiations to ut.cppm; reproduce that block
byte-for-byte in our `generated_files` cppm after `#include "ut.hpp"`.
This is the same forward-port shape as marzer.tomlplusplus carrying a
one-line cut from upstream master — once a >2.3.1 release ships it,
switch `sources` to `*/include/boost/ut.cppm` and drop
`generated_files` (this block comes back verbatim).

`export import std;` is KEPT (with the existing `using std::size_t;`
GCC shim): trialed dropping it to mirror nlohmann.json / marzer.tomlplusplus
but GCC 16.1 then surfaces a cascade of `-Wtemplate-body` errors in
ut.hpp:3288 (`std::empty`), :3282 (`call_steps_` member lookup),
:3322 (`literals::operator""_test` using-decl resolution), and more —
GCC's two-phase lookup in the module purview is stricter than Clang's and
genuinely needs the std module to be import-visible.

Local verification, clean state (`rm -rf tests/examples/boost-ext.ut/{target,.mcpp,mcpp.lock,compile_commands.json}`):

  * mcpp xpkg parse pkgs/b/boost-ext.ut.lua  -> parse OK
  * mcpp test -p boost-ext.ut (gcc 16.1.0, x86_64-windows-gnu)
      -> 'all tests passed (3 asserts in 2 tests)' / 'test result ok. 1 passed'
  * mcpp test -p boost-ext.ut (llvm 22.1.8, x86_64-windows-msvc)
      -> 'all tests passed (3 asserts in 2 tests)' / 'test result ok. 1 passed'

Both Windows default toolchains stay green — dev 3 is a no-op redundancy
on GCC / Clang-on-MSVC and a real fix on Clang-on-Darwin; macOS CI result
will be confirmed by the next push to PR mcpplibs#142.

Design notes in .agents/docs/2026-08-02-add-boost-ext-ut-plan.md §2.3 / §2.4 / §6.
…tic init

The macOS leg (Apple Clang 20.1.7 + libc++) failed with exit 139 at
'Running bin/ut'. A temporary lldb diagnostic workflow pinned it:
    frame #0: reporter_junit::reporter_junit at ut.hpp:1620:31
    stop reason = EXC_BAD_ACCESS (code=1, address=0xffffffffffffffe8)
ut.hpp:1620 is the member-init `std::streambuf* cout_save = std::cout.rdbuf();`.

This is a static-initialization ORDER failure: ut.hpp's module-exported
inline variable `cfg = runner<reporter_junit<printer>>{}` dynamically
initializes before Apple libc++'s `std::cout`, so `cfg`'s member-init
reads an unconstructed std::cout (hence the -0x18 garbage read). MSVC STL
and libstdc++ order their stream init (init_priority / ios_base::Init) so
Windows and Linux legs pass; Apple libc++ does not, and the module
boundary breaks the same-TU ordering the header form relies on.

Fix: add a persistent `std::ios_base::Init` guard object in the module
purview BEFORE `#include "ut.hpp"`. Same-TU dynamic init runs in
declaration order, so the guard constructs std::cout/std::cin/std::cerr
first; when `cfg` (declared later inside ut.hpp) later calls
`std::cout.rdbuf()` the stream objects are fully built. The anonymous-
namespace guard lives for the whole program so stream refcount stays >= 1.

The explicit-template-instantiation block (dev 3, lifted verbatim from
upstream master) is upstream's fix for a separate Clang module linkage
gap and does NOT address this SIOF; it stays as a no-op redundancy.

Local verification (clean state, both Windows default toolchains):
  * mcpp xpkg parse pkgs/b/boost-ext.ut.lua           -> parse OK
  * mcpp test -p boost-ext.ut (llvm 20.1.7, x86_64-windows-msvc)
      -> 'all tests passed (3 asserts in 2 tests)' / 'test result ok. 1 passed'
  * mcpp test -p boost-ext.ut (gcc 16.1.0, x86_64-windows-gnu)
      -> 'all tests passed (3 asserts in 2 tests)' / 'test result ok. 1 passed'
macOS result to be confirmed by the diagnostic workflow on this push.
…R split

The macOS leg still crashed at ut.hpp:1620 (`std::cout.rdbuf()`) even with
the ios_base::Init guard. The lldb backtrace showed the real story:
    stop reason = EXC_BAD_ACCESS (code=1, address=0xffffffffffffffe8)
The address -0x18 means std::cout's vptr is ZERO — the stream was NEVER
constructed. Root cause: `export import std;` inside the module makes
`std::cout` refer to the MODULE's own std entities, not libc++'s — an
ODR split between the std module's std::cout and the library's std::cout.
Apple libc++ does not merge module std entities (libstdc++/MSVC STL do),
so libc++'s ios_base::Init constructs the library copy while the module's
copy stays all-zero; cfg's member-init then dereferences the null vptr.

Fix (matches every other successful module package in this index —
nlohmann.json, marzer.tomlplusplus, neargye.magic_enum): the module TU must
NOT `export import std;`. Pull stdlib via #include only so every std::*
symbol inside ut.hpp is the library's own ODR entity. Consumers `import
std;` themselves (the test member already does).

Removing `export import std;` alone used to explode on GCC with a cascade
of -Wtemplate-body errors, so two enabling changes:
  1. the module's GLOBAL-MODULE-FRAGMENT #includes every stdlib header
     ut.hpp needs (GMF declarations are visible to the purview);
  2. cxxflags = { "-Wno-template-body" } silences GCC 16.1's remaining
     two-phase-lookup pedantry inside ut.hpp's templates (accepted on
     gcc 16.1, ignored by Clang).

Kept from before: `using std::size_t;` (GCC) and the `__argc`/`__argv`
define (Clang-on-Windows), plus the post-v2.3.1 explicit-template-
instantiation block lifted verbatim from upstream master (harmless no-op;
does not fix the macOS ODR split).

Local verification (clean state, both Windows default toolchains):
  * mcpp xpkg parse pkgs/b/boost-ext.ut.lua -> parse OK
  * mcpp test -p boost-ext.ut (llvm 20.1.7, x86_64-windows-msvc)
      -> 'all tests passed (3 asserts in 2 tests)' / 'test result ok. 1 passed'
  * mcpp test -p boost-ext.ut (gcc 16.1.0, x86_64-windows-gnu)
      -> 'all tests passed (3 asserts in 2 tests)' / 'test result ok. 1 passed'
macOS result to be confirmed by the diagnostic workflow on this push.
speak-agent added a commit to mcpp-community/mcpp that referenced this pull request Aug 2, 2026
* fix(build): make the C++ runtime contract one decision (#336)

`[build] static_stdlib = false` was silently ignored for test binaries from
0.0.86 to 2026.8.2.2 while docs/05-mcpp-toml.md kept documenting the opt-out.
The cause is structural: "does this artifact carry its own C++ runtime" was
derived independently in five places — ldStdlibDefault, ldStdlibTest, the
-static-libstdc++ string, the MinGW -static branch, and the
LinkUnit::TestBinary two-way switch in the ninja emitter — and #202's new
semantics landed in some of them and not the others.

Replaced by a three-layer model in src/build/distribution.cppm:

  Role      intrinsic to the link unit (test binaries run here and are
            discarded; archives embed no runtime at all)
  Contract  what the artifact promises about the machine that runs it
  Mechanism (contract x stdlib x binary format) -> flags, as a TOTAL function

Totality is the property that matters: every cell answers, and a cell that
cannot honor what was asked returns `degraded` plus a diagnostic the backend
must print. That turns three silent downgrades into reported ones — including
Linux + clang/libc++, where `static_stdlib = true` emitted no flag at all and
shipped a toolchain-coupled artifact while the manifest, the docs and the
build output all called it self-contained. It now links libc++.a/libc++abi.a/
libunwind.a for real (NEEDED drops to libc/libm/loader).

Also fixes the crash that made the gap visible. On macOS a global object whose
constructor touches std::cout SIGSEGVs at process start under the default
contract: Mach-O runs __init_offsets in link order and has no priority-ordered
init section, so the stream initializer pulled out of libc++.a lands last, and
libc++'s <iostream> has no ios_base::Init guard of its own (libstdc++ and the
MSVC STL do, which is why only macOS breaks). Nor could package code work
around it — std::ios_base::Init is only forward-declared in libc++'s headers,
so the standard's own remedy is unavailable there. mcpp now links a generated
C object first whose constructor calls ios_base::Init::Init(); the reference is
weak, so a toolchain spelling that symbol differently links exactly as before.

New surface: [build] cxx_runtime = "self-contained" | "toolchain-coupled" |
"host-coupled", per role via { default, tests } and per triple via
[target.<triple>].cxx_runtime — beside `linkage`, which is the same axis.
static_stdlib stays a faithful alias.

Analysis: .agents/docs/2026-08-02-issue336-pr142-analysis.md
Unblocks: mcpplibs/mcpp-index#142

* fix(build): a platform limit nobody asked about is not a diagnostic

The MSVC runtime has no self-contained mechanism at all (no /MT emission),
so the default contract degraded on EVERY Windows build and printed a warning
nobody could act on. A diagnostic is for a broken promise — mcpp said the
artifact would be self-contained and it is not. Where mcpp never made the
promise, the cell now stays quiet unless the contract was written down
explicitly. Cells that DO promise something (a missing libc++.a under the
default) still report regardless.

* fix(build): the macOS ordering shim must not be able to break the link

Two Mach-O facts the first CI round found the hard way:

  * an __asm__ label is used VERBATIM — clang does not prepend Mach-O's
    global '_'. The C++ symbol _ZNSt3__18ios_base4InitC1Ev therefore has to
    be written __ZNSt3__18ios_base4InitC1Ev, and getting it wrong is not a
    silent no-op: every macOS link failed with
    'undefined symbol: ZNSt3__18ios_base4InitC1Ev'.

  * plain __attribute__((weak)) on a declaration is NOT Mach-O's
    weak-undefined form, so it did not make the bad reference optional.
    weak_import is.

Both are now correct, but neither is the safety net. The backend only
generates the shim TU when the libc++ archive actually defines the symbol —
checked by scanning the ranlib index in the archive's first member, no
subprocess — so an unexpected libc++ spelling disables the ordering aid and
says so, instead of failing the build. A check upstream of the reference
cannot break a link the way the reference itself can.

* docs: record what shipped and what the CI rounds corrected (#336)

---------

Co-authored-by: sunrisepeak <speakshen@163.com>
@speak-agent

speak-agent commented Aug 2, 2026

Copy link
Copy Markdown
Contributor

摘要

macOS 上的崩溃是 mcpp 的引擎缺陷,不是这个包的问题。 是你们的取证定位到了它 —— 已成为 mcpp-community/mcpp#336,并在 mcpp 2026.8.3.1 修复发布。

由此需要调整的:

  1. 两处根因写错了,需要改:设计文档 §2.3(模块 BMI 未带模板定义)与描述符顶部注释(export import std; 造成 std::cout 的 ODR 分裂)。后者尤其重要 —— 它是下一个人会读到的那份。dev 1 就是按这个不成立的理由做的,而它合入后 macOS 仍然是 exit 139。
  2. -Wno-template-body 需要删:Clang 不认识这个选项(你们自己的诊断日志里就有 unknown warning option),它给两条 Clang 平台每次编译加噪声,并会打断开 -Werror 的消费者。
  3. 实测:在 2026.8.3.1 下把描述符改成逐字使用上游 ut.cppm(无 generated_files、无 shim、无 cxxflags),gcc 16.1.0 与 llvm 22.1.8 两条 Linux 工具链全部通过。所以 §2.1 的 size_t 问题是 Windows 局部现象,不该无条件套给所有平台 —— 描述符本来就有 per-OS 段可以表达它(compat.ftxui 是现成先例)。
  4. 不必须等新 mcpp,有两条路:A 包侧自救(在 wrapper 的 #include "ut.hpp" 之前放构造,0.0.109 上就能用,一次 CI 即可验证);B 只把 validate.ymlMCPP_VERSION 抬到 2026.8.3.1。但请不要动 index.tomlmin_mcpp —— 它是语法契约,为一个行为修复抬它会用 E0006 对整个索引的所有旧 mcpp 用户硬关门。
  5. 合并前:删掉 .github/workflows/diag-ut-macos.yml —— 官方仓库不适合放临时 CI,它会对每个 PR、对所有人跑,并且比调查活得久;那属于 fork(你们本来就有一个)。另外需要 squash,PR 正文已经落后代码两轮迭代。

包身份那部分做得是对的:boost-ext 独立命名空间、name = "ut" 单一原子段、pin 在可复现的 release tag。

以下是详细依据。


The macOS crash was an mcpp engine bug, not a bug in this package

Your forensics on the macOS leg is what located it — the __init_offsets ordering and the zero std::cout vptr are exactly right, and they pointed straight at the defect. Thank you for digging that far; it turned into mcpp-community/mcpp#336 and is fixed and released in mcpp 2026.8.3.1.

The mechanism, confirmed by disassembling iostream.cpp.o:

  • Mach-O has no priority-ordered initializer section (init_priority only orders within a TU), so an initializer pulled out of libc++.a lands last in link order.
  • libc++'s <iostream> carries no ios_base::Init guard of its own. libstdc++ does (iostream:82, static ios_base::Init __ioinit;) and so does the MSVC STL — which is precisely why only macOS broke.
  • std::ios_base::Init is only forward-declared in libc++'s headers (ios:70, no definition anywhere in the tree), so the remedy the standard provides for static-initialization order is not writable by users on libc++ at all.

mcpp now links a generated object first on the link line whose constructor forces the streams up. Being first is the whole point, and only the build system can put something there.


What that means for this PR

Two root-cause narratives in the current head are contradicted by the above, and one of them is in the artifact that outlives the PR:

  1. .agents/docs/…-add-boost-ext-ut-plan.md §2.3 attributes the SIGSEGV to module BMIs not carrying template member definitions ("dispatch 落入未实例化 slot"). The crash address 0xffffffffffffffe8 is this - 0x18, a null-vptr read, not an unresolved dispatch.
  2. The descriptor header comment attributes it to export import std; producing a module-local std::cout distinct from libc++'s ("an ODR split"). The issue's minimal reproducer is #include <iostream> plus a global object with no modules involved at all, and 3eff2899 (which dropped export import std; on that theory) left macOS at exit 139 — it is still 139 at c8cbfa14.

The descriptor comment is the one that matters most: it is what the next contributor reads.

The cost that was paid for the non-fix

Dropping export import std; required a 24-line global-module-fragment #include list plus cxxflags = { "-Wno-template-body" } to silence the GCC cascade that removal caused. That flag is not a Clang option — your own diag run says so:

warning: unknown warning option '-Wno-template-body'; did you mean '-Wno-empty-body'?

So it adds a warning to every compile on both Clang platforms, and breaks any consumer building with -Werror.


Measured with the released 2026.8.3.1

I rebuilt this branch's test member from a cold state with the descriptor reduced to upstream's ut.cppm used verbatim — no generated_files, no shims, no cxxflags:

toolchain result
gcc 16.1.0 / x86_64-linux-gnu Suite 'global': all tests passed (3 asserts in 2 tests)
llvm 22.1.8 / x86_64-linux-gnu (libc++) ✅ same

So on Linux the package currently needs nothing — and §2.1's "GCC 16.1 rejects unqualified size_t" does not reproduce on gcc 16.1.0 there. You saw it on Windows (x86_64-windows-gnu), which makes it a platform-local problem that the descriptor is currently applying to every platform.

⚠️ Re-verification trap. My first attempt at this printed Cached boost-ext.ut v2.3.1 (1 unit) and passed — the dependency build cache had handed me the previous descriptor's output, so the "pass" was meaningless. Clear ~/.mcpp/build-cache/v1/pkg/<namespace> along with the member's .mcpp/ and target/, and only trust a run that says Compiling, not Cached.


A shape that fits the index better

If a platform genuinely still needs a shim, the descriptor already has an axis for that — a per-OS section inside mcpp = { … }, as compat.ftxui does:

mcpp = {
    sources = { "*/include/boost/ut.cppm" },   -- upstream, verbatim
    ...
    windows = {
        cxxflags = { ... },    -- only what Windows needs, only on Windows
    },
},

Baking a Windows-only workaround into a synthesized copy of upstream source that every platform consumes is what that axis exists to avoid. generated_files also has a standing cost: a copy of upstream source enters the trust path and has to be re-synced by hand on every upstream change.

And one level further up — both remaining deviations are upstream bugs, not packaging problems:

  • ut.hpp:687 references __argc/__argv under #if defined(_MSC_VER) while the neighbouring guards at 291 / 311 / 1147 already carry && !defined(__clang__). That line just missed it.
  • the unqualified size_t at namespace scope.

Both are one-line upstream fixes. An index packages, it does not fork — so the durable move is to file them at boost-ext/ut and carry a platform-scoped minimal shim with an explicit drop condition until a release includes them. Your evolution note already does the drop-condition part well; it just needs to point at the right conditions.


Does this PR actually require the new mcpp?

Not necessarily — there are two paths, and I do not think it is my call which one you take:

A. Fix it package-side, works on the pinned 0.0.109. cfg is an inline variable (ut.hpp:2247), so under module attachment it is emitted in the module TU. A construct placed in the wrapper before #include "ut.hpp" therefore precedes it in that object's initializer order. Your S5 experiment called the right function — the disassembly shows bl … std::__1::DoIOSInit::DoIOSInit() and nm shows exactly one definition, so there was never an ODR split — but it landed 3rd in __init_offsets. Whether moving it ahead of the include is enough is one CI run away from being settled, and it is the only experiment that answers whether this PR ever needed the engine fix. It would also protect macOS users still on older mcpp.

B. Move the index's CI pin. validate.yml's MCPP_VERSION: "0.0.109"2026.8.3.1 makes this PR's macOS leg green with no package-side workaround at all.

Please do not raise index.toml's min_mcpp for this. That field is a grammar contract — "the oldest mcpp able to resolve every descriptor in this index" — enforced with a hard E0006 at index-open. No descriptor grammar changed here; raising it would slam the door on every older-mcpp user for the whole index over a codegen fix. Only the CI pin should move.

Worth noting as a real gap: the index can express "this descriptor needs mcpp ≥ X to be parsed", but has no way to say "artifacts this package produces are incorrect on mcpp < X". This package is the first case that needs the latter.


Housekeeping before merge

  • .github/workflows/diag-ut-macos.yml has to go. Debugging a platform you do not have by pushing forensics workflows is a legitimate technique — I had to do exactly the same thing today to fix #336. But the upstream repository is not the place for temporary CI: it runs for everyone, on every PR, and outlives the investigation. That belongs in a fork, and you already had one for it (ZheFeng7110/mcpp-index#1). Only the finished descriptor should arrive here.
  • Squash. 19 commits, 16 of which are diagnostic iteration.
  • The PR body is two iterations stale. It still describes "upstream cppm verbatim + two minimal shims, no upstream line removed" — the file now drops export import std;, adds the GMF list, adds -Wno-template-body, and appends dev 3. The verification table also only covers two Windows toolchains, with no macOS row.
  • dev 3 (the explicit-instantiation block lifted from upstream master) was introduced as the macOS fix, which it is not. If no platform actually needs it, carrying a post-v2.3.1 deviation from the pinned tag is a cost with no stated benefit — worth re-testing without it.
  • "2.2.3" in the design doc's §6 verification table should be "2.3.1".
  • Four new files are missing a trailing newline.

None of this touches the package-identity work, which reads well: boost-ext as its own namespace (not compat, not boost), name = "ut" as a single atomic segment, and the pin to a reproducible release tag are all the right calls.

…t-order workaround (mcpp#336)

Package-side fix per review: cfg (ut.hpp:2247) is an inline variable, so its
initializer is emitted in the module TU; a std::ios_base::Init construct
declared before #include "ut.hpp" precedes cfg's init in this object's
initializer order and forces libc++'s stream construction first. Also
corrects the descriptor's macOS narrative (mcpp#336: missing libc++ stream
guard, NOT an ODR split) and updates the dev list to include dev 4.
…der fix)

Plan B per review: the macOS crash is an mcpp engine bug fixed in 2026.8.3.1
(links a generated object first whose ctor forces libc++'s streams up), so
bump only the CI pin — index.toml min_mcpp stays at 0.0.109 (syntax
contract; a behavior fix must not E0006-gate older-mcpp consumers).
Synced the three workspace matrix mcpp_version entries (keep in sync with
env.MCPP_VERSION).
…acOS init-order workaround (mcpp#336)"

This reverts commit fbf1ad0.
@ZheFeng7110

Copy link
Copy Markdown
Collaborator Author

摘要

方案 B(抬 MCPP_VERSION 到 2026.8.3.1)生效:boost-ext.ut 的 macOS 腿全绿了。但 2026.8.3.1 携带的重写版构建缓存(#317,2026.7.30.2)在本索引的三平台 workspace 上触发了新的引擎缺陷:13/47 个成员(linux)以同一 ninja 错误失败。

ninja: error: '.../build-cache/v1/pkg/compat/compat.zlib@1.3.2/58d45813f79658a1/obj/compress.o',
               needed by 'obj/compress.o', missing and no known rule to make it
  • 这是 mcpp 引擎的缓存缺陷,不是这个包、也不是本索引描述符的问题。0.0.109 用的是旧缓存(整工程指纹),没有逐 TU 的 stage 机制,所以从不受影响 —— 因此「抬版本→大面积失败」是缓存重写带来的副作用,不是 boost-ext.ut 引入的回归。
  • 机制(已定位到源码与碰撞对,见下):缓存条目的对象相对路径由「填充成员自己依赖图的 basename 普查」决定,而 stage 边(消费侧)用「消费成员自己算的路径」;两个成员的图一不同,同一包就落进不同相对路径 → is_cached(校验记录路径)通过、stage 输入(消费路径)缺失 → ninja 硬错误。
  • 失败成员全是「第二次/第 N 次消费某 C 包」的成员,失败路径全部指向先跑成员已成功编译并入库、但消费侧以不同布局寻址的对象。
  • 修复方向:根治在 mcpp 上游(缓存条目应按 resolve_cached 记录的路径供 stage,或缓存路径与成员图解耦);本索引需在以下两条路中选一条,由维护者定夺(我无法替维护者做这个决定):
    1. workspace job 设 MCPP_BUILD_CACHE=local(不读不写,每成员自编依赖,慢但正确)直到 mcpp 修复;
    2. 退回 0.0.109,等 mcpp 修缓存后再合并(macOS 腿会回到 exit 139,需另作处理)。

评审意见里的约束均未违反:index.tomlmin_mcpp 保持 0.0.109 未动;diag-ut-macos.yml 尚未删除(合并前清理)。


失败形态

三平台(linux / macos / windows)完全相同,一次跑完的 47 个成员里 13 个失败(linux 侧):

  • eui-neoeui-neo-app-maineui-neo-markdowneui-neo-sdl2eui-neo-vulkaneui-neo-window → 引用 compat.zlib@…/obj/compress.o
  • opencv-moduleopencv-module-dnnopencv-module-unifont → 引用 compat.ffmpeg@…/…/libavutil/adler32.ocompat.x11@…/…/imKStoUCS.o
  • sdl2compat.x11@…/…/imKStoUCS.o;vulkancompat.x11@…/…/reallocarray.o
  • freetypelibpngcompat.zlib@…/obj/compress.o

关键反证(证明不是「传递依赖」问题):同一 job 内

  • 先跑成员正常:archive 编译 zlib 并 ok;ffmpeg 成员编译 compat.ffmpeg 并 ok;gui-stack 编译 x11 并 ok
  • 部分跨成员复用成功:asio-ssl(先)编译 openssl → openssl 成员(后)Cached compat.openssl v3.5.1 (1 unit)ok;gui-stack 编译 glfw → imgui-module Cached compat.glfw v3.4 (23 units)ok
  • 缓存键含版本且冷启动 miss(mcpp-registry-Linux-2026.8.3.1-…Cache not found),一切在单次 job 内自产自销,无任何 restore 污染;日志无 bmi cache populate failed 警告,即 populate 全部成功。

机制(源码定位)

  1. 新缓存的 stage 边(mcpp src/build/ninja_backend.cppm:979-984):命中的依赖单元发 stage_file 边、不发 compile 边 —— build <本地obj路径> : stage_file <缓存obj路径>。报错正是这条边的输入(缓存对象)缺失。

  2. 对象相对路径取决于成员图(mcpp src/build/plan.cppm:577-585,mcpp#233 的 obj 路径消歧):

    • 全图无同名 basename → 平铺 obj/<name>.o;
    • 有同名冲突 → 带包前缀 obj/<pkg>/<relDir>/<name>.o;
    • 普查(basenameCount)按每个成员自己的整个依赖图计算。
  3. 写侧与读侧路径不一致(mcpp src/build/prepare.cppm):

    • populate 记录的是填充成员布局的路径(:4516);
    • is_cached记录路径逐个校验文件存在(src/bmi_cache.cppm:202-216)→ 通过;
    • stage 边的输入用的是消费成员自己算的路径(:4531)。
    • 两套路径不一致 → is_cached 真、stage 输入缺失 → ninja missing and no known rule to make it

实测碰撞对(已用 GitHub API 核对源文件清单)

填充成员(平铺 → 入库) 消费成员(带前缀 → stage 输入) 冲突来源
zlib archive 图含 bzip2:compress.c 同名 → 入库 obj/compat_zlib/compress.o eui-neo/freetype/libpng 图无 bzip2 → obj/compress.o bzip2 有 compress.c
ffmpeg ffmpeg 成员图独有 → adler32.o 平铺 opencv 图含 zlib → obj/compat_ffmpeg/ffmpeg-8.1.2/libavutil/adler32.o zlib 有 adler32.c
x11 gui-stack 图 → 平铺 opencv/sdl2/vulkan 图 → obj/compat_x11/src/xlibi18n/imKStoUCS.o…/reallocarray.o 图内同名冲突(同机制)

为什么有的成员通过

  • glfw / imgui / openssl 复用成功:两个消费成员的普查结果恰好一致(glfw 各 basename 在两个图里都唯一;openssl 单单元;imgui 复用路径一致)。
  • catch2 第二成员是重编而非报错:feature 差异使缓存键不同 → is_cached miss → 走 compile 边,只是浪费。
  • boost-ext.ut 本身通过:它的图没有任何 C 依赖,不存在第二次消费,也与 macOS 引擎修复(#336)互不干扰。

修复方向

  • 根治在上游:mcpp 的 per-package 缓存(#317)在「异构 workspace 多成员」下,缓存条目路径必须与请求成员的图无关 —— stage 边源路径应取自条目记录(resolve_cached),或把 obj 布局改成规范/全局一致的形态。建议向 mcpp-community/mcpp 提单。
  • 本索引短期:在 mcpp 修复前,CI 要么用 --cache local/MCPP_BUILD_CACHE=local(不读不写、每成员自编依赖,慢但正确),要么退回 0.0.109(macOS 腿需另作处理)。min_mcpp 不动,与本评审意见一致。

以上决定权在维护者,我按要求只陈述事实与两条可选路线。

Sunrisepeak and others added 2 commits August 3, 2026 19:09
diag-ut-macos.yml was a one-shot forensics harness for the boost-ext.ut macOS
SIGSEGV — lldb disassembly, __init_offsets symbolization, a dozen relink
variants — triggered on push to this branch. The crash is diagnosed
(mcpp-community/mcpp#336, fixed in mcpp 2026.8.3.1), so the harness has done
its job. On main it would never fire and only burn macos-15 minutes on manual
dispatch.

Co-authored-by: SPeak Agent <248744407+speak-agent@users.noreply.github.com>
…bal build cache

## The pin

macOS: a global object that touches std::cout during static init crashes on
sight. Mach-O has no priority-ordered init section and libc++'s <iostream>
carries no ios_base::Init guard of its own, so the streams are still all-zero
when an archive member's initializer runs. That is mcpp-community/mcpp#336,
fixed in mcpp 2026.8.3.1 by linking a generated object FIRST whose constructor
brings them up.

boost-ext.ut trips exactly this: ut.hpp:1620 member-initializes
`std::streambuf* cout_save = std::cout.rdbuf();` inside the statically
constructed `cfg::runner<reporter_junit<printer>>`. It is not fixable
package-side — std::ios_base::Init is only forward-declared in libc++'s <ios>
— which this branch established the hard way: dropping `export import std;`
(3eff289) and force-constructing ios_base::Init in the package (fbf1ad0) both
still exit 139 on 0.0.109. The pin is what turns macOS green.

index.toml's min_mcpp/latest_mcpp move with the pin as they always have (mcpplibs#125
moved 0.0.108 -> 0.0.109). Here they especially should: a consumer below the
floor gets a segfault with no diagnostic, which is worse than E0006.

2026.8.3.3 rather than .3.1 — .3.2/.3.3 are cross-compilation fixes (PE
artifact naming, -static decided by host instead of target) that no leg of
this matrix exercises, so taking the newest of the train costs nothing.

## MCPP_BUILD_CACHE: local

mcpp >= 2026.7.30.2 added a global package build cache. It is unusable here
until mcpp-community/mcpp#344 lands:

mcpp#233's object-path disambiguation fires on basename collisions across the
WHOLE build dir — i.e. on which packages the CONSUMER pulls in — while the
cache key deliberately covers only the dependency itself. So one entry can
hold two different layouts. tests/examples/archive pulls zlib AND bzip2 (both
ship compress.c), disambiguates, and stores
obj/compat_zlib/zlib-1.3.2/compress.o; every zlib consumer without bzip2 then
asks the same key for a flat obj/compress.o and ninja dies at graph time with
"missing and no known rule to make it". Reproduced both ways round on
2026.8.3.3 — whichever member runs second fails, in the mirror-image
direction.

That is what took 13/47 (linux), 11/47 (macos) and 8/47 (windows) members down
on this branch's first full-workspace run: eui-neo*, opencv-module*, freetype,
libpng, sdl2, vulkan — all zlib/ffmpeg consumers with a package set different
from whichever member primed the cache.

`local` still caches the std BMI, the expensive one; only package entries are
bypassed. Drop the env once mcpp#344 is fixed.

Co-authored-by: SPeak Agent <248744407+speak-agent@users.noreply.github.com>
Sunrisepeak and others added 2 commits August 3, 2026 19:10
…on-MSVC shim

The wrapper had grown three deviations from upstream v2.3.1's
include/boost/ut.cppm. Two of them turned out to be self-inflicted, and this
reduces the file to upstream's own bytes plus exactly one shim.

## Dropped: the `export import std;` removal + 24 hand-written GMF includes

Introduced as the fix for the macOS SIGSEGV, on the theory that
`export import std;` gave the module its own copy of the std stream entities,
ODR-split from libc++'s. The branch's own CI refutes it: at 3eff289, with
`export import std;` already gone, macOS still reported
`ut ... FAIL (exit 139)` on 0.0.109. The real cause is toolchain-side
(mcpp#336) and is addressed by the pin.

Its cost was real, though: without `export import std;` GCC 16.1 rejected
ut.hpp with a cascade of -Wtemplate-body errors (unqualified `size_t`,
`std::empty`, the literals using-block), which is what forced
`using std::size_t;` and `cxxflags = { "-Wno-template-body" }`. Those existed
only to repair the damage. Verified cold on linux/gcc 16.1.0, on mcpp
2026.8.3.3 AND 0.0.109: with upstream's `export import std;` kept, the module
compiles and the member passes with no cxxflags at all — the compile line
carries neither -Wno-template-body nor -w.

## Dropped: the post-v2.3.1 explicit-template-instantiation block

Lifted from upstream master and carried as a second macOS candidate fix. It
was already present at 61b9441 while macOS still exited 139, so it does not
address anything this index hits, and it ships in no release tag. Out of the
trust path.

## Kept: the __argc / __argv shim

ut.hpp:687 is `#if defined(_MSC_VER)` and reaches for the MSVC builtins
__argc / __argv. Clang on the MSVC ABI sets _MSC_VER but does not provide
them; upstream's neighbouring guards at 291 / 311 / 1147 already gate clang
out and 687 missed it. This index's Windows default toolchain IS Clang on the
MSVC ABI (llvm@20.1.7, *-pc-windows-msvc — not mingw, not cl.exe), so the shim
is load-bearing. Double-gated on `_MSC_VER && __clang__`: real MSVC never
enters it, no non-Windows target ever sees it, and cfg::largc/largv are
reassigned from main()'s argv at runtime so the stand-in values are never
read.

## Docs

README (both languages) claimed the cppm was "used verbatim, no
generated_files" while the descriptor did the opposite — corrected to what the
file now actually is. The design doc is rewritten around the evidence,
including a section on the three refuted approaches so the next reader does
not retry them.

Co-authored-by: SPeak Agent <248744407+speak-agent@users.noreply.github.com>
tests/examples/boost-ext.ut went in at the head of [workspace].members; the
list is alphabetical, so it belongs between asio-ssl and build-mcpp. The two
new member files were also missing their final newline, unlike every
comparable file in the tree.

Co-authored-by: SPeak Agent <248744407+speak-agent@users.noreply.github.com>
@Sunrisepeak
Sunrisepeak merged commit 160c389 into mcpplibs:main Aug 3, 2026
5 checks passed
@ZheFeng7110
ZheFeng7110 deleted the add-boost-ext-ut branch August 3, 2026 13:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants