From 2a12784751ae7751b4c86815fa00323e1adaa834 Mon Sep 17 00:00:00 2001 From: speak-agent Date: Thu, 6 Aug 2026 15:44:05 +0800 Subject: [PATCH 1/9] =?UTF-8?q?feat(build):=20=E5=BA=93=E5=8F=AF=E4=BB=A5?= =?UTF-8?q?=E6=9B=BF=E7=94=A8=E6=88=B7=E6=8B=89=E8=B5=B7=E5=B7=A5=E5=85=B7?= =?UTF-8?q?=E9=93=BE,=E6=9E=84=E5=BB=BA=E7=A8=8B=E5=BA=8F=E5=8F=AF?= =?UTF-8?q?=E4=BB=A5=E5=A3=B0=E6=98=8E=E3=80=8C=E4=B8=80=E7=BB=84=E6=96=87?= =?UTF-8?q?=E4=BB=B6=E3=80=8D(#359)=20(2026.8.6.2)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit #355 让依赖产出的 host 工具可用之后,grpc-m 成了第一个真实使用者,并暴露出 两个缺口:工具**被构建了**却传不到消费者,以及 build.mcpp 无法安全地 glob 自己的输入。两者形状相同 —— **新增一种提供物/输入时,没有任何地方逼你回答 「它怎么传播」「它的指纹怎么取」**。这是本仓库反复付学费的「同一决策在 N 处 推导」的镜像:一个必答问题在**零处**被表达。 ## 提供物进表,传播由 `reexport` 决定 新增 `src/build/provisions.cppm`:ProvisionKind 表(tool / host-module / dep-dir),每行必须回答「裸名可寻址吗」「跨边要不要 reexport」;传播是一条 不动点,与 computeUsageRequirements 同形。 依赖 spec 新增 `reexport = true`(默认 false):把这条边的构建期提供物交给 **本包自己的消费者**。于是 grpc 可以在描述符里声明 protoc + 插件 + 规则模块, 它的用户只写一条依赖。 刻意不复用边上的 `visibility`:它默认就是 "public",搭车等于默认传播,而 「把工具交给消费者」是供应链主张,必须显式写下来。(设计初稿以「include_dirs 默认 private」类比,那条对本仓库不成立 —— prepare.cppm 无条件把 privateBuild.includeDirs 拷进 publicUsage;真正 private-only 的是 build.mcpp 注入的那些。) 裸名不再靠追加顺序决定。全限定 `MCPP_DEP___BIN_` 总是发布; 裸名按与包身份同一套阶梯绑定(mcpplibs → compat → 无命名空间 → 唯一候选), 争用时给出诊断而不是默默选一个 —— 否则「版本错配不可表达」这条性质会从另一 扇门跑回来。 `dep_dir()` 与 host 模块一并沿同一条规则传播(此前分别只覆盖直接依赖、只认 root 的 dependencies)。 ## `rerun-if-changed-glob`:输入可以是一个集合 `build.mcpp` 的重跑键此前只有「文件内容」和「环境变量」两种形态,于是 glob **结构性不安全**:新增一个 .proto 不改变任何已声明文件的哈希,程序不重跑, 新文件静默不生成。 新指令的指纹是**排序后的匹配路径集合**,不含内容、size、mtime —— 内容已由 File 条目覆盖,而 mtime 在 checkout/容器/rsync 下不稳定(本仓库在 file_time_type 的 epoch 上摔过)。构建输出目录与 .git 永不进入集合,否则宽 模式会对着自己的产物无限重跑。 两层必须一起改:工程级 fast path 会整个跳过 prepare,而新增文件不移动任何 mtime —— 那正是它此前静默无效的地方。缓存记录因此新增 `root` 行,fast path 才能按每份缓存自己的根去求值。protocol → 2,cache epoch → 2。 ## 让库能按平台裁剪,让子构建说出真话 `ConditionalConfig` 带着三张依赖表却独缺 `feature-deps`,正是该类型注释为 #258 记录过的失败形状。补上 `[target..feature-deps.]`:否则一旦由 库决定请求什么,不支持的平台就变成用户改不掉的硬错(Windows 的工具子构建 缺陷至今未定位)。feature 本身无条件注册,只有它拉进来的东西是条件性的。 顺带把条件依赖表收进同一条 funnel:此前三个调用点合并构建输入,只有 root 那个还额外合并依赖表,而注释把它说成「out of scope」。 工具子构建失败现在带上 chain、scratch 目录和可直接重跑的命令, `MCPP_TOOL_BUILD_VERBOSE=1` 关掉内层输出过滤 —— Windows 那个缺陷至今未定位 的直接原因就是真正的报错没进日志。 ## 其他 - `path_matches_glob` 从 scanner 的匿名命名空间搬进 `mcpp.modgraph.glob`: glob 输入的指纹必须与 `sources = [...]` 选中同一批文件,两个「应该一致」的 匹配器就是那个病。 - 内置 xlings 升到 2026.8.6.2。 ## 验证 - 新单测:传播规则(未 reexport 不外泄 / 逐跳传递 / 环终止)、裸名阶梯四级、 glob 指纹(增删变、改内容不变、输出目录不参与、跨平台排序一致)。 - 新 e2e 193/194/195。193 已用「让 propagate 忽略 reexport」验证会变红。 - 新增 `EveryDependencySpecKeyIsAccepted`:dep-spec 的键必须同时写进两处, 漏掉第一处时诊断是**误导性**的(bool 被告知「必须是字符串或嵌套表」), #359 加 `reexport` 时就撞上了。 - 本机 e2e 179 passed / 7 failed / 8 skipped;7 条在 main 上逐条复现,均为环境性。 --- .../2026-08-06-provisions-and-build-inputs.md | 223 +++++++++----- .github/actions/bootstrap-mcpp/action.yml | 2 +- .github/actions/setup-macos-llvm/action.yml | 2 +- .github/workflows/bootstrap-macos.yml | 2 +- .github/workflows/ci-fresh-install.yml | 6 +- .github/workflows/ci-linux-e2e.yml | 2 +- .github/workflows/cross-build-test.yml | 4 +- .github/workflows/release.yml | 14 +- docs/05-mcpp-toml.md | 70 ++++- docs/07-build-mcpp.md | 33 +- docs/zh/05-mcpp-toml.md | 58 ++++ docs/zh/07-build-mcpp.md | 30 +- mcpp.toml | 2 +- src/build/build_program.cppm | 89 +++++- src/build/directives.cppm | 87 +++++- src/build/execute.cppm | 7 + src/build/hostprogram.cppm | 13 + src/build/prepare.cppm | 290 ++++++++++++------ src/build/provisions.cppm | 279 +++++++++++++++++ src/manifest/toml.cppm | 37 ++- src/manifest/types.cppm | 12 + src/manifest/xpkg.cppm | 16 +- src/modgraph/glob.cppm | 77 +++++ src/modgraph/scanner.cppm | 63 +--- src/pm/dep_spec.cppm | 19 ++ src/version.cppm | 2 +- src/xlings.cppm | 2 +- tests/e2e/193_provision_reexport.sh | 182 +++++++++++ tests/e2e/194_build_mcpp_glob_input.sh | 125 ++++++++ tests/e2e/195_target_cfg_feature_deps.sh | 138 +++++++++ tests/unit/test_build_directives.cpp | 101 ++++++ tests/unit/test_manifest.cpp | 126 ++++++++ tests/unit/test_provisions.cpp | 208 +++++++++++++ 33 files changed, 2058 insertions(+), 263 deletions(-) create mode 100644 src/build/provisions.cppm create mode 100644 src/modgraph/glob.cppm create mode 100755 tests/e2e/193_provision_reexport.sh create mode 100755 tests/e2e/194_build_mcpp_glob_input.sh create mode 100755 tests/e2e/195_target_cfg_feature_deps.sh create mode 100644 tests/unit/test_provisions.cpp diff --git a/.agents/docs/2026-08-06-provisions-and-build-inputs.md b/.agents/docs/2026-08-06-provisions-and-build-inputs.md index 844b7192..6342003d 100644 --- a/.agents/docs/2026-08-06-provisions-and-build-inputs.md +++ b/.agents/docs/2026-08-06-provisions-and-build-inputs.md @@ -1,9 +1,10 @@ # 依赖提供物与构建期输入:两个缺口,同一个形状 -> 状态：**设计待 review** +> 状态：**已定案，实施中** > 关联：[#359](https://github.com/mcpp-community/mcpp/issues/359)（由 grpc-m 的真实使用暴露） -> 涉及：`src/modgraph/scanner.cppm`（`UsageRequirements`）、`src/build/prepare.cppm`、 -> `src/build/build_program.cppm`、`src/build/directives.cppm` +> 涉及：`src/build/provisions.cppm`（新增）、`src/build/prepare.cppm`、 +> `src/build/build_program.cppm`、`src/build/directives.cppm`、 +> `src/pm/dep_spec.cppm`、`src/manifest/{types,toml,xpkg}.cppm` --- @@ -25,7 +26,7 @@ import mcpp; import grpcgen; int main() { return grpcgen::generate({"helloworld"}) ? 0 : 1; } ``` -后三条**全是为了 codegen**,而且要求用户知道「gRPC 的代码生成需要 protobuf 的 protoc」——这是**库该承担的知识**。目标形态: +后三条全部服务于 codegen,并且要求用户知道「gRPC 的代码生成需要 protobuf 的 protoc」——这是库该承担的知识。目标形态: ```toml grpc = { version = "1.83.0", features = ["codegen"] } @@ -35,143 +36,211 @@ import mcpp; import grpcgen; int main() { return grpcgen::generate_all() ? 0 : 1; } // 扫 proto/** ``` -对照业界:xmake 是 `add_requires("grpc")` + `add_files("proto/*.proto")`;CMake+vcpkg 是 1 条依赖 + `protobuf_generate(...)`。达到目标形态后 mcpp **严格更优** —— 因为它还额外保有「版本错配不可表达」与「交叉编译构造上正确」这两条别人没有的性质。 +对照业界:xmake 是 `add_requires("grpc")` + `add_files("proto/*.proto")`;CMake + vcpkg 是 1 条依赖 + `protobuf_generate(...)`。达到目标形态后 mcpp 严格更优,因为它额外保有「工具与运行时版本错配不可表达」与「交叉编译构造上正确」两条性质。 -两个缺口各挡住一半,**缺一个都到不了**。 +两个缺口各挡住一半,缺一个都到不了。 -## 1. 关键发现:模型已经存在,新东西没接进去 +## 1. 关键发现:一个必答问题在零处被表达 mcpp 早有一套「依赖能提供什么 × 提供给谁」的模型(`src/modgraph/scanner.cppm`): ```cpp struct UsageRequirements { - std::vector includeDirs; - std::vector includeDirsAfter; + std::vector includeDirs, includeDirsAfter; std::vector cflags, cxxflags, ldflags, modules; }; - struct PackageRoot { - UsageRequirements privateBuild; // 只给自己 - UsageRequirements publicUsage; // 沿边传给消费者 + UsageRequirements privateBuild; // 编译自己时用 + UsageRequirements publicUsage; // 沿边流向消费者 UsageRequirements linkUsage; // 链接期 }; ``` -include dirs、defines、link flags、modules 全都通过它传播,规则清楚、单点定义。 - -**#355 引入的两种新提供物没有进入这个模型**: +#355 引入的两种新提供物没有进入任何等价模型: -| 提供物 | 在模型里? | 实际实现 | +| 提供物 | 传播规则在哪 | 实际行为 | |---|---|---| -| include dirs / defines / ldflags / modules | ✅ `UsageRequirements` | 按作用域传播 | -| **host 工具**(`tools = [...]`) | ❌ | `prepare.cppm:4113` 硬编码 `toolEnvByConsumer[edge.consumerPackageIndex]` —— 只给**发出请求的那条边**的消费者 | -| **host 模块**(`host-module = true`) | ❌ | `prepare.cppm:4016` 只遍历 `m->dependencies`,即**只认 root 的**直接依赖 | +| include dirs / defines / ldflags / modules | `UsageRequirements` + 不动点(`prepare.cppm:2906`) | 按作用域传播 | +| host 工具(`tools = [...]`) | 无 | `prepare.cppm:4122` 只发给发出请求的那条边的消费者 | +| host 模块(`host-module = true`) | 无 | `prepare.cppm:4016` 只遍历 root 的 `dependencies` | +| 依赖目录(`dep_dir()`) | 无 | `fillDepDirs` 只覆盖直接依赖 | -于是「库代用户拉起整条 codegen 工具链」在架构上不可能:工具**被构建了**,但环境变量记在库的账上,消费者的 `build.mcpp` 看不见。 +于是「库代用户拉起整条 codegen 工具链」在架构上不可能:工具被构建了,但环境变量记在库的账上,消费者的 `build.mcpp` 看不见。 -> 实测确认(不是推断):一个 path 依赖在自己的 manifest 里写 `compat.protobuf = { tools = ["protoc"] }`,消费者 `mcpp::dep_bin("protobuf","protoc")` 拿到**空串**,`dep_dir` 同样为空。 +> 实测确认:一个 path 依赖在自己的 manifest 里写 `compat.protobuf = { tools = ["protoc"] }`,消费者 `mcpp::dep_bin("protobuf","protoc")` 拿到空串,`dep_dir` 同样为空。 -**根因不是「少了一次传播」,而是:新增一种提供物时,没有任何地方逼你回答「它怎么传播」。** 这与本仓库反复付学费的「同一决策在 N 处推导」是同一形状的镜像——一个必答问题在**零处**被表达。`directives::kTable` 已经用「Scope 是必填字段」解过一次。 +**根因不是少了一次传播,而是新增一种提供物时,没有任何地方逼你回答「它怎么传播」。** 这与本仓库反复付学费的「同一决策在 N 处推导」是同一形状的镜像——一个必答问题在零处被表达。`directives::kTable` 已经用「Scope 是必填字段」解过一次。 ## 2. 缺口 B 同构:输入声明的种类是封闭的 -`build.mcpp` 的缓存键由**声明过的输入**构成,而输入只有两种形态: +`build.mcpp` 的缓存键由声明过的输入构成,而输入只有两种形态: ```cpp // build_program.cppm:283 -os << "in " << hash_file(abs_against_root(root, f)) << ' ' << f << '\n'; // 文件内容 -os << "env " << hash_string(env_value(e)) << ' ' << e << '\n'; // 环境变量 +os << "in " << hash_file(abs_against_root(root, f)) << ' ' << f << '\n'; // 文件内容 +os << "env " << hash_string(env_value(e)) << ' ' << e << '\n'; // 环境变量 ``` -`hash_file` 读的是**文件内容**。于是「我的输出取决于这个目录里有哪些文件」**无法表达**: - -- 对目录调用 `rerun_if_changed` 无效(目录没有可读内容); -- 新增一个 `.proto` 不改变任何已声明文件的哈希 → build.mcpp 不重跑 → **新文件静默不生成**。 +`hash_file` 读的是文件内容,于是「我的输出取决于这里有哪些文件」无法表达:新增一个 `.proto` 不改变任何已声明文件的哈希,build.mcpp 不重跑,新文件静默不生成。实测:glob `proto/**` 后新增 `fresh.proto`,`Finished dev in 0.01s`,产物 0 个。这比「要求用户列名字」更坏,所以 grpc-m 最终选了显式列表。 -实测:glob `proto/**` 后新增 `fresh.proto`,`Finished dev in 0.01s`,产物 0 个。这比「要求用户列名字」更坏,所以 grpc-m 最终选了显式列表。 +同样的形状:新增一种输入时,没有地方回答「它的指纹怎么取」。 -同样的形状:**新增一种输入时,没有地方回答「它的指纹怎么取」。** +--- ## 3. 设计 -一条主张:**两个缺口都收敛成「表 + 必答字段」,与 `directives::kTable` 同一范式**,而不是各打一个补丁。 +一条主张:两个缺口都收敛成「表 + 必答字段」,与 `directives::kTable` 同一范式。 + +### D1. 提供物进一张 `ProvisionKind` 表,传播由 `reexport` 决定 + +#### D1.1 容器:新模块,不是 `UsageRequirements` -### D1. 提供物进 `UsageRequirements`,传播由作用域决定 +`UsageRequirements` 描述的是消费者**编译**时的命令行,进的是 ninja 边;提供物描述的是消费者的 **build.mcpp 程序**能看见什么,进的是 prepare 期子进程的环境。两者的消费者、时机、失效条件都不同,合并会让 `privateBuild / publicUsage / linkUsage` 这三个作用域名字对新字段失去意义(`linkUsage.tools` 没有含义)。 + +因此新增 `src/build/provisions.cppm`,持有: ```cpp -struct UsageRequirements { - // …既有字段… - // #359: host 工具与 host 模块。放在这里而不是旁路,是为了让「它怎么 - // 传播」由所在的作用域回答,与 includeDirs 完全同一条规则。 - std::vector tools; - std::vector hostModules; -}; +enum class ProvisionKind { Tool, HostModule, DepDir }; +``` + +每一种在表里必须给出三件事:环境变量的形态、裸名是否可用、传播是否需要 `reexport`。新增第四种提供物时,这张表逼你回答同样三个问题——这正是 §1 的病所缺的那个位置。 + +分模块而非并进 `prepare.cppm` 的匿名命名空间,理由与 `directives.cppm` 头部记录的一致:该匿名命名空间在 clang 22 + C++20 modules + `-O2` 下会破坏邻居(PR#332 / PR#334)。 + +#### D1.2 语法:`reexport = true`,一个布尔,覆盖三种提供物 + +```toml +# grpc 的描述符 +[feature-deps.codegen] +"compat.protobuf" = { version = "35.1", tools = ["protoc"], reexport = true } +grpc-plugin = { version = "1.83.0", tools = ["grpc_cpp_plugin"], reexport = true } +grpcgen = { version = "1.83.0", host-module = true, reexport = true } +``` + +默认 `false`,即今天的行为。选它而不是新开 `[provides]` 表,理由: + +1. `tools` / `host-module` 已经是这条边上的请求字段,「是否再导出」是同一条边的属性。另开一张表会让「谁请求 / 谁提供」在两处推导——正是本设计要治的病。 +2. 一个词覆盖三种提供物。grpc 的三条依赖各写一次 `reexport = true`,而不是三种不同的键。 +3. `reexport` 在 C++20 模块里就是 `export import` 的名字,读者不需要学新概念。 +4. 粒度已经是 per-edge:库要只导出 protoc 而不导出规则模块,把 `reexport` 写在那一条边上即可。per-kind 的更细粒度是 YAGNI。 + +##### 为什么不能复用边的 `visibility` + +`parseVisibility`(`prepare.cppm:2701`)与 `DependencyEdge::visibility`(`:2670`)**默认都是 Public**。「放进 `publicUsage`、沿 public 边传」落地后就是默认传播,与「必须显式声明」直接冲突。 + +同时更正设计初稿里一处对本仓库的事实误述:manifest 里的 `include_dirs` **并非默认 private**——`prepare.cppm:2868` 把 `publicUsage.includeDirs = privateBuild.includeDirs`,它们默认就是 public 的;真正 private-only 的是 `build.mcpp` 注入的那些(`:2742`)。所以「和 include_dirs 同一条纪律」这个类比不成立,`reexport` 必须是独立的、默认关闭的位。 + +##### 传播规则(不动点,与 include dirs 同构) + ``` +own(P→D) = 该边上请求的 tools / host-module,以及 D 本身的目录 +exported(P) = ⋃ {P→D : edge.reexport} [ own(P→D) ∪ exported(D) ] +visible(P) = ⋃ {P→D} [ own(P→D) ∪ exported(D) ] +``` + +即:一条边写了 `reexport`,就把「这条边提供的东西,以及 D 转手给我的东西」继续交给 P 的消费者;而 P 自己总能看见所有直接依赖提供给它的东西。单调、可用不动点求解,和 `prepare.cppm:2906` 那个循环同形。 + +三条必须写死的语义: + +1. **传播的是可见性,不是自动执行。** `dep_bin()` 只返回路径,跑不跑由消费者的 `build.mcpp` 决定。传播不改变「谁构建了这个工具」,也不改变 tool store 的键。 +2. **provision 的存在性绑定 feature 激活**,与 `featureDefines`(`prepare.cppm:3812`)同一时机。feature 关闭时不应有任何工具被构建。 +3. **`reexport` 只在依赖侧有意义。** root 写它无害但无效果(root 没有消费者)。 + +#### D1.3 裸名:走索引那套命名空间阶梯,而不是「谁最后写谁赢」 + +`env_var_name`(`tool_store.cppm:154`)今天同时发长名与短名:`compat.protobuf` → `MCPP_DEP_COMPAT_PROTOBUF_BIN_PROTOC` 与 `MCPP_DEP_PROTOBUF_BIN_PROTOC`。今天只有 root 亲自声明的工具进环境,撞车在用户眼皮底下;传递传播之后,两条互不相识的库各自提供同名短名工具时,谁赢取决于 vector 的追加顺序,且无任何诊断。这从另一扇门放回了本设计要保住的「版本错配不可表达」性质。 -- 放进 `privateBuild` → 只有该包自己的 `build.mcpp` 能用; -- 放进 `publicUsage` → 沿 **public 边**传给消费者。 +采用与包身份解析同一套机制:**FQN 是标识,裸名是缺省形式,按固定阶梯解析。** -于是 `grpc` 可以在描述符里声明「我的 codegen feature 对外提供 protoc 与 grpc_cpp_plugin」,消费者只写一条依赖。 +- 全限定变量 `MCPP_DEP___BIN_` **总是**发布,永远无歧义; +- 裸名变量 `MCPP_DEP__BIN_` 按阶梯选出唯一归属: -**三条必须写死的语义**,否则这会变成一个安全与可维护性的洞: + 1. `(mcpplibs, X)` —— `kDefaultNamespace` + 2. `(compat, X)` —— `kCompatNamespace` + 3. 其余候选中恰好只剩一个 + 4. 否则不发布裸名变量 -1. **传播的是「可见性」,不是「自动执行」。** `dep_bin()` 只返回路径;跑不跑由消费者的 `build.mcpp` 决定。传播不改变「谁构建了这个工具」,也不改变 tool store 的键。 -2. **必须显式声明,不能默认传播。** 默认传播意味着任意深层依赖都能往消费者的工具命名空间里塞东西——那是供应链问题。库要对外提供,必须自己写明(与 `include_dirs` 默认 private、要 public 得显式是同一条纪律)。 -3. **命名冲突用包名消歧**,`dep_bin(pkg, tool)` 本来就是两段式,无需新语法。 +- 阶梯在第 1/2 级破除了平局(即同名候选不止一个)时,发一条 `provisions/ambiguous` 诊断,写明胜出者与全部候选的 FQN; +- 第 4 级不发布裸名,并把同一条诊断升级为「请改用全限定名」。 -> 顺带修掉一个相邻缺陷:`dep_dir()` 目前只覆盖**直接**依赖,所以传递依赖的数据文件目录取不到(protoc 的 well-known types 就是这么一个目录)。它应与 tools 走同一条传播规则。 +这与索引解析裸包名的阶梯(`(mcpplibs,X) → (compat,X) → (∅,X)`)是同一条规则的同一次应用;第 3 级是必需的补充,否则 `grpc.grpc-plugin` 这类非默认命名空间的包连裸名都拿不到(grpc-m 现在写的正是 `dep_bin("grpc-plugin", ...)`)。`dep_dir()` 沿用同一套。 ### D2. 输入种类进表,指纹由种类决定 +新增一种输入:**glob**。 + ```cpp -enum class InputKind { - File, // 内容哈希(现有) - Directory, // 递归成员集合:相对路径 + size + mtime,不读内容 - Env, // 环境变量(现有) -}; +mcpp::rerun_if_changed_glob("proto/**/*.proto"); ``` -`Directory` 的指纹**只取集合**,不取内容——内容变化由集合里的 `File` 条目负责。这与 Cargo 的 `cargo:rerun-if-changed=` 是同一个解。 +选 glob 而不是「目录 + 可选过滤器」,因为 glob 严格包含目录(`proto/**` 就是目录形态),并且与用户已经在写的 `sources = ["src/**/*.cppm"]` 是同一个概念、同一套匹配器(`scanner.cppm::path_matches_glob` 已支持 `**` 与 `*`)。 -补上之后 glob 从「结构性不安全」变成一等用法,规则包才能提供 `generate_all()`: +**指纹只取排序后的相对路径集合**,不含 mtime、不含 size、不含内容: -```cpp -mcpp::rerun_if_changed_dir("proto"); // 集合变了就重跑 +- size 变化 ⊂ 内容变化,已由 `File` 条目覆盖,纳入只带来误重跑; +- mtime 在 git checkout、容器构建、rsync 下不稳定,而本仓库已经在 `file_time_type` 的 epoch 上摔过一次; +- 「我依赖这里有哪些文件」的语义正好是路径集合,不多不少。 + +必须写死的三条: + +1. **排序与规范化**:`generic_string()` + 字节序排序,否则不同平台的目录遍历顺序会让同一棵树算出不同指纹。 +2. **永不走进构建输出目录与 `.git`。** `mcpp:generated=` 的产物落在 `target/` 下,在项目树内;`rerun_if_changed_glob("**")` 若把它算进去,集合每次都变,build.mcpp 每次重跑。这是 Cargo 的经典坑,必须在引擎侧堵死而不是靠用户写对模式。 +3. **不跟随符号链接**,与既有扫描一致。 + +代价:误重跑的上限是一次 build.mcpp 重跑(秒级),不是全量重编。 + +配套的版本处理(初稿遗漏): + +- `directives::kProtocolVersion` → 2:新增了程序可依赖的 directive; +- `directives::kCacheEpoch` → 2:缓存记录多了一类行,旧引擎读到新条目会忽略它并误判为新鲜。 + +### D3. B3:让库能按平台裁剪 provision,并让子构建说出真话 + +D1 把「是否请求 host 工具」的决定权从用户搬给了库。Windows 上 `compat.protobuf` 不声明 protoc 目标(工具子构建在 Windows 上有一个尚未定位的缺陷,见 #359 的相邻条目),于是库一旦无条件声明,Windows 用户会撞上 `prepare.cppm:4104` 那条硬错:`dependency 'compat.protobuf' has no kind = "bin" target named 'protoc'`。这会让 D1 在 Windows 上从「4 条降到 1 条」变成「本来能构建的现在报错」。 + +采取两条小改动,而不是把 D1 压在一个开放式排查后面: + +**D3a — `[target..feature-deps.]`。** +`ConditionalConfig` 今天已经带 `dependencies` / `devDependencies` / `buildDependencies`,唯独缺 `featureDeps`。这正是该类型自己的注释记录过的失败形状:「条件读取器维护自己的一份键子集,落后了也没人发现」(#258 修的是 `BuildInputs`)。补上它是在补一个已知形状的洞,而不是为一个 issue 加特性。补上后,库可以写: + +```toml +[target.'cfg(not(windows))'.feature-deps.codegen] +"compat.protobuf" = { version = "35.1", tools = ["protoc"], reexport = true } ``` -**代价要写明**:目录指纹用 mtime,而 mtime 在某些场景(容器构建、git checkout)不稳定。因此: -- 只把**成员集合**纳入指纹,不把内容纳入 → 误重跑的代价只是一次 build.mcpp 重跑(秒级),不是全量重编; -- 不递归进符号链接(与既有扫描一致)。 +Windows 上 feature 仍可激活,但不请求工具,于是不触发硬错;用户看到的是 grpcgen 那条自带修复建议的诊断——与今天 Windows 上的处境相同。**D1 因此在 Windows 上是中性的,在其余平台是净收益。** + +**D3b — 工具子构建必须透出内层的真实错误。** +Windows 那个缺陷至今未定位的直接原因是内层 ninja 的输出被汇总吞掉,真正的 scan 报错从未进入日志。透出它是小改动,并且是「将来能定位」的前提。缺陷本身仍然单独开 issue。 -### D3. 为什么这两条必须一起做 +### D4. 为什么这些必须一起做 只做 D1:用户从 4 条降到 1 条,但仍要在 `build.mcpp` 里逐个列 `.proto`。 只做 D2:用户不必列 proto,但仍要写 4 条依赖并知道 gRPC 需要 protobuf 的 protoc。 - -**两条合起来**才是目标形态,也才是「对齐并超过业界」的那一步。 +不做 D3:D1 在 Windows 上是倒退。 ## 4. 实施步骤 | 步 | 内容 | 风险 | |---|---|---| -| 1 | `UsageRequirements` 加 tools / hostModules 两个字段,`privateBuild` 行为保持今天不变 | 低,纯新增 | -| 2 | 沿 public 边聚合(复用 features 的边聚合路径,#242/#243 已有先例) | 中——要确认不会把 private 依赖的工具泄漏出去 | -| 3 | 描述符/manifest 侧:声明「对外提供」的语法 | 中——是新的用户可见语法,需按 Schema Ownership Principle 审 | -| 4 | `dep_dir()` 覆盖传递依赖 | 低 | -| 5 | `InputKind` 表 + `Directory` 指纹 + `rerun_if_changed_dir` | 低 | -| 6 | grpc-m 侧改成 1 条依赖 + `generate_all()`,作为真实验证 | —— | - -步 1–4 是缺口 A,步 5 是缺口 B,步 6 是端到端证据。 +| 1 | `src/build/provisions.cppm`:`ProvisionKind` 表、env 命名、裸名阶梯、传播不动点 | 低,纯新增 | +| 2 | `DependencySpec::reexport` + toml/xpkg 两个解析器 | 低 | +| 3 | `prepare.cppm` 接入:tools / host-module / dep_dir 三处改用不动点结果 | 中——须确认未 `reexport` 的边不外泄 | +| 4 | `ConditionalConfig::featureDeps` + prepare 侧合并(D3a) | 低,与既有条件依赖同路径 | +| 5 | 子构建错误透出(D3b) | 低 | +| 6 | `rerun-if-changed-glob` + 指纹 + protocol/epoch bump + `hostprogram` API | 中——缓存格式变更 | +| 7 | grpc-m 侧改成 1 条依赖 + `generate_all()`,作为真实验证 | ——(需要本设计先发布) | ## 5. 验证 -- **单测**:传播规则(private 不外泄、public 沿边传、冲突消歧)、目录指纹(增删文件变、改内容不变、mtime 抖动不误伤集合)。 -- **e2e**:一个库对外提供工具 + 一个消费者只写一条依赖就能在 `build.mcpp` 里 `dep_bin` 到;新增一个文件后 glob 场景确实重跑。 +- **单测**:传播规则(未 `reexport` 不外泄、`reexport` 沿链传递、feature 关闭时不存在);裸名阶梯(默认命名空间优先、compat 次之、唯一候选兜底、歧义不发布裸名);glob 指纹(增删文件变、改内容不变、输出目录不参与、跨平台排序一致)。 +- **e2e**:一个库 `reexport` 工具 + 一个消费者只写一条依赖就能 `dep_bin` 到;两个不同命名空间的同名包同时提供工具时裸名不被静默绑定;新增一个文件后 glob 场景确实重跑;`[target.*.feature-deps]` 在非匹配平台上不引入依赖。 - **真实场景**:grpc-m 的模板降到 1 条依赖 + 1 行 build.mcpp,且生成产物仍与官方 protoc 逐字节相同(该基线已在 2026.8.5.x 建立)。 ## 6. 明确不做 -- **不让传播默认开启**。库必须显式声明对外提供,理由见 D1 第 2 条。 -- **不把目录内容纳入指纹**。那会把一次 build.mcpp 重跑放大成全量重编,而收益为零(内容变化本来就由 File 条目覆盖)。 -- **不引入「工具版本独立于依赖版本」的语法**。单一版本轴正是「错配不可表达」的来源,是本设计要保住的性质。 -- **不在本轮解决 windows 的工具子构建失败**(见 mcpp-index 的 compat.protobuf windows 块):那是独立缺陷,原因尚未定位。 +- **不让传播默认开启**。理由见 D1.2。 +- **不把目录内容或 mtime/size 纳入 glob 指纹**。理由见 D2。 +- **不引入「工具版本独立于依赖版本」的语法**。单一版本轴正是「错配不可表达」的来源。 +- **不在本轮定位 Windows 的工具子构建失败**。D3b 只让它可被观测,缺陷本身单独开 issue。 +- **不做 per-kind 的 `reexport` 粒度**。per-edge 已经够用。 diff --git a/.github/actions/bootstrap-mcpp/action.yml b/.github/actions/bootstrap-mcpp/action.yml index 543818e8..072f5726 100644 --- a/.github/actions/bootstrap-mcpp/action.yml +++ b/.github/actions/bootstrap-mcpp/action.yml @@ -25,7 +25,7 @@ inputs: # `package.name`, so one of the two was simply unreachable — and which one # depended on the machine, which is why CI failed on `compat:lua` on # Windows and `mcpplibs.capi:lua` on Linux. Never pin below that. - default: '2026.8.5.2' + default: '2026.8.6.2' cache-target: description: also restore/save target/ (build artifacts + BMIs) required: false diff --git a/.github/actions/setup-macos-llvm/action.yml b/.github/actions/setup-macos-llvm/action.yml index e518e61f..e83a8286 100644 --- a/.github/actions/setup-macos-llvm/action.yml +++ b/.github/actions/setup-macos-llvm/action.yml @@ -15,7 +15,7 @@ inputs: # Floor imposed by the index, not a routine bump — see # .github/actions/bootstrap-mcpp/action.yml for why 0.4.69 is required # (two packages named `lua` in one repo need openxlings/xlings#381). - default: '2026.8.5.2' + default: '2026.8.6.2' runs: using: composite diff --git a/.github/workflows/bootstrap-macos.yml b/.github/workflows/bootstrap-macos.yml index cd17ad0b..bb2f34fc 100644 --- a/.github/workflows/bootstrap-macos.yml +++ b/.github/workflows/bootstrap-macos.yml @@ -17,7 +17,7 @@ jobs: # Dormant (workflow_dispatch only), but kept in step with the rest — # check_version_pins.sh holds it there. Floor: 0.4.69, below which the # index cannot resolve two packages that share a short name. - XLINGS_VERSION: '2026.8.5.2' + XLINGS_VERSION: '2026.8.6.2' steps: - uses: actions/checkout@v4 diff --git a/.github/workflows/ci-fresh-install.yml b/.github/workflows/ci-fresh-install.yml index 27ddb4ef..6f751c40 100644 --- a/.github/workflows/ci-fresh-install.yml +++ b/.github/workflows/ci-fresh-install.yml @@ -152,7 +152,7 @@ jobs: env: XLINGS_NON_INTERACTIVE: '1' run: | - curl -fsSL https://raw.githubusercontent.com/openxlings/xlings/main/tools/other/quick_install.sh | bash -s v2026.8.5.2 + curl -fsSL https://raw.githubusercontent.com/openxlings/xlings/main/tools/other/quick_install.sh | bash -s v2026.8.6.2 echo "$HOME/.xlings/subos/current/bin" >> "$GITHUB_PATH" - name: Install mcpp and config mirror @@ -292,7 +292,7 @@ jobs: - name: Install xlings + mcpp run: | - curl -fsSL https://raw.githubusercontent.com/openxlings/xlings/main/tools/other/quick_install.sh | bash -s v2026.8.5.2 + curl -fsSL https://raw.githubusercontent.com/openxlings/xlings/main/tools/other/quick_install.sh | bash -s v2026.8.6.2 # Deliberately NOT writing to $GITHUB_PATH here. On container # images that declare no PATH in their config (opensuse/ # tumbleweed), appending a single dir to GITHUB_PATH makes the @@ -363,7 +363,7 @@ jobs: # (older ones carry minos=15 and refuse to start). # v0.4.51+: in-process sha256 — this image has no sha256sum # binary, so pinned fetches failed before it. - curl -fsSL https://raw.githubusercontent.com/openxlings/xlings/main/tools/other/quick_install.sh | bash -s v2026.8.5.2 + curl -fsSL https://raw.githubusercontent.com/openxlings/xlings/main/tools/other/quick_install.sh | bash -s v2026.8.6.2 echo "$HOME/.xlings/subos/current/bin" >> "$GITHUB_PATH" - name: Install mcpp and config mirror diff --git a/.github/workflows/ci-linux-e2e.yml b/.github/workflows/ci-linux-e2e.yml index 3bd7636e..c395596d 100644 --- a/.github/workflows/ci-linux-e2e.yml +++ b/.github/workflows/ci-linux-e2e.yml @@ -123,7 +123,7 @@ jobs: - name: Bootstrap xlings + released mcpp run: | - curl -fsSL https://raw.githubusercontent.com/openxlings/xlings/main/tools/other/quick_install.sh | bash -s v2026.8.5.2 + curl -fsSL https://raw.githubusercontent.com/openxlings/xlings/main/tools/other/quick_install.sh | bash -s v2026.8.6.2 export PATH="$HOME/.xlings/subos/current/bin:$PATH" xlings update xlings install mcpp -y -g diff --git a/.github/workflows/cross-build-test.yml b/.github/workflows/cross-build-test.yml index 43b7744d..78e0f16b 100644 --- a/.github/workflows/cross-build-test.yml +++ b/.github/workflows/cross-build-test.yml @@ -118,7 +118,7 @@ jobs: # release assets were uploaded in a broken state (records present, # blobs missing → 404 on GET); re-uploaded clean. The stale-INDEX # half is handled by the marker-clear below. - XLINGS_VERSION: '2026.8.5.2' + XLINGS_VERSION: '2026.8.6.2' run: | tarball="xlings-${XLINGS_VERSION}-linux-x86_64.tar.gz" curl -fsSL -o "/tmp/${tarball}" \ @@ -255,7 +255,7 @@ jobs: - name: Bootstrap mcpp via xlings env: XLINGS_NON_INTERACTIVE: '1' - XLINGS_VERSION: '2026.8.5.2' + XLINGS_VERSION: '2026.8.6.2' run: | tarball="xlings-${XLINGS_VERSION}-linux-x86_64.tar.gz" curl -fsSL -o "/tmp/${tarball}" \ diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index a66d4695..e7df99b9 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -96,7 +96,7 @@ jobs: # Pin xlings to a known-good version. The upstream install # script always grabs `latest` (no version override), so we # download + self-install manually to avoid broken releases. - XLINGS_VERSION: '2026.8.5.2' + XLINGS_VERSION: '2026.8.6.2' run: | if [ ! -x "$HOME/.xlings/subos/default/bin/xlings" ]; then tarball="xlings-${XLINGS_VERSION}-linux-x86_64.tar.gz" @@ -288,7 +288,7 @@ jobs: - name: Bootstrap mcpp via xlings env: XLINGS_NON_INTERACTIVE: '1' - XLINGS_VERSION: '2026.8.5.2' + XLINGS_VERSION: '2026.8.6.2' run: | tarball="xlings-${XLINGS_VERSION}-linux-x86_64.tar.gz" curl -fsSL -o "/tmp/${tarball}" \ @@ -358,11 +358,11 @@ jobs: # below are pinned to the same version as XLINGS_VERSION; they are # NOT interpolated from it, so check_version_pins.sh scans for them # explicitly (they were absent from the old lock-step comment). - XLA="xlings-2026.8.5.2-linux-aarch64.tar.gz" + XLA="xlings-2026.8.6.2-linux-aarch64.tar.gz" if curl -fsSL -o "/tmp/$XLA" \ - "https://github.com/openxlings/xlings/releases/download/v2026.8.5.2/$XLA"; then + "https://github.com/openxlings/xlings/releases/download/v2026.8.6.2/$XLA"; then tar -xzf "/tmp/$XLA" -C /tmp - XLBIN=$(find /tmp/xlings-2026.8.5.2-linux-aarch64 -path '*/bin/xlings' -type f | head -1) + XLBIN=$(find /tmp/xlings-2026.8.6.2-linux-aarch64 -path '*/bin/xlings' -type f | head -1) if [ -n "$XLBIN" ]; then mkdir -p "$STAGING/$WRAPPER/registry/bin" cp "$XLBIN" "$STAGING/$WRAPPER/registry/bin/xlings" @@ -440,7 +440,7 @@ jobs: - name: Bootstrap mcpp via xlings env: XLINGS_NON_INTERACTIVE: '1' - XLINGS_VERSION: '2026.8.5.2' + XLINGS_VERSION: '2026.8.6.2' run: | if [ ! -x "$HOME/.xlings/subos/default/bin/xlings" ]; then WORK=$(mktemp -d) @@ -622,7 +622,7 @@ jobs: shell: bash env: XLINGS_NON_INTERACTIVE: '1' - XLINGS_VERSION: '2026.8.5.2' + XLINGS_VERSION: '2026.8.6.2' run: | # Captured before the `cd` below, in POSIX form: this step never # returns to the workspace, and GITHUB_WORKSPACE is a backslash diff --git a/docs/05-mcpp-toml.md b/docs/05-mcpp-toml.md index e5c3baf1..eee57bb9 100644 --- a/docs/05-mcpp-toml.md +++ b/docs/05-mcpp-toml.md @@ -596,7 +596,9 @@ aliases `windows` / `linux` / `macos` / `unix` are never valid target triples, s there is no ambiguity. Use the bare form for a single OS/family; use `cfg(...)` when you need arch/env conditions or combinators. -- **Keys**: `dependencies` / `dev-dependencies` / `build-dependencies`, and +- **Keys**: `dependencies` / `dev-dependencies` / `build-dependencies` / + `feature-deps.` (mcpp 2026.8.6.2+ — see §2.14; the feature is + registered unconditionally, only its dependency set is scoped), and `build` with `cflags` / `cxxflags` / `ldflags` / `sources` (mcpp 0.0.95+ — conditional source globs, e.g. gating `src/x86/**/*.asm` behind `cfg(arch = "x86_64")`; `!`-exclusion globs work here too), plus `flags` and @@ -1064,6 +1066,72 @@ else — the same separation Cargo draws with `[build-dependencies]`. (Before 2026.8.5.2 it was also built as an ordinary library, which made `import mcpp;` inside a rule fail: the bundled module does not exist in that second compile.) +#### `reexport = true` — a library standing up a toolchain for its user (2026.8.6.2+) + +Everything above is declared by whoever *uses* the tool. That is the wrong +place when the knowledge belongs to a library: gRPC's code generation needs +protobuf's `protoc`, and no user of a gRPC package should have to know that. + +`reexport = true` hands an edge's build-time provisions — its `tools`, its +`host-module`, and the dependency's directory — to **this package's own +consumers**: + +```toml +# inside the grpc package's manifest +[feature-deps.codegen] +"compat.protobuf" = { version = "35.1", tools = ["protoc"], reexport = true } +grpc-plugin = { version = "1.83.0", tools = ["grpc_cpp_plugin"], reexport = true } +grpcgen = { version = "1.83.0", host-module = true, reexport = true } +``` + +Its user then writes one line, and imports the rule: + +```toml +[dependencies] +grpc = { version = "1.83.0", features = ["codegen"] } +``` + +```cpp +// build.mcpp +import mcpp; +import grpcgen; +int main() { return grpcgen::generate_all() ? 0 : 1; } +``` + +- **Off by default, and deliberately not the edge's `visibility`.** `visibility` + already defaults to `"public"`, so riding it would let any dependency at any + depth put entries into your build program's tool namespace without saying so. + Handing something to your consumers is a supply-chain statement; it has to be + written down. +- **One hop per declaration.** A re-exported provision reaches the consumers of + the package that declared it. For it to travel further, the next package must + re-export in turn — each package decides only what *it* hands on. +- **Visibility, not execution.** `dep_bin()` returns a path; whether anything + runs is still the consumer's `build.mcpp`'s decision. Nothing changes about + who builds the tool or how the tool store is keyed. +- **Unqualified names are resolved by a ladder, not by luck.** Once two + libraries can re-export, both may offer the tail `protobuf`. The + fully-qualified `MCPP_DEP___BIN_` is always published; the + bare spelling is bound to `mcpplibs.`, else `compat.`, else an + unnamespaced ``, else the single remaining candidate — and when it is + contested mcpp says so instead of picking silently. + +##### Scoping a provision per platform + +A package may declare a `bin` target on some platforms only. Because the +*library* now decides what is requested, an unconditional request turns an +unsupported platform into an error its user cannot edit away. Scope it: + +```toml +[target.'cfg(not(windows))'.feature-deps.codegen] +"compat.protobuf" = { version = "35.1", tools = ["protoc"], reexport = true } +``` + +`[target..feature-deps.]` (2026.8.6.2+) follows the same rules as +the other conditional dependency tables (§2.7.1). The **feature itself is +registered on every platform** — only what it pulls in is conditional — so +requesting it where no predicate matches is not an unknown-feature error. + ## Appendix A. Schema Ownership Principle (admission criteria for new fields) > **Closed syntax, open vocabulary**: whoever owns the parsing semantics defines the keys; whoever owns the domain knowledge defines the values. diff --git a/docs/07-build-mcpp.md b/docs/07-build-mcpp.md index eb90a4d9..b2a66b7c 100644 --- a/docs/07-build-mcpp.md +++ b/docs/07-build-mcpp.md @@ -97,6 +97,7 @@ int main() { | `mcpp::source(p)` | `mcpp:source=` | | `mcpp::include_dir(d)` / `mcpp::include_dir_after(d)` | `mcpp:include-dir=` / `mcpp:include-dir-after=` | | `mcpp::rerun_if_changed(p)` / `mcpp::rerun_if_env_changed(v)` | the matching `rerun-*` directives | +| `mcpp::rerun_if_changed_glob(pat)` *(2026.8.6.2+)* | `mcpp:rerun-if-changed-glob=` — re-run when the **set** of files matching `pat` changes (see below) | | `mcpp::dep_bin(pkg, tool)` *(2026.8.5.1+)* | reads `MCPP_DEP__BIN_` — the absolute path of a **host tool** built by a dependency (see below) | | `mcpp::action{…}.submit()` *(2026.8.5.1+)* | `mcpp:action=` — declares a **build-graph node** instead of doing the work here (see below) | @@ -123,7 +124,37 @@ mcpp builds that `kind = "bin"` target **for the build machine** (even under `mcpp.toml` rather than here for the same reason a dependency does: asking the graph for an extra artifact is a graph-level request, and the graph stays statically analysable. See [05 §2.14](05-mcpp-toml.md) for the full contract, -including `[tools.overrides]`. +including `[tools.overrides]` and `reexport = true` (which is how a library +hands you the whole toolchain so you declare **one** dependency instead of +four). + +### Globbing your inputs: `rerun_if_changed_glob` (2026.8.6.2+) + +The re-run key is built from *declared* inputs. Declare files and it works; +glob a directory and it does not — adding a `.proto` changes no declared file's +hash, so the program never re-runs and the new file is silently never +generated. `rerun_if_changed_glob` is how a program says "my output depends on +which files are here": + +```cpp +import mcpp; +int main() { + mcpp::rerun_if_changed_glob("proto/**/*.proto"); + // … scan the directory, declare one action per file … +} +``` + +The pattern is relative to the manifest directory and uses the same `*` / `**` +grammar as `sources = [...]`. Its fingerprint is the **sorted set of matching +paths** and nothing else: + +- **not contents** — a file whose bytes matter is an ordinary + `rerun_if_changed` input, which already hashes them; +- **not mtime or size** — mtime is unstable across `git checkout`, container + builds and `rsync`, and size is a weaker signal than the hash above. + +The build output tree and `.git` are never part of the set, so a wide pattern +cannot make the program re-run forever against its own outputs. ### Declaring work instead of doing it: `mcpp::action` (2026.8.5.1+) diff --git a/docs/zh/05-mcpp-toml.md b/docs/zh/05-mcpp-toml.md index ddffe660..54d00b98 100644 --- a/docs/zh/05-mcpp-toml.md +++ b/docs/zh/05-mcpp-toml.md @@ -812,6 +812,64 @@ lib 根必须在 `src/.cppm`(或 `[lib] path` 指向的位置);缺失时 划出的是同一条界线。(2026.8.5.2 之前它还会被当作普通库再编一遍,这正是规则里 `import mcpp;` 失败的原因:在那第二次编译里内置模块并不存在。) +#### `reexport = true` —— 由库替用户拉起整条工具链(2026.8.6.2+) + +上面这些都由**使用工具的人**声明。当知识本来属于库时,这个位置就错了:gRPC +的代码生成需要 protobuf 的 `protoc`,而 gRPC 包的任何使用者都不应该知道这件事。 + +`reexport = true` 把一条边上的构建期提供物 —— 它的 `tools`、它的 +`host-module`、以及该依赖的目录 —— 交给**本包自己的消费者**: + +```toml +# 写在 grpc 包自己的 manifest 里 +[feature-deps.codegen] +"compat.protobuf" = { version = "35.1", tools = ["protoc"], reexport = true } +grpc-plugin = { version = "1.83.0", tools = ["grpc_cpp_plugin"], reexport = true } +grpcgen = { version = "1.83.0", host-module = true, reexport = true } +``` + +于是使用者只写一行,再 import 那个规则: + +```toml +[dependencies] +grpc = { version = "1.83.0", features = ["codegen"] } +``` + +```cpp +// build.mcpp +import mcpp; +import grpcgen; +int main() { return grpcgen::generate_all() ? 0 : 1; } +``` + +- **默认关闭,并且刻意不复用边上的 `visibility`。** `visibility` 本身默认就是 + `"public"`,搭它的车意味着任意深度的依赖都能不声不响地往你的构建程序的工具 + 命名空间里塞东西。「把某样东西交给消费者」是一条供应链主张,必须写下来。 +- **一次声明只走一跳。** 被再导出的提供物到达声明它的那个包的消费者;要继续 + 往上走,下一个包必须自己也写 `reexport`。每个包只决定**它**交出什么。 +- **传播的是可见性,不是执行。** `dep_bin()` 只返回路径,跑不跑仍由消费者的 + `build.mcpp` 决定;谁构建了这个工具、tool store 怎么做键,都不改变。 +- **裸名由阶梯决定,而不是靠运气。** 一旦两个库都能再导出,它们可能同时提供 + 尾名 `protobuf`。全限定的 `MCPP_DEP___BIN_` 总是发布;裸名 + 依次绑定到 `mcpplibs.`、`compat.`、无命名空间的 ``,最后才是「剩下 + 的唯一候选」——存在争用时 mcpp 会说出来,而不是默默选一个。 + +##### 按平台裁剪提供物 + +一个包可能只在部分平台声明 `bin` 目标。既然现在是**库**决定请求什么,无条件的 +请求就会把「不支持的平台」变成用户改不掉的硬错。用条件段裁剪: + +```toml +[target.'cfg(not(windows))'.feature-deps.codegen] +"compat.protobuf" = { version = "35.1", tools = ["protoc"], reexport = true } +``` + +`[target..feature-deps.]`(2026.8.6.2+)与 `[target.]` 下的 +其余依赖表(`dependencies` / `dev-dependencies` / `build-dependencies`)遵循同 +一套谓词规则,针对**解析后的 target** 求值。**feature 本身在所有平台都注册** +—— 只有它拉进来的东西是条件性的 —— 因此在没有任何谓词匹配的平台上请求它,不是 +「未知 feature」错误。 + ## 附录 A. Schema 所有权原则(新字段准入标准) > **语法封闭,词汇开放**:谁拥有解析语义谁定义键;谁拥有领域知识谁定义值。 diff --git a/docs/zh/07-build-mcpp.md b/docs/zh/07-build-mcpp.md index e12c254d..2a403159 100644 --- a/docs/zh/07-build-mcpp.md +++ b/docs/zh/07-build-mcpp.md @@ -90,6 +90,7 @@ int main() { | `mcpp::source(p)` | `mcpp:source=` | | `mcpp::include_dir(d)` / `mcpp::include_dir_after(d)` | `mcpp:include-dir=` / `mcpp:include-dir-after=` | | `mcpp::rerun_if_changed(p)` / `mcpp::rerun_if_env_changed(v)` | 对应的 `rerun-*` 指令 | +| `mcpp::rerun_if_changed_glob(pat)` *(2026.8.6.2+)* | `mcpp:rerun-if-changed-glob=` —— 匹配 `pat` 的文件**集合**发生变化时重跑(见下) | | `mcpp::dep_bin(pkg, tool)` *(2026.8.5.1+)* | 读 `MCPP_DEP__BIN_` —— 依赖构建出的 **host 工具**的绝对路径(见下) | | `mcpp::action{…}.submit()` *(2026.8.5.1+)* | `mcpp:action=` —— **声明一个构建图节点**,而不是在这里把活干了(见下) | @@ -114,7 +115,34 @@ int main() { mcpp 会**为构建机器**构建那个 `kind = "bin"` target(即使在 `--target` 下), 全局缓存,并把路径交给你。这个请求写在 `mcpp.toml` 而不是这里,理由和依赖本身 一样:向依赖图索取一个额外产物是**图级别**的请求,而图必须保持可静态分析。 -完整契约(含 `[tools.overrides]`)见 [05 §2.14](05-mcpp-toml.md)。 +完整契约(含 `[tools.overrides]` 与 `reexport = true` —— 库据此把整条工具链交给 +你,于是你只写**一条**依赖而不是四条)见 [05 §2.14](05-mcpp-toml.md)。 + +### 用通配符声明输入:`rerun_if_changed_glob`(2026.8.6.2+) + +重跑键由**声明过的**输入构成。声明具体文件是可行的;而 glob 一个目录不可行 —— +新增一个 `.proto` 不改变任何已声明文件的哈希,于是程序不重跑,新文件静默地永远 +不被生成。`rerun_if_changed_glob` 就是程序用来说「我的输出取决于这里有哪些文件」 +的方式: + +```cpp +import mcpp; +int main() { + mcpp::rerun_if_changed_glob("proto/**/*.proto"); + // … 扫描目录,为每个文件声明一条 action … +} +``` + +模式相对 manifest 目录,`*` / `**` 的文法与 `sources = [...]` 完全一致。它的指纹 +是**排序后的匹配路径集合**,不含其他任何东西: + +- **不含内容** —— 字节内容重要的文件本来就该用 `rerun_if_changed` 声明,那条 + 条目已经在哈希它; +- **不含 mtime 与 size** —— mtime 在 `git checkout`、容器构建、`rsync` 下都不 + 稳定,而 size 是比上面那个哈希更弱的信号。 + +构建输出目录与 `.git` 永远不进入集合,因此再宽的模式也不会让程序对着自己的产物 +无限重跑。 ### 声明工作而不是干活:`mcpp::action`(2026.8.5.1+) diff --git a/mcpp.toml b/mcpp.toml index 54d943b5..8adfdcf2 100644 --- a/mcpp.toml +++ b/mcpp.toml @@ -1,6 +1,6 @@ [package] name = "mcpp" -version = "2026.8.6.1" +version = "2026.8.6.2" description = "Modern C++ build & package management tool" license = "Apache-2.0" authors = ["mcpp-community"] diff --git a/src/build/build_program.cppm b/src/build/build_program.cppm index f878a4db..6af390da 100644 --- a/src/build/build_program.cppm +++ b/src/build/build_program.cppm @@ -85,6 +85,23 @@ std::expected run_build_program( const mcpp::manifest::CppStandardConfig& cppStandard, const BuildProgramEnv& env); +// #359: has any recorded glob input's path SET changed since its build.mcpp +// cache was written? +// +// The project-level fast path skips prepare_build entirely when no source is +// newer than build.ninja, and prepare is where the build.mcpp cache is +// normally consulted. A glob input is precisely an input whose change leaves +// every existing file's mtime alone — adding a .proto — so without this ask, +// the fast path would report "Finished dev in 0.00s" and the new file would +// never be generated. That is the same gap the fast path already closes for +// the build.mcpp source itself; a glob is one more kind of build-program input, +// so it belongs to the same question. +// +// Scans the caches under `/target/.build-mcpp` (the root's own and +// each dependency's). Each cache records the root its globs were relative to, +// so a dependency's glob is evaluated against the dependency's tree. +bool glob_inputs_stale(const std::filesystem::path& projectRoot); + } // namespace mcpp::build namespace mcpp::build { @@ -266,6 +283,20 @@ std::string contract_hash(const std::vector> return mcpp::toolchain::hash_string(s); } +// The project-relative name of the build output tree ("target" by default), so +// a glob input never walks into what a previous run produced. Empty when the +// output lives outside the project, in which case nothing needs excluding. +std::string output_dir_name(const fs::path& root, const fs::path& bdir) { + std::error_code ec; + auto rel = bdir.lexically_relative(root); + if (rel.empty()) return {}; + auto first = rel.begin(); + if (first == rel.end()) return {}; + auto name = first->string(); + if (name == "..") return {}; // outside the project + return name; +} + void write_cache(const fs::path& bdir, const fs::path& root, const std::string& programHash, const std::string& compilerHash, const std::string& ctxHash, @@ -279,10 +310,22 @@ void write_cache(const fs::path& bdir, const fs::path& root, os << "program " << programHash << '\n'; os << "compiler " << compilerHash << '\n'; os << "ctx " << ctxHash << '\n'; + // The root the relative entries below are resolved against. Recorded so a + // reader that is not prepare_build — the fast-path glob check — can + // evaluate a DEPENDENCY's cache against the dependency's own tree. + os << "root " << root.string() << '\n'; for (auto const& f : d.at(Slot::RerunFiles)) os << "in " << mcpp::toolchain::hash_file(abs_against_root(root, f)) << ' ' << f << '\n'; for (auto const& e : d.at(Slot::RerunEnv)) os << "env " << mcpp::toolchain::hash_string(env_value(e)) << ' ' << e << '\n'; + // #359: a glob input's fingerprint is the SET of matching paths. Same + // record shape as `in`/`env`; the value is computed by the table's owner. + { + auto outName = output_dir_name(root, bdir); + for (auto const& g : d.at(Slot::RerunGlobs)) + os << "glob " << dirs::glob_fingerprint(root, g, outName) << ' ' + << g << '\n'; + } dirs::serialize(os, d); } @@ -291,8 +334,10 @@ struct CacheRecord { std::string programHash; std::string compilerHash; std::string ctxHash; // contract env (target/profile/features/out-dir) + std::string rootPath; // what relative entries are resolved against std::vector> inputs; // (hash, path) std::vector> envs; // (hash, name) + std::vector> globs; // (hash, pattern) Directives directives; // A `d` record whose tag this mcpp does not know — the entry was written // by a newer mcpp. Replaying the rest would apply a strict subset of what @@ -320,11 +365,13 @@ CacheRecord read_cache(const fs::path& bdir) { else if (tag == "program") r.programHash = rest; else if (tag == "compiler") r.compilerHash = rest; else if (tag == "ctx") r.ctxHash = rest; - else if (tag == "in" || tag == "env") { + else if (tag == "root") r.rootPath = rest; + else if (tag == "in" || tag == "env" || tag == "glob") { auto sp2 = rest.find(' '); if (sp2 == std::string::npos) continue; std::string h = rest.substr(0, sp2), name = rest.substr(sp2 + 1); - (tag == "in" ? r.inputs : r.envs).emplace_back(h, name); + (tag == "in" ? r.inputs : tag == "env" ? r.envs : r.globs) + .emplace_back(h, name); } else if (tag == "d") { auto sp2 = rest.find(' '); if (sp2 == std::string::npos) continue; @@ -338,7 +385,7 @@ CacheRecord read_cache(const fs::path& bdir) { } // Decide whether the cached run is still valid (so we can skip recompiling/running). -bool cache_fresh(const fs::path& root, const CacheRecord& c, +bool cache_fresh(const fs::path& root, const fs::path& bdir, const CacheRecord& c, const std::string& programHash, const std::string& compilerHash, const std::string& ctxHash) { if (!c.loaded) return false; @@ -351,6 +398,14 @@ bool cache_fresh(const fs::path& root, const CacheRecord& c, if (mcpp::toolchain::hash_file(abs_against_root(root, path)) != h) return false; for (auto const& [h, name] : c.envs) if (mcpp::toolchain::hash_string(env_value(name)) != h) return false; + // #359: the path SET behind each declared glob. A file appearing or + // disappearing changes it; editing one does not (that is what the `in` + // entries above are for). + if (!c.globs.empty()) { + auto outName = output_dir_name(root, bdir); + for (auto const& [h, pattern] : c.globs) + if (dirs::glob_fingerprint(root, pattern, outName) != h) return false; + } // A declared output that vanished invalidates the cache. Driven off the // table's mustExistAfterRun so a future output-shaped directive is covered // without editing this function. @@ -425,7 +480,7 @@ std::expected run_build_program( // Fast path: declared inputs + contract unchanged → reapply cached // directives, no run. CacheRecord cache = read_cache(bdir); - if (cache_fresh(root, cache, programHash, compilerHash, ctxHash)) { + if (cache_fresh(root, bdir, cache, programHash, compilerHash, ctxHash)) { dirs::apply(m, cache.directives); mcpp::ui::info("build.mcpp", "up to date (cached)"); return {}; @@ -781,4 +836,30 @@ std::expected run_build_program( return {}; } +bool glob_inputs_stale(const fs::path& projectRoot) { + std::error_code ec; + const fs::path base = projectRoot / "target" / ".build-mcpp"; + if (!fs::exists(base, ec)) return false; + + // Depth-bounded: the root's cache sits at depth 0 and a dependency's at + // `deps//` (depth 2). Bounding it keeps this off the generated-output + // tree under `out/`, which can hold thousands of files and never holds a + // cache. + fs::recursive_directory_iterator it( + base, fs::directory_options::skip_permission_denied, ec); + if (ec) return false; + for (; it != fs::recursive_directory_iterator(); it.increment(ec)) { + if (ec) break; + if (it.depth() >= 3) { it.disable_recursion_pending(); continue; } + if (it->path().filename() != "build.mcpp.cache") continue; + auto rec = read_cache(it->path().parent_path()); + if (!rec.loaded || rec.globs.empty() || rec.rootPath.empty()) continue; + fs::path recRoot{rec.rootPath}; + auto outName = output_dir_name(recRoot, it->path().parent_path()); + for (auto const& [h, pattern] : rec.globs) + if (dirs::glob_fingerprint(recRoot, pattern, outName) != h) return true; + } + return false; +} + } // namespace mcpp::build diff --git a/src/build/directives.cppm b/src/build/directives.cppm index e03892ce..2ccfe0fb 100644 --- a/src/build/directives.cppm +++ b/src/build/directives.cppm @@ -41,6 +41,8 @@ import std; import mcpp.libs.json; import mcpp.manifest; import mcpp.toolchain.dialect; +import mcpp.toolchain.fingerprint; // hash_string for the glob fingerprint +import mcpp.modgraph.glob; // the one path-glob matcher export namespace mcpp::build::directives { @@ -56,7 +58,8 @@ export namespace mcpp::build::directives { // directive is added that a program may rely on. An engine seeing a HIGHER // number than this must refuse: it cannot know what it is being asked to do, // and "warn and ignore" would turn that into a silently different build. -inline constexpr int kProtocolVersion = 1; +// v2 (#359): adds `rerun-if-changed-glob`. +inline constexpr int kProtocolVersion = 2; // ── Cache-format epoch ───────────────────────────────────────────────────── // @@ -66,7 +69,10 @@ inline constexpr int kProtocolVersion = 1; // Deliberately NOT the mcpp release number: folding the whole version in would // re-run every build program on every release for nothing. Same discipline as // mcpp.build.cache_key::kCacheEpoch. -inline constexpr int kCacheEpoch = 1; +// Epoch 2 (#359): entries gained `glob` records. An engine that does not know +// them would replay a strict subset of the declared inputs and call a stale +// build fresh, which is exactly the silent-wrong-answer this guard exists for. +inline constexpr int kCacheEpoch = 2; // ── Run bound ────────────────────────────────────────────────────────────── // @@ -97,6 +103,12 @@ enum class Slot : std::size_t { IncludeDirsAfter, RerunFiles, RerunEnv, + // #359: an input that is a SET of files rather than one file. The + // fingerprint is the sorted list of matching relative paths — never their + // contents, sizes or timestamps. A program that globs (`proto/**/*.proto`) + // otherwise cannot express "re-run me when a file appears", because no + // declared file's hash changes and the new file is silently never built. + RerunGlobs, // Build-graph nodes (`mcpp:action=`). The value is a JSON payload rather // than a scalar: an action has six fields, and a flat `key=value` line // cannot carry them. The bundled `mcpp` module owns the encoding, which @@ -147,7 +159,7 @@ struct Def { int sinceProtocol; }; -inline constexpr std::array kTable{{ +inline constexpr std::array kTable{{ // wire tag slot scope transform must missingPrefix missingSuffix since {"cxxflag", "cxxflag", Slot::CxxFlags, Scope::PackagePrivate, Transform::Verbatim, false, "", "", 1}, {"cflag", "cflag", Slot::CFlags, Scope::PackagePrivate, Transform::Verbatim, false, "", "", 1}, @@ -160,6 +172,7 @@ inline constexpr std::array kTable{{ {"include-dir-after", "include-dir-after", Slot::IncludeDirsAfter, Scope::PackagePrivate, Transform::AbsPath, false, "", "", 1}, {"rerun-if-changed", "", Slot::RerunFiles, Scope::RerunKey, Transform::Verbatim, false, "", "", 1}, {"rerun-if-env-changed","", Slot::RerunEnv, Scope::RerunKey, Transform::Verbatim, false, "", "", 1}, + {"rerun-if-changed-glob","", Slot::RerunGlobs, Scope::RerunKey, Transform::Verbatim, false, "", "", 2}, {"action", "action", Slot::Actions, Scope::GraphNode, Transform::Verbatim, false, "", "", 1}, }}; @@ -223,6 +236,34 @@ void serialize(std::ostream& os, const Directives& d); // written by a newer mcpp) — the caller treats that as a stale entry. bool accept_cache_record(Directives& d, std::string_view tag, std::string_view value); +// ── Glob inputs (#359) ───────────────────────────────────────────────────── +// +// The fingerprint of `rerun-if-changed-glob=`: the SORTED SET of +// relative paths matching the pattern under `root`, and nothing else. +// +// Deliberately not contents, size or mtime: +// * contents are already covered — a file whose bytes matter is declared as +// an ordinary `rerun-if-changed` input, and size is a strictly weaker +// signal than the hash that entry already carries; +// * mtime is unstable across git checkout, container builds and rsync, and +// this project has already paid for treating a timestamp as identity +// (the file_time_type epoch in the dependency cache). +// The question a glob input asks is "which files are here", so the answer is +// the path set, exactly. +// +// `root`-relative, generic_string, byte-ordered — otherwise the same tree +// fingerprints differently depending on the platform's directory-iteration +// order and separator. +// +// `outputDirName` (typically "target") and ".git" are never walked. A build +// program writes its outputs INSIDE the project, so a pattern like `**` would +// otherwise include what the previous run produced and the set would change on +// every build — a permanent re-run loop, and the classic Cargo footgun. This +// is enforced here rather than left to the author's pattern. +std::string glob_fingerprint(const std::filesystem::path& root, + std::string_view pattern, + std::string_view outputDirName); + // ── Apply ────────────────────────────────────────────────────────────────── // Fold the collected directives into the manifest's buildConfig. The single @@ -457,6 +498,46 @@ bool accept_cache_record(Directives& d, std::string_view tag, std::string_view v return true; } +std::string glob_fingerprint(const std::filesystem::path& root, + std::string_view pattern, + std::string_view outputDirName) { + namespace fs = std::filesystem; + std::vector hits; + std::error_code ec; + // skip_permission_denied only: symlinked directories are NOT followed, the + // same rule the source scan uses, so a self-referential link cannot make + // this walk diverge. + fs::recursive_directory_iterator it( + root, fs::directory_options::skip_permission_denied, ec); + if (ec) return {}; + for (; it != fs::recursive_directory_iterator(); it.increment(ec)) { + if (ec) break; + const auto& p = it->path(); + std::error_code dec; + if (it->is_directory(dec)) { + auto name = p.filename().string(); + if (name == ".git" || (!outputDirName.empty() && name == outputDirName)) { + it.disable_recursion_pending(); + continue; + } + if (it->is_symlink(dec)) it.disable_recursion_pending(); + continue; + } + if (!mcpp::modgraph::path_matches_glob(p, root, pattern)) continue; + std::string rel; + try { + rel = p.lexically_relative(root).generic_string(); + } catch (const std::exception&) { + continue; // unspellable name — see path_matches_glob + } + hits.push_back(std::move(rel)); + } + std::ranges::sort(hits); + std::string joined; + for (auto const& h : hits) { joined += h; joined.push_back('\n'); } + return mcpp::toolchain::hash_string(joined); +} + void apply(mcpp::manifest::Manifest& m, const Directives& d) { auto& bc = m.buildConfig; auto const& cxx = d.at(Slot::CxxFlags); diff --git a/src/build/execute.cppm b/src/build/execute.cppm index bb0be796..c47d3b0a 100644 --- a/src/build/execute.cppm +++ b/src/build/execute.cppm @@ -9,6 +9,7 @@ module; export module mcpp.build.execute; import std; +import mcpp.build.build_program; // #359 glob inputs the mtime sweep cannot see import mcpp.build.prepare; import mcpp.diag; import mcpp.build.plan; @@ -449,6 +450,12 @@ bool sources_newer_than(const std::filesystem::path& projectRoot, auto bt = std::filesystem::last_write_time(bp, ec); if (ec || bt > ninjaTime) return true; } + // #359: a GLOB input changes without any existing file's mtime changing — + // a new .proto appears and every timestamp below is unmoved. The mtime + // sweep therefore cannot see it, and the fast path would report + // "Finished dev in 0.00s" while the new file is never generated. Same + // question as the build.mcpp check above, different kind of input. + if (mcpp::build::glob_inputs_stale(projectRoot)) return true; for (auto& f : mcpp::modgraph::expand_glob(projectRoot, "src/**/*")) { auto ext = f.extension().string(); if (ext != ".cppm" && ext != ".cpp" && ext != ".cc" && diff --git a/src/build/hostprogram.cppm b/src/build/hostprogram.cppm index 135b0be6..d309932e 100644 --- a/src/build/hostprogram.cppm +++ b/src/build/hostprogram.cppm @@ -128,6 +128,19 @@ private: } }; inline void rerun_if_changed(const char* path) { std::printf("mcpp:rerun-if-changed=%s\n", path); } +// mcpp#359: re-run when the SET of files matching `pattern` changes — a file +// appearing or disappearing, not its contents (declare those with +// rerun_if_changed). `pattern` is relative to the manifest directory and uses +// the same `*` / `**` grammar as `sources = [...]`, e.g. "proto/**/*.proto". +// +// Without this a build program that globs is structurally unsafe: adding a +// .proto changes no declared file's hash, so the program does not re-run and +// the new file is silently never generated. The build output tree and .git are +// never part of the set, so watching a wide pattern cannot create a re-run +// loop with the program's own outputs. +inline void rerun_if_changed_glob(const char* pattern) { + std::printf("mcpp:rerun-if-changed-glob=%s\n", pattern); +} inline void rerun_if_env_changed(const char* var) { std::printf("mcpp:rerun-if-env-changed=%s\n", var); } // ── environment contract (read side; values injected by the engine) ───── inline const char* env_or(const char* n) { const char* v = std::getenv(n); return v ? v : ""; } diff --git a/src/build/prepare.cppm b/src/build/prepare.cppm index 4f0b3e68..e556868e 100644 --- a/src/build/prepare.cppm +++ b/src/build/prepare.cppm @@ -36,6 +36,7 @@ import mcpp.build.build_program; import mcpp.build.directives; // directive table: mark / fold_private_tail import mcpp.build.tool_store; // #355 host tools: store layout + key + overrides import mcpp.build.dep_graph; // queries over the resolved edge graph +import mcpp.build.provisions; // #359 build-time provisions: table + propagation import mcpp.build.backend; // BuildOptions for the tool sub-build import mcpp.build.ninja; // make_ninja_backend — driving that sub-build import mcpp.lockfile; @@ -446,11 +447,14 @@ materialize_generated_files(const std::filesystem::path& root, // loadVersionDep() (shared by the main per-dependency loop, the // multi-version mangling secondary, and the SemVer-merge re-fetch — all three // of ITS callers get the merge for free from the one call inside it). -// (Conditional *dependencies* are a separate, root-only concern: they must be -// merged into the dependency map BEFORE resolution even starts, so a -// dependency's own conditional deps are out of scope — see the root cfg -// block that merges `cc.dependencies` etc.) -void merge_conditional_build_inputs(mcpp::manifest::Manifest& m, +// The dependency MAPS ride the same funnel (#359). They used to be merged by +// a hand-written loop at the root call site only, with a comment declaring a +// dependency's own conditional deps "out of scope". That was the #229 shape +// one level up: three call sites merged build inputs, ONE of them also merged +// deps, and nothing said why. A package's `[target.windows.dependencies]` is +// its own statement about itself and means the same thing whether the package +// is the root or someone's dependency. +void merge_conditional_config(mcpp::manifest::Manifest& m, const cfgpred::Ctx& ctx, std::string_view targetTriple) { @@ -465,13 +469,26 @@ void merge_conditional_build_inputs(mcpp::manifest::Manifest& m, // BuildInputs, so conditional sources are mirrored into it here. for (auto const& s : cc.inputs.sources) m.modules.sources.push_back(s); + // insert() keeps an existing unconditional entry: a conditional + // section adds a dependency, it never silently overrides one. + m.dependencies.insert(cc.dependencies.begin(), cc.dependencies.end()); + m.devDependencies.insert(cc.devDependencies.begin(), cc.devDependencies.end()); + m.buildDependencies.insert(cc.buildDependencies.begin(), + cc.buildDependencies.end()); + // #359: `[target..feature-deps.]`. The feature is + // registered by the parser regardless of the predicate; only what it + // pulls in is conditional. + for (auto const& [fname, deps] : cc.featureDeps) { + auto& dst = m.featureDeps[fname]; + dst.insert(deps.begin(), deps.end()); + } } } // Desugar `[build].defines` into `-D` on both C and C++ flag channels. // // ORDER (both halves are load-bearing): this must run AFTER -// merge_conditional_build_inputs — `defines` is a BuildInputs member, so a +// merge_conditional_config — `defines` is a BuildInputs member, so a // matching `[target.'cfg(...)'.build] defines` has been appended by then and // folds in the same pass, landing after the unconditional entries so GNU // last-wins gives the conditional rule precedence — and BEFORE the manifest is @@ -1354,12 +1371,11 @@ prepare_build(bool print_fingerprint, const auto targetPlatform = mcpp::platform::TargetPlatform::for_os( cfgpred::context_for(overrides.target_triple).os); - // ── L1: merge conditional [target.'cfg(...)'.build] sources/flags AND - // root-only [target.'cfg(...)'.dependencies] ───────────────────────────── + // ── L1: merge conditional [target.'cfg(...)'] sections ─────────────────── // Evaluated now (target resolved) against the resolved target — the // --target triple for a cross build, else the host. // - // #229: merge_conditional_build_inputs MUST run here — before + // #229: merge_conditional_config MUST run here — before // `packages[0] = makePackageRoot(*root, *m)` snapshots `m->buildConfig` // into `packages[0].privateBuild`/`.manifest` — because that snapshot, // not `*m`, is what the modgraph scan and per-TU compile-flag assembly @@ -1370,19 +1386,8 @@ prepare_build(bool print_fingerprint, // package's half of the one funnel, not a special case: every package is // merged exactly once, immediately before it is captured into `packages[]`. if (!m->conditionalConfigs.empty()) { - auto cc_ctx = cfgpred::context_for(overrides.target_triple); - merge_conditional_build_inputs(*m, cc_ctx, overrides.target_triple); - for (auto const& cc : m->conditionalConfigs) { - if (!cfgpred::matches(cc.predicate, cc_ctx, overrides.target_triple)) - continue; - // Conditional dependencies (Phase 1b): merge into the manifest maps - // before dependency resolution so they resolve like any dep. insert() - // keeps an existing unconditional entry (no silent override). - // Root-only — a dependency's own conditional deps are out of scope. - m->dependencies.insert(cc.dependencies.begin(), cc.dependencies.end()); - m->devDependencies.insert(cc.devDependencies.begin(), cc.devDependencies.end()); - m->buildDependencies.insert(cc.buildDependencies.begin(), cc.buildDependencies.end()); - } + merge_conditional_config(*m, cfgpred::context_for(overrides.target_triple), + overrides.target_triple); } // `[build].defines` must reach the scanner (P1689) and the compile edge, // and must participate in the fingerprint. Fold before dependency @@ -2655,7 +2660,7 @@ prepare_build(bool print_fingerprint, // just keyed off a different loading branch since path/git deps never // pass through loadVersionDep. if (!manifest->conditionalConfigs.empty()) { - merge_conditional_build_inputs(*manifest, + merge_conditional_config(*manifest, cfgpred::context_for(overrides.target_triple), overrides.target_triple); } @@ -2682,6 +2687,11 @@ prepare_build(bool print_fingerprint, // consumer's request must not be silently dropped, which is the // #242/#243 failure shape. std::vector requestedTools; + // #355 step 5 / #359: does this edge ask for the dependency's lib-root + // interface as a HOST module, and does it hand its build-time + // provisions on to this consumer's own consumers? + bool hostModule = false; + bool reexport = false; }; std::vector dependencyEdges; namespace dg = mcpp::build::dep_graph; @@ -2697,6 +2707,30 @@ prepare_build(bool print_fingerprint, std::map>> hostModulesByConsumer; + // #359: who can see which build-time provision. Computed once by the + // provisioning pass below (a fixpoint over `dependencyEdges`, the same + // shape as computeUsageRequirements) and read by every consumer of the + // three env channels above. Declared here because `fillDepDirs` closes + // over it and is defined long before the pass runs; every call site is + // after it. + namespace prov = mcpp::build::provisions; + prov::Propagation provisionGraph; + // The spellings a given consumer may address a provider by. The qualified + // name always works; the bare tail only when the namespace ladder binds it + // to exactly this package FOR THIS CONSUMER. Scoped per consumer rather + // than globally because two packages sharing a tail only collide inside an + // environment that contains both. + auto bareBindingsFor = [&](std::size_t consumer) { + std::vector fqns; + if (consumer < provisionGraph.visible.size()) + for (auto const& pr : provisionGraph.visible[consumer]) { + if (pr.provider >= packages.size()) continue; + auto const& n = packages[pr.provider].manifest.package.name; + if (std::find(fqns.begin(), fqns.end(), n) == fqns.end()) + fqns.push_back(n); + } + return prov::bind_bare_names(fqns); + }; auto parseVisibility = [](std::string_view visibility) { if (visibility == "private") @@ -2760,12 +2794,36 @@ prepare_build(bool print_fingerprint, // canonical name and its namespace-stripped tail, so // `mcpp::dep_dir("compat.zlib")` and `mcpp::dep_dir("zlib")` both resolve // regardless of which spelling the author used in `deps`. + // + // #359: the set is now the consumer's VISIBLE provisions rather than its + // direct edges, so a re-exported dependency's directory reaches it too. + // That is what makes a rule package able to find data files belonging to a + // dependency the user never declared — protoc's well-known .proto files + // are exactly such a directory, and `grpcgen` reads them through dep_dir. + // + // The bare tail is emitted only when the namespace ladder binds it here. + // Emitting it unconditionally was safe while only the root's own + // declarations reached build.mcpp; with re-export, two packages that never + // heard of each other can share a tail and the later emplace_back would + // silently win. auto fillDepDirs = [&](mcpp::build::BuildProgramEnv& e, std::size_t consumer) { - for (auto d : dg::direct_dependencies(dependencyEdges, consumer)) { - auto const& depPkg = packages[d]; - for (auto const& spelling : - dg::name_spellings(depPkg.manifest.package.name)) - e.depDirs.emplace_back(spelling, depPkg.root); + if (consumer >= provisionGraph.visible.size()) return; + auto bind = bareBindingsFor(consumer); + for (auto const& [tail, b] : bind) { + if (auto note = prov::contest_note(tail, b); !note.empty()) + mcpp::diag::warning("provisions/ambiguous", note); + } + for (auto const& pr : provisionGraph.visible[consumer]) { + if (pr.kind != prov::Kind::DepDir) continue; + if (pr.provider >= packages.size()) continue; + auto const& depPkg = packages[pr.provider]; + auto const& canon = depPkg.manifest.package.name; + e.depDirs.emplace_back(canon, depPkg.root); + auto tail = prov::tail_of(canon); + if (tail == canon) continue; + auto it = bind.find(tail); + if (it != bind.end() && it->second.owner == canon) + e.depDirs.emplace_back(tail, depPkg.root); } }; @@ -2900,6 +2958,8 @@ prepare_build(bool print_fingerprint, .requestedFeatures = spec.features, .defaultFeatures = spec.defaultFeatures, .requestedTools = spec.tools, + .hostModule = spec.hostModule, + .reexport = spec.reexport, }); }; @@ -3664,7 +3724,7 @@ prepare_build(bool print_fingerprint, // BEFORE `propagateLinkFlags`/`makePackageRoot` below, which // snapshot this manifest's flags/sources into `packages[]`. if (!dep_manifest->conditionalConfigs.empty()) { - merge_conditional_build_inputs(*dep_manifest, + merge_conditional_config(*dep_manifest, cfgpred::context_for(overrides.target_triple), overrides.target_triple); } @@ -4015,60 +4075,68 @@ prepare_build(bool print_fingerprint, for (auto const& t : edge.requestedTools) toolRequests[edge.dependencyPackageIndex].insert(t); + // #359: one fixpoint decides who SEES what. `toolRequests` above + // still decides what gets BUILT — the two questions are separate, + // and conflating them is what made a re-exported tool impossible: + // the tool was built, but its path was recorded against the library + // that asked for it rather than the project that needs it. + provisionGraph = prov::propagate(dependencyEdges, packages.size()); + // #355 step 5: dependencies offering HOST build rules. Nothing is // compiled here — the interface is handed to build_program.cppm, // which compiles it in the SAME command as build.mcpp so the BMI // and its consumer agree on standard, dialect and compiler by // construction rather than by luck. - for (auto const& [depName, spec] : m->dependencies) { - if (!spec.hostModule) continue; - for (auto d : dg::direct_dependencies(dependencyEdges, 0)) { - auto const& depPkg = packages[d]; + // + // Driven off the visible set rather than the root manifest, so a + // rule a library re-exports is importable from the consumer's + // build.mcpp without the consumer naming it. The name matching the + // old loop needed is gone with it: the edge already knows which + // package it points at. + for (std::size_t c = 0; c < provisionGraph.visible.size(); ++c) { + for (auto const& pr : provisionGraph.visible[c]) { + if (pr.kind != prov::Kind::HostModule) continue; + if (pr.provider >= packages.size()) continue; + auto const& depPkg = packages[pr.provider]; auto const& canon = depPkg.manifest.package.name; - // Match on either spelling, the same way `deps` keys and - // MCPP_DEP__DIR do — a consumer may have written - // `compat.zlib` or `zlib`. - bool hit = false; - for (auto const& s : dg::name_spellings(canon)) - if (depName == s || depName.ends_with("." + s)) hit = true; - if (!hit) continue; auto rel = mcpp::manifest::resolve_lib_root_path(depPkg.manifest); - hostModulesByConsumer[0].emplace_back(canon, depPkg.root / rel); - - // A build rule is BUILD-TIME ONLY. Registering the module - // is not enough: the package is still an ordinary node of - // the consumer's graph, so its interface was ALSO compiled - // as a normal library and linked into the target. That is - // wrong on its own terms — a rule has no business in the - // consumer's binary — and it made the feature nearly - // unusable, because in that second compile the bundled - // `mcpp` module does not exist: any rule that actually used - // the API it exists to wrap died with - // `fatal error: module 'mcpp' not found` (2026.8.5.1). - // - // Emptying the source globs is how a package is removed - // from the compile set here — the same mechanism the - // feature-gated-sources drop above uses. Resolution is - // untouched: the package still lands on disk, which is - // what `resolve_lib_root_path` just read. - // - // Guarded on the package being reached ONLY from the root's - // host-module edge. A package can legitimately be both a - // rule and a library — for something else in the graph, or - // for the root itself under a second spelling — and - // silently dropping its objects then would surface as an - // undefined reference far from here. - bool hostOnly = true; - for (auto const& e : dependencyEdges) - if (e.dependencyPackageIndex == d - && e.consumerPackageIndex != 0) hostOnly = false; - if (hostOnly) { - auto& dm = packages[d].manifest; - dm.buildConfig.sources.clear(); - dm.buildConfig.featureSources.clear(); - dm.modules.sources.clear(); - } - break; + hostModulesByConsumer[c].emplace_back(canon, depPkg.root / rel); + } + } + + // A build rule is BUILD-TIME ONLY. Registering the module is not + // enough: the package is still an ordinary node of the consumer's + // graph, so its interface was ALSO compiled as a normal library and + // linked into the target. That is wrong on its own terms — a rule + // has no business in the consumer's binary — and it made the + // feature nearly unusable, because in that second compile the + // bundled `mcpp` module does not exist: any rule that actually used + // the API it exists to wrap died with `fatal error: module 'mcpp' + // not found` (2026.8.5.1). + // + // Emptying the source globs is how a package is removed from the + // compile set here — the same mechanism the feature-gated-sources + // drop above uses. Resolution is untouched: the package still lands + // on disk, which is what `resolve_lib_root_path` just read. + // + // Guarded on EVERY edge into the package being a host-module edge. + // A package can legitimately be both a rule and a library, and + // silently dropping its objects then would surface as an undefined + // reference far from here. (The predicate used to be "no consumer + // other than the root", which said the same thing only while the + // root was the only possible requester.) + { + std::set ruleOnly; + for (auto const& e : dependencyEdges) + if (e.hostModule) ruleOnly.insert(e.dependencyPackageIndex); + for (auto const& e : dependencyEdges) + if (!e.hostModule) ruleOnly.erase(e.dependencyPackageIndex); + for (auto d : ruleOnly) { + if (d >= packages.size()) continue; + auto& dm = packages[d].manifest; + dm.buildConfig.sources.clear(); + dm.buildConfig.featureSources.clear(); + dm.modules.sources.clear(); } } @@ -4101,10 +4169,17 @@ prepare_build(bool print_fingerprint, if (t.name == toolName) tgt = &t; } if (!tgt) { + // A package may declare a bin target on some platforms + // only. When the request came from a LIBRARY rather + // than from the user, the user cannot edit it away, so + // point at the knob that library needs (#359 D3a). return std::unexpected(std::format( "dependency '{}' has no `kind = \"bin\"` target named " "'{}' (requested via tools = [...]).\n" - " available bin targets: [{}]", + " available bin targets: [{}]\n" + " If the requesting package is a library, it can " + "scope the request per platform with\n" + " [target.'cfg(...)'.feature-deps.].", depName, toolName, binList.empty() ? std::string("none") : binList)); } @@ -4113,15 +4188,22 @@ prepare_build(bool print_fingerprint, auto varShort = mcpp::build::tool_store::env_var_name(depShort, toolName); + // #359: every consumer that can SEE this tool gets it, not + // just the one whose edge asked for it. The bare spelling + // is emitted only where the namespace ladder binds the tail + // to this package — otherwise two libraries re-exporting a + // same-tailed tool would decide the winner by append order. + const prov::Provision want{ prov::Kind::Tool, depIdx, toolName }; auto record = [&](const std::filesystem::path& p) { - for (auto const& edge : dependencyEdges) { - if (edge.dependencyPackageIndex != depIdx) continue; - if (std::find(edge.requestedTools.begin(), - edge.requestedTools.end(), toolName) - == edge.requestedTools.end()) continue; - auto& v = toolEnvByConsumer[edge.consumerPackageIndex]; + for (std::size_t c = 0; c < provisionGraph.visible.size(); ++c) { + if (!provisionGraph.visible[c].contains(want)) continue; + auto& v = toolEnvByConsumer[c]; v.emplace_back(var, p.string()); - if (varShort != var) v.emplace_back(varShort, p.string()); + if (varShort == var) continue; + auto bind = bareBindingsFor(c); + auto it = bind.find(depShort); + if (it != bind.end() && it->second.owner == depName) + v.emplace_back(varShort, p.string()); } }; @@ -4239,13 +4321,30 @@ prepare_build(bool print_fingerprint, sub.features += f; } + // #359 (D3b): a sub-build failure must be attributable and + // REPRODUCIBLE. The Windows tool sub-build has been failing + // on three abseil TUs since #355 and is still unlocated, + // because what reached the log was a one-line summary with + // no scratch path, no chain, and — on the ninja branch below + // — a filtered view of the inner output. Naming the scratch + // directory is what lets a maintainer re-run the exact inner + // build; MCPP_TOOL_BUILD_VERBOSE turns off the filtering. + auto subContext = [&] { + return std::format( + "\n chain: {}\n sub-build scratch: {}\n" + " re-run it directly: mcpp build -p {} --release\n" + " (set MCPP_TOOL_BUILD_VERBOSE=1 for the inner " + "build's unfiltered output)", + sub.tool_chain, sub.work_dir.string(), + depPkg.root.string()); + }; auto subCtx = prepare_build(/*print_fingerprint=*/false, /*includeDevDeps=*/false, /*extraTargets=*/{}, sub); if (!subCtx) { return std::unexpected(std::format( - "building host tool '{}:{}' failed: {}", - depName, toolName, subCtx.error())); + "building host tool '{}:{}' failed: {}{}", + depName, toolName, subCtx.error(), subContext())); } // Build ONLY the requested target (#274 gave the backend @@ -4265,17 +4364,28 @@ prepare_build(bool print_fingerprint, auto be = mcpp::build::make_ninja_backend(); mcpp::build::BuildOptions bopt; bopt.ninjaTargets = { goal.generic_string() }; + // Unfiltered inner output on demand: the filter drops + // ninja's own progress and command echoes, which is right + // for a normal build and wrong when the question is "what + // did the inner build actually do". + if (const char* v = std::getenv("MCPP_TOOL_BUILD_VERBOSE"); + v && *v && std::string_view(v) != "0") + bopt.verbose = true; auto br = be->build(subCtx->plan, bopt); if (!br) { + auto diag = br.error().diagnosticOutput; + if (diag.empty()) + diag = "(the inner build produced no diagnostic " + "output; re-run with MCPP_TOOL_BUILD_VERBOSE=1)"; return std::unexpected(std::format( - "building host tool '{}:{}' failed: {}\n{}", + "building host tool '{}:{}' failed: {}{}\n{}", depName, toolName, br.error().message, - br.error().diagnosticOutput)); + subContext(), diag)); } if (br->exitCode != 0) { return std::unexpected(std::format( - "building host tool '{}:{}' failed (exit {})", - depName, toolName, br->exitCode)); + "building host tool '{}:{}' failed (exit {}){}", + depName, toolName, br->exitCode, subContext())); } // Publish into the store: build out of place, then move — diff --git a/src/build/provisions.cppm b/src/build/provisions.cppm new file mode 100644 index 00000000..80d4d5ac --- /dev/null +++ b/src/build/provisions.cppm @@ -0,0 +1,279 @@ +// mcpp.build.provisions — what a dependency hands to its consumer's BUILD +// PROGRAM, and how far it travels. +// +// WHY THIS MODULE EXISTS +// +// mcpp has long had a model for "what a dependency provides × who sees it": +// `UsageRequirements` (include dirs, defines, link flags, module names) with +// three scopes and a fixpoint that flows publicUsage into each consumer's +// privateBuild. Adding an entry there forces the author to answer "which +// scope?", which is why include dirs have never silently leaked. +// +// The provisions #355 introduced — host tools (`tools = [...]`) and host build +// rules (`host-module = true`) — went in beside that model rather than into +// it. Each got a hand-written destination: tools were recorded against the +// consumer of the *requesting* edge, host modules only against the root's +// direct dependencies, dependency directories only for direct dependencies. +// None of the three could be re-exported, so a library could not stand up a +// toolchain on its user's behalf, and the user had to name every tool the +// library needed (#359). +// +// The root cause is not a missing propagation. It is that NOTHING FORCED THE +// QUESTION. This codebase has repeatedly paid for "the same decision derived +// in N places" (#233/#240/#242/#344); this is its mirror image — a question +// that must be answered exactly once was answered in ZERO places, so each new +// provision invented its own reach. `directives::kTable` already solved this +// shape once by making Scope a required column. +// +// Here a provision KIND is one row of `kTable`, and every row states whether +// the kind is addressable by a bare name and whether it travels only on an +// explicitly re-exporting edge. Propagation is one fixpoint over the edge +// graph, shared by all kinds. +// +// WHY IT IS A SEPARATE MODULE RATHER THAN MORE OF prepare.cppm +// +// Same reason `mcpp.build.directives` and `mcpp.build.hostprogram` are +// separate: build_program.cppm's anonymous namespace miscompiles its own +// neighbours under clang 22 + C++20 modules + -O2 (PR#332, reproduced by +// PR#334), and prepare_build is already a single function of several thousand +// lines. New policy goes in its own module with its own tests. +// +// See .agents/docs/2026-08-06-provisions-and-build-inputs.md. + +export module mcpp.build.provisions; + +import std; +import mcpp.pm.dep_spec; + +export namespace mcpp::build::provisions { + +// ── The table ────────────────────────────────────────────────────────────── +// +// A provision is something a dependency makes available to a consumer's +// `build.mcpp` PROGRAM — not to its compile commands. That distinction is why +// these do not live in `UsageRequirements`: that type describes a consumer's +// command line and is consumed by the ninja graph, while a provision is +// consumed by a process mcpp runs during prepare. Merging them would leave +// `linkUsage.tools` as a field with no meaning. +enum class Kind { + Tool, // a dependency's `kind = "bin"` target, built for the host + HostModule, // a dependency's lib-root interface, compiled with build.mcpp + DepDir, // a dependency's resolved source directory +}; + +struct Def { + Kind kind; + std::string_view name; + // Does an unqualified spelling address this kind? Tools and dep dirs are + // looked up by name from inside build.mcpp (`dep_bin("protobuf", …)`), so + // they need the bare-name ladder below. A host module is addressed by + // `import ;`, where the module name IS the package name and the + // compiler — not mcpp — resolves it. + bool bareAddressable; + // Does crossing one more edge require `reexport = true` on that edge? + // Every kind does. The column exists so that a future kind cannot be added + // without someone writing an answer down. + bool needsReexport; +}; + +inline constexpr Def kTable[] = { + { Kind::Tool, "tool", true, true }, + { Kind::HostModule, "host-module", false, true }, + { Kind::DepDir, "dep-dir", true, true }, +}; + +inline constexpr const Def& def_of(Kind k) { + for (auto const& d : kTable) + if (d.kind == k) return d; + return kTable[0]; // unreachable: kTable covers the enum +} + +// ── One provision instance ───────────────────────────────────────────────── + +struct Provision { + Kind kind = Kind::Tool; + std::size_t provider = 0; // package index that produces it + std::string tool; // Kind::Tool only; empty otherwise + + // Written out rather than `= default`-ing the spaceship: a defaulted + // operator<=> here made GCC 16 emit two different manglings for + // `__gnu_cxx::operator<=>(__normal_iterator…)` across the module boundary + // and reject the second ("conflicts with a previous mangle"). The ordering + // is only needed to key a std::set, so an explicit `<` costs nothing. + friend bool operator<(const Provision& a, const Provision& b) { + if (a.kind != b.kind) return a.kind < b.kind; + if (a.provider != b.provider) return a.provider < b.provider; + return a.tool < b.tool; + } + friend bool operator==(const Provision& a, const Provision& b) { + return a.kind == b.kind && a.provider == b.provider && a.tool == b.tool; + } +}; + +// ── Propagation ──────────────────────────────────────────────────────────── +// +// own(P→D) = provisions this edge asks D for, plus D's own directory +// exported(P) = ⋃ over edges P→D with reexport: own(P→D) ∪ exported(D) +// visible(P) = ⋃ over all edges P→D: own(P→D) ∪ exported(D) +// +// A package always sees what its direct dependencies hand it; it passes +// something on only when it says so. Monotone in both sets, so the fixpoint +// terminates, and a dependency cycle is harmless (sets stop growing). +// +// Templated on the edge type for the reason `dep_graph` is: `DependencyEdge` +// is a local struct inside prepare_build. An edge must expose +// `consumerPackageIndex`, `dependencyPackageIndex`, `reexport`, +// `requestedTools` and `hostModule`. +struct Propagation { + std::vector> visible; + std::vector> exported; +}; + +template +Propagation propagate(const std::vector& edges, std::size_t packageCount) { + Propagation p; + p.visible.resize(packageCount); + p.exported.resize(packageCount); + + auto own = [](const Edge& e) { + std::set s; + // Every edge exposes the dependency's directory: that is today's + // behaviour for direct dependencies (MCPP_DEP__DIR) and is kept + // unconditional so nothing that works now stops working. + s.insert(Provision{ Kind::DepDir, e.dependencyPackageIndex, {} }); + for (auto const& t : e.requestedTools) + s.insert(Provision{ Kind::Tool, e.dependencyPackageIndex, t }); + if (e.hostModule) + s.insert(Provision{ Kind::HostModule, e.dependencyPackageIndex, {} }); + return s; + }; + + bool changed = true; + while (changed) { + changed = false; + for (auto const& e : edges) { + if (e.consumerPackageIndex >= packageCount) continue; + if (e.dependencyPackageIndex >= packageCount) continue; + auto contributed = own(e); + const auto& fromDep = p.exported[e.dependencyPackageIndex]; + contributed.insert(fromDep.begin(), fromDep.end()); + + auto& vis = p.visible[e.consumerPackageIndex]; + for (auto const& pr : contributed) + if (vis.insert(pr).second) changed = true; + + if (e.reexport) { + auto& exp = p.exported[e.consumerPackageIndex]; + for (auto const& pr : contributed) + if (exp.insert(pr).second) changed = true; + } + } + } + return p; +} + +// ── Bare-name binding ────────────────────────────────────────────────────── +// +// `dep_bin("protobuf", "protoc")` and `dep_dir("protobuf")` address a package +// by its namespace-stripped tail. That was safe while only the root's own +// declarations reached build.mcpp: a collision was between two lines the user +// had written. Once a library can re-export, two packages that have never +// heard of each other can both offer the tail `protobuf`, and "whichever was +// appended last" decides which binary runs — silently, and in exactly the +// place this feature exists to make version mismatch inexpressible. +// +// The binding therefore uses the same mechanism package identity already uses +// for an unqualified name: the fully-qualified spelling is the identity and is +// always available; the bare spelling is a convenience resolved through a +// fixed ladder. +// +// 1. (kDefaultNamespace, X) +// 2. (kCompatNamespace, X) +// 3. (∅, X) — a package declared with no namespace at all +// 4. the single remaining candidate, if there is exactly one +// +// Rung 4 is the addition this context needs and package resolution does not: a +// package in some other namespace (grpc.grpc-plugin) must still be reachable +// as `grpc-plugin`, because that is the spelling rules already in the wild +// use. When it does not apply, the bare spelling is not bound at all and the +// caller must use the qualified one. + +struct BareBinding { + std::string owner; // FQN bound to this tail; empty = unbound + std::vector candidates; // every FQN with this tail, sorted + bool contested = false; // more than one candidate +}; + +inline std::string namespace_of(std::string_view fqn) { + auto dot = fqn.rfind('.'); + if (dot == std::string_view::npos) return {}; + return std::string(fqn.substr(0, dot)); +} + +inline std::string tail_of(std::string_view fqn) { + auto dot = fqn.rfind('.'); + if (dot == std::string_view::npos || dot + 1 >= fqn.size()) + return std::string(fqn); + return std::string(fqn.substr(dot + 1)); +} + +// tail → binding, over every package that provides something. +inline std::map +bind_bare_names(const std::vector& fqns) { + std::map> byTail; + for (auto const& f : fqns) { + auto& v = byTail[tail_of(f)]; + if (std::find(v.begin(), v.end(), f) == v.end()) v.push_back(f); + } + + const std::string ladder[] = { + std::string(mcpp::pm::kDefaultNamespace), + std::string(mcpp::pm::kCompatNamespace), + std::string{}, + }; + + std::map out; + for (auto& [tail, cands] : byTail) { + std::ranges::sort(cands); + BareBinding b; + b.candidates = cands; + b.contested = cands.size() > 1; + if (cands.size() == 1) { + b.owner = cands.front(); + } else { + for (auto const& ns : ladder) { + std::vector rung; + for (auto const& c : cands) + if (namespace_of(c) == ns) rung.push_back(c); + if (rung.size() == 1) { b.owner = rung.front(); break; } + if (rung.size() > 1) break; // ambiguous *within* a rung: unbind + } + } + out.emplace(tail, std::move(b)); + } + return out; +} + +// The human-readable half of a contested binding. Empty when there is nothing +// to say, so callers can `if (auto m = …; !m.empty())`. +inline std::string contest_note(std::string_view tail, const BareBinding& b) { + if (!b.contested) return {}; + std::string list; + for (auto const& c : b.candidates) { + if (!list.empty()) list += ", "; + list += c; + } + if (b.owner.empty()) + return std::format( + "the unqualified name '{}' is offered by more than one package " + "({}) and none of them wins the namespace ladder, so it is NOT " + "bound — address the one you mean by its full name", + tail, list); + return std::format( + "the unqualified name '{}' is offered by more than one package ({}); " + "it resolves to '{}' by the namespace ladder — use the full name to " + "address a different one", + tail, list, b.owner); +} + +} // namespace mcpp::build::provisions diff --git a/src/manifest/toml.cppm b/src/manifest/toml.cppm index d5b8ed03..6495a884 100644 --- a/src/manifest/toml.cppm +++ b/src/manifest/toml.cppm @@ -537,13 +537,19 @@ std::expected parse_string(std::string_view content, // namespaces (so existing fetcher / lockfile lookups by composite name // keep working) and the bare `` for the default namespace (so the // common case stays unchanged). + // MUST list every key `fill_inline_spec` below reads. The two are one + // decision in two places: this predicate also distinguishes an inline dep + // spec from a NESTED namespace table, so a key missing here does not read + // as "unknown option" — the table is taken for a namespace and the user is + // told their value "must be a string, inline dep table, or nested table". + // `Manifest.EveryDependencySpecKeyIsAccepted` holds the two in sync. auto is_dep_spec_key = [](std::string_view k) { return k == "path" || k == "version" || k == "git" || k == "rev" || k == "tag" || k == "branch" || k == "features" || k == "default-features" || k == "workspace" || k == "visibility" || k == "backend" || k == "tools" - || k == "host-module"; + || k == "host-module" || k == "reexport"; }; auto looks_like_inline_dep_spec = [&](const t::Table& sub) { if (sub.empty()) return false; @@ -595,6 +601,13 @@ std::expected parse_string(std::string_view content, if (auto it = sub.find("host-module"); it != sub.end() && it->second.is_bool()) { spec.hostModule = it->second.as_bool(); } + // #359: `reexport = true` — hand this edge's build-time provisions + // (tools, host module, dependency dir) on to THIS package's consumers. + // Off by default; see DependencySpec::reexport for why it is not the + // edge's `visibility`. + if (auto it = sub.find("reexport"); it != sub.end() && it->second.is_bool()) { + spec.reexport = it->second.as_bool(); + } // `backend = ""` — sugar for requesting the dependency's // `backend-` feature (library-level backend selection knob). if (auto it = sub.find("backend"); it != sub.end() && it->second.is_string()) { @@ -652,7 +665,8 @@ std::expected parse_string(std::string_view content, if (!looks_like_inline_dep_spec(sub)) { return std::unexpected(error(origin, std::format( "[{}.{}] must be a version string or table of " - "(path/version/git/rev/tag/branch/features/default-features/visibility/tools)", + "(path/version/git/rev/tag/branch/features/default-features/" + "visibility/backend/workspace/tools/host-module/reexport)", section, key))); } if (auto r = fill_inline_spec(spec, section, key, sub); !r) return r; @@ -1166,13 +1180,30 @@ std::expected parse_string(std::string_view content, if (auto r = read_deps("dependencies", cc.dependencies); !r) return std::unexpected(r.error()); if (auto r = read_deps("dev-dependencies", cc.devDependencies); !r) return std::unexpected(r.error()); if (auto r = read_deps("build-dependencies", cc.buildDependencies); !r) return std::unexpected(r.error()); + // [target..feature-deps.] (#359). The feature + // itself is registered UNCONDITIONALLY: whether the platform + // matches decides what the feature pulls in, not whether the + // feature exists. Otherwise requesting it on a non-matching + // platform would trip the unknown-feature diagnostic. + if (auto f = body.find("feature-deps"); + f != body.end() && f->second.is_table()) { + for (auto& [fname, fval] : f->second.as_table()) { + if (!fval.is_table()) continue; + if (auto r = load_deps_table( + std::format("[target.{}.feature-deps.{}]", triple, fname), + fval.as_table(), cc.featureDeps[std::string(fname)]); !r) + return std::unexpected(r.error()); + m.featuresMap.try_emplace(std::string(fname), + std::vector{}); + } + } if (!cc.inputs.cflags.empty() || !cc.inputs.cxxflags.empty() || !cc.inputs.ldflags.empty() || !cc.inputs.sources.empty() || !cc.inputs.defines.empty() || !cc.inputs.globFlags.empty() || !cc.inputs.includeDirs.empty() || !cc.inputs.includeDirsAfter.empty() || !cc.dependencies.empty() || !cc.devDependencies.empty() - || !cc.buildDependencies.empty()) + || !cc.buildDependencies.empty() || !cc.featureDeps.empty()) m.conditionalConfigs.push_back(std::move(cc)); } } diff --git a/src/manifest/types.cppm b/src/manifest/types.cppm index d5ddb588..9947a7c1 100644 --- a/src/manifest/types.cppm +++ b/src/manifest/types.cppm @@ -443,6 +443,18 @@ struct ConditionalConfig { std::map dependencies; std::map devDependencies; std::map buildDependencies; + // #359: `[target..feature-deps.]`. The conditional channel + // carried three of the four dependency maps and silently lacked the + // fourth, which is the exact failure this struct's `BuildInputs` comment + // above describes for #258 — the conditional reader kept its own subset of + // the keys and fell behind without anyone noticing. + // + // It is load-bearing for build-time provisions: a library that declares a + // host tool behind a feature (`grpc`'s `codegen` pulling protoc) has no + // other way to say "not on this platform", and an unconditional + // declaration turns an unsupported platform into a hard error raised from + // inside the LIBRARY's manifest, which its user cannot work around. + std::map> featureDeps; }; // `[lib]` — library "root" interface convention. diff --git a/src/manifest/xpkg.cppm b/src/manifest/xpkg.cppm index 5291687d..c982080d 100644 --- a/src/manifest/xpkg.cppm +++ b/src/manifest/xpkg.cppm @@ -1533,6 +1533,7 @@ synthesize_from_xpkg_lua(std::string_view luaContent, // original comment was describing. std::string dver; std::vector dtools; + bool dhostModule = false, dreexport = false; if (cur.peek() == '{') { cur.consume('{'); cur.skip_ws_and_comments(); @@ -1553,6 +1554,15 @@ synthesize_from_xpkg_lua(std::string_view luaContent, cur.skip_ws_and_comments(); } cur.consume('}'); + } else if (dk == "host-module" || dk == "reexport") { + // #359: the two build-time-provision knobs. They + // exist here for the reason the `defines` comment + // below states — the xpkg segment and mcpp.toml are + // two spellings of ONE schema, so a key accepted in + // one must not be an unknown-key error in the other. + bool v = cur.read_bareword() == "true"; + if (dk == "host-module") dhostModule = v; + else dreexport = v; } else { // Record rather than swallow — a descriptor author // writing an unsupported dep key deserves to be @@ -1570,8 +1580,10 @@ synthesize_from_xpkg_lua(std::string_view luaContent, } if (!dname.empty()) { DependencySpec spec; - spec.version = dver; - spec.tools = std::move(dtools); + spec.version = dver; + spec.tools = std::move(dtools); + spec.hostModule = dhostModule; + spec.reexport = dreexport; auto selector = mcpp::pm::resolve_dependency_selector( dname, mcpp::pm::DependencySelectorMode::OmittedMcpplibsPriority); diff --git a/src/modgraph/glob.cppm b/src/modgraph/glob.cppm new file mode 100644 index 00000000..a2262735 --- /dev/null +++ b/src/modgraph/glob.cppm @@ -0,0 +1,77 @@ +// mcpp.modgraph.glob — the ONE path-glob matcher. +// +// It lived in scanner.cppm's anonymous namespace, which was fine while the +// scanner was its only user. #359 gives `build.mcpp` a glob-shaped INPUT +// (`rerun-if-changed-glob`), whose fingerprint must select exactly the files a +// `sources = [...]` glob would. Two matchers that "should" agree about `**` is +// the shape this codebase keeps paying for, so there is one. + +export module mcpp.modgraph.glob; + +import std; + +export namespace mcpp::modgraph { + +// Does `candidate` match `glob`, interpreted relative to `root`? +// +// Supports "**" (any number of directory levels) and "*" (within one segment). +// LEXICAL relative: fs::relative() canonicalizes, which would resolve a path +// reached through a directory symlink back to its real location and break the +// match — a glob is about where a file appears in the tree, not where its bits +// live. +bool path_matches_glob(const std::filesystem::path& candidate, + const std::filesystem::path& root, + std::string_view glob) +{ + std::string rel; + try { + rel = candidate.lexically_relative(root).generic_string(); + } catch (const std::exception&) { + // MSVC's narrow conversion throws std::system_error when the native + // (wide) name has no spelling in the ANSI codepage (e.g. a CJK + // filename on an en-US host — mcpp#230 hit this on an issue template + // inside a walked index tree). Such a name can never be spelled in a + // glob or a compile command either: not a match, and never a reason to + // tear down the whole build. + return false; + } + + auto match = [](std::string_view s, std::string_view p) -> bool { + std::function rec = + [&](std::size_t si, std::size_t pi) -> bool { + while (pi < p.size()) { + if (p[pi] == '*' && pi + 1 < p.size() && p[pi + 1] == '*') { + // ** : skip zero or more chars/segments + pi += 2; + if (pi < p.size() && p[pi] == '/') ++pi; + if (pi >= p.size()) return true; + while (si <= s.size()) { + if (rec(si, pi)) return true; + ++si; + } + return false; + } else if (p[pi] == '*') { + // * : skip zero or more chars within segment (not /) + ++pi; + if (pi >= p.size()) { + return s.find('/', si) == std::string_view::npos; + } + while (si <= s.size()) { + if (rec(si, pi)) return true; + if (si < s.size() && s[si] == '/') break; + ++si; + } + return false; + } else { + if (si >= s.size() || s[si] != p[pi]) return false; + ++si; ++pi; + } + } + return si == s.size(); + }; + return rec(0, 0); + }; + return match(rel, glob); +} + +} // namespace mcpp::modgraph diff --git a/src/modgraph/scanner.cppm b/src/modgraph/scanner.cppm index 1dfafef5..6ae3232e 100644 --- a/src/modgraph/scanner.cppm +++ b/src/modgraph/scanner.cppm @@ -11,6 +11,7 @@ export module mcpp.modgraph.scanner; import std; import mcpp.manifest; +import mcpp.modgraph.glob; import mcpp.modgraph.graph; import mcpp.modgraph.p1689; import mcpp.toolchain.detect; @@ -125,68 +126,6 @@ namespace mcpp::modgraph { namespace { -bool path_matches_glob(const std::filesystem::path& candidate, - const std::filesystem::path& root, - std::string_view glob) -{ - // Supports "**" (any number of dirs) and "*" (within one segment). - // Matches relative-path of candidate against glob. LEXICAL relative: - // fs::relative() canonicalizes, which would resolve a path reached - // through a directory symlink back to its real location and break the - // match (globs are about where a file appears in the tree, not where - // its bits live). - std::string rel; - try { - rel = candidate.lexically_relative(root).generic_string(); - } catch (const std::exception&) { - // MSVC's narrow conversion throws std::system_error when the - // native (wide) name has no spelling in the ANSI codepage (e.g. a - // CJK filename on an en-US host — mcpp#230 hit this on an issue - // template inside a walked index tree). Such a name can never be - // spelled in a glob or a compile command either: not a match, and - // never a reason to tear down the whole build. - return false; - } - - auto match = [](std::string_view s, std::string_view p) -> bool { - // Simple recursive matcher. - std::function rec = - [&](std::size_t si, std::size_t pi) -> bool { - while (pi < p.size()) { - if (p[pi] == '*' && pi + 1 < p.size() && p[pi + 1] == '*') { - // ** : skip zero or more chars/segments - pi += 2; - if (pi < p.size() && p[pi] == '/') ++pi; - if (pi >= p.size()) return true; - while (si <= s.size()) { - if (rec(si, pi)) return true; - ++si; - } - return false; - } else if (p[pi] == '*') { - // * : skip zero or more chars within segment (not /) - ++pi; - if (pi >= p.size()) { - return s.find('/', si) == std::string_view::npos; - } - while (si <= s.size()) { - if (rec(si, pi)) return true; - if (si < s.size() && s[si] == '/') break; - ++si; - } - return false; - } else { - if (si >= s.size() || s[si] != p[pi]) return false; - ++si; ++pi; - } - } - return si == s.size(); - }; - return rec(0, 0); - }; - return match(rel, glob); -} - // Trim leading/trailing whitespace. std::string_view trim(std::string_view s) { std::size_t i = 0, j = s.size(); diff --git a/src/pm/dep_spec.cppm b/src/pm/dep_spec.cppm index 9c190ca9..88abc594 100644 --- a/src/pm/dep_spec.cppm +++ b/src/pm/dep_spec.cppm @@ -68,6 +68,25 @@ struct DependencySpec { // something to verify — the same class of failure as `module X CRC // mismatch`, which this project has paid for before. bool hostModule = false; + // #359: re-export everything this edge provides at BUILD time — the tools + // above, the host module above, and the dependency's directory — to this + // package's own consumers, transitively. + // + // This is what lets a library stand up a toolchain on its user's behalf: + // `grpc` declares protoc + the codegen rule with `reexport = true`, and the + // user writes one dependency line instead of four. It is the knob vcpkg's + // `"host": true` and Conan's `tool_requires` have no equivalent of. + // + // Default FALSE, and deliberately NOT reusing the edge's `visibility`: + // visibility already defaults to "public", so riding it would mean any + // dependency at any depth could push entries into its consumer's tool + // namespace without saying so. That is a supply-chain property, not a + // convenience, so it must be written down. (The initial design proposed + // riding public edges on the analogy that `include_dirs` are private by + // default — they are not: prepare.cppm copies privateBuild.includeDirs + // into publicUsage unconditionally. Only build.mcpp-INJECTED include dirs + // are private-only.) + bool reexport = false; bool defaultFeatures = true; // consumer opt-out: `default-features = false` // suppresses the dep's own [features].default seed // (Cargo parity). Explicit `features = [...]` still apply. diff --git a/src/version.cppm b/src/version.cppm index 3741777b..4e26374e 100644 --- a/src/version.cppm +++ b/src/version.cppm @@ -31,6 +31,6 @@ import std; export namespace mcpp { -inline constexpr std::string_view MCPP_VERSION = "2026.8.6.1"; +inline constexpr std::string_view MCPP_VERSION = "2026.8.6.2"; } // namespace mcpp diff --git a/src/xlings.cppm b/src/xlings.cppm index 492673d9..58256d2c 100644 --- a/src/xlings.cppm +++ b/src/xlings.cppm @@ -44,7 +44,7 @@ namespace pinned { // in lock-step by hand; that list was already missing both composite // actions, which is how CI's sandbox sat on 0.4.30 unnoticed while // everything else had moved on. Don't reintroduce a hand-maintained list. - inline constexpr std::string_view kXlingsVersion = "2026.8.5.2"; + inline constexpr std::string_view kXlingsVersion = "2026.8.6.2"; inline constexpr std::string_view kNasmVersion = "3.02"; } diff --git a/tests/e2e/193_provision_reexport.sh b/tests/e2e/193_provision_reexport.sh new file mode 100755 index 00000000..81dbd854 --- /dev/null +++ b/tests/e2e/193_provision_reexport.sh @@ -0,0 +1,182 @@ +#!/usr/bin/env bash +# requires: gcc +# 193_provision_reexport.sh — #359: a library standing up a toolchain on its +# user's behalf. +# +# Before this, `tools = [...]` and `host-module = true` were recorded against +# the consumer of the edge that ASKED. A library could therefore build a tool +# but not hand it on, so its user had to declare every tool the library needed +# — for grpc that meant four dependency lines and knowing that gRPC codegen +# runs protobuf's protoc, which is the library's knowledge, not the user's. +# +# Covered here: +# 1. WITHOUT `reexport`, a library's tool stays with the library. That is the +# supply-chain default: an arbitrary transitive dependency must not be +# able to put entries in your build program's tool namespace. +# 2. WITH `reexport = true`, the consumer's build.mcpp sees the tool, the +# re-exported rule module, and the re-exported package's directory — +# while declaring ONE dependency. +# +# See .agents/docs/2026-08-06-provisions-and-build-inputs.md. +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +cd "$TMP" + +export MCPP_HOME="$TMP/mcpphome" +mkdir -p "$MCPP_HOME" +if [ -d "$HOME/.mcpp/registry" ]; then + ln -s "$HOME/.mcpp/registry" "$MCPP_HOME/registry" +fi + +# ── the tool package (what a library depends on, and the user never names) ── +mkdir -p toolpkg/src +cat > toolpkg/mcpp.toml <<'EOF' +[package] +name = "toolpkg" +version = "0.1.0" + +[build] +sources = ["src/lib.cpp"] + +[targets.codegen] +kind = "bin" +main = "src/codegen.cpp" +EOF +printf 'int toolpkg_lib(){return 1;}\n' > toolpkg/src/lib.cpp +cat > toolpkg/src/codegen.cpp <<'EOF' +#include +int main(int argc, char** argv) { + if (argc < 2) return 2; + FILE* f = std::fopen(argv[1], "w"); + if (!f) return 3; + std::fprintf(f, "int generated_answer() { return 42; }\n"); + std::fclose(f); + return 0; +} +EOF + +# ── the rule package: an importable build rule, distributed as a package ──── +mkdir -p rulepkg/src +cat > rulepkg/mcpp.toml <<'EOF' +[package] +name = "rulepkg" +version = "0.1.0" + +[lib] +path = "src/rulepkg.cppm" +EOF +cat > rulepkg/src/rulepkg.cppm <<'EOF' +export module rulepkg; +import std; +import mcpp; +export namespace rulepkg { +// The rule knows which tool it needs. Its consumer does not have to. +bool generate() { + const char* tool = mcpp::dep_bin("toolpkg", "codegen"); + if (!tool || !*tool) { + std::println(std::cerr, "rulepkg: no codegen tool"); + return false; + } + // The tool package's own tree must be reachable too — that is where a real + // rule finds its data files (protoc's well-known .proto files). + if (std::string(mcpp::dep_dir("toolpkg")).empty()) { + std::println(std::cerr, "rulepkg: no dep_dir for toolpkg"); + return false; + } + std::string out = std::string(mcpp::out_dir()) + "/gen.cpp"; + std::string cmd = std::string("\"") + tool + "\" \"" + out + "\""; + if (std::system(cmd.c_str()) != 0) return false; + mcpp::generated(out.c_str()); + return true; +} +} +EOF + +# ── the library: it owns the knowledge, and re-exports what its user needs ── +mkdir -p lib/src +cat > lib/mcpp.toml <<'EOF' +[package] +name = "mylib" +version = "0.1.0" + +[build] +sources = ["src/mylib.cpp"] + +[dependencies] +toolpkg = { path = "../toolpkg", tools = ["codegen"] } +rulepkg = { path = "../rulepkg", host-module = true } +EOF +printf 'int mylib_fn(){return 1;}\n' > lib/src/mylib.cpp + +# ── 1. no reexport: the tool is built for the library, not handed on ──────── +mkdir -p app/src +cat > app/mcpp.toml <<'EOF' +[package] +name = "app" +version = "0.1.0" + +[dependencies] +mylib = { path = "../lib" } +EOF +cat > app/src/main.cpp <<'EOF' +#include +int generated_answer(); +int main() { std::printf("ANSWER=%d\n", generated_answer()); } +EOF +cat > app/build.mcpp <<'EOF' +#include +import mcpp; +int main() { + const char* tool = mcpp::dep_bin("toolpkg", "codegen"); + std::printf("SEES_TOOL=%d\n", (tool && *tool) ? 1 : 0); + return 1; // fail on purpose: the build must not proceed either way +} +EOF +cd app +if "$MCPP" build > b1.log 2>&1; then + cat b1.log; echo "FAIL: build.mcpp returned 1 but the build succeeded"; exit 1 +fi +if grep -q "SEES_TOOL=1" b1.log; then + cat b1.log + echo "FAIL: a library's tool leaked to its consumer without reexport" + exit 1 +fi +grep -q "SEES_TOOL=0" b1.log || { + cat b1.log; echo "FAIL: build.mcpp did not run"; exit 1; } + +# ── 2. reexport: ONE dependency line is enough ────────────────────────────── +cd "$TMP" +cat > lib/mcpp.toml <<'EOF' +[package] +name = "mylib" +version = "0.1.0" + +[build] +sources = ["src/mylib.cpp"] + +[dependencies] +toolpkg = { path = "../toolpkg", tools = ["codegen"], reexport = true } +rulepkg = { path = "../rulepkg", host-module = true, reexport = true } +EOF +# The consumer names neither the tool nor the rule's dependencies — only the +# rule's module, which is what it actually calls. +cat > app/build.mcpp <<'EOF' +import mcpp; +import rulepkg; +int main() { return rulepkg::generate() ? 0 : 1; } +EOF +cd app && rm -rf target +"$MCPP" build > b2.log 2>&1 || { + cat b2.log; echo "FAIL: build with re-exported provisions failed"; exit 1; } +out="$("$MCPP" run 2>&1 | grep '^ANSWER=' | tail -1)" +[[ "$out" == "ANSWER=42" ]] || { + echo "FAIL: the re-exported tool's output was not linked: $out"; exit 1; } + +# The rule is BUILD-TIME only: it must not have been compiled into the binary. +if grep -q "rulepkg" <(nm -C "target"/*/*/bin/app 2>/dev/null || true); then + echo "FAIL: the rule package was linked into the consumer's binary"; exit 1 +fi + +echo "PASS: 193_provision_reexport" diff --git a/tests/e2e/194_build_mcpp_glob_input.sh b/tests/e2e/194_build_mcpp_glob_input.sh new file mode 100755 index 00000000..3fcf73e7 --- /dev/null +++ b/tests/e2e/194_build_mcpp_glob_input.sh @@ -0,0 +1,125 @@ +#!/usr/bin/env bash +# requires: gcc +# 194_build_mcpp_glob_input.sh — #359: an input that is a SET of files. +# +# `build.mcpp`'s re-run key was built from declared FILES and environment +# variables only, both fingerprinted by content. A program that globs its +# inputs therefore could not express "re-run me when a file appears": adding a +# .proto changed no declared file's hash, so the program did not re-run and the +# new file was silently never generated. Measured before the fix: +# `Finished dev in 0.01s`, zero artifacts. That is worse than requiring the +# author to list names, which is why grpc-m chose the explicit list. +# +# Two layers have to agree, and the second is easy to miss: the build.mcpp +# cache AND the project-level fast path, which skips prepare entirely when no +# source is newer than build.ninja. A new .proto moves no existing mtime, so +# the fast path is exactly where this silently did nothing. +# +# Covered here: +# 1. adding a matching file re-runs the program and changes the artifact +# 2. removing one re-runs it too (the set shrank) +# 3. editing a matched file's CONTENT does not re-run on the glob's account +# — content is what an ordinary rerun-if-changed entry is for +# 4. the program's own outputs live inside the project and must never be part +# of the set, or the build would re-run forever +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +cd "$TMP" + +mkdir -p app/src app/inputs +cat > app/mcpp.toml <<'EOF' +[package] +name = "app" +version = "0.1.0" +EOF +cat > app/src/main.cpp <<'EOF' +#include +int count_inputs(); +int main() { std::printf("COUNT=%d\n", count_inputs()); } +EOF +# The program watches a directory of ".in" files and generates one function +# returning how many it found. It never names them: that is the point. +cat > app/build.mcpp <<'EOF' +import std; +import mcpp; +int main() { + namespace fs = std::filesystem; + mcpp::rerun_if_changed_glob("inputs/**/*.in"); + std::string root = mcpp::manifest_dir(); + int n = 0; + std::error_code ec; + for (auto const& e : fs::recursive_directory_iterator(root + "/inputs", ec)) + if (e.path().extension() == ".in") ++n; + std::string out = std::string(mcpp::out_dir()) + "/count.cpp"; + { std::ofstream os(out); os << "int count_inputs() { return " << n << "; }\n"; } + // Written into the project tree on purpose: a real rule does this, and it + // is what would make a naive glob re-run forever. + { std::ofstream os(root + "/marker.txt"); os << n; } + mcpp::generated(out.c_str()); + return 0; +} +EOF + +cd app +# Logs live OUTSIDE the project: the wide-glob case below asserts that the +# build's own outputs do not perturb the set, and a redirect creating b6.log +# inside the tree would perturb it for real — the test would then be checking +# the shell, not mcpp. +LOGS="$TMP/logs"; mkdir -p "$LOGS" +printf 'a\n' > inputs/a.in + +ran() { grep -q "build.mcpp running" "$1"; } +cached() { grep -q "build.mcpp.*cached" "$1"; } + +"$MCPP" build > "$LOGS/b1.log" 2>&1 || { cat "$LOGS/b1.log"; echo "FAIL: first build failed"; exit 1; } +out="$("$MCPP" run 2>&1 | grep '^COUNT=' | tail -1)" +[[ "$out" == "COUNT=1" ]] || { echo "FAIL: expected COUNT=1, got '$out'"; exit 1; } + +# ── 1. a new matching file re-runs the program ────────────────────────────── +printf 'b\n' > inputs/b.in +"$MCPP" build > "$LOGS/b2.log" 2>&1 || { cat "$LOGS/b2.log"; echo "FAIL: build after adding a file failed"; exit 1; } +ran "$LOGS/b2.log" || { + cat "$LOGS/b2.log" + echo "FAIL: adding a matching file did not re-run build.mcpp" + exit 1 +} +out="$("$MCPP" run 2>&1 | grep '^COUNT=' | tail -1)" +[[ "$out" == "COUNT=2" ]] || { echo "FAIL: expected COUNT=2, got '$out'"; exit 1; } + +# ── 2. removing one re-runs it as well ───────────────────────────────────── +rm inputs/b.in +"$MCPP" build > "$LOGS/b3.log" 2>&1 || { cat "$LOGS/b3.log"; echo "FAIL: build after removing a file failed"; exit 1; } +ran "$LOGS/b3.log" || { cat "$LOGS/b3.log"; echo "FAIL: removing a matching file did not re-run build.mcpp"; exit 1; } +out="$("$MCPP" run 2>&1 | grep '^COUNT=' | tail -1)" +[[ "$out" == "COUNT=1" ]] || { echo "FAIL: expected COUNT=1 after removal, got '$out'"; exit 1; } + +# ── 3. editing content does not re-run on the glob's account ─────────────── +# The set is unchanged, so the glob has nothing to say. (A program that cares +# about contents declares the file with rerun_if_changed, which hashes them.) +printf 'a totally different body\n' > inputs/a.in +"$MCPP" build > "$LOGS/b4.log" 2>&1 || { cat "$LOGS/b4.log"; echo "FAIL: build after an edit failed"; exit 1; } +ran "$LOGS/b4.log" && { + cat "$LOGS/b4.log" + echo "FAIL: editing a matched file's content re-ran the program via the glob" + exit 1 +} + +# ── 4. no re-run loop from the program's own outputs ─────────────────────── +# marker.txt is written into the project by the program itself, and target/ +# holds everything it generated. With a deliberately maximal pattern, a second +# build with nothing changed must still be a no-op; if the output tree were +# part of the set it never would be. +sed 's|"inputs/\*\*/\*\.in"|"**"|' build.mcpp > build.mcpp.new +mv build.mcpp.new build.mcpp +"$MCPP" build > "$LOGS/b5.log" 2>&1 || { cat "$LOGS/b5.log"; echo "FAIL: wide-glob build failed"; exit 1; } +ran "$LOGS/b5.log" || { cat "$LOGS/b5.log"; echo "FAIL: the edited program did not run"; exit 1; } +"$MCPP" build > "$LOGS/b6.log" 2>&1 || { cat "$LOGS/b6.log"; echo "FAIL: second wide-glob build failed"; exit 1; } +ran "$LOGS/b6.log" && { + cat "$LOGS/b6.log" + echo "FAIL: a wide glob re-runs forever — the build output tree is in its set" + exit 1 +} + +echo "PASS: 194_build_mcpp_glob_input" diff --git a/tests/e2e/195_target_cfg_feature_deps.sh b/tests/e2e/195_target_cfg_feature_deps.sh new file mode 100755 index 00000000..da1c4495 --- /dev/null +++ b/tests/e2e/195_target_cfg_feature_deps.sh @@ -0,0 +1,138 @@ +#!/usr/bin/env bash +# 195_target_cfg_feature_deps.sh — #359 D3a: `[target..feature-deps.]`. +# +# The conditional channel carried `dependencies`, `dev-dependencies` and +# `build-dependencies` and silently lacked `feature-deps`. That is the shape +# ConditionalConfig's own comment records for #258: the conditional reader +# keeps its own subset of the keys and falls behind without anyone noticing. +# +# It became load-bearing once a library could re-export build-time provisions. +# A library that puts a host tool behind a feature has no other way to say "not +# on this platform", and an unconditional declaration turns an unsupported +# platform into a hard error raised from inside the LIBRARY's manifest — which +# its user cannot edit and cannot work around. +# +# Two properties: +# 1. a matching predicate's feature-deps are pulled in when the feature is on +# 2. a non-matching predicate's are not — but the FEATURE still exists, so +# requesting it on that platform is not an unknown-feature error +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +cd "$TMP" + +mkdir -p widget/src +cat > widget/mcpp.toml <<'EOF' +[package] +name = "widget" +version = "0.1.0" +[targets.widget] +kind = "lib" +EOF +cat > widget/src/widget.cppm <<'EOF' +export module widget; +export int widget_anchor() { return 0; } +EOF + +mkdir -p lib/src +cat > lib/mcpp.toml <<'EOF' +[package] +name = "mylib" +version = "0.1.0" + +[build] +sources = ["src/mylib.cpp"] + +# Exactly one of these matches on any of the three CI platforms, so the +# `codegen` feature always pulls widget in. +[target.'cfg(unix)'.feature-deps.codegen] +widget = { path = "../widget", visibility = "public" } +[target.'cfg(windows)'.feature-deps.codegen] +widget = { path = "../widget", visibility = "public" } + +# Never matches. If a non-matching section were merged, resolving this bogus +# path would fail the build. +[target.'cfg(arch = "no_such_arch")'.feature-deps.codegen] +ghost = { path = "../this_path_does_not_exist" } +EOF +printf 'int mylib_fn(){return 1;}\n' > lib/src/mylib.cpp + +# ── 1. feature on: the matching platform's feature-dep is resolved ────────── +mkdir -p app/src +cat > app/mcpp.toml <<'EOF' +[package] +name = "app" +version = "0.1.0" + +[dependencies] +mylib = { path = "../lib", features = ["codegen"] } +EOF +cat > app/src/main.cpp <<'EOF' +import widget; // only reachable if the cfg-matched feature-dep resolved +int main() { return widget_anchor(); } +EOF +cd app +"$MCPP" build > b1.log 2>&1 || { + cat b1.log + echo "FAIL: a matching [target.*.feature-deps] was not merged" + exit 1 +} +if [ -f mcpp.lock ] && grep -q 'ghost' mcpp.lock; then + echo "FAIL: a non-matching predicate's feature-dep was pulled in"; exit 1 +fi + +# ── 2. feature off: nothing is pulled in ─────────────────────────────────── +cd "$TMP" +mkdir -p plain/src +cat > plain/mcpp.toml <<'EOF' +[package] +name = "plain" +version = "0.1.0" + +[dependencies] +mylib = { path = "../lib" } +EOF +printf 'int main(){}\n' > plain/src/main.cpp +cd plain +"$MCPP" build > b2.log 2>&1 || { cat b2.log; echo "FAIL: plain build failed"; exit 1; } +if [ -f mcpp.lock ] && grep -q 'widget' mcpp.lock; then + echo "FAIL: an inactive feature's conditional dep was resolved anyway"; exit 1 +fi + +# ── 3. the feature exists on EVERY platform ──────────────────────────────── +# Only what it pulls in is conditional. A consumer requesting it where no +# predicate matches must not be told the feature does not exist — that error +# would be the library's platform support leaking into its user's manifest. +cd "$TMP" +mkdir -p nomatch/src +cat > nomatch/mcpp.toml <<'EOF' +[package] +name = "nomatchlib" +version = "0.1.0" + +[build] +sources = ["src/nm.cpp"] + +[target.'cfg(arch = "no_such_arch")'.feature-deps.codegen] +ghost = { path = "../this_path_does_not_exist" } +EOF +printf 'int nm(){return 1;}\n' > nomatch/src/nm.cpp +mkdir -p nmapp/src +cat > nmapp/mcpp.toml <<'EOF' +[package] +name = "nmapp" +version = "0.1.0" + +[dependencies] +nomatchlib = { path = "../nomatch", features = ["codegen"] } +EOF +printf 'int main(){}\n' > nmapp/src/main.cpp +cd nmapp +"$MCPP" build --strict > b3.log 2>&1 || { + cat b3.log + echo "FAIL: requesting a conditionally-populated feature failed where no predicate matches" + exit 1 +} + +echo "PASS: 195_target_cfg_feature_deps" diff --git a/tests/unit/test_build_directives.cpp b/tests/unit/test_build_directives.cpp index f8f642b0..f94b2340 100644 --- a/tests/unit/test_build_directives.cpp +++ b/tests/unit/test_build_directives.cpp @@ -360,3 +360,104 @@ TEST(BuildDirectives, RunTimeoutDefaultsToABoundAndIsOverridable) { EXPECT_GT(dirs::run_timeout().count(), 0); EXPECT_EQ(dirs::run_timeout().count(), dirs::kDefaultRunTimeoutSecs * 1000); } + +// ── Glob inputs (#359) ───────────────────────────────────────────────────── +// +// A build program that globs its inputs was structurally unsafe: adding a +// .proto changed no declared file's hash, so the program did not re-run and +// the new file was silently never generated. Measured before the fix: +// `Finished dev in 0.01s`, zero artifacts. +// +// The fingerprint is the SET of matching paths. These tests pin what is in it +// and — just as importantly — what is not. + +namespace { + +struct GlobTree { + std::filesystem::path root; + explicit GlobTree(std::string_view name) { + root = std::filesystem::temp_directory_path() / name; + std::filesystem::remove_all(root); + std::filesystem::create_directories(root / "proto"); + } + ~GlobTree() { std::error_code ec; std::filesystem::remove_all(root, ec); } + void write(std::string_view rel, std::string_view text) { + auto p = root / rel; + std::filesystem::create_directories(p.parent_path()); + std::ofstream os(p, std::ios::trunc); + os << text; + } + std::string fp(std::string_view pattern, std::string_view outDir = "target") { + return dirs::glob_fingerprint(root, pattern, outDir); + } +}; + +} // namespace + +TEST(BuildDirectives, GlobFingerprintChangesWhenAFileAppearsOrDisappears) { + GlobTree t{"mcpp_glob_fp_membership"}; + t.write("proto/a.proto", "syntax=\"proto3\";"); + auto one = t.fp("proto/**/*.proto"); + + t.write("proto/b.proto", "syntax=\"proto3\";"); + auto two = t.fp("proto/**/*.proto"); + EXPECT_NE(one, two); + + std::filesystem::remove(t.root / "proto/b.proto"); + EXPECT_EQ(t.fp("proto/**/*.proto"), one); +} + +TEST(BuildDirectives, GlobFingerprintIgnoresContentSizeAndTimestamp) { + // Contents are covered by the ordinary `rerun-if-changed` entry for that + // file. Folding them in here would only add false re-runs — and mtime is + // unstable across git checkout, container builds and rsync, which this + // project has already paid for once (the file_time_type epoch in the + // dependency cache). + GlobTree t{"mcpp_glob_fp_content"}; + t.write("proto/a.proto", "syntax=\"proto3\";"); + auto before = t.fp("proto/**/*.proto"); + t.write("proto/a.proto", "syntax=\"proto3\"; message Much { string longer = 1; }"); + std::filesystem::last_write_time( + t.root / "proto/a.proto", + std::filesystem::file_time_type::clock::now() + std::chrono::hours(1)); + EXPECT_EQ(t.fp("proto/**/*.proto"), before); +} + +TEST(BuildDirectives, GlobFingerprintNeverWalksTheBuildOutputTree) { + // A build program writes its outputs INSIDE the project. If a wide pattern + // included them the set would change on every build and the program would + // re-run forever — the classic Cargo footgun. Enforced by the engine + // rather than left to the author's pattern. + GlobTree t{"mcpp_glob_fp_outdir"}; + t.write("proto/a.proto", "x"); + auto before = t.fp("**"); + t.write("target/.build-mcpp/out/a.pb.cc", "generated"); + t.write("target/.build-mcpp/out/a.pb.h", "generated"); + EXPECT_EQ(t.fp("**"), before); + + // .git is excluded for the same reason: it changes on every commit and + // never means the build program's inputs changed. + t.write(".git/HEAD", "ref: refs/heads/main"); + EXPECT_EQ(t.fp("**"), before); +} + +TEST(BuildDirectives, GlobFingerprintIsIndependentOfDirectoryIterationOrder) { + // The set is sorted before hashing, so two trees with the same members + // agree regardless of the order the platform hands them back. + GlobTree a{"mcpp_glob_fp_order_a"}; + GlobTree b{"mcpp_glob_fp_order_b"}; + for (auto n : { "z.proto", "a.proto", "m.proto" }) a.write(std::string("proto/") + n, "x"); + for (auto n : { "a.proto", "m.proto", "z.proto" }) b.write(std::string("proto/") + n, "y"); + EXPECT_EQ(a.fp("proto/**/*.proto"), b.fp("proto/**/*.proto")); +} + +TEST(BuildDirectives, GlobDirectiveParsesIntoItsOwnSlot) { + auto d = parse("mcpp:rerun-if-changed-glob=proto/**/*.proto\n"); + ASSERT_EQ(d.at(dirs::Slot::RerunGlobs).size(), 1u); + EXPECT_EQ(d.at(dirs::Slot::RerunGlobs)[0], "proto/**/*.proto"); + // A re-run key is not a build input, so it must not be persisted as a `d` + // record — otherwise a cache hit would replay it as one. + std::ostringstream os; + dirs::serialize(os, d); + EXPECT_EQ(os.str().find("proto/**"), std::string::npos) << os.str(); +} diff --git a/tests/unit/test_manifest.cpp b/tests/unit/test_manifest.cpp index 08e6587b..f7cb8683 100644 --- a/tests/unit/test_manifest.cpp +++ b/tests/unit/test_manifest.cpp @@ -2,6 +2,7 @@ import std; import mcpp.manifest; +import mcpp.pm.dep_spec; import mcpp.platform.axis; import mcpp.platform; @@ -3034,3 +3035,128 @@ pinned = { url = "https://example.com/i.git", artifact = "https://example.com/re EXPECT_EQ(pinned.artifact, "https://example.com/res/i"); EXPECT_FALSE(pinned.artifact_applicable()); } + +// #359: `reexport = true` on a dependency edge — the knob that lets a library +// hand its build-time provisions (tools, host module, dependency dir) to its +// own consumers. Off by default, deliberately: the edge's `visibility` already +// defaults to "public", so riding that would have made every dependency at +// every depth able to push entries into a consumer's tool namespace silently. +TEST(Manifest, DependencyReexportIsOptIn) { + auto tmp = std::filesystem::temp_directory_path() / "mcpp_dep_reexport"; + std::filesystem::create_directories(tmp); + auto path = tmp / "mcpp.toml"; + { + std::ofstream os(path); + os << R"( +[package] +name = "reexporter" +version = "0.1.0" + +[dependencies.compat] +protobuf = { version = "35.1", tools = ["protoc"], reexport = true } +zlib = { version = "1.3.1", tools = ["minigzip"] } +)"; + } + auto m = mcpp::manifest::load(path); + ASSERT_TRUE(m) << (m ? "" : m.error().message); + + const mcpp::pm::DependencySpec* pb = nullptr; + const mcpp::pm::DependencySpec* zl = nullptr; + for (auto const& [k, spec] : m->dependencies) { + if (spec.shortName == "protobuf") pb = &spec; + if (spec.shortName == "zlib") zl = &spec; + } + ASSERT_NE(pb, nullptr); + ASSERT_NE(zl, nullptr); + EXPECT_TRUE(pb->reexport); + EXPECT_FALSE(zl->reexport); +} + +// #359 D3a: the conditional channel carried three of the four dependency maps +// and silently lacked `feature-deps`. A library that puts a host tool behind a +// feature has no other way to say "not on this platform", and an unconditional +// declaration turns an unsupported platform into a hard error raised from +// inside the LIBRARY's manifest, which its user cannot work around. +// +// The FEATURE is registered regardless of the predicate — only what it pulls +// in is conditional — so requesting it on a non-matching platform is not an +// unknown-feature error. +TEST(Manifest, ConditionalFeatureDepsAreParsedAndTheFeatureIsRegistered) { + auto tmp = std::filesystem::temp_directory_path() / "mcpp_cond_feature_deps"; + std::filesystem::create_directories(tmp); + auto path = tmp / "mcpp.toml"; + { + std::ofstream os(path); + os << R"( +[package] +name = "condfeatdeps" +version = "0.1.0" + +[target.'cfg(not(windows))'.feature-deps.codegen] +"compat.protobuf" = { version = "35.1", tools = ["protoc"], reexport = true } +)"; + } + auto m = mcpp::manifest::load(path); + ASSERT_TRUE(m) << (m ? "" : m.error().message); + + EXPECT_TRUE(m->featuresMap.contains("codegen")); + // Not folded into the unconditional map: that happens in prepare, after + // the predicate is evaluated against the RESOLVED target. + EXPECT_TRUE(m->featureDeps.empty()); + + ASSERT_EQ(m->conditionalConfigs.size(), 1u); + auto const& cc = m->conditionalConfigs[0]; + EXPECT_EQ(cc.predicate, "cfg(not(windows))"); + ASSERT_TRUE(cc.featureDeps.contains("codegen")); + ASSERT_EQ(cc.featureDeps.at("codegen").size(), 1u); + auto const& spec = cc.featureDeps.at("codegen").begin()->second; + EXPECT_EQ(spec.shortName, "protobuf"); + ASSERT_EQ(spec.tools.size(), 1u); + EXPECT_EQ(spec.tools[0], "protoc"); + EXPECT_TRUE(spec.reexport); +} + +// A dependency-spec key must be listed in TWO places in the TOML reader: the +// predicate that tells an inline spec from a nested namespace table, and the +// filler that reads the value. Miss the first and the diagnostic is actively +// misleading — the table is taken for a namespace and the user is told their +// bool "must be a string, inline dep table, or nested table". #359 hit exactly +// that while adding `reexport`. +// +// One manifest using every key at once is the cheapest way to keep the two in +// sync: a key added to the filler but not the predicate fails this test. +TEST(Manifest, EveryDependencySpecKeyIsAccepted) { + auto tmp = std::filesystem::temp_directory_path() / "mcpp_dep_spec_keys"; + std::filesystem::create_directories(tmp); + auto path = tmp / "mcpp.toml"; + { + std::ofstream os(path); + os << R"( +[package] +name = "depspeckeys" +version = "0.1.0" + +[dependencies.compat] +everything = { version = "1.0.0", features = ["x"], default-features = false, visibility = "private", backend = "openblas", tools = ["t"], host-module = true, reexport = true } +bygit = { git = "https://example.invalid/x.git", tag = "v1", visibility = "interface" } +bypath = { path = "../sibling" } +)"; + } + auto m = mcpp::manifest::load(path); + ASSERT_TRUE(m) << (m ? "" : m.error().message); + + const mcpp::pm::DependencySpec* all = nullptr; + for (auto const& [k, spec] : m->dependencies) + if (spec.shortName == "everything") all = &spec; + ASSERT_NE(all, nullptr); + EXPECT_EQ(all->version, "1.0.0"); + EXPECT_EQ(all->visibility, "private"); + EXPECT_FALSE(all->defaultFeatures); + EXPECT_TRUE(all->hostModule); + EXPECT_TRUE(all->reexport); + ASSERT_EQ(all->tools.size(), 1u); + EXPECT_EQ(all->tools[0], "t"); + // `backend = "openblas"` is sugar for requesting the backend- feature. + EXPECT_NE(std::find(all->features.begin(), all->features.end(), "backend-openblas"), + all->features.end()); +} diff --git a/tests/unit/test_provisions.cpp b/tests/unit/test_provisions.cpp new file mode 100644 index 00000000..3d4bda6f --- /dev/null +++ b/tests/unit/test_provisions.cpp @@ -0,0 +1,208 @@ +#include + +import std; +import mcpp.build.provisions; + +// A build-time provision — a host tool, a host build rule, a dependency's +// directory — is something a dependency hands to its consumer's build PROGRAM. +// Before #359 each kind invented its own reach, and none could be re-exported, +// so a library could not stand up a toolchain on its user's behalf. +// +// Two properties are load-bearing and both are supply-chain properties, which +// is why they are pinned here rather than left to an e2e: +// * nothing propagates unless the edge says `reexport = true`; +// * an unqualified name is bound by a fixed ladder, never by "whoever was +// appended last" — otherwise two libraries that have never heard of each +// other could decide which `protoc` runs. + +namespace prov = mcpp::build::provisions; + +namespace { + +struct Edge { + std::size_t consumerPackageIndex = 0; + std::size_t dependencyPackageIndex = 0; + std::vector requestedTools; + bool hostModule = false; + bool reexport = false; +}; + +bool sees_tool(const prov::Propagation& p, std::size_t consumer, + std::size_t provider, std::string_view tool) { + if (consumer >= p.visible.size()) return false; + return p.visible[consumer].contains( + prov::Provision{ prov::Kind::Tool, provider, std::string(tool) }); +} + +bool sees_dir(const prov::Propagation& p, std::size_t consumer, + std::size_t provider) { + if (consumer >= p.visible.size()) return false; + return p.visible[consumer].contains( + prov::Provision{ prov::Kind::DepDir, provider, {} }); +} + +} // namespace + +// ── Table integrity ──────────────────────────────────────────────────────── + +TEST(Provisions, EveryKindAnswersBothQuestions) { + // The table exists so a new provision kind cannot be added without someone + // stating how far it travels. A row that answered neither would compile + // fine and silently pick a default, which is the exact failure #359 fixed. + std::set seen; + for (auto const& d : prov::kTable) { + EXPECT_FALSE(d.name.empty()); + EXPECT_TRUE(d.needsReexport) << d.name; + seen.insert(d.kind); + } + EXPECT_EQ(seen.size(), 3u); + EXPECT_TRUE(prov::def_of(prov::Kind::Tool).bareAddressable); + EXPECT_TRUE(prov::def_of(prov::Kind::DepDir).bareAddressable); + // A host module is addressed by `import ;` — the compiler resolves + // it, so there is no bare-name channel for mcpp to bind. + EXPECT_FALSE(prov::def_of(prov::Kind::HostModule).bareAddressable); +} + +// ── Propagation ──────────────────────────────────────────────────────────── + +TEST(Provisions, ToolIsVisibleToTheRequesterWithoutAnyReexport) { + // 0=root, 1=protobuf. root --tools=[protoc]--> protobuf + std::vector edges{ { 0, 1, { "protoc" }, false, false } }; + auto p = prov::propagate(edges, 2); + EXPECT_TRUE(sees_tool(p, 0, 1, "protoc")); + // Nothing to export: the root has no consumers, and it never said it + // re-exported anything anyway. + EXPECT_TRUE(p.exported[0].empty()); +} + +TEST(Provisions, WithoutReexportALibrarysToolStaysWithTheLibrary) { + // 0=app, 1=lib, 2=protobuf. app -> lib --tools=[protoc]--> protobuf + std::vector edges{ + { 0, 1, {}, false, false }, + { 1, 2, { "protoc" }, false, /*reexport=*/false }, + }; + auto p = prov::propagate(edges, 3); + EXPECT_TRUE(sees_tool(p, 1, 2, "protoc")); + // THE test: an ordinary dependency must not push entries into its + // consumer's tool namespace. That is a supply-chain rule, not a + // convenience — which is why `reexport` defaults to false. + EXPECT_FALSE(sees_tool(p, 0, 2, "protoc")); +} + +TEST(Provisions, ReexportHandsTheToolToTheLibrarysConsumer) { + std::vector edges{ + { 0, 1, {}, false, false }, + { 1, 2, { "protoc" }, false, /*reexport=*/true }, + }; + auto p = prov::propagate(edges, 3); + EXPECT_TRUE(sees_tool(p, 0, 2, "protoc")); + EXPECT_TRUE(sees_tool(p, 1, 2, "protoc")); +} + +TEST(Provisions, ReexportTravelsFurtherOnlyWhenEachHopSaysSo) { + // 0=app, 1=mid, 2=lib, 3=protobuf. + // lib re-exports protoc to mid; mid does NOT re-export to app. + std::vector edges{ + { 0, 1, {}, false, /*reexport=*/false }, + { 1, 2, {}, false, /*reexport=*/false }, + { 2, 3, { "protoc" }, false, /*reexport=*/true }, + }; + auto p = prov::propagate(edges, 4); + EXPECT_TRUE(sees_tool(p, 2, 3, "protoc")); + EXPECT_TRUE(sees_tool(p, 1, 3, "protoc")); + EXPECT_FALSE(sees_tool(p, 0, 3, "protoc")); + + // Flip the middle hop and it reaches all the way. Each package decides + // what IT hands to ITS consumers; nothing decides on someone else's + // behalf. + edges[1].reexport = true; + auto q = prov::propagate(edges, 4); + EXPECT_TRUE(sees_tool(q, 0, 3, "protoc")); +} + +TEST(Provisions, HostModuleFollowsTheSameRule) { + std::vector edges{ + { 0, 1, {}, false, false }, + { 1, 2, {}, /*hostModule=*/true, /*reexport=*/true }, + }; + auto p = prov::propagate(edges, 3); + const prov::Provision rule{ prov::Kind::HostModule, 2, {} }; + EXPECT_TRUE(p.visible[0].contains(rule)); + EXPECT_TRUE(p.visible[1].contains(rule)); +} + +TEST(Provisions, EveryEdgeExposesTheDependencyDirectory) { + // dep_dir() has always worked for a DIRECT dependency without anyone + // declaring anything; that stays true. Re-export is what extends it. + std::vector edges{ + { 0, 1, {}, false, false }, + { 1, 2, {}, false, /*reexport=*/true }, + { 1, 3, {}, false, /*reexport=*/false }, + }; + auto p = prov::propagate(edges, 4); + EXPECT_TRUE(sees_dir(p, 0, 1)); + EXPECT_TRUE(sees_dir(p, 0, 2)); + EXPECT_FALSE(sees_dir(p, 0, 3)); + EXPECT_TRUE(sees_dir(p, 1, 3)); +} + +TEST(Provisions, ACycleTerminates) { + // Resolution should not produce one, but the fixpoint must not depend on + // that: sets only grow and are bounded, so a cycle simply stops changing. + std::vector edges{ + { 0, 1, { "a" }, false, true }, + { 1, 0, { "b" }, false, true }, + }; + auto p = prov::propagate(edges, 2); + EXPECT_TRUE(sees_tool(p, 0, 1, "a")); + EXPECT_TRUE(sees_tool(p, 1, 0, "b")); +} + +// ── Bare-name binding ────────────────────────────────────────────────────── + +TEST(Provisions, ALoneCandidateOwnsItsBareName) { + auto b = prov::bind_bare_names({ "compat.protobuf" }); + ASSERT_TRUE(b.contains("protobuf")); + EXPECT_EQ(b["protobuf"].owner, "compat.protobuf"); + EXPECT_FALSE(b["protobuf"].contested); + EXPECT_TRUE(prov::contest_note("protobuf", b["protobuf"]).empty()); +} + +TEST(Provisions, ANonDefaultNamespaceStillGetsItsBareName) { + // grpc-m's rule calls dep_bin("grpc-plugin", …); (grpc, grpc-plugin) is on + // no rung of the package-identity ladder, so without the + // unique-candidate rung the spelling already in the wild would break. + auto b = prov::bind_bare_names({ "grpc.grpc-plugin" }); + EXPECT_EQ(b["grpc-plugin"].owner, "grpc.grpc-plugin"); +} + +TEST(Provisions, TheLadderPicksTheDefaultNamespaceFirst) { + auto b = prov::bind_bare_names({ "compat.zlib", "mcpplibs.zlib", "acme.zlib" }); + EXPECT_EQ(b["zlib"].owner, "mcpplibs.zlib"); + EXPECT_TRUE(b["zlib"].contested); + // Contested but bound: the user is told which one won and how to name the + // other, rather than getting whichever the loop appended last. + auto note = prov::contest_note("zlib", b["zlib"]); + EXPECT_NE(note.find("mcpplibs.zlib"), std::string::npos); + EXPECT_NE(note.find("acme.zlib"), std::string::npos); +} + +TEST(Provisions, CompatIsTheSecondRung) { + auto b = prov::bind_bare_names({ "compat.zlib", "acme.zlib" }); + EXPECT_EQ(b["zlib"].owner, "compat.zlib"); +} + +TEST(Provisions, TwoUnreachableNamespacesLeaveTheBareNameUnbound) { + // Neither is on the ladder and neither is unique. Binding either one would + // be an arbitrary choice that decides which binary runs. + auto b = prov::bind_bare_names({ "acme.protoc-ish", "other.protoc-ish" }); + EXPECT_TRUE(b["protoc-ish"].owner.empty()); + EXPECT_TRUE(b["protoc-ish"].contested); + auto note = prov::contest_note("protoc-ish", b["protoc-ish"]); + EXPECT_NE(note.find("NOT"), std::string::npos); +} + +TEST(Provisions, AnUnnamespacedPackageIsTheThirdRung) { + auto b = prov::bind_bare_names({ "grpcgen", "acme.grpcgen" }); + EXPECT_EQ(b["grpcgen"].owner, "grpcgen"); +} From c1a3e451dbbecc33be401b0a52030076c53ab056 Mon Sep 17 00:00:00 2001 From: speak-agent Date: Thu, 6 Aug 2026 15:48:08 +0800 Subject: [PATCH 2/9] =?UTF-8?q?fix(ci):=20=E8=87=AA=E4=B8=BE=20pin=20?= =?UTF-8?q?=E6=8C=87=E5=90=91=E7=B4=A2=E5=BC=95=E9=87=8C=E5=B7=B2=E7=BB=8F?= =?UTF-8?q?=E4=B8=8D=E5=AD=98=E5=9C=A8=E7=9A=84=E7=89=88=E6=9C=AC?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `ci-aarch64-fresh-install` 的 self-host 步报 `xlings: version '2026.8.5.3' not found for 'mcpp' — available: 2026.8.6.1`。 自举 pin 是「从哪个已发布版本开始自举」,按设计不随每次发布走;但它是**上界 之下的选择**,不是一个可以无限期不动的常量 —— xim-pkgindex 对 mcpp 只暴露最 新条目,pin 落在它之外就成了「装不上」。判据因此是「该版本此刻仍可解析」, 而不是「它是不是最新」。 --- .xlings.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.xlings.json b/.xlings.json index 220779fd..fade76ab 100644 --- a/.xlings.json +++ b/.xlings.json @@ -1,5 +1,5 @@ { "workspace": { - "mcpp": "2026.8.5.3" + "mcpp": "2026.8.6.1" } } From 4838952980ae1d48c32137ca4a4d6c375602a7ba Mon Sep 17 00:00:00 2001 From: speak-agent Date: Thu, 6 Aug 2026 15:51:59 +0800 Subject: [PATCH 3/9] =?UTF-8?q?feat(build):=20feature-dep=20=E4=B8=8E?= =?UTF-8?q?=E5=B7=B2=E5=A3=B0=E6=98=8E=E4=BE=9D=E8=B5=96=E5=90=8C=E9=94=AE?= =?UTF-8?q?=E6=97=B6=E6=8C=89=E5=8A=A0=E6=B3=95=E5=90=88=E5=B9=B6?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `mergeActiveFeatureDeps` 用的是 `try_emplace`,键已存在就丢弃 feature 的 spec。这条规则对 version/path/git 是对的(条件段不该静默覆盖无条件段),对 `tools` / `reexport` 却丢掉了这个 feature 存在的全部理由。 gRPC 就是反例,而且是本 issue 的头号用例:它**无条件**依赖 compat.protobuf, 而 `codegen` feature 需要往**同一条边**加 `tools = ["protoc"], reexport = true`。 挪到无条件条目上不可行 —— 那会让每个 grpc 消费者都构建 protoc(~157 个额外 TU),而这正是 tools 默认关闭要避免的。 于是:tools / features 取并集,host-module / reexport 取或,身份字段不合并。 与逐边 feature 请求本来遵循的规则一致。 e2e 193 补第 3 段覆盖它,并已用「不合并 tools」验证会变红 (`rulepkg: no codegen tool`)。 --- .../2026-08-06-provisions-and-build-inputs.md | 12 +++++ docs/05-mcpp-toml.md | 6 +++ docs/zh/05-mcpp-toml.md | 4 ++ src/build/prepare.cppm | 29 +++++++++- tests/e2e/193_provision_reexport.sh | 54 +++++++++++++++++++ 5 files changed, 104 insertions(+), 1 deletion(-) diff --git a/.agents/docs/2026-08-06-provisions-and-build-inputs.md b/.agents/docs/2026-08-06-provisions-and-build-inputs.md index 6342003d..4b0927e4 100644 --- a/.agents/docs/2026-08-06-provisions-and-build-inputs.md +++ b/.agents/docs/2026-08-06-provisions-and-build-inputs.md @@ -146,6 +146,18 @@ visible(P) = ⋃ {P→D} [ own(P→D) ∪ exported(D) ] 2. **provision 的存在性绑定 feature 激活**,与 `featureDefines`(`prepare.cppm:3812`)同一时机。feature 关闭时不应有任何工具被构建。 3. **`reexport` 只在依赖侧有意义。** root 写它无害但无效果(root 没有消费者)。 +##### feature-dep 与已声明依赖同键时按加法合并 + +`mergeActiveFeatureDeps` 原本用 `try_emplace`,即「键已存在就丢弃 feature 的 +spec」。gRPC 恰好是反例:它**无条件**依赖 `compat.protobuf`,而 `codegen` +feature 需要往**同一条边**加 `tools = ["protoc"], reexport = true`。把这个请求 +挪到无条件条目上不可行(那会让每个消费者都构建 protoc),丢弃又会静默丢掉这个 +feature 存在的全部理由。 + +因此:`tools` / `features` 取并集,`host-module` / `reexport` 取或; +`version` / `path` / `git` 这类**身份字段不合并**,「条件段绝不静默覆盖无条件 +段」这条纪律保持不变。与逐边 feature 请求本来就遵循的规则一致。 + #### D1.3 裸名:走索引那套命名空间阶梯,而不是「谁最后写谁赢」 `env_var_name`(`tool_store.cppm:154`)今天同时发长名与短名:`compat.protobuf` → `MCPP_DEP_COMPAT_PROTOBUF_BIN_PROTOC` 与 `MCPP_DEP_PROTOBUF_BIN_PROTOC`。今天只有 root 亲自声明的工具进环境,撞车在用户眼皮底下;传递传播之后,两条互不相识的库各自提供同名短名工具时,谁赢取决于 vector 的追加顺序,且无任何诊断。这从另一扇门放回了本设计要保住的「版本错配不可表达」性质。 diff --git a/docs/05-mcpp-toml.md b/docs/05-mcpp-toml.md index eee57bb9..e62e483c 100644 --- a/docs/05-mcpp-toml.md +++ b/docs/05-mcpp-toml.md @@ -1106,6 +1106,12 @@ int main() { return grpcgen::generate_all() ? 0 : 1; } - **One hop per declaration.** A re-exported provision reaches the consumers of the package that declared it. For it to travel further, the next package must re-export in turn — each package decides only what *it* hands on. +- **A feature may add a request to a dependency you already declare.** gRPC + depends on protobuf unconditionally and its `codegen` feature adds + `tools = ["protoc"], reexport = true` to that same edge. `tools` and + `features` union, `host-module` and `reexport` OR together; `version` / + `path` / `git` do not merge, so a feature still cannot silently override the + unconditional entry's identity. - **Visibility, not execution.** `dep_bin()` returns a path; whether anything runs is still the consumer's `build.mcpp`'s decision. Nothing changes about who builds the tool or how the tool store is keyed. diff --git a/docs/zh/05-mcpp-toml.md b/docs/zh/05-mcpp-toml.md index 54d00b98..425db082 100644 --- a/docs/zh/05-mcpp-toml.md +++ b/docs/zh/05-mcpp-toml.md @@ -847,6 +847,10 @@ int main() { return grpcgen::generate_all() ? 0 : 1; } 命名空间里塞东西。「把某样东西交给消费者」是一条供应链主张,必须写下来。 - **一次声明只走一跳。** 被再导出的提供物到达声明它的那个包的消费者;要继续 往上走,下一个包必须自己也写 `reexport`。每个包只决定**它**交出什么。 +- **feature 可以往一条已经声明过的依赖上追加请求。** gRPC 无条件依赖 protobuf, + 而它的 `codegen` feature 往同一条边加 `tools = ["protoc"], reexport = true`。 + `tools` 与 `features` 取并集,`host-module` 与 `reexport` 取或;`version` / + `path` / `git` 不合并 —— feature 仍然无法静默覆盖无条件条目的身份。 - **传播的是可见性,不是执行。** `dep_bin()` 只返回路径,跑不跑仍由消费者的 `build.mcpp` 决定;谁构建了这个工具、tool store 怎么做键,都不改变。 - **裸名由阶梯决定,而不是靠运气。** 一旦两个库都能再导出,它们可能同时提供 diff --git a/src/build/prepare.cppm b/src/build/prepare.cppm index e556868e..272e2cfa 100644 --- a/src/build/prepare.cppm +++ b/src/build/prepare.cppm @@ -3148,7 +3148,34 @@ prepare_build(bool print_fingerprint, for (auto& f : activateFeatures(pm, requested, seedDefault)) { auto it = pm.featureDeps.find(f); if (it == pm.featureDeps.end()) continue; - for (auto& [k, spec] : it->second) pm.dependencies.try_emplace(k, spec); + for (auto& [k, spec] : it->second) { + auto [pos, fresh] = pm.dependencies.try_emplace(k, spec); + if (fresh) continue; + // #359: the key already exists unconditionally, and dropping + // the feature's spec here loses REQUESTS the feature exists to + // make. gRPC is the shape: it depends on compat.protobuf + // always, and its `codegen` feature has to add + // `tools = ["protoc"], reexport = true` to that same edge — + // which is precisely what must NOT be paid for by a consumer + // who did not ask for codegen, so moving it to the + // unconditional entry is not an option either. + // + // Additive fields merge; identity fields (version/path/git) do + // not, keeping "a conditional section never silently + // overrides an unconditional one" intact. Same rule the + // per-edge feature request already follows. + auto& dst = pos->second; + for (auto const& t : spec.tools) + if (std::find(dst.tools.begin(), dst.tools.end(), t) + == dst.tools.end()) + dst.tools.push_back(t); + for (auto const& f2 : spec.features) + if (std::find(dst.features.begin(), dst.features.end(), f2) + == dst.features.end()) + dst.features.push_back(f2); + dst.hostModule = dst.hostModule || spec.hostModule; + dst.reexport = dst.reexport || spec.reexport; + } } }; diff --git a/tests/e2e/193_provision_reexport.sh b/tests/e2e/193_provision_reexport.sh index 81dbd854..c6ed3623 100755 --- a/tests/e2e/193_provision_reexport.sh +++ b/tests/e2e/193_provision_reexport.sh @@ -179,4 +179,58 @@ if grep -q "rulepkg" <(nm -C "target"/*/*/bin/app 2>/dev/null || true); then echo "FAIL: the rule package was linked into the consumer's binary"; exit 1 fi +# ── 3. a feature ADDING a request to an already-declared dependency ──────── +# The real shape: gRPC depends on protobuf always, and its `codegen` feature +# has to add `tools = ["protoc"], reexport = true` to that SAME edge. Moving +# the request to the unconditional entry is not an option — it would make every +# consumer build protoc — and dropping the feature's spec (try_emplace keeps +# the existing key) would silently lose the request. +cd "$TMP" +cat > lib/mcpp.toml <<'EOF' +[package] +name = "mylib" +version = "0.1.0" + +[build] +sources = ["src/mylib.cpp"] + +# Unconditional: the library links against this package no matter what. +[dependencies] +toolpkg = { path = "../toolpkg" } +rulepkg = { path = "../rulepkg", host-module = true, reexport = true } + +# The feature adds a REQUEST to the edge above, and nothing else. +[feature-deps.codegen] +toolpkg = { path = "../toolpkg", tools = ["codegen"], reexport = true } +EOF +cat > app/mcpp.toml <<'EOF' +[package] +name = "app" +version = "0.1.0" + +[dependencies] +mylib = { path = "../lib", features = ["codegen"] } +EOF +cd app && rm -rf target +"$MCPP" build > b3.log 2>&1 || { + cat b3.log + echo "FAIL: a feature could not add a tool request to an existing dependency" + exit 1 +} +out="$("$MCPP" run 2>&1 | grep '^ANSWER=' | tail -1)" +[[ "$out" == "ANSWER=42" ]] || { echo "FAIL: expected ANSWER=42, got '$out'"; exit 1; } + +# … and without the feature, nothing is built. +cd "$TMP" +sed 's/, features = \["codegen"\]//' app/mcpp.toml > app/mcpp.toml.new +mv app/mcpp.toml.new app/mcpp.toml +cd app && rm -rf target +if "$MCPP" build > b4.log 2>&1; then + cat b4.log + echo "FAIL: the build succeeded without the codegen feature — the tool ran anyway" + exit 1 +fi +grep -q "host tool" b4.log && { + cat b4.log; echo "FAIL: a tool was built for a consumer that did not enable the feature"; exit 1; } + echo "PASS: 193_provision_reexport" From 6345e3c7ded7c6ab813f8cb6f39ef33fc97536be Mon Sep 17 00:00:00 2001 From: speak-agent Date: Thu, 6 Aug 2026 16:01:13 +0800 Subject: [PATCH 4/9] =?UTF-8?q?test(e2e):=20195=20=E7=94=A8=E8=AF=8A?= =?UTF-8?q?=E6=96=AD=E8=80=8C=E4=B8=8D=E6=98=AF=20--strict=20=E5=88=A4?= =?UTF-8?q?=E5=AE=9A?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `--strict` 会把**所有**降级提升为错误,包括与本测试无关的那些 —— Windows 上 clang 报「this toolchain and platform combination emits no GNU depfile」,于是 195 在 Windows 上因为一件它并不关心的事而红。 本测试要断言的是「没有任何谓词匹配的平台上,请求该 feature 不是未知 feature 错误」,那就直接断言那条诊断不出现。 --- tests/e2e/195_target_cfg_feature_deps.sh | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/tests/e2e/195_target_cfg_feature_deps.sh b/tests/e2e/195_target_cfg_feature_deps.sh index da1c4495..9b3ad322 100755 --- a/tests/e2e/195_target_cfg_feature_deps.sh +++ b/tests/e2e/195_target_cfg_feature_deps.sh @@ -129,10 +129,19 @@ nomatchlib = { path = "../nomatch", features = ["codegen"] } EOF printf 'int main(){}\n' > nmapp/src/main.cpp cd nmapp -"$MCPP" build --strict > b3.log 2>&1 || { +# Asserted on the DIAGNOSTIC rather than with --strict: --strict promotes every +# degradation, including unrelated ones (clang on Windows reports "this +# toolchain and platform combination emits no GNU depfile"), so it would make +# this test fail for a reason it is not about. +"$MCPP" build > b3.log 2>&1 || { cat b3.log echo "FAIL: requesting a conditionally-populated feature failed where no predicate matches" exit 1 } +grep -q "does not declare requested feature" b3.log && { + cat b3.log + echo "FAIL: the feature was reported as undeclared where no predicate matches" + exit 1 +} echo "PASS: 195_target_cfg_feature_deps" From c5cdcec09d99833024e7213b47c036d6a9f3b27e Mon Sep 17 00:00:00 2001 From: speak-agent Date: Thu, 6 Aug 2026 16:10:08 +0800 Subject: [PATCH 5/9] =?UTF-8?q?fix(manifest):=20=E4=B8=8D=E8=AE=A4?= =?UTF-8?q?=E8=AF=86=E7=9A=84=E4=BE=9D=E8=B5=96=E9=94=AE=E9=99=8D=E7=BA=A7?= =?UTF-8?q?=E8=80=8C=E4=B8=8D=E6=98=AF=E8=AE=A9=E6=95=B4=E4=BB=BD=20manife?= =?UTF-8?q?st=20=E5=8A=A0=E8=BD=BD=E5=A4=B1=E8=B4=A5?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `reexport` 要写进**已发布包**的 manifest(grpc 的 [feature-deps.codegen]), 于是冒出一个此前没人问过的问题:比它旧的 mcpp 读到会怎样? 答案曾经是整份加载失败,而且报错误导 —— `reexport = true` 被告知 「must be a string, inline dep table, or nested table」。 根因是**一个谓词兼任两职**:`looks_like_inline_dep_spec` 既判定「内联 spec 还是 嵌套命名空间表」,又枚举「哪些键有意义」。不认识的键因此走不到「未知选项」那条 路上 —— 表直接判不出是 spec,被当成命名空间。 后果不是提示不友好,而是**任何已发布包都永远无法采用新键**。这与 #349 确立的 是同一条性质:数据不得决定程序是否可用。 判据换轴:内联 spec 的判定是「它是否指名了一个来源」(path/version/git/workspace)。 嵌套命名空间表的键是**包名**,不会有包叫 `version`,所以不会误判。识别为 spec 之后,不认识的键记为降级(--strict 仍拒绝),与 xpkg 读取器的 xpkgUnknownKeys 「record rather than swallow」同一条纪律。 救不了已经发布出去的旧客户端,但从这一版起这类问题不再复发。 --- .../2026-08-06-provisions-and-build-inputs.md | 24 ++++++ docs/05-mcpp-toml.md | 13 ++++ docs/zh/05-mcpp-toml.md | 10 +++ src/manifest/toml.cppm | 53 ++++++++++---- tests/unit/test_manifest.cpp | 73 +++++++++++++++++++ 5 files changed, 160 insertions(+), 13 deletions(-) diff --git a/.agents/docs/2026-08-06-provisions-and-build-inputs.md b/.agents/docs/2026-08-06-provisions-and-build-inputs.md index 4b0927e4..43a50d9f 100644 --- a/.agents/docs/2026-08-06-provisions-and-build-inputs.md +++ b/.agents/docs/2026-08-06-provisions-and-build-inputs.md @@ -177,6 +177,30 @@ feature 存在的全部理由。 这与索引解析裸包名的阶梯(`(mcpplibs,X) → (compat,X) → (∅,X)`)是同一条规则的同一次应用;第 3 级是必需的补充,否则 `grpc.grpc-plugin` 这类非默认命名空间的包连裸名都拿不到(grpc-m 现在写的正是 `dep_bin("grpc-plugin", ...)`)。`dep_dir()` 沿用同一套。 +### D1.4 发布一个新键不能让旧客户端加载失败 + +`reexport` 要写进**已发布包**的 manifest(grpc 的 `[feature-deps.codegen]`), +于是冒出一个此前没人问过的问题:比它旧的 mcpp 读到这份 manifest 会怎样? + +答案曾经是**整份加载失败**,而且报错是误导性的: +`reexport = true` 被告知「must be a string, inline dep table, or nested table」。 + +根因是**一个谓词兼任两职**:`looks_like_inline_dep_spec` 既判定「这是内联依赖 +spec 还是嵌套命名空间表」,又枚举「哪些键有意义」。于是「不认识的键」不会走到 +「未知选项」那条路上——表直接判不出是 spec,被当成命名空间。 + +后果不是提示不友好,而是**任何已发布包都永远无法采用新键**。这与 #349 确立的 +性质是同一条:**数据不得决定程序是否可用**。 + +判据换轴:**内联 spec 的判定是「它是否指名了一个来源」**(`path` / `version` / +`git` / `workspace`)。嵌套命名空间表的键是**包名**,不会有包叫 `version`,所以 +这个判据不会误判。识别为 spec 之后,不认识的键**记为降级**(`--strict` 仍拒 +绝),而不是让整份 manifest 加载失败——与 xpkg 读取器的 `xpkgUnknownKeys` +(「record rather than swallow」)同一条纪律。 + +这不能救**已经发布出去的**旧客户端(它们的解析器就是那样),但从这一版起, +这类问题不再复发。 + ### D2. 输入种类进表,指纹由种类决定 新增一种输入:**glob**。 diff --git a/docs/05-mcpp-toml.md b/docs/05-mcpp-toml.md index e62e483c..0695538b 100644 --- a/docs/05-mcpp-toml.md +++ b/docs/05-mcpp-toml.md @@ -1122,6 +1122,19 @@ int main() { return grpcgen::generate_all() ? 0 : 1; } unnamespaced ``, else the single remaining candidate — and when it is contested mcpp says so instead of picking silently. +##### Older mcpp reading a manifest that uses this + +An unrecognized dependency key is reported as a **degradation** and ignored +(mcpp 2026.8.6.2+), so a package written for a newer mcpp still loads and the +parts this reader understands still apply. Before that release it was a hard +load failure with a misleading message, which is why a published package could +not adopt a new key at all — the same property the index floor establishes: +data must not decide whether the program works. + +Consequently a package that *relies* on `reexport` for its ergonomics still +needs a client new enough to implement it; what changed is that everything else +about that package keeps working on an older one. + ##### Scoping a provision per platform A package may declare a `bin` target on some platforms only. Because the diff --git a/docs/zh/05-mcpp-toml.md b/docs/zh/05-mcpp-toml.md index 425db082..4fb251e7 100644 --- a/docs/zh/05-mcpp-toml.md +++ b/docs/zh/05-mcpp-toml.md @@ -858,6 +858,16 @@ int main() { return grpcgen::generate_all() ? 0 : 1; } 依次绑定到 `mcpplibs.`、`compat.`、无命名空间的 ``,最后才是「剩下 的唯一候选」——存在争用时 mcpp 会说出来,而不是默默选一个。 +##### 旧版 mcpp 读到用了这些键的 manifest + +不认识的依赖键会被**记为降级**并忽略(mcpp 2026.8.6.2+),因此一份为更新的 +mcpp 写的包仍然能加载,这个读取器认识的部分照常生效。在那之前它是**整份加载 +失败**且报错误导,这正是「已发布的包永远无法采用新键」的原因——与索引下限确立 +的是同一条性质:**数据不得决定程序是否可用**。 + +因此,一个**依赖** `reexport` 才有那套人机工程的包,仍然需要足够新的客户端; +变化在于该包的其余部分在旧客户端上不再一起失效。 + ##### 按平台裁剪提供物 一个包可能只在部分平台声明 `bin` 目标。既然现在是**库**决定请求什么,无条件的 diff --git a/src/manifest/toml.cppm b/src/manifest/toml.cppm index 6495a884..dac1d68d 100644 --- a/src/manifest/toml.cppm +++ b/src/manifest/toml.cppm @@ -537,11 +537,7 @@ std::expected parse_string(std::string_view content, // namespaces (so existing fetcher / lockfile lookups by composite name // keep working) and the bare `` for the default namespace (so the // common case stays unchanged). - // MUST list every key `fill_inline_spec` below reads. The two are one - // decision in two places: this predicate also distinguishes an inline dep - // spec from a NESTED namespace table, so a key missing here does not read - // as "unknown option" — the table is taken for a namespace and the user is - // told their value "must be a string, inline dep table, or nested table". + // MUST list every key `fill_inline_spec` below reads. // `Manifest.EveryDependencySpecKeyIsAccepted` holds the two in sync. auto is_dep_spec_key = [](std::string_view k) { return k == "path" || k == "version" || k == "git" @@ -551,12 +547,30 @@ std::expected parse_string(std::string_view content, || k == "backend" || k == "tools" || k == "host-module" || k == "reexport"; }; - auto looks_like_inline_dep_spec = [&](const t::Table& sub) { + // What makes a table an inline dep spec is that it names a SOURCE. This + // used to be "every key is known", which quietly coupled two unrelated + // things: the discriminator (spec vs nested namespace table) and the + // vocabulary (which keys mean something). + // + // The coupling is a compatibility hazard, not a style problem. A manifest + // using a key introduced after the reader was built did not get "unknown + // option" — the table failed the discriminator, was taken for a NAMESPACE, + // and the user was told their `reexport = true` "must be a string, inline + // dep table, or nested table". Worse, a published package cannot adopt a + // new key at all, because every older client fails to load it outright + // rather than ignoring what it does not understand. That is the same + // property #349 established for the index floor: data must not be able to + // decide whether the program works. + // + // An identity key is an unambiguous discriminator: a nested namespace + // table's keys are PACKAGE names, and no package is named `version` / + // `path` / `git` / `workspace`. + auto looks_like_inline_dep_spec = [](const t::Table& sub) { if (sub.empty()) return false; - for (auto& [sk, sv] : sub) { - if (!is_dep_spec_key(sk)) return false; - } - return true; + for (auto& [sk, sv] : sub) + if (sk == "path" || sk == "version" || sk == "git" || sk == "workspace") + return true; + return false; }; auto fill_inline_spec = [&](DependencySpec& spec, @@ -564,6 +578,18 @@ std::expected parse_string(std::string_view content, std::string_view fqName, const t::Table& sub) -> std::expected { + // Now that the discriminator no longer doubles as the vocabulary, an + // unrecognized key can be REPORTED — as a degradation, so `--strict` + // still refuses it, while an ordinary build of a package written for a + // newer mcpp proceeds with the part this one understands. Same + // discipline as the xpkg reader's `xpkgUnknownKeys`: record rather + // than swallow, and never fail the whole load over it. + for (auto& [sk, sv] : sub) { + if (is_dep_spec_key(sk)) continue; + m.schemaWarnings.push_back(std::format( + "[{}.\"{}\"] has unrecognized key '{}' (ignored). It may be a " + "typo, or a field a newer mcpp understands.", section, fqName, sk)); + } if (auto it = sub.find("path"); it != sub.end() && it->second.is_string()) spec.path = it->second.as_string(); if (auto it = sub.find("version"); it != sub.end() && it->second.is_string()) spec.version = it->second.as_string(); if (auto it = sub.find("git"); it != sub.end() && it->second.is_string()) spec.git = it->second.as_string(); @@ -664,9 +690,10 @@ std::expected parse_string(std::string_view content, auto& sub = value.as_table(); if (!looks_like_inline_dep_spec(sub)) { return std::unexpected(error(origin, std::format( - "[{}.{}] must be a version string or table of " - "(path/version/git/rev/tag/branch/features/default-features/" - "visibility/backend/workspace/tools/host-module/reexport)", + "[{}.{}] must be a version string, or a table naming a " + "source (one of path/version/git/workspace) alongside any " + "of rev/tag/branch/features/default-features/visibility/" + "backend/tools/host-module/reexport", section, key))); } if (auto r = fill_inline_spec(spec, section, key, sub); !r) return r; diff --git a/tests/unit/test_manifest.cpp b/tests/unit/test_manifest.cpp index f7cb8683..b92c8d49 100644 --- a/tests/unit/test_manifest.cpp +++ b/tests/unit/test_manifest.cpp @@ -3160,3 +3160,76 @@ bypath = { path = "../sibling" } EXPECT_NE(std::find(all->features.begin(), all->features.end(), "backend-openblas"), all->features.end()); } + +// #359: a manifest written for a NEWER mcpp must still load. +// +// What made this impossible was that one predicate served two purposes: it +// decided "inline dep spec vs nested namespace table" AND enumerated the +// vocabulary. A key the reader did not know therefore did not read as an +// unknown option — the table failed the discriminator, was taken for a +// namespace, and the user was told their `reexport = true` "must be a string, +// inline dep table, or nested table". A published package could consequently +// never adopt a new key: every older client failed to load it outright. +// +// The discriminator is now "does it name a source", which a namespace table +// cannot accidentally satisfy, and an unrecognized key is a recorded +// degradation (--strict still refuses it). +TEST(Manifest, UnknownDependencyKeyDegradesInsteadOfFailingTheLoad) { + auto tmp = std::filesystem::temp_directory_path() / "mcpp_dep_future_key"; + std::filesystem::create_directories(tmp); + auto path = tmp / "mcpp.toml"; + { + std::ofstream os(path); + os << R"( +[package] +name = "futurekey" +version = "0.1.0" + +[dependencies] +zlib = { version = "1.3.1", some-future-field = true } +)"; + } + auto m = mcpp::manifest::load(path); + ASSERT_TRUE(m) << (m ? "" : m.error().message); + + const mcpp::pm::DependencySpec* z = nullptr; + for (auto const& [k, spec] : m->dependencies) + if (spec.shortName == "zlib") z = &spec; + ASSERT_NE(z, nullptr); + EXPECT_EQ(z->version, "1.3.1"); // the part this reader understands still applies + + bool warned = false; + for (auto const& w : m->schemaWarnings) + if (w.find("some-future-field") != std::string::npos) warned = true; + EXPECT_TRUE(warned) << "the ignored key must be reported, not swallowed"; +} + +// The discriminator must not mistake a namespace subtable for an inline spec. +// `[dependencies.compat] zlib = "1.3.1"` has one key, `zlib`, which names no +// source — so it is a namespace, and the entry below it is the dependency. +TEST(Manifest, NamespaceSubtableIsNotMistakenForAnInlineSpec) { + auto tmp = std::filesystem::temp_directory_path() / "mcpp_ns_vs_spec"; + std::filesystem::create_directories(tmp); + auto path = tmp / "mcpp.toml"; + { + std::ofstream os(path); + os << R"( +[package] +name = "nsvsspec" +version = "0.1.0" + +[dependencies.compat] +zlib = "1.3.1" +gtest = { version = "1.15.2", features = ["main"] } +)"; + } + auto m = mcpp::manifest::load(path); + ASSERT_TRUE(m) << (m ? "" : m.error().message); + std::set got; + for (auto const& [k, spec] : m->dependencies) { + got.insert(spec.shortName); + EXPECT_EQ(spec.namespace_, "compat") << spec.shortName; + } + EXPECT_TRUE(got.contains("zlib")); + EXPECT_TRUE(got.contains("gtest")); +} From ca0425fa679884d6b5f5e1c347901befab8785f1 Mon Sep 17 00:00:00 2001 From: speak-agent Date: Thu, 6 Aug 2026 16:24:47 +0800 Subject: [PATCH 6/9] =?UTF-8?q?fix(toolchain):=20=E5=9B=BA=E5=AE=9A?= =?UTF-8?q?=E7=9A=84=20host=20=E5=B7=A5=E5=85=B7=E9=93=BE=E8=A3=85?= =?UTF-8?q?=E4=B8=8D=E4=B8=8A=E6=97=B6,=E5=9B=9E=E9=80=80=E5=88=B0?= =?UTF-8?q?=E5=B7=B2=E7=BB=8F=E5=9C=A8=E6=9C=AC=E6=9C=BA=E5=8F=AF=E7=94=A8?= =?UTF-8?q?=E7=9A=84=20target=20=E5=B7=A5=E5=85=B7=E9=93=BE?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `ci-aarch64-fresh-install` 在 aarch64 上构建 xlings 时死在: [error] download failed for xim:gcc@16.1.0: HTTP 404 error: host toolchain for build.mcpp ('gcc@16.1.0'): ... 链条是:图里有包带 build.mcpp ⇒ 需要 HOST 编译器 ⇒ 取 `[toolchain] default` ⇒ 那是 `gcc@16.1.0`,而 xim 的 gcc 声明 `archs = { "x86_64" }` ⇒ 404。 `[toolchain]` 的键是 OS,**没有 arch 这根轴**(#367),所以 aarch64 问的和 x86_64 问的是同一个包。结果是:凡是依赖图里出现 build.mcpp 的工程,在 aarch64 上就是死路。 但这台机器上**已经有**一个能用的编译器:当解析出的 target 与 host 只差 libc 环境(aarch64-linux-musl 跑在 aarch64 Linux 上),target 工具链就是本机原生的 —— 它在这里能跑,它产出的二进制在这里也能跑,拿它编译一个构建程序完全成立。 因此:固定的 host 工具链取不到时,若 arch 与 os 与 host 相同,就用 target 工具 链,并发一条命名双方的告警。成功路径一字未改;此前的死路变成一次可解释的降级。 #367 记录的仍是根因(host 工具链选择缺 arch 轴),这条只是让它不再是绝路。 --- src/build/prepare.cppm | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+) diff --git a/src/build/prepare.cppm b/src/build/prepare.cppm index 272e2cfa..780e9752 100644 --- a/src/build/prepare.cppm +++ b/src/build/prepare.cppm @@ -1812,6 +1812,34 @@ prepare_build(bool print_fingerprint, mcpp::fetcher::InstallProgressHandler progress; auto payload = fetcher.resolve_xpkg_path(pkg.target(), /*autoInstall=*/true, &progress); if (!payload) { + // The pinned host toolchain is not obtainable here. Before giving + // up, ask whether we already HAVE a compiler that runs on this + // machine: when the resolved target differs from the host only in + // its libc environment (aarch64-linux-musl on an aarch64 Linux + // host), the target toolchain is native — it runs here and the + // binaries it produces run here — so it is a perfectly good + // compiler for a build program. + // + // Without this, a whole architecture is a dead end for any project + // whose graph contains a build.mcpp: `[toolchain]` is keyed by OS + // with no arch axis (#367), so `default = "gcc@16.1.0"` is what + // aarch64 asks for too, and that package declares + // `archs = { "x86_64" }` — the download 404s and the build stops. + // A toolchain that is already on disk and already correct should + // not be unusable because a second, redundant one is missing. + auto hostT = mcpp::toolchain::triple::host_triple(); + auto tgtT = mcpp::toolchain::triple::parse(overrides.target_triple); + if (tgtT && tgtT->arch == hostT.arch && tgtT->os == hostT.os) { + mcpp::diag::warning("toolchain/host-fallback", std::format( + "the pinned host toolchain '{}' could not be provisioned " + "({}), so build.mcpp will be compiled with the target " + "toolchain instead — it is native here ({} vs host {}). " + "Pin a host toolchain that exists on this architecture to " + "silence this.", + *tcSpec, payload.error().message, tgtT->str(), hostT.str())); + hostTcCache = std::pair{explicit_compiler, *tc}; + return *hostTcCache; + } return std::unexpected(std::format( "host toolchain for build.mcpp ('{}'): {}", *tcSpec, payload.error().message)); From 6d354e8f00ac806c8b3bafcfe7007a9ce75d3bde Mon Sep 17 00:00:00 2001 From: speak-agent Date: Thu, 6 Aug 2026 16:30:52 +0800 Subject: [PATCH 7/9] =?UTF-8?q?test(ci):=20aarch64=20=E8=87=AA=E4=B8=BE?= =?UTF-8?q?=E6=AD=A5=E7=9A=84=20xlings=20=E4=B8=80=E5=8D=8A=E7=94=A8?= =?UTF-8?q?=E7=9A=84=E6=98=AF**=E5=B7=B2=E5=AE=89=E8=A3=85**=E7=9A=84=20mc?= =?UTF-8?q?pp,=E4=B8=8D=E6=98=AF=E8=A2=AB=E5=AE=A1=E7=9A=84=E4=BB=A3?= =?UTF-8?q?=E7=A0=81?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 同一条 job 里,mcpp 用 "$m"(刚构建出来的)验证,紧接着的 xlings 构建却用 裸 `mcpp` —— 即已安装的那个。于是这一半从来没看见过 PR。 这正是上面那段注释为 clone ref 记录过的缺陷,只是位置往下挪了几行;暴露方式 也一样:一个**只在这条构建里出现**的 aarch64 失败,其修复在这里无法验证,因为 跑它的二进制早于该修复。 顺带两处:`m` 改为绝对路径(`cd /tmp/xlings-src` 之后还要用),并显式把 MCPP_HOME 传下去 —— mcpp 从**二进制所在位置**推导 home,放在 /tmp/mcpp-src 下的二进制否则会认领一个空 home,把整套生态重新自举一遍。 --- .../workflows/ci-aarch64-fresh-install.yml | 19 +++++++++++++++++-- 1 file changed, 17 insertions(+), 2 deletions(-) diff --git a/.github/workflows/ci-aarch64-fresh-install.yml b/.github/workflows/ci-aarch64-fresh-install.yml index ad878224..fe3882f3 100644 --- a/.github/workflows/ci-aarch64-fresh-install.yml +++ b/.github/workflows/ci-aarch64-fresh-install.yml @@ -120,12 +120,27 @@ jobs: echo "self-hosting $repo @ $ref" mcpp self config --mirror GLOBAL 2>/dev/null || true mcpp build --target aarch64-linux-musl - m=$(find target/aarch64-linux-musl -type f -path '*/bin/mcpp' | head -1) + # Absolute: it is used again after `cd /tmp/xlings-src` below. + m=$(find "$PWD/target/aarch64-linux-musl" -type f -path '*/bin/mcpp' | head -1) file "$m" | grep -q "ARM aarch64" || { echo "expected aarch64 mcpp"; exit 1; } "$m" --version git clone --depth 1 https://github.com/openxlings/xlings /tmp/xlings-src cd /tmp/xlings-src - mcpp build --target aarch64-linux-musl + # "$m", not `mcpp`: the just-built binary is the code under review, + # and building xlings with the INSTALLED one meant this half of the + # gate never saw the PR — the same defect the clone-ref comment above + # records, one line further down. It surfaced the same way: a fix for + # an aarch64-only failure in exactly this build could not be + # validated here, because the binary running it predated the fix. + # + # MCPP_HOME must be carried over explicitly: mcpp derives it from the + # BINARY's location, so a binary sitting in /tmp/mcpp-src/target would + # otherwise adopt an empty home and re-bootstrap the whole ecosystem + # instead of reusing what the fresh-install steps above provisioned. + export MCPP_HOME=$(mcpp self env | awk -F'= *' '/^MCPP_HOME/{print $2; exit}') + echo "reusing MCPP_HOME=$MCPP_HOME" + test -d "$MCPP_HOME" || { echo "could not determine MCPP_HOME"; exit 1; } + "$m" build --target aarch64-linux-musl x=$(find target/aarch64-linux-musl -type f -path '*/bin/xlings' | head -1) file "$x" | grep -q "ARM aarch64" || { echo "expected aarch64 xlings"; exit 1; } "$x" --version From a4790be2ec7a7507bb9c1ecf129be44f2763a4c3 Mon Sep 17 00:00:00 2001 From: speak-agent Date: Thu, 6 Aug 2026 16:47:07 +0800 Subject: [PATCH 8/9] =?UTF-8?q?fix(toolchain):=20aarch64=20=E4=B8=8A?= =?UTF-8?q?=E5=8E=9F=E7=94=9F=20GCC=20=E7=9A=84=E8=BD=BD=E8=8D=B7=E6=98=AF?= =?UTF-8?q?=20musl-gcc,=E4=B8=8D=E6=98=AF=20glibc=20gcc=20(#367)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `ci-aarch64-fresh-install` 报 `download failed for xim:gcc@16.1.0: HTTP 404`。 **先前的归因是错的**:我判成「`[toolchain]` 缺 arch 轴」,并动手给 manifest 加了 一根选择器轴。生态其实**早就支持 aarch64** —— xlings-res 的 musl-gcc 发行版里 就有 `musl-gcc-16.1.0-linux-aarch64.tar.gz`,而 `xim:gcc` 声明的是 `archs = { "x86_64" }`。所以缺的不是让用户去写 arch,而是 mcpp 问错了包。 同一台机器上 target 一侧是对的: `gcc@16.1.0 → aarch64-linux-musl → xim-x-musl-gcc`。因为 target 一侧把 triple 注入了 spec,`to_xim_package` 的 musl 分支便按「同一 target,两种载荷形态」选中 了原生包。而 build.mcpp 的 **host** 解析刻意不注入 target,于是落到最后那条 glibc 分支,拿到只有 x86_64 资产的 `gcc`。 「这个 spec 在这台机器上对应哪个载荷」正是 `to_xim_package` 存在的意义,上面 那条 musl 分支已经在回答它的另一半。所以修在这里:Linux 上瞄准本机的 GCC spec, 在非 x86_64 架构上解析到 `musl-gcc`。用户不该被要求去编码「某个工具链包是为 哪些架构构建的」。 同时撤回上一版那个「host 工具链装不上就回退到 target 工具链」的兜底 —— 它是 错误归因的产物;正确解析之后它是死代码,而一条失败路径上的隐式回退不该白留。 判定抽成自由函数并接收 host arch,而不是读编译期常量:这样 aarch64 的答案能在 x86_64 机器上被测到 —— 而「测一个手边没有的架构」正是重点。 --- src/build/prepare.cppm | 28 --------------- src/toolchain/registry.cppm | 49 +++++++++++++++++++++++++- tests/unit/test_toolchain_registry.cpp | 37 +++++++++++++++++++ 3 files changed, 85 insertions(+), 29 deletions(-) diff --git a/src/build/prepare.cppm b/src/build/prepare.cppm index 780e9752..272e2cfa 100644 --- a/src/build/prepare.cppm +++ b/src/build/prepare.cppm @@ -1812,34 +1812,6 @@ prepare_build(bool print_fingerprint, mcpp::fetcher::InstallProgressHandler progress; auto payload = fetcher.resolve_xpkg_path(pkg.target(), /*autoInstall=*/true, &progress); if (!payload) { - // The pinned host toolchain is not obtainable here. Before giving - // up, ask whether we already HAVE a compiler that runs on this - // machine: when the resolved target differs from the host only in - // its libc environment (aarch64-linux-musl on an aarch64 Linux - // host), the target toolchain is native — it runs here and the - // binaries it produces run here — so it is a perfectly good - // compiler for a build program. - // - // Without this, a whole architecture is a dead end for any project - // whose graph contains a build.mcpp: `[toolchain]` is keyed by OS - // with no arch axis (#367), so `default = "gcc@16.1.0"` is what - // aarch64 asks for too, and that package declares - // `archs = { "x86_64" }` — the download 404s and the build stops. - // A toolchain that is already on disk and already correct should - // not be unusable because a second, redundant one is missing. - auto hostT = mcpp::toolchain::triple::host_triple(); - auto tgtT = mcpp::toolchain::triple::parse(overrides.target_triple); - if (tgtT && tgtT->arch == hostT.arch && tgtT->os == hostT.os) { - mcpp::diag::warning("toolchain/host-fallback", std::format( - "the pinned host toolchain '{}' could not be provisioned " - "({}), so build.mcpp will be compiled with the target " - "toolchain instead — it is native here ({} vs host {}). " - "Pin a host toolchain that exists on this architecture to " - "silence this.", - *tcSpec, payload.error().message, tgtT->str(), hostT.str())); - hostTcCache = std::pair{explicit_compiler, *tc}; - return *hostTcCache; - } return std::unexpected(std::format( "host toolchain for build.mcpp ('{}'): {}", *tcSpec, payload.error().message)); diff --git a/src/toolchain/registry.cppm b/src/toolchain/registry.cppm index fff4e79f..b3f563ea 100644 --- a/src/toolchain/registry.cppm +++ b/src/toolchain/registry.cppm @@ -86,6 +86,20 @@ void print_compat_hint(const ToolchainSpec& spec); // The (family, target, host) → xim package mapping — the distribution layer. XimToolchainPackage to_xim_package(const ToolchainSpec& spec); +// #367: does a GCC spec aimed at the NATIVE Linux host resolve to the +// `musl-gcc` payload rather than the glibc `gcc` one? +// +// `xim:gcc` publishes x86_64 assets only (`archs = { "x86_64" }`); the GCC the +// ecosystem ships for other Linux architectures is `musl-gcc`, which does +// publish them (aarch64 included, verified against xlings-res). Asking for +// `gcc` on aarch64 404s. +// +// A free function taking the host arch rather than reading the compile-time +// constant, so the aarch64 answer is testable on an x86_64 machine — the whole +// point being an architecture the developer is not sitting in front of. +bool gcc_native_payload_is_musl(std::string_view hostArch, bool isLinux, + const triple::Triple& target); + ToolchainSpec with_resolved_xim_version(const ToolchainSpec& spec, std::string_view ximVersion); @@ -209,6 +223,16 @@ void print_compat_hint(const ToolchainSpec& spec) { compat::print_hint_once(spec.compatHint); } +bool gcc_native_payload_is_musl(std::string_view hostArch, bool isLinux, + const triple::Triple& target) { + if (!isLinux) return false; + if (hostArch == "x86_64") return false; // the glibc payload exists here + // "Native" = no explicit target, or one naming this same machine. A CROSS + // target keeps its own payload rule (the `-gcc` packages above). + return target.empty() + || (target.os == "linux" && target.arch == hostArch); +} + XimToolchainPackage to_xim_package(const ToolchainSpec& spec) { XimToolchainPackage pkg; pkg.displaySpec = spec.display(); @@ -271,7 +295,30 @@ XimToolchainPackage to_xim_package(const ToolchainSpec& spec) { return pkg; } - // Host target (or linux-gnu): the glibc gcc package. + // Host target (or linux-gnu): the glibc gcc package — on x86_64. + // + // #367: `xim:gcc` declares `archs = { "x86_64" }` and publishes assets for + // that arch only. The GCC the ecosystem ships for other Linux + // architectures is `musl-gcc`, which does publish them (aarch64 included). + // Asking for `gcc` on aarch64 therefore 404s — and because a `build.mcpp` + // host compile resolves the toolchain spec with NO target injection, it + // landed here and made every project whose graph contains a build program + // unbuildable on aarch64, with `--target aarch64-linux-musl` fixing only + // the target half. + // + // "Which payload backs this spec on this machine" is exactly the question + // this function exists to answer — the musl branch above already answers + // its half the same way ("same target, two payload shapes"). The knowledge + // belongs here rather than in every manifest: a user should not have to + // encode which architectures a toolchain package was built for. + if (gcc_native_payload_is_musl(mcpp::platform::host_arch, + mcpp::platform::is_linux, t)) { + const auto mt = host_musl_triple(); + pkg.ximName = "musl-gcc"; + pkg.frontendCandidates = { mt.str() + "-g++", "g++" }; + return pkg; // no glibc specs fixup: that payload is not glibc-linked + } + pkg.ximName = "gcc"; pkg.frontendCandidates = {"g++"}; pkg.needsGccPostInstallFixup = true; diff --git a/tests/unit/test_toolchain_registry.cpp b/tests/unit/test_toolchain_registry.cpp index 97afde7e..8e961ae7 100644 --- a/tests/unit/test_toolchain_registry.cpp +++ b/tests/unit/test_toolchain_registry.cpp @@ -3,6 +3,7 @@ import std; import mcpp.platform; import mcpp.toolchain.registry; +import mcpp.toolchain.triple; using namespace mcpp::toolchain; @@ -187,3 +188,39 @@ TEST(ToolchainRegistry, IdentifiesToolchainPayloadsAndSkipsOthers) { EXPECT_FALSE(identify_xim_payload("python").has_value()); EXPECT_FALSE(identify_xim_payload("linux-headers").has_value()); } + +// #367: which GCC payload backs a NATIVE Linux build. +// +// `xim:gcc` declares `archs = { "x86_64" }` and publishes assets for that arch +// only; the GCC the ecosystem ships for other Linux architectures is +// `musl-gcc` (xlings-res carries musl-gcc-16.1.0-linux-aarch64.tar.gz). Asking +// for `gcc` on aarch64 therefore 404s — which is what made every project whose +// graph contains a `build.mcpp` unbuildable there, since a build program's +// host compile resolves the spec with no target injection and landed on the +// glibc package. +// +// Tested through a free function taking the host arch rather than the +// compile-time constant, precisely so the aarch64 answer is checkable from an +// x86_64 machine. +TEST(ToolchainRegistry, NativeGccPayloadFollowsWhatTheArchActuallyPublishes) { + using mcpp::toolchain::gcc_native_payload_is_musl; + const mcpp::toolchain::triple::Triple none{}; + + // x86_64: unchanged — the glibc package is the one that exists. + EXPECT_FALSE(gcc_native_payload_is_musl("x86_64", true, none)); + EXPECT_FALSE(gcc_native_payload_is_musl( + "x86_64", true, {"x86_64", "linux", "gnu"})); + + // aarch64: the glibc package has no asset, musl-gcc does. + EXPECT_TRUE(gcc_native_payload_is_musl("aarch64", true, none)); + EXPECT_TRUE(gcc_native_payload_is_musl( + "aarch64", true, {"aarch64", "linux", "gnu"})); + + // A CROSS target keeps its own payload rule — this is about the native + // one, and `-gcc` packages answer for the rest. + EXPECT_FALSE(gcc_native_payload_is_musl( + "aarch64", true, {"x86_64", "linux", "gnu"})); + + // Non-Linux hosts are out of scope: macOS uses llvm, Windows mingw/msvc. + EXPECT_FALSE(gcc_native_payload_is_musl("aarch64", false, none)); +} From 532fbb67a2a55fc562ea1dc5ca15b43081881151 Mon Sep 17 00:00:00 2001 From: speak-agent Date: Thu, 6 Aug 2026 16:53:49 +0800 Subject: [PATCH 9/9] =?UTF-8?q?docs(release):=20=E8=87=AA=E4=B8=BE=20pin?= =?UTF-8?q?=20=E7=9A=84=E5=A4=B1=E8=B4=A5=E5=8F=88=E8=A2=AB=E5=BD=92?= =?UTF-8?q?=E5=9B=A0=E6=88=90=E3=80=8C=E7=B4=A2=E5=BC=95=E5=88=A0=E4=BA=86?= =?UTF-8?q?=E7=89=88=E6=9C=AC=E3=80=8D,=E7=AC=AC=E4=B8=89=E6=AC=A1?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 09-release.md §5 已经为 3b1cb6b 记过一次这个更正,而本 PR 的 c1a3e45 又写了一遍 「指向索引里已经不存在的版本」。它是错的:openxlings/xim-pkgindex 与 d2learn/xim-pkgindex 都列着 63 个 mcpp 版本,`2026.8.5.3` 在里面。 值得注意的是 `available:` 实际列出的东西 —— 只有一个版本,正是那次 job 刚装上 的那个。这是「已安装版本」的形状,不是索引清单;而那一步解析的是 .xlings.json 的 **workspace** pin,workspace 作用域正是既有的那个陷阱。真实机制未坐实,不写 成结论。 pin 的 bump 本身没问题(§4 明确说这是合理的做法,而且它确实解开了那条 job)。 教训更窄,而且反复被重学:**在断言「索引删了它」之前,先读索引。** 一条 curl 就能定。 --- docs/09-release.md | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/docs/09-release.md b/docs/09-release.md index 3f575b4f..88f9eb06 100644 --- a/docs/09-release.md +++ b/docs/09-release.md @@ -141,6 +141,20 @@ would again let the index guard and the installed version drift apart. > §3, seen from the other side. Nothing about a release removes older versions, > and no reasoning should be built on the idea that it does. +> **The same misdiagnosis, again (2026-08-06).** `ci-aarch64-fresh-install` +> failed with `xlings: version '2026.8.5.3' not found for 'mcpp' — available: +> 2026.8.6.1`, and the fix commit again claimed the index had dropped the +> version. It had not: both `openxlings/xim-pkgindex` and `d2learn/xim-pkgindex` +> list 63 mcpp versions including `2026.8.5.3`. Note what `available:` actually +> enumerated — a single version, the one that job had just installed — which is +> the shape of an *installed-versions* view, not an index listing. That step +> resolves a `.xlings.json` **workspace** pin, and a workspace-scoped resolve is +> the documented scope trap. +> +> The bump itself was fine (§4 endorses it, and it did unblock the job). The +> lesson is narrower and keeps being relearned: **before concluding "the index +> dropped it", read the index.** One `curl` of `pkgs/m/mcpp.lua` settles it. + ## 6. Checklist ```