Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
99 changes: 99 additions & 0 deletions .agents/docs/2026-08-05-add-grpc-plan.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,99 @@
# 新增 mcpplibs.grpc 1.83.0(2026-08-05)

[gRPC 收录可行性分析](2026-08-04-grpc-feasibility-analysis.md) 的终点。索引侧此前已备齐依赖
([abseil+protobuf](2026-08-04-add-abseil-and-protobuf-plan.md)、
[re2+upb](2026-08-04-add-re2-and-protobuf-upb-plan.md)、
[c-ares](2026-08-04-add-c-ares-plan.md)),本次加入 gRPC 本体。

产出:`pkgs/g/grpc.lua`(Form A,指向 [mcpplibs/grpc-m](https://github.com/mcpplibs/grpc-m)
的 release)+ workspace 成员 `tests/examples/grpc-module`。

## 1. 为什么是独立仓库

这是本次唯一一个不能用 compat 描述符表达的库,理由只有一条但很硬:**gRPC 不发布任何自包含的
源码产物**。v1.83.0 完全没有 release asset,而其 tag 归档里 abseil / protobuf / re2 /
boringssl / zlib 全是**空的 submodule 占位**(每个只有一个目录条目),`url` + `sha256` 无处可指。

`grpc-m` 的 release tarball 就是那个缺失的产物:上游 `src/` 与 `include/` **零补丁** vendor,
外加 gRPC 自己真正带内容的两块 third_party(`address_sorting`、`xxhash`)。

**而它不 vendor 的东西才是它属于本索引的理由**:abseil、protobuf(+upb)、re2、c-ares、
OpenSSL、zlib 全部取自本索引的包。于是一个同时直接使用 protobuf 或 abseil 的消费者链进去的是
**同一份**,而不是和第二份 vendored 副本撞车。

## 2. 构建形态

无 CMake、无 Bazel、无 configure —— 这不是取巧,而是核实过的事实:gRPC 源码树里**没有任何
`.h.in` 或 `config.h.cmake`**,且其 upb 生成码上游已 check-in,所以 mcpp 只需要 include 路径。
1001 个 TU 全部由解析出的工具链编译,**没有任何外部构建系统参与**,因此不存在
`compat.openssl` 那种"外部 `c++` 编出的产物与 mcpp 链接侧 C++ ABI 不一致"的风险
(那正是可行性分析里否掉 `install()` + CMake 路线的原因)。

源码清单是**上游自己的**:`add_library(gpr)` + `add_library(grpc)` + `add_library(grpc++)` +
`add_library(address_sorting)` 之并(995 条,c-ares 的 7 条另计)。`grpc-m` 的
`tools/gen_sources.py` 能从上游 checkout 重新生成它,`--check` 在该仓 CI 里运行,证明 manifest
与 vendored 源码树没有漂移。

刻意排除一个文件:`src/core/ext/upb-gen/google/protobuf/descriptor.upb_minitable.c` ——
与 `compat.protobuf` 的 `upb` feature 已编译的 bootstrap 版本**逐字节完全相同**,两份同编会重复符号。

## 3. `import grpc;` 与顺序约束

`grpc-m` 提供真实的 C++23 模块层(`src/grpc.cppm`),它同时也充当 mcpp `kind = "lib"` 约定要求的
lib root。导出面只取**公开 API**:`namespace grpc` 里还有约 280 个实现细节名字
(`grpc::internal`、`CallOp*` 机器),整表扫描会把它们一起导出。每个名字都由编译器验证 ——
`export using ::grpc::X;` 在 X 不存在时直接编译失败 —— 这已经当场抓出三个并不存在的名字
(`GrpcLibraryCodegen`、`DisableDefaultHealthCheckService`,以及补 include 之前的
`ServerReaderWriter`)。

**`import grpc;` 必须放在该 TU 所有 textual `#include` 之后**,标准库头也算。全局模块片段把大半个
标准库带进了 BMI,import 之后再 textual include 会让同一批声明到达两次。两种顺序都实测过
(gcc 16.1.0):

- import 在前 → `redefinition of std::__is_constant_evaluated` 加一大片 `<limits>` 冲突
- 仅有 std 头跟在 import 之后 → `std::string` 上的 `ambiguous overload for operator==`
- **include 全部在前、import 在最后 → 正常**,因为导出实体属于**全局模块**,textual 视图与
import 视图是同一批实体

这不是本包发明的约束:gRPC 的代码生成器产出的是**头文件**,任何真实程序都会同时用到两个世界。

## 4. `ares` feature 的方向是被迫的

c-ares **默认开启**,与上游 gRPC 及各发行版一致;`default-features = false` 关闭。

方向不能反过来:mcpp 的 feature 是**只增不减**的,若把 `-DGRPC_ARES=0` 写进基础 flag,再让
feature 翻成 1,两个 `-DGRPC_ARES` 会同时出现在命令行。因此 manifest 对 `GRPC_ARES` **只字不提**
(`port_platform.h` 用 `#ifndef` 兜底成 1,所以"沉默"就是开启态),关闭态由 `build.mcpp` 在
feature 缺席时发 `GRPC_ARES=0`;而那 7 个解析器 TU 走相反方向 —— 声明式地放在
`[features.ares].sources` 里。

## 5. 验证结论

`grpc-m` 侧(与本索引 CI 同款配置:mcpp 2026.8.3.3、gcc@16.1.0、`MCPP_BUILD_CACHE=local`,
且**从已发布索引解析、无任何本地重定向**),1412 个目标文件入链:

```
mcpp test -> grpc::Version() = 1.83.0 ; module: OK
examples/helloworld -> server listening on 127.0.0.1:34733
Greeter replied: Hello mcpp
Greeter rejected the empty name: name must not be empty
```

helloworld 刻意做成**单进程**:在真实 loopback 端口上起真实服务端,经真实 HTTP/2 通道发起真实
unary RPC,所以 `mcpp run` 用退出码回答"gRPC 到底能不能用"。**错误路径也走线** —— 空 name 必须
从服务端回 `INVALID_ARGUMENT` —— 因此一个"什么都答 OK"的桩通不过。

索引侧成员 `tests/examples/grpc-module` 全程只用 `import grpc;`,不含任何 protoc 产物:
gRPC 的 codegen 需要 mcpp 无法交付给消费者的宿主工具,所以生成物那条路径由 grpc-m 自己的
example 覆盖,索引成员验证模块面。

## 6. 遗留:codegen 工具的分发

`protoc` 与 `grpc_cpp_plugin` 目前仍需用户自备(`grpc-m` 的模板与 example 把生成产物**签入**,
所以开箱即用)。根本原因是 mcpp 没有把依赖的构建产物交给消费者的机制:`mcpp::dep_dir()` 给的是
依赖的**源码目录**,而 `kind = "bin"` 的 target 不对消费者暴露。

有利条件已核实:protobuf 上游发布**全平台预编译 protoc**,所以缺口只有 `grpc_cpp_plugin`
(本次为生成 helloworld 的 stub,已用 vendored 源码在本地编出过它,157 TU 的 libprotoc + 3 个
grpc compiler TU)。后续可做成 `xim:grpc-tools` 预编译包,消费者以 `[xlings] deps` 声明,
再由 `build.mcpp` 调用 —— 这条链路 mcpp 已经具备,只差把工具打包。
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ Two kinds of packages live here:

| Shape | Examples |
|------|------|
| Native module library (Form A) | [`mcpplibs.xpkg`](pkgs/x/xpkg.lua) · [`mcpplibs.tinyhttps`](pkgs/t/tinyhttps.lua) · [`tensorvia-cpu`](pkgs/t/tensorvia-cpu.lua) · [`ffmpeg`](pkgs/f/ffmpeg.lua) (module layer; sources compiled directly through `compat.ffmpeg`) · [`opencv`](pkgs/o/opencv.lua) (single repository: the module layer and the full OpenCV 5 source build both live in the package, and only this descriptor stays on the index side) |
| Native module library (Form A) | [`mcpplibs.xpkg`](pkgs/x/xpkg.lua) · [`mcpplibs.tinyhttps`](pkgs/t/tinyhttps.lua) · [`tensorvia-cpu`](pkgs/t/tensorvia-cpu.lua) · [`ffmpeg`](pkgs/f/ffmpeg.lua) (module layer; sources compiled directly through `compat.ffmpeg`) · [`opencv`](pkgs/o/opencv.lua) (single repository: the module layer and the full OpenCV 5 source build both live in the package, and only this descriptor stays on the index side) · [`mcpplibs.grpc`](pkgs/g/grpc.lua) (gRPC 1.83.0 — the one library here that CANNOT be a compat descriptor: upstream publishes no self-contained source artifact, its tag archive carrying abseil/protobuf/re2/boringssl/zlib as empty submodule placeholders, so [grpc-m](https://github.com/mcpplibs/grpc-m)'s release tarball IS that artifact. It vendors only gRPC's own source and takes the five dependencies from this index, so a consumer that also uses protobuf links one copy rather than two) |
| C-source compat (with `features`) | [`compat.cjson`](pkgs/c/compat.cjson.lua) · [`compat.zlib`](pkgs/c/compat.zlib.lua) |
| C++-source compat, one depending on the other | [`compat.abseil`](pkgs/c/compat.abseil.lua) (151 TUs; a wildcard over `absl/**` trimmed by upstream's test/benchmark naming conventions) · [`compat.protobuf`](pkgs/c/compat.protobuf.lua) (the libprotobuf runtime, 79 TUs transcribed from upstream's own `src/file_lists.cmake`; declares `compat.abseil` as a dependency because protobuf's public headers include `absl/…`, and its `gzip` feature defines `HAVE_ZLIB` and pulls `compat.zlib`, while `upb` adds protobuf's 64-TU C runtime out of the same tarball) · [`compat.re2`](pkgs/c/compat.re2.lua) (22 TUs, upstream's own `RE2_SOURCES`) |
| header-only (with `features`) | [`compat.eigen`](pkgs/c/compat.eigen.lua) |
Expand Down
2 changes: 1 addition & 1 deletion README.zh-CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ mcpp self config --mirror CN # 切换至国内镜像,默认使用 GLOBAL 上

| 形态 | 示例 |
|------|------|
| 原生模块库(Form A) | [`mcpplibs.xpkg`](pkgs/x/xpkg.lua) · [`mcpplibs.tinyhttps`](pkgs/t/tinyhttps.lua) · [`tensorvia-cpu`](pkgs/t/tensorvia-cpu.lua) · [`ffmpeg`](pkgs/f/ffmpeg.lua)(模块层,源码经 `compat.ffmpeg` 直编) · [`opencv`](pkgs/o/opencv.lua)(单仓库:模块层与 OpenCV 5 全源码构建同在包内,索引侧只留本描述符) |
| 原生模块库(Form A) | [`mcpplibs.xpkg`](pkgs/x/xpkg.lua) · [`mcpplibs.tinyhttps`](pkgs/t/tinyhttps.lua) · [`tensorvia-cpu`](pkgs/t/tensorvia-cpu.lua) · [`ffmpeg`](pkgs/f/ffmpeg.lua)(模块层,源码经 `compat.ffmpeg` 直编) · [`opencv`](pkgs/o/opencv.lua)(单仓库:模块层与 OpenCV 5 全源码构建同在包内,索引侧只留本描述符) · [`mcpplibs.grpc`](pkgs/g/grpc.lua)(gRPC 1.83.0 —— 本索引里唯一**无法**做成 compat 描述符的库:上游不发布任何自包含源码产物,其 tag 归档里 abseil/protobuf/re2/boringssl/zlib 全是空 submodule 占位,因此 [grpc-m](https://github.com/mcpplibs/grpc-m) 的 release tarball 才是那个产物。它只 vendor gRPC 自己的源码,五个依赖全取自本索引,故同时直接使用 protobuf 的消费者链进去的是同一份而非两份)|
| C 源码 compat(含 `features`) | [`compat.cjson`](pkgs/c/compat.cjson.lua) · [`compat.zlib`](pkgs/c/compat.zlib.lua) |
| C++ 源码 compat(彼此依赖) | [`compat.abseil`](pkgs/c/compat.abseil.lua)(151 TU;对 `absl/**` 取通配后,按上游自身的 test/benchmark 命名约定裁剪) · [`compat.protobuf`](pkgs/c/compat.protobuf.lua)(libprotobuf 运行时,79 TU 逐条转录自上游 `src/file_lists.cmake`;因 protobuf 公开头文件 include 了 `absl/…`,故显式依赖 `compat.abseil`;`gzip` feature 定义 `HAVE_ZLIB` 并拉入 `compat.zlib`,`upb` feature 则从同一个 tarball 里再编出 protobuf 的 64 TU C 运行时) · [`compat.re2`](pkgs/c/compat.re2.lua)(22 TU,取自上游自身的 `RE2_SOURCES`) |
| header-only(含 `features`) | [`compat.eigen`](pkgs/c/compat.eigen.lua) |
Expand Down
1 change: 1 addition & 0 deletions mcpp.toml
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@ members = [
"tests/examples/godot-cpp-module",
"tests/examples/godot-cpp-module-v10",
"tests/examples/godot-cpp-v10",
"tests/examples/grpc-module",
"tests/examples/gui-stack",
"tests/examples/imgui",
"tests/examples/imgui-module",
Expand Down
76 changes: 76 additions & 0 deletions pkgs/g/grpc.lua
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
-- Form A descriptor: the public gRPC package ships its own mcpp.toml, so this
-- file carries metadata and a download address and nothing else. mcpp's default
-- lookup finds <verdir>/*/mcpp.toml inside the GitHub source tarball wrap.
--
-- WHY THIS IS A SEPARATE REPOSITORY. Every other heavy library in this index is
-- a `compat` descriptor pointing at an upstream tarball. gRPC cannot be: it
-- publishes NO self-contained source artifact. v1.83.0 has no release assets at
-- all, and its tag archive carries abseil, protobuf, re2, boringssl and zlib as
-- EMPTY submodule placeholders (one directory entry each), so there is nothing
-- for `url` + `sha256` to point at. grpc-m's release tarball IS that artifact:
-- upstream's src/ and include/ vendored with zero patches, plus the two
-- third_party pieces gRPC really does ship (address_sorting, xxhash).
--
-- WHAT IT DOES NOT VENDOR is the reason it belongs in this index rather than
-- standing alone. abseil, protobuf(+upb), re2, c-ares, OpenSSL and zlib are all
-- taken from the packages here, so a consumer that also uses protobuf or abseil
-- directly links ONE copy instead of colliding with a second vendored set. The
-- descriptors it depends on landed in #147 (abseil, protobuf), #148 (re2, the
-- protobuf `upb` feature) and #149 (c-ares).
--
-- The package builds without CMake, Bazel or a configure step: gRPC's tree
-- contains no .h.in or config.h.cmake, and its generated upb code is checked in
-- upstream, so mcpp needs only include paths. 1001 TUs, all compiled by the
-- resolved toolchain — no external build system runs, so nothing inherits a
-- foreign C++ ABI.
--
-- Exposes `import grpc;` over gRPC's public C++ API. The import must come AFTER
-- every textual #include in a TU (gRPC's codegen emits headers, so real
-- programs always mix the two); the package's README explains why and the
-- module's own header comment carries the measured failure modes.
--
-- linux + macOS only; see the note in the xpm table.
--
-- Optional feature `ares` is ON by default, matching upstream gRPC:
-- `grpc = { version = "1.83.0", default-features = false }` drops the c-ares
-- resolver and its dependency, which is upstream's own `grpc_no_ares=true`.
package = {
spec = "1",
name = "grpc",
namespace = "mcpplibs",
description = "gRPC 1.83.0 — vendored upstream source build with import grpc; (abseil/protobuf/re2/c-ares/OpenSSL come from this index)",
licenses = {"Apache-2.0"},
repo = "https://github.com/mcpplibs/grpc-m",
type = "package",

xpm = {
linux = {
["1.83.0"] = {
url = {
GLOBAL = "https://github.com/mcpplibs/grpc-m/archive/refs/tags/v1.83.0.tar.gz",
CN = "https://gitcode.com/mcpp-res/grpc/releases/download/1.83.0/grpc-m-1.83.0.tar.gz",
},
sha256 = "99cf3bda0a4e025f674f08dd68183538c2f655c3f72a55e1e2c1f4e05a282f43",
},
},
macosx = {
["1.83.0"] = {
url = {
GLOBAL = "https://github.com/mcpplibs/grpc-m/archive/refs/tags/v1.83.0.tar.gz",
CN = "https://gitcode.com/mcpp-res/grpc/releases/download/1.83.0/grpc-m-1.83.0.tar.gz",
},
sha256 = "99cf3bda0a4e025f674f08dd68183538c2f655c3f72a55e1e2c1f4e05a282f43",
},
},
-- No windows block, and the reason is a DEPENDENCY rather than gRPC:
-- compat.openssl has no windows xpm entry ("windows deferred —
-- requires prebuilt MSVC libs"), so resolution there fails with
-- E_NOT_FOUND: package 'compat:openssl@3.5.1' not found
-- before anything is compiled. gRPC's secure build cannot drop TLS, so
-- this package's platform coverage is exactly compat.openssl's, and it
-- widens the day that entry lands — the package already carries its
-- windows compile/link flags.
},

-- (no `mcpp` field -- default lookup will find <verdir>/*/mcpp.toml)
}
34 changes: 34 additions & 0 deletions tests/examples/grpc-module/mcpp.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
# grpc test project: consumes the mcpplibs.grpc Form-A package (grpc-m) and
# asserts behavior under `mcpp test`.
#
# The `default` namespace redirect points the built-in mcpplibs namespace at
# this checkout, so the descriptor under test is pkgs/g/grpc.lua — not whatever
# the published index happens to hold. grpc's own compat.* dependencies still
# resolve from the published index, which is the point: this member proves the
# descriptor works against the real dependency set.
#
# NO protoc output anywhere: gRPC's codegen needs host tools mcpp cannot hand a
# consumer, so this exercises the module surface instead. grpc-m's own
# examples/helloworld covers the generated-stub path end to end.
[package]
name = "grpc-module-tests"
version = "0.1.0"

# linux + macOS only: the package has no windows xpm entry (its compat.openssl
# dependency has none yet), so on windows this member carries no dependency at
# all and the test compiles to a no-op main() — the same shape
# tests/examples/openssl uses for the same underlying reason.
[target.'cfg(linux)'.dependencies.mcpplibs]
grpc = "1.83.0"

[target.'cfg(linux)'.build]
cxxflags = ["-DHAVE_GRPC=1"]

[target.'cfg(macos)'.dependencies.mcpplibs]
grpc = "1.83.0"

[target.'cfg(macos)'.build]
cxxflags = ["-DHAVE_GRPC=1"]

[indices]
default = { path = "../../.." }
45 changes: 45 additions & 0 deletions tests/examples/grpc-module/tests/module.cpp
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
// Behavioral test for the mcpplibs.grpc package, driven entirely through
// `import grpc;` — there is no <grpcpp/...> include in this file. Every call
// reaches a real symbol in the linked gRPC, so a package that resolved and
// "built" but linked nothing could not pass.
//
// Note the ORDER: textual #includes first, `import grpc;` last. The module
// carries the standard library in its BMI, so a std header included after the
// import arrives twice and GCC fails on std::string's operator==.
// HAVE_GRPC comes from this project's own cfg-gated cxxflags — the package is
// linux/macOS-only (see mcpp.toml), so elsewhere this file is an empty main().
#ifdef HAVE_GRPC

#include <cstdio>
#include <string>

import grpc;

int main() {
// A version string read out of the library, not a header constant.
const std::string version = grpc::Version();
std::printf("grpc::Version() = %s\n", version.c_str());
if (version.rfind("1.83", 0) != 0) return 1;

// grpc::Status, error path included — an always-OK stub cannot pass.
const grpc::Status bad(grpc::StatusCode::UNAVAILABLE, "down");
if (bad.ok() || bad.error_code() != grpc::StatusCode::UNAVAILABLE ||
bad.error_message() != "down" || !grpc::Status::OK.ok()) {
return 1;
}

// A real channel object. Channels connect lazily, so no server is needed;
// IDLE before the first RPC proves this is a live object from the library.
auto channel = grpc::CreateChannel("127.0.0.1:1", grpc::InsecureChannelCredentials());
if (!channel) return 1;
if (channel->GetState(false) != GRPC_CHANNEL_IDLE) return 1;

std::puts("grpc module: OK");
return 0;
}

#else // !HAVE_GRPC

int main() { return 0; }

#endif
Loading