Skip to content

feat: add compat.abseil 20250512.1 + compat.protobuf 35.1 - #147

Merged
Sunrisepeak merged 2 commits into
mainfrom
feat/add-abseil-protobuf
Aug 4, 2026
Merged

feat: add compat.abseil 20250512.1 + compat.protobuf 35.1#147
Sunrisepeak merged 2 commits into
mainfrom
feat/add-abseil-protobuf

Conversation

@Sunrisepeak

Copy link
Copy Markdown
Member

Groundwork for gRPC: its two largest prerequisites, added as ordinary compat packages.

Why these two don't need a dedicated repo (and gRPC does)

The starting question was whether gRPC could be added as compat.grpc.lua. The analysis
(.agents/docs/2026-08-04-grpc-feasibility-analysis.md) says no — but it also found that
abseil and protobuf are a completely different case:

abseil / protobuf gRPC
self-contained release asset abseil-cpp-20250512.1.tar.gz 2.2MB · protobuf-35.1.tar.gz 7.1MB ❌ no release assets at all; the tag archive has abseil/protobuf/re2/boringssl/zlib as empty submodule placeholders (1 entry each)
configure step ❌ none — platform handling is in-source #ifdef needs per-OS config
protoc bootstrap ❌ none — the 14 .pb.cc (well-known types + descriptor.pb.cc) are checked in plugins must be built

So these two land in the standard Shape A template, while gRPC still needs the grpc-m
adaptation-repo route (opencv-m precedent). That plan is recorded in the analysis doc.

Versions track upstream verbatim — a correctness requirement here

20250512.1 is the Abseil LTS tag. protobuf 35.1's MODULE.bazel names that same LTS,
and gRPC 1.83.0 pins protobuf at commit 35cd01f (== tag v35.1) and abseil at 76bb243
("Abseil LTS Branch, May 2025, Patch 1" == 20250512.1). Keeping the upstream spelling is
what lets a future gRPC package link one abseil instead of colliding with a second
vendored copy.

Source lists

  • protobuf transcribes libprotobuf_srcs from upstream's own src/file_lists.cmake
    (79 TUs). A directory glob cannot substitute: src/google/protobuf/**/*.cc would also
    sweep in libprotoc (157 more TUs) and the tests, while two runtime TUs live under
    compiler/ (importer.cc, parser.cc — the .proto text parser is part of the
    runtime), so nothing separates the two libraries by directory.
  • abseil globs absl/**/*.cc and trims by upstream's own test/benchmark naming
    conventions plus 14 named stragglers. Four excluded TUs define their own main() — a
    dependency's objects all enter the consumer's link, so any one would collide with the
    consumer's main() and break every dependent build.

compat.protobuf depends on compat.abseil (protobuf's public headers include absl/…)
and gates Gzip streams behind a gzip feature — io/gzip_stream.cc is wrapped head-to-toe
in #if HAVE_ZLIB, so the default build compiles it to an empty TU and pulls no zlib at all.

Verification

Cold, with the CI configuration (pinned mcpp 2026.8.3.3, gcc@16.1.0,
MCPP_INDEX_MIRROR=GLOBAL, MCPP_BUILD_CACHE=local), target/ + .mcpp/ removed first:

::: abseil :::         Compiling compat.abseil v20250512.1   test result ok. 1 passed; 0 failed  (18.57s)
::: protobuf :::       Compiling compat.protobuf v35.1       test result ok. 1 passed; 0 failed  (63.35s)
::: protobuf-gzip :::  Compiling compat.protobuf v35.1       test result ok. 1 passed; 0 failed  (54.48s)
  • Object counts checked, not assumed: 151 .o linked for compat.abseil, 80 for compat.protobuf.
  • The protobuf test uses no protoc output anywhere: .proto text through the runtime's own
    parser → DynamicMessage → wire/TextFormat/JSON round-trips → well-known types → UTF-8
    validation (reaching utf8_range.c). A truncated payload must be rejected, so a no-op parser
    cannot pass.
  • gzip verified negatively too: reverting the member to a bare protobuf = "35.1" fails
    the link with undefined GzipOutputStream/GzipInputStream symbols.
  • CN mirrors closed-loop checked: mcpp-res/abseil@20250512.1 and mcpp-res/protobuf@35.1
    both return http=200 and are byte-identical to GLOBAL.
  • Lint: all 70 descriptors pass syntax / mirror / name checks, cross-package refs OK, and both
    new descriptors pass mcpp xpkg parse on the pinned mcpp.

Two known mcpp-side issues (documented, neither affects CI)

  1. randen_round_keys.cc prints a false module 'binascii' imported but not provided warning —
    the Python generator pasted inside a /* … */ block comment starts with import binascii at
    column 0, and mcpp's text scan strips // comments and raw strings but not block comments.
    It cannot be silenced from the descriptor: scan_overrides rejects an entry that declares
    neither provides nor imports, which is exactly this file's true answer.
  2. A new instance of [bug] 全局 build cache:obj 布局随消费方包组合变化(#233 消歧),而 cache key 不含消费方 → 同一 key 下第二个消费者必挂 'missing and no known rule' mcpp-community/mcpp#344 — abseil's str_format/parser.cc and protobuf's
    compiler/parser.cc share a basename, and the package build cache key doesn't cover the
    collision context. CI is unaffected (it already sets MCPP_BUILD_CACHE: local for this
    very reason), and upstream fixed it in 2026.8.3.4, so MCPP_VERSION is deliberately left alone.

Both are written up in .agents/docs/2026-08-04-add-abseil-and-protobuf-plan.md.

Groundwork for gRPC: its two largest prerequisites, as ordinary compat
packages. gRPC itself still needs a dedicated adaptation repo (see
.agents/docs/2026-08-04-grpc-feasibility-analysis.md) — these two do not,
and the analysis says why:

  * both publish a real self-contained release ASSET (not the tag archive):
    abseil-cpp-20250512.1.tar.gz 2.2MB, protobuf-35.1.tar.gz 7.1MB. No
    submodules; protobuf's third_party/ carries utf8_range as actual source.
  * no configure step — platform handling is in-source #ifdef, so all three
    xpm blocks share one tarball and one sha256.
  * no protoc bootstrap — the 14 .pb.cc for the well-known types and
    descriptor.pb.cc are CHECKED IN upstream.

By contrast gRPC's tag archive has abseil/protobuf/re2/boringssl/zlib as
EMPTY submodule placeholders (1 entry each), which is what would force a
self-hosted repack.

Versions track upstream verbatim, which here is a correctness requirement
rather than a style choice: 20250512.1 IS the Abseil LTS tag, protobuf
35.1's MODULE.bazel names that same LTS, and gRPC 1.83.0 pins protobuf at
commit 35cd01f (== tag v35.1) and abseil at 76bb243 ("Abseil LTS Branch,
May 2025, Patch 1" == 20250512.1). So a later gRPC package links ONE
abseil instead of colliding with a second vendored copy.

Source lists are taken the way each upstream affords:

  * protobuf transcribes libprotobuf_srcs from upstream's own
    src/file_lists.cmake (79 TUs). A directory glob cannot substitute:
    src/google/protobuf/**/*.cc would also sweep in libprotoc (157 more
    TUs) and the tests, while two runtime TUs live under compiler/
    (importer.cc, parser.cc — the .proto text parser is part of the
    runtime), so nothing separates the two libraries by directory.
  * abseil globs absl/**/*.cc and trims by upstream's own test/benchmark
    naming conventions plus 14 named stragglers. Four of the excluded TUs
    define their own main() — a dependency's objects all enter the
    consumer's link, so any one of them would collide with the consumer's
    main() and break every dependent build.

compat.protobuf declares compat.abseil as a dependency because protobuf's
public headers include "absl/…" directly, and gates GzipInputStream /
GzipOutputStream behind a `gzip` feature: io/gzip_stream.cc is wrapped
head-to-toe in #if HAVE_ZLIB, so the default build compiles it to an empty
TU and pulls no zlib at all.

Verified cold with the pinned mcpp 2026.8.3.3, gcc@16.1.0,
MCPP_INDEX_MIRROR=GLOBAL and MCPP_BUILD_CACHE=local (the CI configuration),
target/ and .mcpp/ removed first:

  abseil        test result ok. 1 passed; 0 failed  (18.57s)
  protobuf      test result ok. 1 passed; 0 failed  (63.35s)
  protobuf-gzip test result ok. 1 passed; 0 failed  (54.48s)

Object counts checked rather than assumed: 151 .o linked for compat.abseil,
80 for compat.protobuf. The protobuf test uses NO protoc output anywhere —
.proto text through the runtime's own parser, then DynamicMessage,
wire/TextFormat/JSON round-trips, well-known types and UTF-8 validation
(which reaches utf8_range.c); a truncated payload must be rejected, so a
no-op parser cannot pass. The gzip feature was verified negatively too:
reverting the member to a bare `protobuf = "35.1"` fails the link with
undefined GzipOutputStream/GzipInputStream symbols.

CN mirrors published and closed-loop checked: mcpp-res/abseil@20250512.1
and mcpp-res/protobuf@35.1 both return http=200 and are byte-identical to
GLOBAL.

Two known mcpp-side issues are documented in the plan doc, neither a defect
in these descriptors and neither affecting CI: a false `module 'binascii'
imported but not provided` warning from a Python snippet inside a block
comment in randen_round_keys.cc (mcpp's text scan does not strip block
comments, and scan_overrides cannot express "no module edges"), and a new
instance of mcpp#344 — abseil's str_format/parser.cc and protobuf's
compiler/parser.cc share a basename, which the package build cache
mishandles. CI already bypasses that cache via MCPP_BUILD_CACHE: local, and
upstream fixed it in 2026.8.3.4, so MCPP_VERSION is deliberately left alone.
… does

CI's linux leg passed; macOS and Windows did not, both for the same reason —
each upstream's own build system carries platform settings that the
descriptors did not restate.

macOS, compat.abseil. cctz's time_zone_lookup.cc reads the system time zone
through CFTimeZoneCopyDefault / CFStringGet* / CFRelease on Apple platforms,
so CoreFoundation belongs on the link line; upstream's
absl/time/CMakeLists.txt carries
`$<$<PLATFORM_ID:Darwin,…>:-Wl,-framework,CoreFoundation>`. Without it the
package compiles fine and the failure lands in the CONSUMER's link:

  ld64.lld: error: undefined symbol: CFTimeZoneCopyDefault
  ld64.lld: error: undefined symbol: CFStringGetCString        (+4 more)

Windows, both packages. <windows.h> defines min/max as function-like MACROS,
which turns absl/time/time.cc's `std::numeric_limits<time_t>::max()` into

  error: too few arguments provided to function-like macro invocation
  note: macro 'max' defined here   (Windows Kits …/minwindef.h)

NOMINMAX is therefore load-bearing rather than hygiene. It comes from
upstream's ABSL_MSVC_FLAGS (absl/copts/GENERATED_AbseilCopts.cmake), along
with WIN32_LEAN_AND_MEAN and _CRT_SECURE_NO_WARNINGS. compat.protobuf gets
the same three: it reaches <windows.h> through io/io_win32.cc and port.h
while using std::min/std::max throughout, and upstream's protobuf CMake only
gets away with staying silent because its Abseil dependency's copts supply
them — here each package carries its own compile flags.

Only the macosx/windows blocks changed. Re-verified on linux with the pinned
mcpp 2026.8.3.3 + MCPP_BUILD_CACHE=local, target/ removed first: abseil,
protobuf and protobuf-gzip all `test result ok`. Lint and `mcpp xpkg parse`
still pass. The other two platforms are verifiable only in CI.
@Sunrisepeak
Sunrisepeak merged commit 163bce4 into main Aug 4, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant