Skip to content

Namespace ancillary codegen output into parallel kind-trees (view::, oneofs::, ext::)#54

Open
iainmcgin wants to merge 3 commits intomainfrom
prototype/codegen-namespacing
Open

Namespace ancillary codegen output into parallel kind-trees (view::, oneofs::, ext::)#54
iainmcgin wants to merge 3 commits intomainfrom
prototype/codegen-namespacing

Conversation

@iainmcgin
Copy link
Copy Markdown
Collaborator

@iainmcgin iainmcgin commented Apr 17, 2026

Summary

Ancillary types emitted by buffa-codegen — borrowed views, oneof enums, oneof view enums, extension consts, and the per-file registration fn — previously lived in the same Rust namespace as user-declared proto types and could collide with them. For example, a message FooView in a .proto would clash with the generated view type of sibling message Foo; message DataType { oneof data_type {...} } would put a struct and an enum of the same name side by side.

This PR moves every ancillary generated type into a dedicated kind-namespaced parallel tree (view::, oneofs::, ext::, view::oneofs::), structurally deconflicting the codegen output from proto-derived names. The design rule (now documented in DESIGN.md) is:

Ancillary kinds are outermost; modifiers (only view today) wrap them; inside every tree the proto message path is mirrored.

Layout changes

Before After
pkg::FooView pkg::view::FooView
pkg::foo::BarView (nested msg view) pkg::view::foo::BarView
pkg::foo::KindOneof (oneof enum) pkg::oneofs::foo::Kind
pkg::foo::KindOneofView (oneof view enum) pkg::view::oneofs::foo::Kind
pkg::FOO (extension const) pkg::ext::FOO
pkg::register_types pkg::ext::register_types

Suffix policy

  • Message view structs keep the View suffix (pkg::view::foo::FooView) — this is the one documented exception to the "tree disambiguates, so drop the suffix" rule. Rationale: users commonly import a message and its view into the same scope (use pkg::{Foo, view::FooView};), and a scope-level rename would be friction for every call site.
  • Oneof enums and oneof view enums both drop their suffixes — the parallel-tree position is the disambiguator, and neither name is commonly co-imported with an unrelated type called Kind.

Co-import ergonomics

use pkg::{Foo, view::FooView};
fn process(m: &Foo, v: FooView<'_>) { /* ... */ }

Codegen internals

  • GeneratedFile gains kind: GeneratedFileKind and package: String fields. GeneratedFileKind has five variants: Owned, View, Ext, Oneofs, ViewOneofs. Each proto produces five sibling output files (.rs, .__view.rs, .__ext.rs, .__oneofs.rs, .__view_oneofs.rs); empty bodies when the kind has no content for that proto.
  • generate_module_tree accepts &[ModuleTreeEntry<'_>] and coalesces per-package streams across multiple .proto files so packages that span files (e.g. google.rpc) merge their view:: / ext:: / oneofs:: / view::oneofs:: blocks into single wrappers per kind.
  • CodeGenError::OneofNameConflict and the reserved-name machinery around oneof-vs-nested-type collisions are deleted — the collisions they guarded against are now structurally impossible.
  • Checked-in generated code (WKTs + bootstrap descriptor types) fully regenerated. Bootstrap uses generate_views=false, so its .__view*.rs siblings are empty stubs; the file count is kept consistent for the module tree stitcher.

Deconfliction wins

Patterns that were previously rejected at codegen time (or required OneofNameConflict errors with proto renames) now compile as-is:

  • message DataType { oneof data_type {...} }
  • message Foo { nested message Kind {...} oneof kind {...} }
  • message M { nested message FooView {...} oneof foo {...} }
  • Sibling oneofs my_field and my_field_view

DESIGN.md

Adds a "Generated code layout — parallel trees per kind" section covering the formula, the five-sibling invariant, the path-table worked example for my.pkg.Foo, the View-suffix exception with rationale, and the three structural benefits (collision-free / predictable / feature-gatable per kind).

Migration for downstream consumers

// Before
use pkg::{Foo, FooView, FOO, register_types};
use pkg::foo::{KindOneof, InnerMessage};

// After
use pkg::Foo;
use pkg::view::FooView;
use pkg::ext::{FOO, register_types};
use pkg::oneofs::foo::Kind;
use pkg::foo::InnerMessage;

Test plan

  • cargo check --workspace clean
  • task test — 1448 tests pass (18 suites), 0 failed
  • task test-codegen — snapshot tests pass
  • task lintcargo fmt --check and cargo clippy --workspace --all-targets -- -D warnings both clean
  • task lint-md — markdownlint clean
  • task conformance — 6 CONFORMANCE SUITE PASSED lines, baseline numbers match exactly (std 5549 / no_std 5529 / view 2797 binary+JSON; 883/883/0 text)
  • task stress-googleapis — 235 files compile cleanly (including multi-file google.rpc)
  • Three previously-inverted tests/naming.rs collision tests are further updated to coexist assertions; OneofNameConflict test references removed with the variant

Diff breakdown by category

Total: 113 files, +6339 / -3562.

Category Files +lines -lines Notes
Core codegen source (buffa-codegen/src/*.rs, non-test) 7 744 320 Path resolution + module-tree stitching for the full kind matrix, including oneofs and view-oneofs wrappers. Net +424.
Codegen tests (snapshot + integration) 8 423 261 Snapshot updates + coexist regression tests for the deconfliction patterns. Net +162.
buffa-build API adaptation 1 103 42 Plumbing the five-kind ModuleTreeEntry through the build-script path. Net +61.
buffa-test consumer test suite 14 289 205 Mechanical import-path migration across the end-to-end suite. Net +84.
buffa-types / buffa-descriptor hand-maintained 5 96 37 lib.rs / mod.rs hand-stitch the pub mod view { pub mod oneofs {} }, pub mod oneofs, pub mod ext wrappers. Net +59.
Conformance runner + examples (addressbook, envelope, logging) 7 230 33 Path reference updates + regenerated logging example. Net +197.
protoc-gen-buffa-packaging 1 44 13 Emits five ModuleTreeEntrys per proto. Net +31.
Googleapis stress tooling (gen_lib_rs.py) 1 74 25 Groups the two new sibling kinds and emits the oneofs / view_oneofs wrappers. Net +49.
DESIGN.md 1 45 0 New "Generated code layout" section. Net +45.
Subtotal — human-authored 45 2048 936 Net +1112
Regenerated WKT types (buffa-types/src/generated/) 35 3169 2588 7 protos × 5 siblings, with inlined view/oneof code relocated to sibling files. Net +581 (mostly because the sibling-file scheme adds some boilerplate per kind).
Regenerated bootstrap descriptor types (buffa-descriptor/src/generated/) 12 38 4 Regenerated mod.rs + sibling stubs. Net +34.
Regenerated examples/logging output 21 1084 34 task gen-logging-example output after the layout move. Net +1050.
Subtotal — regenerated 68 4291 2626 Net +1665

Human-authored change: ~2048 / 936 (32% / 26%) of the total. The rest is codegen-regenerated output where the file count expands because every proto now emits five siblings, but the per-file content shrinks correspondingly.

Breaking changes

API-breaking for downstream consumers of generated code. Pre-1.0 this is expected; the migration is mechanical (adjust import paths per the table above).

@iainmcgin iainmcgin force-pushed the prototype/codegen-namespacing branch from ae6dd3b to e46e6b9 Compare April 18, 2026 00:15
@iainmcgin iainmcgin force-pushed the prototype/codegen-namespacing branch from e46e6b9 to 8319105 Compare April 22, 2026 20:19
@iainmcgin iainmcgin changed the title Namespace ancillary codegen output into view:: and ext:: sub-modules Namespace ancillary codegen output into parallel kind-trees (view::, oneofs::, ext::) Apr 22, 2026
Moves generated types that previously collided with user-declared proto
names into dedicated sub-modules, structurally deconflicting the codegen
output from proto-derived names.

Layout changes:

- Views: `pkg::FooView` → `pkg::view::FooView` (top-level), and
  `pkg::foo::BarView` → `pkg::view::foo::BarView` (nested). Oneof view
  enums follow the same tree: `pkg::view::foo::KindView`. The `View`
  suffix is retained so that owned and view types can be imported into
  the same scope without renaming.

- Extensions: `pkg::FOO` → `pkg::ext::FOO`, and per-file
  `pkg::register_types` → `pkg::ext::register_types`.

- Oneof enums: `foo::KindOneof` → `foo::Kind`. The suffix is no longer
  needed for disambiguation; the owner message's sub-module provides the
  namespace.

Oneof enums themselves remain in the owner message's sub-module
(`pkg::foo::Kind`). The view-tree move makes several previously-illegal
proto patterns legal — two regression tests in `naming.rs` capture this.

Codegen internals:

- `GeneratedFile` gains `kind: GeneratedFileKind` + `package: String`
  fields. Each generated proto now produces three sibling output files
  (`.rs`, `.__view.rs`, `.__ext.rs`); `generate_module_tree` coalesces
  them per package so multiple `.proto` files that share a Rust module
  (e.g. `google.rpc`) merge their `view::` / `ext::` blocks cleanly.

- `generate_module_tree` signature widens from `&[(&str, &str)]` to
  `&[ModuleTreeEntry<'_>]` to carry the kind.

- `MessageScope::deeper()` is used for depth-aware path resolution in the
  parallel view tree.

Checked-in generated code regenerated (WKTs + bootstrap descriptor types).
Conformance suite, googleapis stress test, and the existing test suite
(1442 tests) all pass.
Post-rebase onto main (which now includes #39): convert the
prelude_shadow include to include_generated! and update the oneof
enum path to oneofs::picker::option::Value (no Oneof suffix under
namespacing).
@iainmcgin iainmcgin force-pushed the prototype/codegen-namespacing branch from 8319105 to e62b8d0 Compare April 23, 2026 17:27
@iainmcgin iainmcgin marked this pull request as ready for review April 23, 2026 17:27
@iainmcgin iainmcgin requested a review from rpb-ant April 23, 2026 17:31
The benchmarks/buffa and benchmarks/gen-datasets crates use hand-written
include!() blocks (not the auto-generated _include.rs), so they were
silently broken under namespacing — generated owned-struct code
references super::oneofs::... which only resolves when the four
.__view/.__oneofs/.__view_oneofs/.__ext files are all included. CI did
not catch this because these crates are outside the workspace.

- Add a local include_generated! macro to both crates (mirrors the
  buffa-test pattern).
- Update view-type and oneof-enum imports in benches/protobuf.rs and
  gen-datasets/main.rs to the namespaced paths.
@iainmcgin iainmcgin force-pushed the prototype/codegen-namespacing branch from 9ae726f to 0482699 Compare April 23, 2026 18:13
@iainmcgin iainmcgin enabled auto-merge (squash) April 23, 2026 18:36
@rpb-ant rpb-ant self-assigned this Apr 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants