Skip to content

descriptor: custom options, Any pack/unpack, and symbol-to-file lookup#140

Merged
iainmcgin merged 1 commit into
mainfrom
reflect/options-symbol-any
May 21, 2026
Merged

descriptor: custom options, Any pack/unpack, and symbol-to-file lookup#140
iainmcgin merged 1 commit into
mainfrom
reflect/options-symbol-any

Conversation

@iainmcgin
Copy link
Copy Markdown
Collaborator

What

The final three reflection-parity items against protobuf-go / protobuf-es, so the answer to "what does Go's reflection do that buffa's doesn't" is only ever the documented deferrals (proto2 declared defaults, text format on dynamic messages, descriptor-tree navigation).

1. Custom options

Each linked descriptor (Field/Message/Enum/EnumValue/Service/Method/Oneof) exposes options() -> Option<&XxxOptions> — the raw generated options message, boxed and cloned at link time (None for the common no-options case). Matches Go's desc.Options().

Custom options are extensions of the *Options message and survive on its unknown fields. DynamicMessage::from_options(pool, field.options()?) re-decodes the options as a DynamicMessage of the corresponding google.protobuf.*Options type (resolved by MessageName::FULL_NAME), so dyn_opts.get(ext.field()) reads a custom option by name — no hardcoded type strings. Requires descriptor.proto plus the option-defining proto in the pool. This is the headline use case (protovalidate, (google.api.http) transcoding, PII annotations) and leans entirely on the extension machinery from #139.

2. Any pack/unpack

DynamicMessage::pack_any / unpack_any — the binary counterpart to the Any JSON @type expansion, for CEL dyn evaluation. unpack_any resolves the type_url's last path segment against the pool and decodes the value bytes; pack_any wraps a message with the type.googleapis.com/ prefix. AnyError (#[non_exhaustive], Clone) distinguishes NotAny / AnyNotRegistered / MissingTypeUrl / UnknownType / Decode { type_url, source }.

3. Symbol → file

DescriptorPool::file_containing_symbol is now backed by a precomputed symbol_file index — O(log n), replacing an O(symbols × files) parent-walk scan that only covered messages and enums. It indexes the full set gRPC server reflection's FindFileContainingSymbol accepts: messages, fields, oneofs, enums, enum values (in their parent scope per protobuf naming), services, methods, and extensions. Unblocks a ServerReflection adapter in connect-rust.

Verification

  • 22 new e2e tests across the three features (custom options on field/message/method, Any pack/unpack round-trip + all error paths, symbol→file for every symbol kind).
  • Reflect conformance suite holds at 2783 successes, 0 expected failures, 0 unexpected.
  • Workspace clippy clean, reflect-only no_std clean, fmt clean, doctests pass.
  • Both review agents' Critical/High/Medium findings addressed.

Net change

+~560/-36. Stacked on #139 (extension reflection) — the custom-option read path depends on it.

@github-actions
Copy link
Copy Markdown

github-actions Bot commented May 21, 2026

All contributors have signed the CLA ✍️ ✅
Posted by the CLA Assistant Lite bot.

@iainmcgin iainmcgin force-pushed the reflect/extensions branch from fbe41fe to 2d0c3a6 Compare May 21, 2026 19:56
Base automatically changed from reflect/extensions to main May 21, 2026 20:32
@iainmcgin iainmcgin force-pushed the reflect/options-symbol-any branch from 3f5d534 to ba7f3f1 Compare May 21, 2026 20:48
The final reflection-parity items against protobuf-go and protobuf-es.

Custom options: each linked descriptor (Field/Message/Enum/EnumValue/
Service/Method/Oneof) exposes options() -> Option<&XxxOptions>, the raw
generated options message cloned (boxed) at link time — None for the
common no-options case. Matches Go's desc.Options(). Custom options are
extensions of the *Options message and survive on its unknown fields;
DynamicMessage::from_options(pool, field.options()?) re-decodes the
options as a DynamicMessage of the corresponding google.protobuf.*Options
type (resolved by MessageName::FULL_NAME), so dyn_opts.get(ext.field())
reads a custom option by name with no hardcoded type strings. Requires
descriptor.proto plus the option-defining proto in the pool.

Any pack/unpack: DynamicMessage::pack_any / unpack_any, the binary
counterpart to the Any JSON @type expansion (CEL dyn evaluation).
unpack_any resolves the type_url's last segment against the pool and
decodes the value bytes; pack_any wraps with the type.googleapis.com/
prefix. AnyError (#[non_exhaustive], Clone) distinguishes
NotAny / AnyNotRegistered / MissingTypeUrl / UnknownType /
Decode { type_url, source }.

Symbol → file: DescriptorPool::file_containing_symbol is now backed by a
precomputed symbol_file index (O(log n), replacing an O(symbols × files)
parent-walk scan that only covered messages and enums). Indexes the full
set gRPC server reflection's FindFileContainingSymbol accepts: messages,
fields, oneofs, enums, enum values (parent scope per protobuf naming),
services, methods, and extensions. Unblocks a ServerReflection adapter
in connect-rust.

Conformance: reflect suite holds at 2783 successes, 0 expected failures,
0 unexpected. 22 new e2e tests across options/Any/symbol.
@iainmcgin iainmcgin force-pushed the reflect/options-symbol-any branch from ba7f3f1 to ca8f3e5 Compare May 21, 2026 21:16
@iainmcgin iainmcgin marked this pull request as ready for review May 21, 2026 21:32
@iainmcgin iainmcgin requested a review from azdagron May 21, 2026 21:32
@iainmcgin iainmcgin merged commit 0b5e9be into main May 21, 2026
7 checks passed
@iainmcgin iainmcgin deleted the reflect/options-symbol-any branch May 21, 2026 21:57
@github-actions github-actions Bot locked and limited conversation to collaborators May 21, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants