descriptor: add reflection runtime — DynamicMessage, ReflectMessage, ReflectCow#132
Merged
Conversation
Adds a #[doc(hidden)] buffa::json_helpers::wkt module with the shared formatting and parsing primitives for the well-known types' JSON forms: Timestamp RFC 3339 (fmt_timestamp, parse_timestamp), Duration decimal seconds (fmt_duration, parse_duration, validate_duration), FieldMask camelCase (snake_to_camel, camel_to_snake, field_mask_path_round_trips), and the Howard Hinnant civil-calendar helpers (days_to_date, date_to_days). buffa-types' typed serde impls (Timestamp, Duration, FieldMask) now call into this module rather than carrying their own implementations. The adapters preserve the Option-returning private API the existing test suite (~50 call sites) targets, so no test churn. Sharing the implementation is load-bearing: the conformance suite exercises the typed JSON path today, and a forthcoming reflective JSON codec on DynamicMessage will exercise the same forms. A divergence between the two (one accepting a fractional-second precision the other rejects, or two civil-calendar implementations disagreeing on a leap-year edge) would be a user-visible inconsistency. With the implementation shared, drift is impossible. The module is #[doc(hidden)] because the supported entry points are the typed serde impls and (forthcoming) DynamicMessage's JSON codec — these helpers operate on raw scalars and have no semver contract.
Two changes that lay the foundation for runtime reflection. Linked descriptor types (buffa-descriptor/src/desc.rs): MessageDescriptor, FieldDescriptor, FieldKind, SingularKind, OneofDescriptor, EnumDescriptor, EnumValueDescriptor, ServiceDescriptor, MethodDescriptor — the processed, feature-resolved form of the raw FileDescriptorProto tree. Where the raw protos use string type_name references and unresolved FeatureSet options, these types use pool indices (MessageIndex, EnumIndex, ServiceIndex) and pre-resolved edition features (presence, packed, delimited, enum openness). FieldKind flattens protobuf's orthogonal type x label x map-entry axes into a single Copy discriminant that maps 1:1 to runtime representation, the same approach protobuf-es takes with its fieldKind union. Fields are private with #[inline] accessor methods, matching the buffa convention for hand-written API types (SizeCache, UnknownFields, Tag). Construction is gated to DescriptorPool (forthcoming) — downstream test fixtures go through DescriptorPool::decode from FDS bytes, so they don't skip the feature-resolution and validation passes. Field indices within a message are u16, capping fields-per-message at 65,535. Field numbers stay u32 per the protobuf spec. Feature resolution dedup: The shared core (file/message/enum/oneof feature resolution, edition defaults, FeatureSet merge) moves from buffa-codegen/src/features.rs to buffa-descriptor/src/features.rs and is re-exported from buffa-codegen. buffa-codegen retains the codegen-only resolve_field, which overlays the referenced enum's enum_type from CodeGenContext::is_enum_closed — a lookup built during codegen and not available to a runtime pool. A divergence between codegen and the runtime pool would mean generated code and reflective code disagree on packed encoding, presence, or enum openness — sharing the implementation makes that impossible.
DescriptorPool builds the linked descriptor types from a FileDescriptorSet. Construction is three-pass: 1. Register: walk every file, recording the fully-qualified name of every message and enum (including nested ones) and assigning each a pool index. Forward references and cross-file references resolve in pass 2. 2. Link: walk again, building the linked MessageDescriptor for each message — resolving type_name strings to indices, classifying fields as singular/list/map, resolving editions features down the file -> message -> field chain, validating field numbers and the u16 field-count cap. 3. Link services: services reference message types by name for their input/output, so they link after the type passes. The pool retains the original FileDescriptorProtos after linking (file_by_name() accessor) so gRPC server reflection can serve the raw bytes. DescriptorPool::decode treats its input as untrusted — it's the entry point for consumers loading descriptors from a schema registry, gRPC server reflection peer, or on-disk policy bundle. Malformed input returns PoolError, never panics: out-of-range field numbers, negative extension ranges, dangling type names, and unparseable wire bytes are all handled. The pass-1/pass-2 walk-order invariant is asserted in release builds because a desync silently corrupts every cross-reference in the pool. Behind the new `reflect` feature (default-off). Tests in tests/pool_e2e.rs against a protoc-compiled FileDescriptorSet exercising proto3 presence, editions feature resolution, packed encoding, map entries, oneofs (including synthetic), service descriptors, idempotent re-add, and wrong-kind/missing lookups.
…ReflectCow
The reflection runtime in buffa-descriptor/src/reflect/. Behind the
`reflect` feature.
Value model (value.rs):
- Value (owned) / ValueRef<'_> (borrowed, ≤32B compile-time-asserted)
with the wire-level scalar types plus List, Map, Message containers.
- MapValue: a sorted-Vec<(MapKey, Value)> newtype with a sorted/no-
duplicates invariant. get_str(&str) is allocation-free (binary
search, no MapKey constructed) — the CEL m["key"] hot path.
MapValue::new() is const fn so the absent-map default is a real
static — no leak pattern, no OnceLock, no unsafe.
- ReflectList / ReflectMap traits over the container variants, with
&dyn trait objects in ValueRef::List/Map. This is the load-bearing
shape decision: a future vtable-mode `impl ReflectMessage for
FooView<'a>` holds RepeatedView<'a, T>/MapView<'a, K, V> which
cannot yield a &[Value] without materializing. Trait objects let
bridge mode (Vec<Value>/MapValue) and a future view impl share the
same ValueRef shape. Both &dyn and &[Value] are 16-byte fat
pointers, so the size budget holds.
- MapKeyRef<'a>: borrowed MapKey with the spec-restricted variant set
so ReflectMap::for_each consumers match exhaustively over only the
valid key types.
Trait surface (message.rs):
- ReflectMessage: dyn-safe, storage-agnostic. Accessors take
&FieldDescriptor, return ValueRef; for_each_set takes &mut dyn FnMut.
which_oneof() default-implemented mirrors protoreflect.WhichOneof.
- ReflectMessageMut: set/clear with oneof-sibling clearing.
- ReflectCow::{Borrowed,Owned}: clone-on-write reflective handle. Owned
is boxed to keep ReflectCow at 24 bytes (one fat pointer + tag), the
budget that keeps ValueRef at 32.
- Reflectable: the codegen entry point. Bridge-mode reflect() round-
trips through encode/decode; the call site `foo.reflect().get(fd)`
is the same in vtable mode (deferred).
DynamicMessage (dynamic.rs):
- BTreeMap<u32, Value> + Arc<DescriptorPool> + MessageIndex. Descriptor-
driven encode/decode preserving unknown fields and field-number
ordering. Validates wire type before dispatch (mismatch → unknown
field, per the protobuf spec). Singular message merge, oneof
last-wins, oneof explicit presence.
- from_message/to_message bridge for generated types.
Tests in tests/dynamic_e2e.rs and buffa-test/tests/reflect_bridge.rs
(generated <-> dynamic round-trip).
|
All contributors have signed the CLA ✍️ ✅ |
# Conflicts: # buffa-descriptor/src/lib.rs
azdagron
approved these changes
May 20, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
The reflection runtime in
buffa-descriptor/src/reflect/. Behind thereflectfeature.Value model (
value.rs)Value(owned) /ValueRef<'_>(borrowed, ≤32B compile-time-asserted) with the wire-level scalar types plusList,Map,Messagecontainers.MapValue: a sorted-Vec<(MapKey, Value)>newtype with a sorted/no-duplicates invariant.get_str(&str)is allocation-free (binary search, noMapKeyconstructed) — the CELm["key"]hot path.MapValue::new()isconst fnso the absent-map default is a realstatic— no leak pattern, noOnceLock, nounsafe.ReflectList/ReflectMaptraits with&dyntrait objects inValueRef::List/Map. This is the load-bearing shape decision: a future vtable-modeimpl ReflectMessage for FooView<'a>holdsRepeatedView<'a, T>/MapView<'a, K, V>which cannot yield a&[Value]without materializing. Trait objects let bridge mode (Vec<Value>/MapValue) and a future view impl share the sameValueRefshape. Both&dynand&[Value]are 16-byte fat pointers, so the size budget holds.MapKeyRef<'a>: borrowedMapKeywith the spec-restricted variant set soReflectMap::for_eachconsumers match exhaustively over only the valid key types.Trait surface (
message.rs)ReflectMessage: dyn-safe, storage-agnostic. Accessors take&FieldDescriptor, returnValueRef;for_each_settakes&mut dyn FnMut.which_oneof()default-implemented mirrorsprotoreflect.WhichOneof.ReflectMessageMut:set/clearwith oneof-sibling clearing.ReflectCow::{Borrowed, Owned}: clone-on-write reflective handle.Ownedis boxed to keepReflectCowat 24 bytes (one fat pointer + tag), the budget that keepsValueRefat 32.Reflectable: the codegen entry point. Bridge-modereflect()round-trips through encode/decode; the call sitefoo.reflect().get(fd)is the same in vtable mode (deferred).DynamicMessage(dynamic.rs)BTreeMap<u32, Value>+Arc<DescriptorPool>+MessageIndex. Descriptor-driven encode/decode preserving unknown fields and field-number ordering. Validates wire type before dispatch (mismatch → unknown field, per the protobuf spec). Singular message merge, oneof last-wins, oneof explicit presence.from_message/to_messagebridge for generated types.Verification
tests/dynamic_e2e.rsexercises scalar/container round-trips, unknown-field preservation, get/has/for_each_set, empty-containerhas()semantics, andwhich_oneof().buffa-test/tests/reflect_bridge.rsround-trips a generatedPerson/InventorythroughDynamicMessage.BUFFA_VIA_REFLECT=1mode lands in PR 6).no_stdbuild clean.Net change
+2388/-1. ~600 lines are tests + protoc fixtures. The four runtime modules are split by concern but share theValue/ValueReftypes, so they land together — splitting would create import cycles or non-compiling intermediate states.