Skip to content

descriptor: reject ambiguous field identities#300

Merged
iainmcgin merged 3 commits into
anthropics:mainfrom
fallintoplace:agent/validate-descriptor-field-identity
Jul 16, 2026
Merged

descriptor: reject ambiguous field identities#300
iainmcgin merged 3 commits into
anthropics:mainfrom
fallintoplace:agent/validate-descriptor-field-identity

Conversation

@fallintoplace

@fallintoplace fallintoplace commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

What this does

Rejects duplicate field numbers and collisions across proto and JSON field names before building the message lookup indexes.

It also rejects negative and out-of-range oneof_index values instead of silently leaving fields with oneof presence but no oneof membership. DescriptorPool stages additions on a clone, so each invalid descriptor is rejected without changing an existing pool.

Coverage

The regression tests cover duplicate field numbers with different wire kinds, duplicate proto names, duplicate JSON names, proto/JSON cross-collisions, positive and negative oneof indices, and transactional rollback.

Validation

  • cargo fmt --all -- --check
  • cargo clippy --workspace --all-targets -- -D warnings
  • cargo test --workspace
  • cargo test -p buffa-descriptor --features reflect
  • cargo check --workspace --all-features

@github-actions

Copy link
Copy Markdown

All contributors have signed the CLA ✍️ ✅
Posted by the CLA Assistant Lite bot.

@fallintoplace fallintoplace changed the title Reject ambiguous descriptor fields in DescriptorPool descriptor: reject ambiguous field identities Jul 10, 2026
…leniency

The check rejected two fields resolving to one JSON name in any message, which
refuses descriptor sets protoc itself produces. Verified against protoc 33.5:
a proto2 conflict between derived JSON names is a warning and the set is
emitted, and a proto3 message that sets deprecated_legacy_json_field_conflicts
is accepted outright. Both loaded fine in 0.8.x and would have failed here.

The failure mode made it worse than a build break. Codegen never builds a pool,
so such a schema still compiles; the panic lands at runtime on first
descriptor_pool() access, from generated code whose doc comment blames a
codegen bug for what is a legal schema.

Gate on the signal protobuf already defines for this: json_format resolves to
LegacyBestEffort for proto2 and Allow for proto3 and editions, which is the
same distinction protoc draws, and the deprecated option is its explicit
opt-out. protoc rejects every conflict this leaves through, so a hand-built or
third-party set with genuinely ambiguous JSON names is still caught while
protoc's own output always loads.

Duplicate proto names and field numbers stay unconditional: those are invalid
in every syntax and protoc never emits them.

@iainmcgin iainmcgin left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[claude code] Thanks for this — three of the four rules are exactly right, and the oneof_index one is a genuine catch: link_field silently dropped an out-of-u16 index to None while presence resolution keyed off the raw oneof_index.is_some(), so a field could end up with oneof presence and no oneof membership. That's a real latent bug, not just validation.

The JSON-name rule needed reshaping, and I've done that maintainer-side in 9b3aa07 rather than sending it back to you. It rejected descriptor sets protoc itself emits. Verified against protoc 33.5:

case protoc 33.5 buf 1.70 this PR (before)
proto2, derived JSON-name conflict warning, emits emits rejected
proto3 + deprecated_legacy_json_field_conflicts accepts rejects rejected
proto3, no opt-out error error rejected
proto2, explicit duplicate json_name error error rejected

I fed both protoc-emitted sets to DescriptorPool::decode on this branch and both came back REJECTED. Since protoc hard-errors on every genuine conflict itself, the rule could only ever fire on sets protoc had deliberately blessed.

The failure mode made it sharper than a build break: buffa-codegen never constructs a pool, so such a schema still compiles clean. The panic lands at runtime on first descriptor_pool() access — from generated code whose doc comment says a panic there "indicates a codegen bug, not consumer input", which would be actively wrong.

The fix turned out to need no new API, because protobuf already models this distinction and buffa already resolves it: json_format is LegacyBestEffort for proto2 and Allow for proto3/editions — the same line protoc draws — and deprecated_legacy_json_field_conflicts is protoc's explicit per-message opt-out. So the rule is now gated on msg_features.json_format == JsonFormat::Allow && !deprecated_legacy_json_field_conflicts. protoc rejects everything that leaves through, so a hand-built or third-party set with genuinely ambiguous JSON names is still caught, while protoc's output always loads. One condition, ~10 lines, no new public surface.

Worth noting a constraint I hit: protoc populates json_name for every field, so a descriptor set cannot distinguish an explicit json_name from a derived one. Exact protoc parity is therefore unreachable — this rule is a deliberate superset, slightly more lenient than protoc for hand-built proto2 sets, which is the safe direction.

Also added: three tests pinning the new behaviour (proto2 conflicts accepted, the opt-out honoured, duplicate proto names still rejected in proto2 — the leniency is scoped to JSON names only). The first two fail against the strict rule, so they genuinely guard it. And a changelog fragment: this is a user-visible tightening shipping in 0.9.0 and needed one — check-changelog only verifies CHANGELOG.md is in sync with existing fragments, so its passing wasn't evidence one wasn't needed. Merged current main in as well; the branch predated today's five merges.

Your other three rules are untouched and land as written. Approving.

@iainmcgin
iainmcgin enabled auto-merge July 16, 2026 19:58
@iainmcgin
iainmcgin added this pull request to the merge queue Jul 16, 2026
Merged via the queue into anthropics:main with commit 940dfac Jul 16, 2026
9 checks passed
@github-actions github-actions Bot locked and limited conversation to collaborators Jul 16, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants