Skip to content

feat(protogen): preserve stable field numbers and auto-emit reserved statements on regeneration#403

Merged
wenchy merged 1 commit into
masterfrom
reserve-field-number
May 26, 2026
Merged

feat(protogen): preserve stable field numbers and auto-emit reserved statements on regeneration#403
wenchy merged 1 commit into
masterfrom
reserve-field-number

Conversation

@Kybxd
Copy link
Copy Markdown
Collaborator

@Kybxd Kybxd commented May 9, 2026

Background / Motivation

When Tableau regenerates .proto files from Excel workbooks, it previously re-assigned field numbers strictly from 1..N in declaration order. This broke wire-format compatibility in two ways:

  1. Reordering / inserting columns silently reassigned field numbers of unchanged columns, breaking existing serialized data.
  2. Deleting columns freed the corresponding numbers for reuse by future new columns, violating protobuf's golden rule "never reuse a deleted field number".

This PR makes regeneration wire-compatible by preserving old field numbers and emitting reserved statements as needed.

What changed

internal/protogen/exporter.gosheetExporter.assignFieldNumbers:

  • Stable numbers for surviving fields. When an oldMD (previously generated MessageDescriptor) is available, fields whose name matches an existing field keep their old number, regardless of column order.
  • Reserve deleted fields. Any field present in oldMD but no longer in the current sheet has its number added to a reserved set.
  • Carry over pre-existing reserved ranges. Reserved ranges already declared in oldMD are preserved verbatim across regenerations.
  • Auto-reserve gaps. Numbers in [1, maxUsedNumber] that are neither active nor already reserved are auto-reserved — this defends against older Tableau versions that deleted fields without reserving them.
  • Collision-free new numbers. New fields get the next number strictly greater than any active-or-reserved number, so they never collide with a reserved entry.

Supporting pieces:

  • New helper formatReservedNumbers collapses a sorted number list into the compact proto syntax (2, 5, 7 to 9, 12).
  • New helper sheetExporter.printReserved emits a single reserved …; line at the correct indentation.
  • The three call-sites (exportStruct, exportUnion, exportMessager, and the nested-message branch in exportField) now consume the returned reserved slice and print it.

Tests

internal/protogen/exporter_test.go:

  • New Test_formatReservedNumbers covers empty / single / non-consecutive / consecutive / mixed / multi-range formatting.
  • New Test_sheetExporter_assignFieldNumbers uses protodesc.NewFile to synthesize MessageDescriptors with arbitrary active fields and reserved ranges, covering 10 semantic scenarios:
    • oldMD == nil → sequential assignment from 1.
    • All old fields kept → reuse old numbers.
    • New field appended → next number after max used.
    • Deleted field → number becomes reserved.
    • Pre-existing reserved N right after active fields → new field skips it.
    • Pre-existing reserved N higher than all active fields → all gaps below are auto-reserved; new field starts after N.
    • Pre-existing reserved + deletion in the same round → merged correctly.
    • Pre-existing consecutive reserved 2 to 4 → carried over verbatim.
    • Active-field gap (1, 2, 4) without pre-existing reserved → gap auto-reserved.
    • Field rename → old number reserved, new name gets next number.
  • Existing exportStruct / exportUnion / exportMessager tests extended with field-number-compatibility-* sub-tests (insertion-in-the-middle, delete-and-add, sub-message nesting).

Compatibility

  • Wire-format compatible: no existing field number is ever moved or reused.
  • Backward compatible with .proto files generated by older Tableau versions that left gaps without reserved: those gaps are now auto-reserved on the next regeneration.
  • No public API change; behavior is gated on whether oldMD is discoverable (i.e., requires PreserveFieldNumbers + an existing generated proto in the registry), so fresh generations are unaffected.

@codecov
Copy link
Copy Markdown

codecov Bot commented May 9, 2026

Codecov Report

❌ Patch coverage is 97.50000% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 74.78%. Comparing base (24a1dc8) to head (6e4a53f).

Files with missing lines Patch % Lines
internal/protogen/exporter.go 97.50% 1 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master     #403      +/-   ##
==========================================
+ Coverage   74.63%   74.78%   +0.15%     
==========================================
  Files          88       88              
  Lines        9185     9249      +64     
==========================================
+ Hits         6855     6917      +62     
- Misses       1755     1756       +1     
- Partials      575      576       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@Kybxd Kybxd requested a review from wenchy May 11, 2026 02:59
@Kybxd Kybxd force-pushed the reserve-field-number branch from 0c89e3a to 6e4a53f Compare May 26, 2026 09:06
Copy link
Copy Markdown
Member

@wenchy wenchy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@wenchy wenchy merged commit e51127c into master May 26, 2026
7 checks passed
@wenchy wenchy deleted the reserve-field-number branch May 26, 2026 14:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

protogen: Field number compatibility broken across multiple regenerations: deleted field numbers are not marked as reserved and may be reused

2 participants