feat(#492): implement SetStatistics, RemoveStatistics, AddEncryptionKey, RemoveEncryptionKey updates by tanmayrauth · Pull Request #902 · apache/iceberg-go

tanmayrauth · 2026-04-15T04:07:00Z

Adds the four remaining table update types required for REST catalog compatibility. Without these, any commit response containing these actions fails with "unknown update action".

Changes:

table/encryption.go: new file; defines EncryptionKey type (key-id, key-metadata, key-algorithm) separate from statistics types
table/statistics.go: remove EncryptionKey (moved to encryption.go)
table/updates.go: add constants and update structs for set-statistics, remove-statistics, add-encryption-key, remove-encryption-key; register all four in UnmarshalJSON
table/metadata.go:
- add EncryptionKeys() iter.Seq2[string, EncryptionKey] to Metadata interface and implement on commonMetadata
- add EncryptionKeyMap field to commonMetadata with JSON tag encryption-keys (omitempty; inert on V1/V2 tables)
- add encryptionKeyMap to MetadataBuilder; initialize in NewMetadataBuilder alongside refs; copy in MetadataBuilderFromBase; write through in buildCommonMetadata
- add SetStatistics (upsert by snapshot-id), RemoveStatistics, AddEncryptionKey, RemoveEncryptionKey builder methods
- include EncryptionKeyMap in commonMetadata.Equals
table/updates_test.go: unmarshal and Apply tests for all four update types including replace-not-append semantics for SetStatistics and no-op remove cases

Feature #492

tanmayrauth · 2026-04-15T20:29:23Z

@zeroshade Can you please review this change and merge if the changes are okay?

zeroshade · 2026-04-15T21:34:47Z

+type EncryptionKey struct {
+	KeyID        string  `json:"key-id"`
+	KeyMetadata  *string `json:"key-metadata,omitempty"`
+	KeyAlgorithm *string `json:"key-algorithm,omitempty"`
 }


These aren't the correct json keys according to the spec, for example it should be encrypted-key-metadata not key-metadata, there is no KeyAlgorithm in the spec and you're missing the "Properties" member. Please update to follow the v3 spec

Fixed. Renamed to encrypted-key-metadata, removed key-algorithm, added properties field per the V3 spec.

zeroshade · 2026-04-15T21:36:21Z

 	SnapshotRefs       map[string]SnapshotRef    `json:"refs,omitempty"`
 	StatisticsList     []StatisticsFile          `json:"statistics,omitempty"`
 	PartitionStatsList []PartitionStatisticsFile `json:"partition-statistics,omitempty"`
+	EncryptionKeyMap   map[string]EncryptionKey  `json:"encryption-keys,omitempty"`


According to the spec, encryption-keys is a list, not a map

Right, I've changed from map[string]EncryptionKey to []EncryptionKey throughout metadata, builder, and interface.

laskoviymishka

maps.Equal on EncryptionKey is broken — *string fields compared by pointer address, not value. V3 gating missing for encryption keys.

laskoviymishka · 2026-04-15T21:46:55Z

 		slices.Equal(c.SnapshotLog, other.SnapshotLog) && slices.Equal(c.MetadataLog, other.MetadataLog) &&
-		iceinternal.SliceEqualHelper(c.SortOrderList, other.SortOrderList)
+		iceinternal.SliceEqualHelper(c.SortOrderList, other.SortOrderList) &&
+		maps.Equal(c.EncryptionKeyMap, other.EncryptionKeyMap)


i think this is a bug

EncryptionKey has *string fields (KeyMetadata, KeyAlgorithm). maps.Equal uses == — compares pointer addresses, not values. Two keys deserialized from the same JSON will have different pointers. Equals() returns false for identical metadata. Use maps.EqualFunc with reflect.DeepEqual like SnapshotRefs three lines above.

Now using iceinternal.SliceEqualHelper with an Equals() method on EncryptionKey that does proper value comparison of *string fields and map[string]string properties.

laskoviymishka · 2026-04-15T21:47:31Z

+}
+
+// AddEncryptionKey adds or replaces an encryption key indexed by its key-id.
+func (b *MetadataBuilder) AddEncryptionKey(key EncryptionKey) error {


Encryption keys are V3+. No format version check. The codebase already gates V3 features — see minFormatVersionRowLineage. Add a b.formatVersion < 3 guard.

Fixed. AddEncryptionKey now rejects calls when formatVersion < 3 with a descriptive error, consistent with the existing minFormatVersionRowLineage pattern.

…Key, RemoveEncryptionKey updates Adds the four remaining table update types required for REST catalog compatibility. Without these, any commit response containing these actions fails with "unknown update action". Changes: - table/encryption.go: new file; defines EncryptionKey type (key-id, key-metadata, key-algorithm) separate from statistics types - table/statistics.go: remove EncryptionKey (moved to encryption.go) - table/updates.go: add constants and update structs for set-statistics, remove-statistics, add-encryption-key, remove-encryption-key; register all four in UnmarshalJSON - table/metadata.go: - add EncryptionKeys() iter.Seq2[string, EncryptionKey] to Metadata interface and implement on commonMetadata - add EncryptionKeyMap field to commonMetadata with JSON tag encryption-keys (omitempty; inert on V1/V2 tables) - add encryptionKeyMap to MetadataBuilder; initialize in NewMetadataBuilder alongside refs; copy in MetadataBuilderFromBase; write through in buildCommonMetadata - add SetStatistics (upsert by snapshot-id), RemoveStatistics, AddEncryptionKey, RemoveEncryptionKey builder methods - include EncryptionKeyMap in commonMetadata.Equals - table/updates_test.go: unmarshal and Apply tests for all four update types including replace-not-append semantics for SetStatistics and no-op remove cases

- Fix EncryptionKey struct: rename key-metadata to encrypted-key-metadata, remove key-algorithm, add encrypted-by-id and properties fields - Change encryption-keys from map to list per spec - Fix equality check using Equals method instead of maps.Equal - Add V3 format version guard to AddEncryptionKey

laskoviymishka

zeroshade

Just one nitpick and then we're good

…isticsUpdate (#1018) Adds SetPartitionStatisticsUpdate and RemovePartitionStatisticsUpdate in table/updates.go with MetadataBuilder methods and UnmarshalJSON wiring. Mirrors the existing SetStatisticsUpdate / RemoveStatisticsUpdate from #902. Closes #1009

tanmayrauth requested a review from zeroshade as a code owner April 15, 2026 04:07

tanmayrauth mentioned this pull request Apr 15, 2026

feat: missing table updates #492

Closed

2 tasks

zeroshade requested changes Apr 15, 2026

View reviewed changes

laskoviymishka requested changes Apr 15, 2026

View reviewed changes

tanmayrauth added 2 commits April 15, 2026 20:49

tanmayrauth force-pushed the feat/missing-table-updates-492 branch from a9efcfd to 98adfba Compare April 16, 2026 04:13

laskoviymishka approved these changes Apr 16, 2026

View reviewed changes

zeroshade requested changes Apr 16, 2026

View reviewed changes

Comment thread table/encryption.go Outdated

zeroshade added 2 commits April 16, 2026 13:32

Update table/encryption.go

1948680

lint fix

50172d5

zeroshade approved these changes Apr 16, 2026

View reviewed changes

zeroshade merged commit c5f82b5 into apache:main Apr 16, 2026
14 checks passed

laskoviymishka mentioned this pull request Apr 27, 2026

feat: v2 spec completion tracking issue #829

Open

96 tasks

This was referenced May 5, 2026

feat(table): SetPartitionStatisticsUpdate and RemovePartitionStatisticsUpdate #1009

Closed

feat: v3 support tracking issue #589

Open

tanmayrauth mentioned this pull request May 5, 2026

feat(table): add SetPartitionStatisticsUpdate and RemovePartitionStatisticsUpdate #1018

Merged

Conversation

tanmayrauth commented Apr 15, 2026

Uh oh!

tanmayrauth commented Apr 15, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

laskoviymishka left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

laskoviymishka left a comment

Choose a reason for hiding this comment

Uh oh!

zeroshade left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

zeroshade left a comment •

edited

Loading