(feature) REST Catalog: Add encryption key rotation endpoint

[obelix74]

Proposed Change

Summary

This proposal adds a dedicated REST Catalog API endpoint for encryption key rotation, complementing the add and remove encryption key actions introduced in #12987. The
endpoint provides an atomic operation for rotating encryption keys with "forward-only" semantics.

Motivation / Use Case

Encryption key rotation is a critical security operation required by enterprise security policies and compliance frameworks:

Compliance Requirements

Framework	Requirement
NIST 800-57	Cryptoperiod limits based on key usage
SOC 2	Key management controls including rotation

Security Use Cases

1. Scheduled Rotation: Proactive key rotation per security policy (e.g., annually, quarterly)
2. Key Compromise Response: Immediate rotation when a key is suspected to be compromised
3. Personnel Changes: Rotate keys when key custodians leave the organization
4. Cryptoperiod Expiration: Rotate before keys exceed their recommended usage lifetime

Why a Dedicated Endpoint?

While #12987 added add and remove encryption key actions, key rotation requires:

• Atomicity: Add new key + set as current must be a single atomic operation
• Clear Intent: Rotation is a distinct operation from simply adding a key
• Audit Trail: Security teams need clear rotation events in audit logs
• Cross-Engine Consistency: All engines (Spark, Flink, Trino, etc.) should implement rotation identically
  Without a dedicated endpoint, each catalog implementation may implement rotation differently, leading to divergent behavior across the ecosystem.

There is a RFC underway in the Apache Polaris community to add Table Encryption and this API is being proposed there. Each catalog might create their own API. RFC: Table-Level Encryption Support in Apache Polaris

Proposed API

Endpoint

POST /v1/{prefix}/namespaces/{namespace}/tables/{table}/encryption/rotate

Request

{
"new-key-id": "arn:aws:kms:us-east-1:123456789:key/def-456"
}

Field	Type	Required	Description
new-key-id	string	Yes	The new master key ID to use for future encryption operations

Response

{
"previous-key-id": "arn:aws:kms:us-east-1:123456789:key/abc-123",
"current-key-id": "arn:aws:kms:us-east-1:123456789:key/def-456",
"rotated-at": "2025-01-15T10:30:00Z",
"active-key-count": 2
}

Field	Type	Description
previous-key-id	string	The key ID that was current before rotation
current-key-id	string	The new current key ID (from request)
rotated-at	string (ISO 8601)	Timestamp when rotation was performed
active-key-count	integer	Total number of active keys (for reading historical data)

Error Responses

Status	Error Code	Description
400	InvalidKeyId	The new-key-id format is invalid
404	TableNotFound	Table does not exist
409	KeyAlreadyCurrent	The specified key is already the current key
422	TableNotEncrypted	Table does not have encryption enabled
503	KmsUnavailable	Unable to verify key with KMS

Semantics

Forward-Only Key Rotation

This endpoint implements forward-only key rotation.

Key behaviors:

1. New snapshots use the new key for encryption
2. Existing snapshots remain encrypted with their original keys (no re-encryption)
3. All historical keys are retained in the encryption-keys array for reading historical data
4. Time-travel queries continue to work (old keys decrypt old snapshots)

Alternatives Considered

Without dedicated endpoint:

 catalog.add_encryption_key(table, new_key_id)
  catalog.update_table_properties(table, {"encryption.current-key-id": new_key_id})

Drawbacks:

• Not atomic: Failure between calls leaves table in inconsistent state
• Requires client-side coordination
• No clear audit trail for "rotation" as a distinct event
• Different engines may implement differently

---

/cc @rdblue @ggershinsky @amogh-jahagirdar @flyrain

Proposal document

No response

Specifications

• Table
• View
• REST
• Puffin
• Encryption
• Other