Apache Iceberg version
None
Query engine
None
Please describe the bug 馃悶
Currently, rewrite_table_path produces an unreadable copy of an encrypted table.
The procedure completes normally and returns a file list, but reading the registered target fails:
java.lang.RuntimeException: GCM tag check failed. Possible reasons: wrong decryption
key; or corrupt/tampered data. AES GCM doesn't differentiate between these two.
Caused by: javax.crypto.AEADBadTagException: Tag mismatch
Root cause:
- The copied manifest list is encrypted with a key that is never recorded. A
fresh data encryption key is generated for it, but the key is only persisted when
ManifestListWriter.toManifestListFile() is called, and the rewrite path uses the
writer as a plain FileAppender and never calls it. The key is discarded when the
writer closes.
- The target inherits the source's key registry. The rewrite copies the
encryption-keys list and the snapshot's key-id into the target metadata
unchanged, so the target points at keys that decrypt the source's manifest
lists, not its own.
For now, I wonder if we can just disallow this procedure for encrypted tables so encryption users do not run into this - #17988.
Willingness to contribute
Apache Iceberg version
None
Query engine
None
Please describe the bug 馃悶
Currently,
rewrite_table_pathproduces an unreadable copy of an encrypted table.The procedure completes normally and returns a file list, but reading the registered target fails:
Root cause:
fresh data encryption key is generated for it, but the key is only persisted when
ManifestListWriter.toManifestListFile()is called, and the rewrite path uses thewriter as a plain
FileAppenderand never calls it. The key is discarded when thewriter closes.
encryption-keyslist and the snapshot'skey-idinto the target metadataunchanged, so the target points at keys that decrypt the source's manifest
lists, not its own.
For now, I wonder if we can just disallow this procedure for encrypted tables so encryption users do not run into this - #17988.
Willingness to contribute