Apache Iceberg Rust version
None
Describe the bug
Hi team, spec requires puffin footer to be either uncompressed or LZ4 compressed. But currently we don't implement lz4, which means the compression flag is incorrect.
To Reproduce
Example to reproduce
#[tokio::test]
async fn test_compress_footer_fails_because_lz4_is_unsupported() {
let temp_dir = TempDir::new().unwrap();
let file_io = FileIO::new_with_fs();
let path = temp_dir.path().join("compressed_footer.bin");
let output_file = file_io.new_output(path.to_str().unwrap()).unwrap();
// compress_footer=true sets the footer codec to LZ4, which is unimplemented.
let writer = PuffinWriter::new(&output_file, HashMap::new(), /*compress_footer=*/true)
.await
.unwrap();
writer.close().await.unwrap();
}
Error message
thread 'puffin::writer::tests::test_compress_footer_fails_because_lz4_is_unsupported' (400849) panicked at crates/iceberg/src/puffin/writer.rs:350:30:
called `Result::unwrap()` on an `Err` value: FeatureUnsupported => LZ4 compression is not supported currently
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
failures:
puffin::writer::tests::test_compress_footer_fails_because_lz4_is_unsupported
Expected behavior
When compression option specified, footer payload should be compressed and written without issue.
Willingness to contribute
None
Apache Iceberg Rust version
None
Describe the bug
Hi team, spec requires puffin footer to be either uncompressed or LZ4 compressed. But currently we don't implement lz4, which means the compression flag is incorrect.
To Reproduce
Example to reproduce
Error message
Expected behavior
When compression option specified, footer payload should be compressed and written without issue.
Willingness to contribute
None