Skip to content

BUG: failed to write compressed puffin footer #2419

@dentiny

Description

@dentiny

Apache Iceberg Rust version

None

Describe the bug

Hi team, spec requires puffin footer to be either uncompressed or LZ4 compressed. But currently we don't implement lz4, which means the compression flag is incorrect.

To Reproduce

Example to reproduce

#[tokio::test]
    async fn test_compress_footer_fails_because_lz4_is_unsupported() {
        let temp_dir = TempDir::new().unwrap();
        let file_io = FileIO::new_with_fs();
        let path = temp_dir.path().join("compressed_footer.bin");
        let output_file = file_io.new_output(path.to_str().unwrap()).unwrap();

        // compress_footer=true sets the footer codec to LZ4, which is unimplemented.
        let writer = PuffinWriter::new(&output_file, HashMap::new(), /*compress_footer=*/true)
            .await
            .unwrap();
        writer.close().await.unwrap();
    }

Error message

thread 'puffin::writer::tests::test_compress_footer_fails_because_lz4_is_unsupported' (400849) panicked at crates/iceberg/src/puffin/writer.rs:350:30:
called `Result::unwrap()` on an `Err` value: FeatureUnsupported => LZ4 compression is not supported currently

note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace


failures:
    puffin::writer::tests::test_compress_footer_fails_because_lz4_is_unsupported

Expected behavior

When compression option specified, footer payload should be compressed and written without issue.

Willingness to contribute

None

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions