Search before asking
Please describe the bug
Rust declares IndexFileMeta.file_size as i32 even though the index-manifest Avro schema declares _FILE_SIZE as long and Java Paimon uses long. The fast Avro decoder reads the complete long and then casts it to i32.
As a result, Rust builders reject an individual index file larger than i32::MAX, while a Java-written manifest containing a larger value is silently truncated when Rust reads it. For example, 2,147,483,648 cannot be represented correctly.
Expected behavior: Rust should preserve the full signed 64-bit file size used by the storage schema and Java implementation.
Solution
Use i64 for IndexFileMeta.file_size, remove the narrowing cast in the fast decoder, and use checked i64 conversions in all index-file writers. Keep the existing Avro long schema unchanged and add a boundary round-trip regression test.
Are you willing to submit a PR?
Search before asking
Please describe the bug
Rust declares IndexFileMeta.file_size as i32 even though the index-manifest Avro schema declares _FILE_SIZE as long and Java Paimon uses long. The fast Avro decoder reads the complete long and then casts it to i32.
As a result, Rust builders reject an individual index file larger than i32::MAX, while a Java-written manifest containing a larger value is silently truncated when Rust reads it. For example, 2,147,483,648 cannot be represented correctly.
Expected behavior: Rust should preserve the full signed 64-bit file size used by the storage schema and Java implementation.
Solution
Use i64 for IndexFileMeta.file_size, remove the narrowing cast in the fast decoder, and use checked i64 conversions in all index-file writers. Keep the existing Avro long schema unchanged and add a boundary round-trip regression test.
Are you willing to submit a PR?