Problem
WriterPropertiesBuilder::set_column_encoding accepts any Encoding for any column, but incompatible encoding/physical-type combinations can panic when the encoder is constructed. For example, configuring RLE for an INT64 column reaches RleValueEncoder::new, whose ensure_phys_ty! check panics. Likewise, DELTA_BINARY_PACKED on a BYTE_ARRAY column panics in DeltaBitPackEncoder::new.
Downstream callers that accept user-selected encodings need to duplicate parquet's compatibility matrix to validate input and return a normal error before writing. That matrix can drift as parquet adds encodings or changes support.
Requested API/behavior
Please expose a stable, fallible API that validates whether an Encoding is supported for a physical type or column descriptor, or make writer/encoder construction return ParquetError rather than panic for incompatible combinations. Downstream callers could then delegate validation to parquet instead of maintaining a copy.
This arose while implementing per-column encoding selection in datafusion-contrib/tpcgen-rs#405.
Problem
WriterPropertiesBuilder::set_column_encodingaccepts anyEncodingfor any column, but incompatible encoding/physical-type combinations can panic when the encoder is constructed. For example, configuringRLEfor anINT64column reachesRleValueEncoder::new, whoseensure_phys_ty!check panics. Likewise,DELTA_BINARY_PACKEDon aBYTE_ARRAYcolumn panics inDeltaBitPackEncoder::new.Downstream callers that accept user-selected encodings need to duplicate parquet's compatibility matrix to validate input and return a normal error before writing. That matrix can drift as parquet adds encodings or changes support.
Requested API/behavior
Please expose a stable, fallible API that validates whether an
Encodingis supported for a physical type or column descriptor, or make writer/encoder construction returnParquetErrorrather than panic for incompatible combinations. Downstream callers could then delegate validation to parquet instead of maintaining a copy.This arose while implementing per-column encoding selection in datafusion-contrib/tpcgen-rs#405.