Skip to content

Use FixedSizeBinary instead of Binary for int96 conversion when convertInt96ToArrowTimestamp is false #3088

@doki23

Description

@doki23

Describe the enhancement requested

public TypeMapping convertINT96(PrimitiveTypeName primitiveTypeName) throws RuntimeException {
  if (convertInt96ToArrowTimestamp) {
    return field(new ArrowType.Timestamp(TimeUnit.NANOSECOND, null));
  } else {
    return field(new ArrowType.Binary());
  }
}

When converting a Parquet type to an Arrow type, if the original type is int96 and the option convertInt96ToArrowTimestamp is set to false, the resulting Arrow type defaults to Binary. However, it might be more appropriate to use FixedSizeBinary instead.

Component(s)

No response

Metadata

Metadata

Assignees

No one assigned

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions