Skip to content

readStreaming ignores its useUnalignedBuffers parameter #178

Description

@maltzsama

Describe the bug, including details regarding any error messages, version, and platform.

Description

ArrowReader.readStreaming(_:useUnalignedBuffers:) declares a
useUnalignedBuffers parameter but never uses it. The ByteBuffer is
constructed with allowReadingUnalignedBuffers: true regardless of what
the caller passes.

readFile(_:useUnalignedBuffers:) and fromMessage(_:dataBody:result:useUnalignedBuffers:)
both forward the parameter correctly, so the streaming path is inconsistent
with the rest of the reader API.

Location

Sources/Arrow/ArrowReader.swift, in readStreaming:

var dataBuffer = ByteBuffer(
    data: streamData,
    allowReadingUnalignedBuffers: true
)

Open question

It is not clear whether the hardcoded true is intentional. The streaming
reader slices the input at message boundaries (input[offset...]), and
ArrowWriter.writeStreaming does not pad the schema message to an 8-byte
boundary, so subsequent messages can start at unaligned offsets. If the
hardcoded value was introduced to avoid a trap on unaligned reads, then
forwarding the parameter would reintroduce that problem and the correct fix
would be to remove or deprecate the parameter instead.

Guidance from a maintainer on which behaviour is intended would be helpful
before opening a PR.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions