Skip to content

[Python] Convert to_pylist fast paths per range instead of per element #50448

Description

@viirya

Describe the enhancement requested

Follow-up of #50326 (mechanism introduced in GH-50327). The scalar-free conversion currently makes one _getitem_py virtual call per element. Several per-element overheads could be batched per range, as suggested in the GH-50327 review:

  1. Replace the per-element entry point with a cdef list _getitem_range_py(self, int64_t offset, int64_t length) used by to_pylist and the list/struct/map container loops, cutting per-element function-call and prologue overhead.
  2. Hoist the numeric type-id dispatch out of the loop: NumericArray._getitem_py currently walks an if/elif ladder per element even though the type is invariant; a per-range implementation (or per-concrete-class overrides) makes the loop monomorphic.
  3. Skip the per-element IsNull check when null_count() == 0.
  4. Read the offsets buffer pointer once per range in the list/map paths instead of calling value_offset() twice per row.

None of these change semantics; they compound on top of the structural win in GH-50327 (no per-element Scalar / per-row Array wrapper allocation).

Component(s)

Python

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions