Skip to content

refactor!: Remove explicit pyarrow dependency - #384

Merged
Oliver Borchert (borchero) merged 8 commits into
pycapsuelfrom
remove-pyarrow
Jul 31, 2026
Merged

refactor!: Remove explicit pyarrow dependency#384
Oliver Borchert (borchero) merged 8 commits into
pycapsuelfrom
remove-pyarrow

Conversation

@borchero

@borchero Oliver Borchert (borchero) commented Jul 29, 2026

Copy link
Copy Markdown
Member

Motivation

With #385, there is no need for pyarrow anymore.

The positive LoC in this PR stem merely from the fact that we can "demote" pyarrow to a non-optional test dependency which results in it being installed in more environments.

@codecov

codecov Bot commented Jul 30, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 100.00%. Comparing base (6ec3629) to head (841c771).

Additional details and impacted files
@@             Coverage Diff              @@
##           pycapsuel      #384    +/-   ##
============================================
  Coverage     100.00%   100.00%            
============================================
  Files             56        56            
  Lines           3522      3416   -106     
============================================
- Hits            3522      3416   -106     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@borchero
Oliver Borchert (borchero) changed the base branch from main to pycapsuel July 30, 2026 17:12

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR removes the library’s explicit PyArrow integration surface (Schema/Column helpers) and shifts PyArrow-related schema conversion to the Arrow PyCapsule interface introduced in #385, while keeping pyarrow available as a test dependency via Pixi.

Changes:

  • Removed Schema.to_pyarrow_schema and all Column.pyarrow_* helpers, along with the _compat.pa shim.
  • Updated tests and docs to stop referencing to_pyarrow_schema, and consolidated PyArrow schema checks around pyarrow.schema(SchemaClass).
  • Adjusted dependency configuration: removed the pyarrow optional extra and added pyarrow to Pixi’s feature.test dependencies.

Reviewed changes

Copilot reviewed 25 out of 26 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
tests/columns/test_pyarrow.py Removed PyArrow-specific schema conversion test suite tied to the deleted API.
tests/columns/test_arrow_pycapsule.py Switched to direct pyarrow import and removed optional-skipping in favor of always-on test dependency.
tests/column_types/test_object.py Removed test asserting Object.pyarrow_dtype behavior (API removed).
pyproject.toml Removed pyarrow optional extra and updated mypy override module list.
pixi.toml Moved pyarrow from optionals to the test feature dependencies.
docs/index.md Updated external-tool integration wording to remove direct PyArrow mention.
docs/api/schema/conversion.rst Removed Schema.to_pyarrow_schema from conversion API docs.
docs/_templates/classes/column.rst Updated Column API doc exclusions to drop PyArrow members.
dataframely/schema.py Removed to_pyarrow_schema classmethod and PyArrow compat import.
dataframely/columns/_base.py Removed abstract pyarrow_dtype and pyarrow_field API from the Column base class.
dataframely/_compat.py Removed the optional import shim for pyarrow as pa and stopped exporting pa.
dataframely/columns/any.py Removed PyArrow field/dtype helpers from Any.
dataframely/columns/array.py Removed PyArrow dtype construction helpers from Array.
dataframely/columns/binary.py Removed pyarrow_dtype from Binary.
dataframely/columns/bool.py Removed pyarrow_dtype from Bool.
dataframely/columns/categorical.py Removed pyarrow_dtype from Categorical.
dataframely/columns/datetime.py Removed PyArrow dtype mappings from date/time/datetime/duration columns.
dataframely/columns/decimal.py Removed pyarrow_dtype from Decimal.
dataframely/columns/enum.py Removed pyarrow_dtype dictionary encoding logic from Enum.
dataframely/columns/float.py Removed pyarrow_dtype from float columns.
dataframely/columns/integer.py Removed pyarrow_dtype from integer columns.
dataframely/columns/list.py Removed pyarrow_dtype from List.
dataframely/columns/object.py Removed pyarrow_dtype from Object.
dataframely/columns/string.py Removed pyarrow_dtype from String.
dataframely/columns/struct.py Removed pyarrow_dtype from Struct.

Comment thread pyproject.toml
Comment thread docs/api/schema/conversion.rst
Comment thread docs/api/schema/conversion.rst
@borchero
Oliver Borchert (borchero) merged commit 0efd98a into main Jul 31, 2026
33 checks passed
@borchero
Oliver Borchert (borchero) deleted the remove-pyarrow branch July 31, 2026 12:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants