refactor!: Remove explicit pyarrow dependency - #384
Merged
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## pycapsuel #384 +/- ##
============================================
Coverage 100.00% 100.00%
============================================
Files 56 56
Lines 3522 3416 -106
============================================
- Hits 3522 3416 -106 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Oliver Borchert (borchero)
marked this pull request as ready for review
July 30, 2026 17:15
Oliver Borchert (borchero)
requested review from
Andreas Albert (AndreasAlbertQC) and
Daniel Elsner (delsner)
as code owners
July 30, 2026 17:15
Contributor
There was a problem hiding this comment.
Pull request overview
This PR removes the library’s explicit PyArrow integration surface (Schema/Column helpers) and shifts PyArrow-related schema conversion to the Arrow PyCapsule interface introduced in #385, while keeping pyarrow available as a test dependency via Pixi.
Changes:
- Removed
Schema.to_pyarrow_schemaand allColumn.pyarrow_*helpers, along with the_compat.pashim. - Updated tests and docs to stop referencing
to_pyarrow_schema, and consolidated PyArrow schema checks aroundpyarrow.schema(SchemaClass). - Adjusted dependency configuration: removed the
pyarrowoptional extra and addedpyarrowto Pixi’sfeature.testdependencies.
Reviewed changes
Copilot reviewed 25 out of 26 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/columns/test_pyarrow.py | Removed PyArrow-specific schema conversion test suite tied to the deleted API. |
| tests/columns/test_arrow_pycapsule.py | Switched to direct pyarrow import and removed optional-skipping in favor of always-on test dependency. |
| tests/column_types/test_object.py | Removed test asserting Object.pyarrow_dtype behavior (API removed). |
| pyproject.toml | Removed pyarrow optional extra and updated mypy override module list. |
| pixi.toml | Moved pyarrow from optionals to the test feature dependencies. |
| docs/index.md | Updated external-tool integration wording to remove direct PyArrow mention. |
| docs/api/schema/conversion.rst | Removed Schema.to_pyarrow_schema from conversion API docs. |
| docs/_templates/classes/column.rst | Updated Column API doc exclusions to drop PyArrow members. |
| dataframely/schema.py | Removed to_pyarrow_schema classmethod and PyArrow compat import. |
| dataframely/columns/_base.py | Removed abstract pyarrow_dtype and pyarrow_field API from the Column base class. |
| dataframely/_compat.py | Removed the optional import shim for pyarrow as pa and stopped exporting pa. |
| dataframely/columns/any.py | Removed PyArrow field/dtype helpers from Any. |
| dataframely/columns/array.py | Removed PyArrow dtype construction helpers from Array. |
| dataframely/columns/binary.py | Removed pyarrow_dtype from Binary. |
| dataframely/columns/bool.py | Removed pyarrow_dtype from Bool. |
| dataframely/columns/categorical.py | Removed pyarrow_dtype from Categorical. |
| dataframely/columns/datetime.py | Removed PyArrow dtype mappings from date/time/datetime/duration columns. |
| dataframely/columns/decimal.py | Removed pyarrow_dtype from Decimal. |
| dataframely/columns/enum.py | Removed pyarrow_dtype dictionary encoding logic from Enum. |
| dataframely/columns/float.py | Removed pyarrow_dtype from float columns. |
| dataframely/columns/integer.py | Removed pyarrow_dtype from integer columns. |
| dataframely/columns/list.py | Removed pyarrow_dtype from List. |
| dataframely/columns/object.py | Removed pyarrow_dtype from Object. |
| dataframely/columns/string.py | Removed pyarrow_dtype from String. |
| dataframely/columns/struct.py | Removed pyarrow_dtype from Struct. |
Andreas Albert (AndreasAlbertQC)
approved these changes
Jul 31, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
With #385, there is no need for
pyarrowanymore.The positive LoC in this PR stem merely from the fact that we can "demote"
pyarrowto a non-optional test dependency which results in it being installed in more environments.