feat: Implement Arrow PyCapsule interface for schemas

[Oliver Borchert (borchero)]

Motivation

Currently, we implement to_polars_schema, to_pyarrow_schema, and there could be a need to implement this conversion for more packages going forward (e.g. for https://github.com/borchero/ducklake-sdk).

In order not to be required to implement the conversion manually, for every other package, we can leverage the Arrow PyCapsule Interface to obtain a "canonical" representation of the schema in Arrow. This schema can then be imported by other packages supporting the interface. Today, this includes, e.g., polars, pyarrow, and ducklake-sdk.

Changes

• Implement __arrow_c_schema__ on dy.Schema. The implementation is done in our custom Rust code as we need to "correct" nullability information which would be lost by calling __arrow_c_schema__ on a pl.Schema. Building the PyCapsule in Python would be possible but very annoying with ctypes.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 100.00%. Comparing base (84d3166) to head (6ec3629).

Additional details and impacted files

@@            Coverage Diff            @@
##              main      #385   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files           56        56           
  Lines         3507      3522   +15     
=========================================
+ Hits          3507      3522   +15     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[Copilot]

Pull request overview

Adds support for the Arrow PyCapsule (C Data) schema export on dy.Schema so downstream libraries (e.g. pyarrow, polars) can import a canonical Arrow schema directly, preserving metadata and correctly propagating nested nullability.

Changes:

• Implement SchemaMeta.__arrow_c_schema__ in Python, backed by a native Rust arrow_c_schema function that builds an arrow_schema PyCapsule.
• Add Arrow nullability export hooks (_arrow_nullability) on columns, including nested columns, plus new tests validating metadata and nullability.
• Update documentation to describe Arrow PyCapsule-based interoperability; adjust Rust dependencies (including a temporary Polars fork patch).

Reviewed changes

Copilot reviewed 12 out of 13 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
tests/columns/test_arrow_pycapsule.py	Adds coverage for Arrow PyCapsule schema import behavior (metadata + nullability).
src/lib.rs	Registers the new native arrow_c_schema function in the Python module.
src/arrow.rs	Implements native Arrow schema PyCapsule export and recursive nullability application.
docs/index.md	Mentions Arrow PyCapsule as an external integration mechanism.
docs/guides/quickstart.md	Documents how to use pyarrow.schema(HouseSchema) / polars.Schema(HouseSchema) via the PyCapsule interface.
dataframely/columns/struct.py	Adds nested nullability export for struct fields.
dataframely/columns/list.py	Adds nested nullability export for list value fields.
dataframely/columns/array.py	Adds nested nullability export for arrays.
dataframely/columns/_base.py	Introduces the default _arrow_nullability hook for all columns.
dataframely/_native.pyi	Adds typing for the new native arrow_c_schema binding and Nullability.
dataframely/_base_schema.py	Adds __arrow_c_schema__ implementation on schema classes.
Cargo.toml	Enables additional pyo3-polars dtype features and temporarily patches Polars crates to a fork.
Cargo.lock	Updates lockfile for the dependency changes and patched sources.