Skip to content

Relax the arrow version requirement in comet-udf-sdk #5253

Description

@andygrove

Follow-up from review of #4459 (thread).

comet-udf-sdk inherits arrow = { workspace = true }, currently 58.4.0. Because the SDK is compiled into the user's cdylib, Cargo has to unify its arrow requirement with the user's, so a UDF author is effectively pinned to arrow 58.x whatever their own project uses.

Nothing about the ABI requires that. Only FFI_ArrowArray and FFI_ArrowSchema cross the boundary, and those are #[repr(C)] renderings of the Arrow C Data Interface, stable across arrow-rs versions. As @paleolimbot noted, the requirement could in principle be relaxed back to whenever those types were added.

Not urgent: the crate is publish = false and users depend on it by git, so there are no external consumers to unblock yet. It becomes the blocking constraint if and when the SDK is published.

Things to work out:

  • A relaxed range on one workspace member needs an override rather than workspace = true, and it should not perturb how the rest of the workspace resolves.
  • The floor needs establishing by actually compiling against it, not by reading changelogs.
  • CI would want a job building the SDK against the floor version, otherwise the range is a claim nothing tests.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions