Skip to content

ci: fix publish paimon-datafusion and python issue#253

Merged
JingsongLi merged 1 commit intoapache:release-0.1from
luoyuxia:fix-publish-python-rust-issue
Apr 16, 2026
Merged

ci: fix publish paimon-datafusion and python issue#253
JingsongLi merged 1 commit intoapache:release-0.1from
luoyuxia:fix-publish-python-rust-issue

Conversation

@luoyuxia
Copy link
Copy Markdown
Contributor

@luoyuxia luoyuxia commented Apr 16, 2026

Purpose

Fix CI issues that block the 0.1.0 release:

  1. paimon-datafusion cannot be published to crates.io because it depends on the unpublished paimon
  2. Python wheel builds fail on both x86_64 and aarch64 due to OpenSSL dependency issues

Brief change log

Rust crate publishing (release-rust.yml):

  • Run cargo publish --dry-run only for paimon (dry-run fails for paimon-datafusion because it depends on the unpublished paimon)
  • After publishing paimon, actively poll crates.io until the new version is indexed (up to 5 minutes, every 30 seconds) before proceeding to publish paimon-datafusion

Workspace dependency refactor:

  • Add paimon = { version = "0.1.0", path = "crates/paimon" } to [workspace.dependencies] in root Cargo.toml
  • crates/integrations/datafusion/Cargo.toml: switch from paimon = { path = "../../paimon" } to paimon = { workspace = true }
  • Reason: publishing to crates.io requires internal workspace dependencies to specify both version and path

Python wheel builds (release_python_binding.yml):

  • sdist: use apt-get install libssl-dev (more reliable than the previous fallback)
  • x86_64 (manylinux2014): install openssl11-devel via yum and create pkg-config symlinks (system OpenSSL 1.0.2 is too old for openssl-sys)
  • aarch64 (manylinux_2_28 cross-compilation):
    • Download precompiled OpenSSL 1.1.1 arm64 packages from Debian's official repository
    • Extract .deb packages with ar (no package manager needed inside the manylinux container)
    • Install headers and libraries into the cross-compilation sysroot (/usr/aarch64-unknown-linux-gnu/)
    • Properly merge architecture-independent headers with arch-specific configs (opensslconf.h, configuration.h)
    • Set CFLAGS_aarch64_unknown_linux_gnu="-D__ARM_ARCH=8" to fix ring crate ARM assembly compilation

Tests

Verified by running the release workflows on my own fork:

  • Rust crates publish in the correct order (paimon → wait for crates.io → paimon-datafusion)
  • Python wheels build successfully for both x86_64 (manylinux2014) and aarch64 (manylinux_2_28)

API and Format

No API or storage format changes.

Documentation

No documentation updates required (CI-only changes).

@JingsongLi JingsongLi closed this Apr 16, 2026
@JingsongLi JingsongLi reopened this Apr 16, 2026
@JingsongLi
Copy link
Copy Markdown
Contributor

+1

@JingsongLi JingsongLi merged commit ca89546 into apache:release-0.1 Apr 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants