Skip to content

ci: introduce spark paimon integration for integration test#107

Open
luoyuxia wants to merge 1 commit intoapache:mainfrom
luoyuxia:introduce-it-infra
Open

ci: introduce spark paimon integration for integration test#107
luoyuxia wants to merge 1 commit intoapache:mainfrom
luoyuxia:introduce-it-infra

Conversation

@luoyuxia
Copy link
Contributor

@luoyuxia luoyuxia commented Mar 4, 2026

Purpose

Linked issue: close #108

Inspired by iceberg-rust apache/iceberg-rust#766, this pr only introduce base testing infra for reading, will add integration tests in follow pull requests.

Brief change log

  • Add Docker Compose configuration for Spark + Paimon environment
  • Add Makefile targets (docker-up, docker-down) for managing test containers
  • Add Spark Dockerfile with configurable SPARK_VERSION and PAIMON_VERSION build args
  • Add provision.py script to create test tables and data
  • Add CI integration job to run Docker containers (with cleanup on completion)
  • Use /tmp/paimon-warehouse as shared volume between Spark and host for test data

Tests

API and Format

Documentation

@luoyuxia luoyuxia force-pushed the introduce-it-infra branch 5 times, most recently from b4dfdaa to 075385d Compare March 5, 2026 01:59
@luoyuxia luoyuxia force-pushed the introduce-it-infra branch from 075385d to cb95dd9 Compare March 5, 2026 02:02
@luoyuxia
Copy link
Contributor Author

luoyuxia commented Mar 5, 2026

cc @XiaoHongbo-Hope

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

introduce base spark paimon integration for teting

1 participant