Skip to content

Comments

[HUDI-6091] Add Java 11 and 17 to bundle validation#8481

Merged
yihua merged 5 commits intoapache:masterfrom
yihua:HUDI-6091-bundle-validation-image-java11-17
Apr 21, 2023
Merged

[HUDI-6091] Add Java 11 and 17 to bundle validation#8481
yihua merged 5 commits intoapache:masterfrom
yihua:HUDI-6091-bundle-validation-image-java11-17

Conversation

@yihua
Copy link
Contributor

@yihua yihua commented Apr 17, 2023

Change Logs

This PR adds the support of running bundle validation with Spark on Java 11 and 17 runtime in Github Actions. The following changes are made.

  • Moves the Dockerfile for building apachehudi/hudi-ci-bundle-validation-base to base/ and revises it to install openjdk11 and openjdk17;
  • Adds scripts to build new images (hudi-ci-bundle-validation-base) for the following tags: flink1136hive313spark313, flink1146hive313spark323, flink1153hive313spark331 (the images have been pushed to DockerHub);
  • Adds base-dev/Dockerfile for faster development locally;
  • Changes the validate.sh script to allow running bundle validation on specified Java runtime (Java 11/17), by changing JAVA_HOME to the corresponding version;
  • Changes the ci_run.sh to pass in specified Java runtime version;
  • Adds steps in validate-bundles in .github/workflows/bot.yml to run bundle validation on Java 11 (Spark 3.1, 3.2, and 3.3) and 17 (Spark 3.3).

Impact

Certifies that Hudi bundles with Spark run on Java 11 and 17 runtime without any issues.

Risk level

none

Documentation Update

Adds README.md for building and uploading bundle image.

Contributor's checklist

  • Read through contributor's guide
  • Change Logs and Impact were stated clearly
  • Adequate tests were added if applicable
  • CI passed

@yihua yihua force-pushed the HUDI-6091-bundle-validation-image-java11-17 branch 3 times, most recently from 65f9692 to ff65d74 Compare April 18, 2023 17:23
@yihua yihua force-pushed the HUDI-6091-bundle-validation-image-java11-17 branch from ff65d74 to 632fbc2 Compare April 18, 2023 19:23
@yihua yihua force-pushed the HUDI-6091-bundle-validation-image-java11-17 branch from 1a7bd62 to 9c3a908 Compare April 20, 2023 00:10
@yihua yihua changed the title [DNM][HUDI-6091] Add Java 11 and 17 to bundle validation image [HUDI-6091] Add Java 11 and 17 to bundle validation Apr 20, 2023
@yihua yihua requested a review from xushiyan April 20, 2023 00:34
@yihua yihua added engine:spark Spark integration test-coverage labels Apr 20, 2023
@apache apache deleted a comment from hudi-bot Apr 20, 2023
* under the License.
-->

# Bundle Validation for Hudi
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good docs to have!

@apache apache deleted a comment from hudi-bot Apr 20, 2023
@apache apache deleted a comment from hudi-bot Apr 21, 2023
@hudi-bot
Copy link
Collaborator

CI report:

Bot commands @hudi-bot supports the following commands:
  • @hudi-bot run azure re-run the last Azure build

@yihua
Copy link
Contributor Author

yihua commented Apr 21, 2023

Flink IT failed due to flakiness, which is irrelevant to this PR.

@yihua yihua merged commit dc07466 into apache:master Apr 21, 2023
yihua added a commit to yihua/hudi that referenced this pull request May 15, 2023
This commit adds the support of running bundle validation with Spark on Java 11 and 17 runtime in Github Actions.
yihua added a commit to yihua/hudi that referenced this pull request May 15, 2023
This commit adds the support of running bundle validation with Spark on Java 11 and 17 runtime in Github Actions.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

engine:spark Spark integration

Projects

Archived in project

Development

Successfully merging this pull request may close these issues.

3 participants