Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BIOFORMATS-image: rotate between JDK 8, 11 and 17 #196

Merged
merged 1 commit into from
Jun 6, 2023

Conversation

sbesson
Copy link
Member

@sbesson sbesson commented Oct 26, 2022

As discussed on Monday Formats meeting, this is the minimal configuration change that should allow the nightly OME CI builds to start testing Bio-Formats on JDK 17 in addition to JDK 8 and JDK 11.

The rotation is currently set so that JDK 17 builds would happen on Wednesday and Saturday to minimize the impact on the OME standup review as I am still expecting some repository failures.

If the plan above is acceptable, next proposed step would be to manually deploy the changes on https://merge-ci.openmicroscopy.org/jenkins/ and mark ome/bioformats#3815 as ready for review for inclusion in the nightly builds.

@sbesson sbesson requested review from jburel and dgault October 26, 2022 09:41
Copy link
Member

@jburel jburel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes make sense

@sbesson
Copy link
Member Author

sbesson commented Oct 27, 2022

After including ome/bioformats#3815 in the nightly CI builds, confirmed this caused no regression in the JDK 8 tests. Applied the configuration change proposed in this PR manually to https://merge-ci.openmicroscopy.org/jenkins/job/BIOFORMATS-image/ to have an assessment of the state of JDK 17 and we can review next steps here at the weekly Formats meeting next Monday

@sbesson
Copy link
Member Author

sbesson commented Nov 3, 2022

From the discussion at the last Formats meeting, @melissalinkert raises a potential issue as ideally any change would need to be validated against all JDK versions. With the current implementation, that would be at least four days between the proposal and the inclusion in the development line which certainly adds some unnecessary overhead.

At the moment, repository CI jobs are consistently taking between 4 and 4.5 hours. Speeding up individual job execution is always a nice-to-have b

  • extend the scope of the test-repo job to run against a matrix of all formats x all jdk versions. This would mean typical completion times for this job would now be ~12-14h
  • allow the JDK to be configurable, pin the first execution of the test-repository job to a given JDK but launch two additional executions of the jobs as a follow-up
  • split into separate jobs, one per JDK

At the moment the CI job builds the image using https://github.com/ome/bio-formats-build/blob/master/Dockerfile using different base images. Independently of the chosen solution, a requirement will be to have pre-build images with different JDKs.
An additional consideration is that we do not necessarily need Bio-Formats to be built and tested on each JDK by this CI infrastructure. The compilation should be tested individually via GitHub Actions at the level of each repository. Due to the nature of the Java ecosystem, it is also probably valuable to tests components built using the minimal supported JDK version (8) against different target JDKs. One thought would be to modify the base Dockerfile to use multi-stage builds and create minimal runtime images that could be tagged and consumed by the downstream jobs:

FROM  ${BUILD_IMAGE} as build
...

FROM openjdk:8-slim-bullseye as jdk8
COPY
...

FROM  openjdk:11-slim-bullseye as jdk11
COPY ...

FROM  openjdk:17-slim-bullseye as jdk17
COPY ...

@jburel jburel merged commit 1dd0fff into ome:master Jun 6, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants