Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FLINK-25002][build] Add java 17 add-opens/add-exports JVM arguments #22794

Merged
merged 5 commits into from
Jun 16, 2023

Conversation

zentol
Copy link
Contributor

@zentol zentol commented Jun 15, 2023

This PR sets of all required add-opens/add-exports statements for Maven, surefire, test-internal JVMs and flink-dist.

If a module requires such a statement for surefire/testing then it must define a list of statements in the surefire.module.config property of the module's pom. This property is set on the test JVMs via surefire, but is also exposed as a system property to the test itself such that utils like the TestJvmProcess can forward them.
I documented a hint for each statement as to why it is required.

flink-dist is configured separately via flink-conf.yaml.

The required statements were purely determined by running tests and checking what fails.
"Completeness" was validated on my personal CI; we may not be covering all tests since some only run on the apache branches. We'll just amend missing ones in the future if necessary.

@zentol zentol requested a review from XComp June 15, 2023 13:29
pom.xml Show resolved Hide resolved
@flinkbot
Copy link
Collaborator

flinkbot commented Jun 15, 2023

CI report:

Bot commands The @flinkbot bot supports the following commands:
  • @flinkbot run azure re-run the last Azure build

Copy link
Contributor

@XComp XComp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally, it looks fine. I think it's hard to review that one without the build failures. In this sense, I trust you that you did proper checking here. I just added a few comments to verify to make sure that I understand it correctly.

flink-dist/src/main/resources/flink-conf.yaml Show resolved Hide resolved
flink-dist/src/main/resources/flink-conf.yaml Outdated Show resolved Hide resolved
.mvn/jvm.config Show resolved Hide resolved
flink-core/pom.xml Outdated Show resolved Hide resolved
flink-core/pom.xml Show resolved Hide resolved
flink-core/pom.xml Show resolved Hide resolved
flink-core/pom.xml Show resolved Hide resolved
flink-core/pom.xml Show resolved Hide resolved
@zentol
Copy link
Contributor Author

zentol commented Jun 15, 2023

oh boy CI is broken pretty badly. I think this is the first time I'm running all this stuff on Java 8 👀

@zentol
Copy link
Contributor Author

zentol commented Jun 15, 2023

oh boy CI is broken pretty badly. I think this is the first time I'm running all this stuff on Java 8 eyes

Fixed. Bit odd that surefire just hangs if the JVM crashes immediately 🤔

.mvn/jvm.config Show resolved Hide resolved
.mvn/jvm.config Show resolved Hide resolved
Copy link
Contributor

@XComp XComp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, I did a few test runs (but didn't check all the configuration changes). The change looks reasonable from my side. 👍

The CI failure seems to be unrelated to the change. Multiple Yarn tests are failing in the end. It looks like the first test failure didn't clean the Yarn cluster properly (I created FLINK-32366 to cover it).

flink-yarn/pom.xml Outdated Show resolved Hide resolved
flink-table/flink-table-runtime/pom.xml Outdated Show resolved Hide resolved
@zentol
Copy link
Contributor Author

zentol commented Jun 16, 2023

hmm the yarn tests seems to reliably get stuck now; investigating...
It's quite odd because the tests pass just fine on the JDK 17 branch. 🤔

@zentol
Copy link
Contributor Author

zentol commented Jun 16, 2023

Yarn failure should be fixed; we were missing an IgnoreUnrecognizedVMOptions for yarn TMs.

@XComp
Copy link
Contributor

XComp commented Jun 16, 2023

yikes, good that you checked 👍 I'm gonna close FLINK-32366 again

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants