Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

trapme: tests failing when a testimage is not specified on a CUSTOMIZED_SDK_URL #5179

Open
sxa opened this issue Mar 26, 2024 · 0 comments
Open

Comments

@sxa
Copy link
Member

sxa commented Mar 26, 2024

I've been re-running the tests suites manually on some custom builds in order to test some modified compiler options on Linux as per adoptium/temurin-build#3685

By default if you just specify the JDK tarball a number of the tests in the sanity suite will fail - for example this was the output from sanity.openjk JDk22 without the testimage specified:

18:39:09  	jdk_lang_0 - Test results: passed: 999; failed: 4; error: 7
18:39:09  	jdk_lang_1 - Test results: passed: 999; failed: 4; error: 7
18:39:09  	jdk_lang_native_0 - Test results: error: 1
18:39:09  	jdk_lang_native_1 - Test results: error: 1
18:39:09  	jdk_foreign_0 - Test results: passed: 56; failed: 40; error: 4
18:39:09  	jdk_foreign_1 - Test results: passed: 56; failed: 40; error: 4

(Note, I'm not 100% certain all of these are caused by the missing test image but many of them are, and a subsequent run with the fix had all of these passing)

Since this is "non-obvious" when you're just trying to run a test suite I think it would be useful to trap this situation or otherwise warn the user if a testimage has not been specified and the tests may require it. The above list may be adequate for determining which suites require it. Presumably these would also be a list of "expected failures" when using someone else's third party tarball which does not include a testimage download too and therefore would not be able to pass the tests.

The description of CUSTOMIZED_SDK_URL is also Customized SDK url, need to set when SDK_RESOURCE=customized and makes no mention of the special format to specify the testimage tarball (specify it after the main jdk tarball with a space character separating them) so it would be good to update the description to make that clear. I think chaging the behavior here would be a great usability improvement to stop people tripping up and having unexpected test case failures

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant