8323515: Create test alias "all" for all test roots #17422

shipilev · 2024-01-15T11:05:09Z

Since recent work to improve tier4 performance, we actually test tier{1,2,3,4} often, which includes all the tests in current tree. It would be more convenient to just have the all test group/alias, so that we can do make test TEST=all. This also gives a parallelism / run time benefit, as we do not wait for tests in each tier to complete before moving to next tier.

Sample run on out-of-the-box Linux x86_64 fastdebug is below. For some environments one also needs to supply a few keywords like !printer to skip tests that cannot complete without failure due to misconfiguration. I left the keywords as is to show how would a failing run look. There is also an existing shortcut in build system that allows to run this with make test-all.

% make test TEST=all

Test selection 'all', will run:
* jtreg:test/hotspot/jtreg:all
* jtreg:test/jdk:all
* jtreg:test/langtools:all
* jtreg:test/jaxp:all
* jtreg:test/lib-test:all

(...about 6 hours later...)

==============================
Test summary
==============================
   TEST                                              TOTAL  PASS  FAIL ERROR   
>> jtreg:test/hotspot/jtreg:all                       6731  6702    29     0 <<
>> jtreg:test/jdk:all                                 9962  9951    11     0 <<
   jtreg:test/langtools:all                           4469  4469     0     0   
   jtreg:test/jaxp:all                                 513   513     0     0   
   jtreg:test/lib-test:all                              32    32     0     0   
==============================
TEST FAILURE

Progress

Change must be properly reviewed (1 review required, with at least 1 Reviewer)
Change must not contain extraneous whitespace
Commit message must refer to an issue

Issue

JDK-8323515: Create test alias "all" for all test roots (Enhancement - P4)

Reviewers

David Holmes (@dholmes-ora - Reviewer) ⚠️ Review applies to 7f6797b6
Alan Bateman (@AlanBateman - Reviewer)
Joe Wang (@JoeWang-Java - Reviewer)
Leonid Mesnik (@lmesnik - Reviewer)

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.org/jdk.git pull/17422/head:pull/17422
$ git checkout pull/17422

Update a local copy of the PR:
$ git checkout pull/17422
$ git pull https://git.openjdk.org/jdk.git pull/17422/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 17422

View PR using the GUI difftool:
$ git pr show -t 17422

Using diff file

Download this PR as a diff file:
https://git.openjdk.org/jdk/pull/17422.diff

Webrev

Link to Webrev Comment

bridgekeeper · 2024-01-15T11:06:58Z

👋 Welcome back shade! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

openjdk · 2024-01-15T11:09:14Z

@shipilev The following labels will be automatically applied to this pull request:

compiler
core-libs
hotspot

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing lists. If you would like to change these labels, use the /label pull request command.

mlbridge · 2024-01-15T16:13:33Z

Webrevs

dholmes-ora

Okay - change is harmless with no ongoing maintenance cost.

test/jdk/TEST.groups

openjdk · 2024-01-15T22:39:38Z

@shipilev This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8323515: Create test alias "all" for all test roots

Reviewed-by: dholmes, alanb, joehw, lmesnik

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 111 new commits pushed to the master branch:

3d82363: 8300088: [IMPROVE] OPEN_MAX is no longer the max limit on macOS >= 10.6 for RLIMIT_NOFILE
2a01c79: 8324513: Inline ContiguousSpace::object_iterate_from
fbaaac6: 8314164: java/net/HttpURLConnection/HttpURLConnectionExpectContinueTest.java fails intermittently in timeout
791b427: 8323640: [TESTBUG]testMemoryFailCount in jdk/internal/platform/docker/TestDockerMemoryMetrics.java always fail because OOM killed
bcaad51: 8318228: RISC-V: C2 ConvF2HF
5acd37f: 8324207: Serial: Remove Space::set_saved_mark_word
f5e6d11: 8324210: Serial: Remove unused methods in Generation
bcb340d: 8324286: Fix backsliding on use of nullptr instead of NULL
3696765: 8323964: runtime/Thread/ThreadCountLimit.java fails intermittently on AIX
5a74c2a: 8323438: Enhance assertions for Windows sync API failures
... and 101 more: https://git.openjdk.org/jdk/compare/cd0fe377417be65dbf1338d8b47da8817985c7d8...master

As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

➡️ To integrate this PR with the above commit message to the master branch, type /integrate in a new comment.

shipilev · 2024-01-17T12:10:14Z

Any other reviews needed for this? Nominally, this changes the test groups in langtools, so maybe @lahodaj or @biboudis want to take a look. For jaxp, @JoeWang-Java, maybe?

JoeWang-Java

Thanks for the reminder. The new alias is nice, easier to run all tier tests. I often run xml-only as well that includes jaxp_all plus a small set of jaxp tests in jdk_all (test/jdk/javax/xml/jaxp). But that's just me, jdk_all already covers those tests.

lahodaj · 2024-01-23T11:42:45Z

I have nothing against it, but I don't think I know enough about details of jtreg configuration to provide approval. OTOH, I personally don't see a strong need for it.

shipilev · 2024-01-23T15:40:42Z

All right, thanks! @lmesnik, I realized I forgot to ask if you had objections to this.

shipilev · 2024-01-23T17:03:13Z

Thank you all!

/integrate

openjdk · 2024-01-23T17:03:36Z

Going to push as commit 8b9bf75.
Since your change was applied there have been 111 commits pushed to the master branch:

3d82363: 8300088: [IMPROVE] OPEN_MAX is no longer the max limit on macOS >= 10.6 for RLIMIT_NOFILE
2a01c79: 8324513: Inline ContiguousSpace::object_iterate_from
fbaaac6: 8314164: java/net/HttpURLConnection/HttpURLConnectionExpectContinueTest.java fails intermittently in timeout
791b427: 8323640: [TESTBUG]testMemoryFailCount in jdk/internal/platform/docker/TestDockerMemoryMetrics.java always fail because OOM killed
bcaad51: 8318228: RISC-V: C2 ConvF2HF
5acd37f: 8324207: Serial: Remove Space::set_saved_mark_word
f5e6d11: 8324210: Serial: Remove unused methods in Generation
bcb340d: 8324286: Fix backsliding on use of nullptr instead of NULL
3696765: 8323964: runtime/Thread/ThreadCountLimit.java fails intermittently on AIX
5a74c2a: 8323438: Enhance assertions for Windows sync API failures
... and 101 more: https://git.openjdk.org/jdk/compare/cd0fe377417be65dbf1338d8b47da8817985c7d8...master

Your commit was automatically rebased without conflicts.

openjdk · 2024-01-23T17:03:42Z

@shipilev Pushed as commit 8b9bf75.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

DamonFool · 2024-01-24T15:24:35Z

Hi @shipilev , plese see #17558 .
Thanks.

Fix

7f6797b

openjdk bot added hotspot hotspot-dev@openjdk.org core-libs core-libs-dev@openjdk.org compiler compiler-dev@openjdk.org labels Jan 15, 2024

shipilev marked this pull request as ready for review January 15, 2024 16:09

shipilev mentioned this pull request Jan 15, 2024

8323717: Introduce test keyword for tests that need external dependencies #17421

Closed

5 tasks

openjdk bot added the rfr Pull request is ready for review label Jan 15, 2024

dholmes-ora approved these changes Jan 15, 2024

View reviewed changes

test/jdk/TEST.groups Show resolved Hide resolved

openjdk bot added the ready Pull request is ready to be integrated label Jan 15, 2024

shipilev added 2 commits January 16, 2024 09:47

jdk_all and lib_test_all groups

78f5f9b

Catch-all -> All tests

def2f39

AlanBateman approved these changes Jan 16, 2024

View reviewed changes

JoeWang-Java approved these changes Jan 17, 2024

View reviewed changes

lmesnik approved these changes Jan 23, 2024

View reviewed changes

openjdk bot added the integrated Pull request has been integrated label Jan 23, 2024

openjdk bot closed this Jan 23, 2024

openjdk bot removed ready Pull request is ready to be integrated rfr Pull request is ready for review labels Jan 23, 2024

shipilev deleted the JDK-8323515-tests-all branch January 31, 2024 16:14

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

8323515: Create test alias "all" for all test roots #17422

8323515: Create test alias "all" for all test roots #17422

shipilev commented Jan 15, 2024 •

edited by openjdk bot

Loading

bridgekeeper bot commented Jan 15, 2024

openjdk bot commented Jan 15, 2024

mlbridge bot commented Jan 15, 2024 •

edited

Loading

dholmes-ora left a comment

openjdk bot commented Jan 15, 2024 •

edited

Loading

shipilev commented Jan 17, 2024

JoeWang-Java left a comment

lahodaj commented Jan 23, 2024

shipilev commented Jan 23, 2024

shipilev commented Jan 23, 2024

openjdk bot commented Jan 23, 2024

openjdk bot commented Jan 23, 2024

DamonFool commented Jan 24, 2024

8323515: Create test alias "all" for all test roots #17422

8323515: Create test alias "all" for all test roots #17422

Conversation

shipilev commented Jan 15, 2024 • edited by openjdk bot Loading

Progress

Issue

Reviewers

Reviewing

Webrev

bridgekeeper bot commented Jan 15, 2024

openjdk bot commented Jan 15, 2024

mlbridge bot commented Jan 15, 2024 • edited Loading

Webrevs

dholmes-ora left a comment

Choose a reason for hiding this comment

openjdk bot commented Jan 15, 2024 • edited Loading

shipilev commented Jan 17, 2024

JoeWang-Java left a comment

Choose a reason for hiding this comment

lahodaj commented Jan 23, 2024

shipilev commented Jan 23, 2024

shipilev commented Jan 23, 2024

openjdk bot commented Jan 23, 2024

openjdk bot commented Jan 23, 2024

DamonFool commented Jan 24, 2024

shipilev commented Jan 15, 2024 •

edited by openjdk bot

Loading

mlbridge bot commented Jan 15, 2024 •

edited

Loading

openjdk bot commented Jan 15, 2024 •

edited

Loading