Skip to content

Conversation

AndreyBozhko
Copy link
Contributor

Description

If a test fails due to some exception during static initialization, the logs contain message that looks like

Reproduce with: gradlew :solr:core:test --tests "org.apache.solr.response.JSONWriterTest.classMethod"

which is not a valid command to re-run the test.

This PR changes the message to be

Reproduce with: gradlew :solr:core:test --tests "org.apache.solr.response.JSONWriterTest"

Checklist

Please review the following and check all that apply:

  • I have reviewed the guidelines for How to Contribute and my code conforms to the standards described there to the best of my ability.
  • I have created a Jira issue and added the issue ID to my pull request title.
  • I have given Solr maintainers access to contribute to my PR branch. (optional but recommended)
  • I have developed this patch against the main branch.
  • I have run ./gradlew check.
  • I have added tests for my changes.
  • I have added documentation for the Reference Guide

if (result.resultType == TestResult.ResultType.FAILURE) {
def testOrClassName = desc.name == "classMethod" ? desc.className : "${desc.className}.${desc.name}"
failedTests << [
"name": "${desc.className}.${desc.name}",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about also changing this?

Suggested change
"name": "${desc.className}.${desc.name}",
"name": "$testOrClassName",

(I spotted this only because in https://github.com/apache/solr/actions/runs/9274510162/job/25517863554?pr=2483 it says "... ERROR: The following test(s) have failed: ... org.apache.solr.ltr.feature.TestUserTermScoreWithQ.classMethod (:solr:modules:ltr) ..." currently.)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, its still probably good to have a pretty clear indication that it was a classMethod, right? I think it would be a bit confusing otherwise to see that both org.apache.solr.ltr.feature.TestUserTermScoreWithQ.testUserTermScoreWithQ and org.apache.solr.ltr.feature.TestUserTermScoreWithQ failed

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It just happened so that in the GHA check (https://github.com/apache/solr/actions/runs/9274510162/job/25517863554) for this PR, a couple of tests failed - and one could see how the failures are reported:

  • lines 479-478
> Task :solr:modules:ltr:test

org.apache.solr.ltr.feature.TestUserTermScoreWithQ > testUserTermScoreWithQ FAILED
    java.lang.Exception: Test abandoned because suite timeout was reached.
        at __randomizedtesting.SeedInfo.seed([1B9234167BEFFA61]:0)

org.apache.solr.ltr.feature.TestUserTermScoreWithQ > classMethod FAILED
    java.lang.Exception: Suite timeout exceeded (>= 600000 msec).
        at __randomizedtesting.SeedInfo.seed([1B9234167BEFFA61]:0)
  • line 1017
2> NOTE: reproduce with: gradlew test --tests TestUserTermScoreWithQ.testUserTermScoreWithQ -Dtests.seed=1B9234167BEFFA61 -Dtests.locale=ar-BH -Dtests.timezone=WET -Dtests.asserts=true -Dtests.file.encoding=UTF-8
  • line 1404
2> NOTE: reproduce with: gradlew test --tests TestUserTermScoreWithQ -Dtests.seed=1B9234167BEFFA61 -Dtests.locale=ar-BH -Dtests.timezone=WET -Dtests.asserts=true -Dtests.file.encoding=UTF-8
  • lines 1413-1420
ERROR: The following test(s) have failed:
  - org.apache.solr.ltr.feature.TestUserTermScoreWithQ.testUserTermScoreWithQ (:solr:modules:ltr)
    Test output: /tmp/src/solr/solr/modules/ltr/build/test-results/test/outputs/OUTPUT-org.apache.solr.ltr.feature.TestUserTermScoreWithQ.txt
    Reproduce with: gradlew :solr:modules:ltr:test --tests "org.apache.solr.ltr.feature.TestUserTermScoreWithQ.testUserTermScoreWithQ" -Ptests.jvms=96 "-Ptests.jvmargs=-XX:TieredStopAtLevel=1 -XX:+UseParallelGC -XX:ActiveProcessorCount=1 -XX:ReservedCodeCacheSize=120m" -Ptests.seed=1B9234167BEFFA61 -Ptests.timeoutSuite=600000! -Ptests.file.encoding=US-ASCII

  - org.apache.solr.ltr.feature.TestUserTermScoreWithQ.classMethod (:solr:modules:ltr)
    Test output: /tmp/src/solr/solr/modules/ltr/build/test-results/test/outputs/OUTPUT-org.apache.solr.ltr.feature.TestUserTermScoreWithQ.txt
    Reproduce with: gradlew :solr:modules:ltr:test --tests "org.apache.solr.ltr.feature.TestUserTermScoreWithQ" -Ptests.jvms=96 "-Ptests.jvmargs=-XX:TieredStopAtLevel=1 -XX:+UseParallelGC -XX:ActiveProcessorCount=1 -XX:ReservedCodeCacheSize=120m" -Ptests.seed=1B9234167BEFFA61 -Ptests.timeoutSuite=600000! -Ptests.file.encoding=US-ASCII

Note that this PR only adjusts the line 1420 from the last block, - it now becomes consistent with line 1404 which already knew not to append "classMethod" to the class name.

@madrob
Copy link
Contributor

madrob commented May 29, 2024

See also https://issues.apache.org/jira/browse/SOLR-15447

@gus-asf
Copy link
Contributor

gus-asf commented May 29, 2024

if we are touching this lets add a ./ on the front of the command so one can actually cut and paste it :)

@AndreyBozhko
Copy link
Contributor Author

Thanks @madrob - any particular reason why the patch from SOLR-15447 was never applied?

@AndreyBozhko
Copy link
Contributor Author

Looks like this may be superseded by #2487

@dsmiley
Copy link
Contributor

dsmiley commented May 31, 2024

Yes I think this should be closed as incorporated by #2487 . Feel free to provide any further feedback there!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants