From 026b0b926dfd40038f2cee932f38b917eb25b77e Mon Sep 17 00:00:00 2001 From: Gengliang Wang Date: Sun, 19 Jul 2020 12:00:23 +0900 Subject: [PATCH] [SPARK-32253][INFRA] Show errors only for the sbt tests of github actions ### What changes were proposed in this pull request? Make the test result log of github action more readable by showing errors from SBT only. 1. Add "--error" flag to sbt in github action to set the log level as "ERROR" 2. Show only failed test cases in stderr output of github action. According to https://www.scalatest.org/user_guide/using_the_runner, with SBT option `-eNCXEHLOPQMDF ` we can drop all the following events: ``` N - drop TestStarting events C - drop TestSucceeded events X - drop TestIgnored events E - drop TestPending events H - drop SuiteStarting events L - drop SuiteCompleted events O - drop InfoProvided events P - drop ScopeOpened events Q - drop ScopeClosed events R - drop ScopePending events M - drop MarkupProvided events ``` and enable the following two mode: ``` D - show all durations F - show full stack traces ``` ### Why are the changes needed? Currently, the output of github action is very long and we have to scroll down to find the failed test cases. Even more, the log may be truncated. In such a case, we will have to wait until all the jobs are completed and then download all the raw logs. ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? Before changes, all the warnings in compiling are shown: ![image](https://user-images.githubusercontent.com/1097932/87846810-98ec8900-c887-11ea-913b-164b84df62cd.png) as well as all the passed and ignored test cases: ![image](https://user-images.githubusercontent.com/1097932/87846834-ca655480-c887-11ea-9c29-977f802e4c82.png) After changes, sbt test only shows the summary for a successful job: ![image](https://user-images.githubusercontent.com/1097932/87846961-e74e5780-c888-11ea-82d5-cf1da1740181.png) ![image](https://user-images.githubusercontent.com/1097932/87745273-5735e280-c7a2-11ea-8ac9-b4b0e3cb458d.png) If there is a test failure, a full stack track is shown as well as a test failure summary at the end of test log: ![image](https://user-images.githubusercontent.com/1097932/87751143-3aa1a680-c7b2-11ea-9d09-52637a322270.png) ![image](https://user-images.githubusercontent.com/1097932/87752404-1f846600-c7b5-11ea-8106-8ddaf3cc3f7e.png) Closes #29133 from gengliangwang/shortLog. Authored-by: Gengliang Wang Signed-off-by: HyukjinKwon --- dev/run-tests.py | 3 +++ project/SparkBuild.scala | 7 +++++++ 2 files changed, 10 insertions(+) diff --git a/dev/run-tests.py b/dev/run-tests.py index 065a27c0e853b..8e29f89c3a0d0 100755 --- a/dev/run-tests.py +++ b/dev/run-tests.py @@ -653,6 +653,9 @@ def main(): # If we're running the tests in Github Actions, attempt to detect and test # only the affected modules. if test_env == "github_actions": + # Set the log level of sbt as ERROR to make the output more readable. + if build_tool == "sbt": + extra_profiles.append("--error") if os.environ["GITHUB_BASE_REF"] != "": # Pull requests changed_files = identify_changed_files_from_git_commits( diff --git a/project/SparkBuild.scala b/project/SparkBuild.scala index d19b514d662fa..198405a1d29ca 100644 --- a/project/SparkBuild.scala +++ b/project/SparkBuild.scala @@ -1027,6 +1027,13 @@ object TestSettings { }.getOrElse(Nil): _*), // Show full stack trace and duration in test cases. testOptions in Test += Tests.Argument("-oDF"), + // Show only the failed test cases with full stack traces in github action to make the log more + // readable. + // Check https://www.scalatest.org/user_guide/using_the_runner for the details of options . + testOptions in Test += Tests.Argument(TestFrameworks.ScalaTest, + sys.env.get("GITHUB_ACTIONS").map { _ => + Seq("-eNCXEHLOPQMDF") + }.getOrElse(Nil): _*), testOptions in Test += Tests.Argument(TestFrameworks.JUnit, "-v", "-a"), // Required to detect Junit tests for each project, see also https://github.com/sbt/junit-interface/issues/35 crossPaths := false,