Skip to content

Commit

Permalink
[SPARK-47112][INFRA] Write logs into a file in SparkR Windows build
Browse files Browse the repository at this point in the history
### What changes were proposed in this pull request?

We used to write Log4J logs into `target/unit-tests.log` instead of console. This seems to be broken in SparkR Windows job. This PR fixes it.

### Why are the changes needed?

https://github.com/apache/spark/actions/runs/7977185456/job/21779508822#step:10:89
This write too many logs, and difficult to see the real test cases.

### Does this PR introduce _any_ user-facing change?

No, dev-only.

### How was this patch tested?

In my fork

### Was this patch authored or co-authored using generative AI tooling?

No.

Closes #45192 from HyukjinKwon/reduce-logs.

Authored-by: Hyukjin Kwon <gurwls223@apache.org>
Signed-off-by: Hyukjin Kwon <gurwls223@apache.org>
  • Loading branch information
HyukjinKwon committed Feb 21, 2024
1 parent b3e3462 commit 0d938de
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion .github/workflows/build_sparkr_window.yml
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,7 @@ jobs:
run: |
set HADOOP_HOME=%USERPROFILE%\hadoop-3.3.5
set PATH=%HADOOP_HOME%\bin;%PATH%
.\bin\spark-submit2.cmd --driver-java-options "-Dlog4j.configuration=file:///%CD:\=/%/R/log4j2.properties" --conf spark.hadoop.fs.defaultFS="file:///" R\pkg\tests\run-all.R
.\bin\spark-submit2.cmd --driver-java-options "-Dlog4j.configurationFile=file:///%CD:\=/%/R/log4j2.properties" --conf spark.hadoop.fs.defaultFS="file:///" R\pkg\tests\run-all.R
shell: cmd
env:
NOT_CRAN: true
Expand Down

0 comments on commit 0d938de

Please sign in to comment.