Skip to content

Commit

Permalink
[MINOR][CORE] Change the log level to WARN for the message which is s…
Browse files Browse the repository at this point in the history
…hown in case users attemp to add a JAR twice

### What changes were proposed in this pull request?

This PR changes the log level to WARN for the message which is shown in case users attempt to add a JAR twice by `sc.addJar`.

### Why are the changes needed?

It's for consistency. `sc.addFile` and `sc.addArchive` show the message as warning in such case.

### Does this PR introduce _any_ user-facing change?

Yes. The message will appear even when the log level of the logger is WARN. But I don't think it affects the behavior.

### How was this patch tested?

Manually confirmed like as follows:
```
sc.addJar("file:////tmp/test.jar")
sc.addJar("file:////tmp/test.jar")
22/02/09 00:56:28 WARN SparkContext: The JAR file:////tmp/test.jar at spark://192.168.1.204:44533/jars/test.jar has been added already. Overwriting of added jar is not supported in the current version.
```

Closes #35443 from sarutak/add-jar-warn.

Authored-by: Kousuke Saruta <sarutak@oss.nttdata.com>
Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
  • Loading branch information
sarutak authored and dongjoon-hyun committed Apr 3, 2022
1 parent 61074d4 commit acbfd03
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion core/src/main/scala/org/apache/spark/SparkContext.scala
Original file line number Diff line number Diff line change
Expand Up @@ -2023,7 +2023,7 @@ class SparkContext(config: SparkConf) extends Logging {
}
if (existed.nonEmpty) {
val jarMessage = if (scheme != "ivy") "JAR" else "dependency jars of Ivy URI"
logInfo(s"The $jarMessage $path at ${existed.mkString(",")} has been added already." +
logWarning(s"The $jarMessage $path at ${existed.mkString(",")} has been added already." +
" Overwriting of added jar is not supported in the current version.")
}
}
Expand Down

0 comments on commit acbfd03

Please sign in to comment.