-
Notifications
You must be signed in to change notification settings - Fork 28k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[SPARK-48145][CORE] Remove logDebug and logTrace with MDC in JAVA structured logging framework #46405
Conversation
cc @panbingkun as well. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is this consistent with SPIP, @gengliangwang ?
Yes, the SPIP only suggests the logInfo/logWarn/logError APIs. LogDebug/LogTrace is optional. Since #46390 is not migrating the debug/trace level logs, I suggest merging this one to reduce unnecessary changes in it. |
In Scala side, the logDebug/logTrace APIs was introduced for debugging cache purposes: #45990 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
+1, LGTM. Thank you!
cc @cloud-fan as a release manager for Apache Spark 4.0.0-preview |
Merged to master for Apache Spark 4.0.0-preview. |
Thank you, @gengliangwang . |
Got it. |
…uctured logging framework ### What changes were proposed in this pull request? Since we are targeting on migration INFO/WARN/ERROR level logs to structure logging, this PR removes the logDebug and logTrace methods from the JAVA structured logging framework. ### Why are the changes needed? In the log migration PR apache#46390, there are unnecessary changes such as updating ``` logger.debug("Task {} need to spill {} for {}", taskAttemptId, Utils.bytesToString(required - got), requestingConsumer); ``` to ``` LOGGER.debug("Task {} need to spill {} for {}", String.valueOf(taskAttemptId), Utils.bytesToString(required - got), requestingConsumer.toString()); ``` With this PR, we can avoid such changes during log migrations. ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? Existing UT. ### Was this patch authored or co-authored using generative AI tooling? No Closes apache#46405 from gengliangwang/updateJavaLog. Authored-by: Gengliang Wang <gengliang@apache.org> Signed-off-by: Dongjoon Hyun <dhyun@apple.com>
What changes were proposed in this pull request?
Since we are targeting on migration INFO/WARN/ERROR level logs to structure logging, this PR removes the logDebug and logTrace methods from the JAVA structured logging framework.
Why are the changes needed?
In the log migration PR #46390, there are unnecessary changes such as updating
to
With this PR, we can avoid such changes during log migrations.
Does this PR introduce any user-facing change?
No
How was this patch tested?
Existing UT.
Was this patch authored or co-authored using generative AI tooling?
No