-
Notifications
You must be signed in to change notification settings - Fork 28k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[SPARK-48074][Core] Improve the readability of JSON loggings #46322
Conversation
cc @panbingkun @dtenedor as well |
common/utils/src/test/scala/org/apache/spark/util/StructuredLoggingSuite.scala
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sorry but I'm not sure this is better because I prefer level
at the original position.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
+1, LGTM (Pending CIs).
Thank you. I removed the following from the PR title consistently.
- Move the order
level
field, so that users can read the messages right after the timestamps.
Merged to master for Apache spark 4.0.0. |
+1, LGTM |
### What changes were proposed in this pull request? Improve the readability of JSON loggings via: 1. Use UTC in the timestamp so that the timestamp field value is more concise. 2. Display the simple name of the logger instead of the full qualified name. 3. Display the stack trace elements with `class`/`method`/`file`/`line` instead of stringifying everything into a long string. Before changes: <img width="2129" alt="image" src="https://github.com/apache/spark/assets/1097932/a7458293-2f84-4699-bbc2-3183a078a543"> After changes: <img width="2341" alt="image" src="https://github.com/apache/spark/assets/1097932/23742bc0-4e1e-491c-83c9-b6695f41e57e"> ### Why are the changes needed? Improve the readability of Spark logs ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? Existing UT and manually reviews ### Was this patch authored or co-authored using generative AI tooling? No Closes apache#46322 from gengliangwang/improveLayout. Authored-by: Gengliang Wang <gengliang@apache.org> Signed-off-by: Dongjoon Hyun <dhyun@apple.com>
What changes were proposed in this pull request?
Improve the readability of JSON loggings via:
class
/method
/file
/line
instead of stringifying everything into a long string.Before changes:
After changes:
Why are the changes needed?
Improve the readability of Spark logs
Does this PR introduce any user-facing change?
No
How was this patch tested?
Existing UT and manually reviews
Was this patch authored or co-authored using generative AI tooling?
No