-
Notifications
You must be signed in to change notification settings - Fork 5.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Serve] fix missing message body for json log formats #42729
[Serve] fix missing message body for json log formats #42729
Conversation
Signed-off-by: Gene Su <e870252314@gmail.com>
@GeneDer so with these changes, what would be the difference in the example screenshot you posted? Would there just be empty |
Yes, the screenshot shows the differences. The first highlighted part is without this change. You see those "Finished recovering deployments...", "Starting proxy with name..." missing. The second run on the bottom shows the fix revealed the missing logs :) |
@@ -81,10 +81,10 @@ def format(self, record: logging.LogRecord) -> str: | |||
SERVE_LOG_APPLICATION | |||
] | |||
|
|||
if SERVE_LOG_MESSAGE in record.__dict__: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@edoakes Basically this if statement is blocking the message from populating since the formatter is not yet called and the "message" field doesn't exist on the log record yet
@edoakes @sihanwang41 Just in case the notification is missed. Would be nice to get this in 🙏 |
Why are these changes needed?
Found quite a lot of logs misses log message body when working on the logs observability (on Workspaces).
![image](https://private-user-images.githubusercontent.com/7553988/299860871-f2e45ebc-9535-4319-a006-c182ad9cd19a.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MTk2MjI5MjcsIm5iZiI6MTcxOTYyMjYyNywicGF0aCI6Ii83NTUzOTg4LzI5OTg2MDg3MS1mMmU0NWViYy05NTM1LTQzMTktYTAwNi1jMTgyYWQ5Y2QxOWEucG5nP1gtQW16LUFsZ29yaXRobT1BV1M0LUhNQUMtU0hBMjU2JlgtQW16LUNyZWRlbnRpYWw9QUtJQVZDT0RZTFNBNTNQUUs0WkElMkYyMDI0MDYyOSUyRnVzLWVhc3QtMSUyRnMzJTJGYXdzNF9yZXF1ZXN0JlgtQW16LURhdGU9MjAyNDA2MjlUMDA1NzA3WiZYLUFtei1FeHBpcmVzPTMwMCZYLUFtei1TaWduYXR1cmU9YzM5NDIxZjFhNTQxNzkxMWYwNDA1ZjRkZDc1MjJhNTlkNDQ4YjdiNWNiODYzMTA5OWI1ZjdlNjFjOWRkYmNlMSZYLUFtei1TaWduZWRIZWFkZXJzPWhvc3QmYWN0b3JfaWQ9MCZrZXlfaWQ9MCZyZXBvX2lkPTAifQ.9Ug1WUuKRZMn-kGDAO67Xlw3I5Moh-zIRE_MmO5fDPg)
After a bit more digging, it seems like in some occasion, the
message
attribute would not be presented on therecord.__dict__
so caused the if statement to be skipped and logged a line without themessage
body. Internally inLogRecord
, themessage
field is "...computed asmsg % args
. This is set when Formatter.format() is invoked." This PR useslogging.Formatter
to format the message directly instead of relying on Python's string substitution so the message can always be computed. Also did it in the TDD way to ensure there is a test covering it.LogRecord Doc: https://docs.python.org/3/library/logging.html#logrecord-objects
Related issue number
Checks
git commit -s
) in this PR.scripts/format.sh
to lint the changes in this PR.method in Tune, I've added it in
doc/source/tune/api/
under thecorresponding
.rst
file.