-
Notifications
You must be signed in to change notification settings - Fork 3.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
log: long-ish line broken up in multiple messages #61681
Comments
that is properly impossible - the crdb-v1 log format is ambiguous and cannot be parsed reliably an faithfully. That being said I agree that the log retrieval API is still broken. For one, it still accumulates all the log entries in RAM server-side before returning them to the client. That's insane. I think the next step here is to simply remove that API altogether and replace it by a better one. When looking at this issue together with #59558 which you filed earlier, I think the better thing we can do is to fix / extend the |
Why is the "crdb-v1" format relevant, given that we've moved on? Also I never understood why that RPC service needs to parse anything at all.
I personally care about how log files look independently of |
The RPC code only knows how to parse crdb-v1. It happens to work with crdb-v2 because crdb-v2 was designed to be compatible with crdb-v1 parsers.
Because it returns structured protobuf entries, not the raw text of the log file. This is a misdesign from the start.
Hm that is also right. Let's discuss this soon. |
Here are a few notes from our discussion today:
|
64900: util/log: fix the long line splitting behavior with UTF-8 sequences r=bdarnell a=knz Alleviates #61681. Fixes #64896. Release note (cli change): Long lines in the `crdb-v2` output logging format are now split when they exceed 16K, instead of 1K as previously. Release note (bug fix): The automatic long line splitting behavior of the new `crdb-v2` logging format now avoids splitting lines in the middle of UTF-8 sequences, and avoids unbalanced redaction markers. Co-authored-by: Raphael 'kena' Poss <knz@thaumogen.net>
I've got a long log line, but not all that long - maybe a few KB. The logging infra seems to break it into multiple messages. This makes for a much worse experience when I need to read it, grep in it, etc.
Was there a good reason why we did this - particularly with seemingly such a low char limit?
I know we used to have problem #50166 with some crap RPC service truncating 64K lines. But a) the limit there was much larger and b) shouldn't we fix that damn service instead?
Jira issue: CRDB-2674
Epic CRDB-21266
The text was updated successfully, but these errors were encountered: