-
-
Notifications
You must be signed in to change notification settings - Fork 4k
systemd-journald drops all bytes after '\0' #4863
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Comments
… it to 48K This adds a new setting LineMax= to journald.conf, and sets it by default to 48K. When we convert stream-based stdout/stderr logging into record-based log entries, read up to the specified amount of bytes before forcing a line-break. This also makes three related changes: - When a NUL byte is read we'll not recognize this as alternative line break, instead of silently dropping everything after it. (see systemd#4863) - The reason for a line-break is now encoded in the log record, if it wasn't a plain newline. Specifically, we distuingish "nul", "max-reached" and "eof", for line breaks due to NUL byte, due to the maximum line length as configured with LineMax= or due to end of stream. This data is stored in the new implicit _LINE_BREAK= field. It's not synthesized for plain \n line breaks. - A randomized 128bit ID is assigned to each log stream. With these three changes in place it's (mostly) possible to reconstruct the original byte streams from log data, as (most) of the context of the conversion from the byte stream to log records is saved now. (So, the only bits we still drop are empty lines. Which might be something to look into in a future change, and which is outside of the scope of this work) Fixes: https://bugs.freedesktop.org/show_bug.cgi?id=86465 Fixes: systemd#4863 Replaces: systemd#4875
|
… it to 48K This adds a new setting LineMax= to journald.conf, and sets it by default to 48K. When we convert stream-based stdout/stderr logging into record-based log entries, read up to the specified amount of bytes before forcing a line-break. This also makes three related changes: - When a NUL byte is read we'll not recognize this as alternative line break, instead of silently dropping everything after it. (see systemd#4863) - The reason for a line-break is now encoded in the log record, if it wasn't a plain newline. Specifically, we distuingish "nul", "max-reached" and "eof", for line breaks due to NUL byte, due to the maximum line length as configured with LineMax= or due to end of stream. This data is stored in the new implicit _LINE_BREAK= field. It's not synthesized for plain \n line breaks. - A randomized 128bit ID is assigned to each log stream. With these three changes in place it's (mostly) possible to reconstruct the original byte streams from log data, as (most) of the context of the conversion from the byte stream to log records is saved now. (So, the only bits we still drop are empty lines. Which might be something to look into in a future change, and which is outside of the scope of this work) Fixes: https://bugs.freedesktop.org/show_bug.cgi?id=86465 See: systemd#4863 Replaces: systemd#4875
… it to 48K This adds a new setting LineMax= to journald.conf, and sets it by default to 48K. When we convert stream-based stdout/stderr logging into record-based log entries, read up to the specified amount of bytes before forcing a line-break. This also makes three related changes: - When a NUL byte is read we'll not recognize this as alternative line break, instead of silently dropping everything after it. (see systemd#4863) - The reason for a line-break is now encoded in the log record, if it wasn't a plain newline. Specifically, we distuingish "nul", "line-max" and "eof", for line breaks due to NUL byte, due to the maximum line length as configured with LineMax= or due to end of stream. This data is stored in the new implicit _LINE_BREAK= field. It's not synthesized for plain \n line breaks. - A randomized 128bit ID is assigned to each log stream. With these three changes in place it's (mostly) possible to reconstruct the original byte streams from log data, as (most) of the context of the conversion from the byte stream to log records is saved now. (So, the only bits we still drop are empty lines. Which might be something to look into in a future change, and which is outside of the scope of this work) Fixes: https://bugs.freedesktop.org/show_bug.cgi?id=86465 See: systemd#4863 Replaces: systemd#4875
… it to 48K (#6838) This adds a new setting LineMax= to journald.conf, and sets it by default to 48K. When we convert stream-based stdout/stderr logging into record-based log entries, read up to the specified amount of bytes before forcing a line-break. This also makes three related changes: - When a NUL byte is read we'll not recognize this as alternative line break, instead of silently dropping everything after it. (see #4863) - The reason for a line-break is now encoded in the log record, if it wasn't a plain newline. Specifically, we distuingish "nul", "line-max" and "eof", for line breaks due to NUL byte, due to the maximum line length as configured with LineMax= or due to end of stream. This data is stored in the new implicit _LINE_BREAK= field. It's not synthesized for plain \n line breaks. - A randomized 128bit ID is assigned to each log stream. With these three changes in place it's (mostly) possible to reconstruct the original byte streams from log data, as (most) of the context of the conversion from the byte stream to log records is saved now. (So, the only bits we still drop are empty lines. Which might be something to look into in a future change, and which is outside of the scope of this work) Fixes: https://bugs.freedesktop.org/show_bug.cgi?id=86465 See: #4863 Replaces: #4875
Since ec20fe5 we will no consider '\n', '\0' as well as line size limits and EOF all as fully equivalent EOL markers. This means the command proposed will now cleanly result in multiple log lines generated. Here's an example: $ printf 'ab\0cd\nef' | systemd-cat --identifier drop-after-zero
$ journalctl -n 3
-- Logs begin at Thu 2016-08-18 23:09:37 CEST, end at Thu 2018-05-24 13:21:25 CEST. --
Mai 24 13:21:25 sigma drop-after-zero[51513]: ab
Mai 24 13:21:25 sigma drop-after-zero[51513]: cd
Mai 24 13:21:25 sigma drop-after-zero[51513]: ef I am pretty sure this new behaviour makes a ton of sense, and no data is lost. In fact the structure log message contains enough information to even reconstruct whether the line was broken due to \n, \0, line limit or EOF through the _LINE_BREAK= property:
|
It's not fixed. That's why I wrote in the description that I should update it and we discussed it in #6838 (comment). |
ah, indeed. ok, let's clarify on this issue hence, that the offending command is this:
And that indeed remains unfixed. Sorry for the confusion |
Hmm, so while I agree that simply ignoring everything coming after a NUL byte in a BSD syslog message is definitely the wrong approach, I am not sure what the right approach though is. If we get an UDP or AF_UNIX datagram with an embedded NUL byte, does this means the datagram contains two log records and the NUL is the separator? or does it mean that NUL shall be considered part of the log message? |
I think the latter. According to https://tools.ietf.org/html/rfc5424
|
$ printf '<13>Sep 15 15:07:58 HOST: null\0here' | nc -w1 -u -U /run/systemd/journal/dev-log $ journalctl -n1 -o short-unix --no-hostname 1528452686.141537 HOST[18159]: null\000here This mostly fixes systemd#4863. The standard allows NUL-bytes and arbitrary crap to be embeded in messages. We forward all the crap to syslog, but strip whitespace and escape invalid characters to generate a printable message to store as the MESSAGE= field. The common case of messages that don't need stripping or escaping is optimized.
This allows the original stream to be recreated and/or verified. The new field is written if any stripping or escaping was done or if the input message contained embeded NULs. $ printf '<13>Sep 15 15:07:58 HOST: x\0y' | nc -w1 -u -U /run/systemd/journal/dev-log $ journalctl -o json-pretty ... { ... "MESSAGE" : "x", "_ORIGINAL_MESSAGE" : [ 60, 49, 51, 62, 83, 101, 112, 32, 49, 53, 32, 49, 53, 58, 48, 55, 58, 53, 56, 32, 72, 79, 83, 84, 58, 32, 120, 0, 121 ] } $ journalctl -o export ... | cat -v ... MESSAGE=x _ORIGINAL_MESSAGE ^]^@^@^@^@^@^@^@<13>Sep 15 15:07:58 HOST: x^@y This mostly fixes systemd#4863.
… it to 48K (systemd#6838) This adds a new setting LineMax= to journald.conf, and sets it by default to 48K. When we convert stream-based stdout/stderr logging into record-based log entries, read up to the specified amount of bytes before forcing a line-break. This also makes three related changes: - When a NUL byte is read we'll not recognize this as alternative line break, instead of silently dropping everything after it. (see systemd#4863) - The reason for a line-break is now encoded in the log record, if it wasn't a plain newline. Specifically, we distuingish "nul", "line-max" and "eof", for line breaks due to NUL byte, due to the maximum line length as configured with LineMax= or due to end of stream. This data is stored in the new implicit _LINE_BREAK= field. It's not synthesized for plain \n line breaks. - A randomized 128bit ID is assigned to each log stream. With these three changes in place it's (mostly) possible to reconstruct the original byte streams from log data, as (most) of the context of the conversion from the byte stream to log records is saved now. (So, the only bits we still drop are empty lines. Which might be something to look into in a future change, and which is outside of the scope of this work) Fixes: https://bugs.freedesktop.org/show_bug.cgi?id=86465 See: systemd#4863 Replaces: systemd#4875 (cherry picked from commit ec20fe5) Resolves: #1442262 [msekleta: I had to manually rewrite upstream commit, because git did very poor job merging old and new code and identified a lot of merge conflicts in a code that was totally unrelated.]
As shown in #10918, |
The issue was also reported in #12484. I don't think it makes much sense to keep this one open. |
I'm sorry for not being specific. I will have to update the description so that it won't cause confusion like #6838 (comment).
From #1460
The text was updated successfully, but these errors were encountered: