Add support for log tags in newest CRI spec #8265
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
CRI spec has changed to include tags (to tell if the line is full or partial): ``` 2016-10-06T00:17:09.669794202Z stdout F The content of the log entry 1 2016-10-06T00:17:09.669794202Z stdout P First line of log entry 2 2016-10-06T00:17:09.669794202Z stdout P Second line of the log entry 2 2016-10-06T00:17:10.113242941Z stderr F Last line of the log entry 2 ``` I still have to determine how this affects the older format, obviously this change is breaking it, but to be seen if CRI is still supporting it. For more details you can check: https://github.com/kubernetes/community/blob/master/contributors/design-proposals/node/kubelet-cri-logging.md
exekias
added
review
Filebeat
|
Could we make it a config option on which version should be used? Like this we could keep |
|
That makes sense, I will add a parameter, where did you get these numbers from? |
|
https://github.com/elastic/beats/pull/8265/files#diff-d53e98303996a81ae177747740c9f9deR63 (I know, there is more to it then just the split :-) |
|
Aah got it, thanks! Yes, I was thinking of |
|
I wonder if users are aware of CRI versions? Will a user know that tags got added to the log? What config will he look for? +1 on the propose config as I don't have a better suggestion. One thing we could do later is have a dynamic implementation, meaning that we implement it in a way that with the first log message we detect, for example by checking if there is a valid tag, if it is the old or new CRI. |
Setting `cri.parse_flags` will configure the input to parse CRI flags (for partial lines)
|
I have requested more info about the change here: containerd/cri#830, the format is not really backward compatible, as you cannot tell if the log really started with |
kvch
approved these changes
Sep 10, 2018
exekias
added
v6.4.1
and removed
needs_backport
exekias
added a commit
to exekias/beats
that referenced
this pull request
Sep 10, 2018
* Add support for log tags in newest CRI spec CRI spec has changed to include tags (to tell if the line is full or partial): ``` 2016-10-06T00:17:09.669794202Z stdout F The content of the log entry 1 2016-10-06T00:17:09.669794202Z stdout P First line of log entry 2 2016-10-06T00:17:09.669794202Z stdout P Second line of the log entry 2 2016-10-06T00:17:10.113242941Z stderr F Last line of the log entry 2 ``` I still have to determine how this affects the older format, obviously this change is breaking it, but to be seen if CRI is still supporting it. For more details you can check: https://github.com/kubernetes/community/blob/master/contributors/design-proposals/node/kubelet-cri-logging.md (cherry picked from commit 0a20f58)
exekias
added a commit
to exekias/beats
that referenced
this pull request
Sep 10, 2018
* Add support for log tags in newest CRI spec CRI spec has changed to include tags (to tell if the line is full or partial): ``` 2016-10-06T00:17:09.669794202Z stdout F The content of the log entry 1 2016-10-06T00:17:09.669794202Z stdout P First line of log entry 2 2016-10-06T00:17:09.669794202Z stdout P Second line of the log entry 2 2016-10-06T00:17:10.113242941Z stderr F Last line of the log entry 2 ``` I still have to determine how this affects the older format, obviously this change is breaking it, but to be seen if CRI is still supporting it. For more details you can check: https://github.com/kubernetes/community/blob/master/contributors/design-proposals/node/kubelet-cri-logging.md (cherry picked from commit 0a20f58)
exekias
added a commit
that referenced
this pull request
Sep 11, 2018
* Add support for log tags in newest CRI spec CRI spec has changed to include tags (to tell if the line is full or partial): ``` 2016-10-06T00:17:09.669794202Z stdout F The content of the log entry 1 2016-10-06T00:17:09.669794202Z stdout P First line of log entry 2 2016-10-06T00:17:09.669794202Z stdout P Second line of the log entry 2 2016-10-06T00:17:10.113242941Z stderr F Last line of the log entry 2 ``` I still have to determine how this affects the older format, obviously this change is breaking it, but to be seen if CRI is still supporting it. For more details you can check: https://github.com/kubernetes/community/blob/master/contributors/design-proposals/node/kubelet-cri-logging.md (cherry picked from commit 0a20f58)
exekias
added a commit
that referenced
this pull request
Sep 11, 2018
* Add support for log tags in newest CRI spec CRI spec has changed to include tags (to tell if the line is full or partial): ``` 2016-10-06T00:17:09.669794202Z stdout F The content of the log entry 1 2016-10-06T00:17:09.669794202Z stdout P First line of log entry 2 2016-10-06T00:17:09.669794202Z stdout P Second line of the log entry 2 2016-10-06T00:17:10.113242941Z stderr F Last line of the log entry 2 ``` I still have to determine how this affects the older format, obviously this change is breaking it, but to be seen if CRI is still supporting it. For more details you can check: https://github.com/kubernetes/community/blob/master/contributors/design-proposals/node/kubelet-cri-logging.md (cherry picked from commit 0a20f58)
|
I've pusehd a snapshot for |
leweafan
pushed a commit
to leweafan/beats
that referenced
this pull request
Apr 28, 2023
) * Add support for log tags in newest CRI spec CRI spec has changed to include tags (to tell if the line is full or partial): ``` 2016-10-06T00:17:09.669794202Z stdout F The content of the log entry 1 2016-10-06T00:17:09.669794202Z stdout P First line of log entry 2 2016-10-06T00:17:09.669794202Z stdout P Second line of the log entry 2 2016-10-06T00:17:10.113242941Z stderr F Last line of the log entry 2 ``` I still have to determine how this affects the older format, obviously this change is breaking it, but to be seen if CRI is still supporting it. For more details you can check: https://github.com/kubernetes/community/blob/master/contributors/design-proposals/node/kubelet-cri-logging.md (cherry picked from commit c06e91b)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
CRI spec has changed to include tags (to tell if the line is full or
partial):
In order to support both the previous and this new format, this PR introduces a new flag you can pass to the
dockerinput (cri.parse_flags):cri.parse_flagsis disabled by default, we should flip this to enabled by default in 7.0.For more details you can check:
https://github.com/kubernetes/community/blob/master/contributors/design-proposals/node/kubelet-cri-logging.md
Fixes #8257