From 3d5e04850dce7df69b63527b17f42336b3782804 Mon Sep 17 00:00:00 2001 From: Xiang Dai <764524258@qq.com> Date: Tue, 1 Oct 2019 14:12:14 +0800 Subject: [PATCH 1/3] docs(stage): add docker and cri Signed-off-by: Xiang Dai <764524258@qq.com> --- docs/clients/promtail/stages/cri.md | 41 ++++++++++++++++++++++++++ docs/clients/promtail/stages/docker.md | 31 +++++++++++++++++++ 2 files changed, 72 insertions(+) create mode 100644 docs/clients/promtail/stages/cri.md create mode 100644 docs/clients/promtail/stages/docker.md diff --git a/docs/clients/promtail/stages/cri.md b/docs/clients/promtail/stages/cri.md new file mode 100644 index 000000000000..2d3df56c0ac3 --- /dev/null +++ b/docs/clients/promtail/stages/cri.md @@ -0,0 +1,41 @@ +# `cri` stage + +The `cri` stage is a parsing stage that reads the log line as the way that cri generated. + +## Format + +Each log from cri is string format and covert following three parts: +1. `log`: the content of log +2. `stream`: stdout/stderr +3. `time`: the timestamp string of log + +Each part is splited with blank with others and no blanks in each part. + +So in the following logs, only the first is cri format which `cri` stage can format: +``` +"2019-01-01T01:00:00.000000001Z stderr P test\ngood" +"2019-01-01 T01:00:00.000000001Z stderr testgood" +"2019-01-01T01:00:00.000000001Z testgood" +``` + +More case refer to [entry_parser_test](../../../../pkg/promtail/api/entry_parser_test.go). + +## Examples + +### Using log line + +```yaml +cri: {} +``` + +Given the following log line: + +``` +"2019-04-30T02:12:41.8443515Z stdout xx message" +``` + +The following key-value pairs would be created in the set of extracted data: + +- `output`: `message` +- `stream`: `stdout` +- `timestamp`: `2019-04-30T02:12:41.8443515` diff --git a/docs/clients/promtail/stages/docker.md b/docs/clients/promtail/stages/docker.md new file mode 100644 index 000000000000..2fa5347631ea --- /dev/null +++ b/docs/clients/promtail/stages/docker.md @@ -0,0 +1,31 @@ +# `docker` stage + +The `docker` stage is a parsing stage that reads the log line as the way that docker generated. +It works similar with `json` stage but no need to set anything. + +## Format + +Each log from docker is json format and covert following three keys: +1. `log`: the content of log +2. `stream`: stdout/stderr +3. `time`: the timestamp string of log + +## Examples + +### Using log line + +```yaml +docker: {} +``` + +Given the following log line: + +``` +{"log":"log message\n","stream":"stderr","time":"2019-04-30T02:12:41.8443515Z"} +``` + +The following key-value pairs would be created in the set of extracted data: + +- `output`: `log message\n` +- `stream`: `stderr` +- `timestamp`: `2019-04-30T02:12:41.8443515` From 2c9fb2ca5fca7527e0c2652f26fa20fd652ec690 Mon Sep 17 00:00:00 2001 From: Xiang Dai <764524258@qq.com> Date: Tue, 1 Oct 2019 15:22:58 +0800 Subject: [PATCH 2/3] add link Signed-off-by: Xiang Dai <764524258@qq.com> --- docs/clients/promtail/pipelines.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/clients/promtail/pipelines.md b/docs/clients/promtail/pipelines.md index d46da9c6475c..43b84b404de8 100644 --- a/docs/clients/promtail/pipelines.md +++ b/docs/clients/promtail/pipelines.md @@ -188,6 +188,8 @@ given log entry. Parsing stages: + * [docker](./stages/docker.md): Extract data by parsing the log line as docker format. + * [cri](./stages/cri.md): Extract data by parsing the log line as cri format. * [regex](./stages/regex.md): Extract data using a regular expression. * [json](./stages/json.md): Extract data by parsing the log line as JSON. From 3a9677bd099b7b89d02f1223b59fcc69601486f5 Mon Sep 17 00:00:00 2001 From: Robert Fratto Date: Tue, 1 Oct 2019 09:01:30 -0400 Subject: [PATCH 3/3] docs: minor grammar fixes and standardize cri/docker pages --- docs/clients/promtail/pipelines.md | 4 ++-- docs/clients/promtail/stages/cri.md | 29 +++++++++++++++----------- docs/clients/promtail/stages/docker.md | 25 ++++++++++++++-------- 3 files changed, 35 insertions(+), 23 deletions(-) diff --git a/docs/clients/promtail/pipelines.md b/docs/clients/promtail/pipelines.md index 43b84b404de8..72bd64883bfe 100644 --- a/docs/clients/promtail/pipelines.md +++ b/docs/clients/promtail/pipelines.md @@ -188,8 +188,8 @@ given log entry. Parsing stages: - * [docker](./stages/docker.md): Extract data by parsing the log line as docker format. - * [cri](./stages/cri.md): Extract data by parsing the log line as cri format. + * [docker](./stages/docker.md): Extract data by parsing the log line using the standard Docker format. + * [cri](./stages/cri.md): Extract data by parsing the log line using the standard CRI format. * [regex](./stages/regex.md): Extract data using a regular expression. * [json](./stages/json.md): Extract data by parsing the log line as JSON. diff --git a/docs/clients/promtail/stages/cri.md b/docs/clients/promtail/stages/cri.md index 2d3df56c0ac3..a054b9f39542 100644 --- a/docs/clients/promtail/stages/cri.md +++ b/docs/clients/promtail/stages/cri.md @@ -1,31 +1,36 @@ # `cri` stage -The `cri` stage is a parsing stage that reads the log line as the way that cri generated. +The `cri` stage is a parsing stage that reads the log line using the standard CRI logging format. -## Format +## Schema -Each log from cri is string format and covert following three parts: -1. `log`: the content of log -2. `stream`: stdout/stderr -3. `time`: the timestamp string of log +```yaml +cri: {} +``` + +Unlike most stages, the `cri` stage provides no configuration options and only +supports the specific CRI log format. CRI specifies log lines log lines as +space-delimited values with the following components: -Each part is splited with blank with others and no blanks in each part. +1. `time`: The timestamp string of the log +2. `stream`: Either stdout or stderr +3. `log`: The contents of the log line + +No whitespace is permitted between the components. In the following exmaple, +only the first log line can be properly formatted using the `cri` stage: -So in the following logs, only the first is cri format which `cri` stage can format: ``` "2019-01-01T01:00:00.000000001Z stderr P test\ngood" "2019-01-01 T01:00:00.000000001Z stderr testgood" "2019-01-01T01:00:00.000000001Z testgood" ``` -More case refer to [entry_parser_test](../../../../pkg/promtail/api/entry_parser_test.go). - ## Examples -### Using log line +For the given pipeline: ```yaml -cri: {} +- cri: {} ``` Given the following log line: diff --git a/docs/clients/promtail/stages/docker.md b/docs/clients/promtail/stages/docker.md index 2fa5347631ea..da0fc7ee05fc 100644 --- a/docs/clients/promtail/stages/docker.md +++ b/docs/clients/promtail/stages/docker.md @@ -1,21 +1,28 @@ # `docker` stage -The `docker` stage is a parsing stage that reads the log line as the way that docker generated. -It works similar with `json` stage but no need to set anything. +The `docker` stage is a parsing stage that reads log lines in the standard +format of Docker log files. -## Format +## Schema -Each log from docker is json format and covert following three keys: -1. `log`: the content of log -2. `stream`: stdout/stderr -3. `time`: the timestamp string of log +```yaml +docker: {} +``` + +Unlike most stages, the `docker` stage provides no configuration options and +only supports the specific Docker log format. Each log line from Docker is +written as JSON with the following keys: + +1. `log`: The content of log line +2. `stream`: Either `stdout` or `stderr` +3. `time`: The timestamp string of the log line ## Examples -### Using log line +For the given pipeline: ```yaml -docker: {} +- docker: {} ``` Given the following log line: