Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,7 @@ Processor 由一个 name 和多个配置组成,不同类型的 Processor 配

- `date`: 解析格式化的时间字符串字段,例如 `2024-07-12T16:18:53.048`。
- `epoch`: 解析数字时间戳字段,例如 `1720772378893`。
- `decolorize`: 移除日志数据中的 ANSI 颜色代码。
- `dissect`: 对 log 数据字段进行拆分。
- `gsub`: 对 log 数据字段进行替换。
- `join`: 对 log 中的 array 类型字段进行合并。
Expand Down Expand Up @@ -110,6 +111,21 @@ processors:
- `resolution`: 时间戳精度,支持 `s`, `sec` , `second` , `ms`, `millisecond`, `milli`, `us`, `microsecond`, `micro`, `ns`, `nanosecond`, `nano`。默认为 `ms`。
- `ignore_missing`: 忽略字段不存在的情况。默认为 `false`。如果字段不存在,并且此配置为 false,则会抛出异常。

### `decolorize`

`decolorize` Processor 用于移除日志数据中的 ANSI 颜色代码。示例配置如下:

```yaml
processors:
- decolorize:
fields:
- message
```

如上所示,`decolorize` Processor 的配置包含以下字段:

- `fields`: 需要移除颜色代码的字段名列表。

### `dissect`

`dissect` Processor 用于对 log 数据字段进行拆分,示例配置如下:
Expand Down
26 changes: 21 additions & 5 deletions versioned_docs/version-0.10/user-guide/logs/pipeline-config.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,7 @@ We currently provide the following built-in Processors:

- `date`: parses formatted time string fields, such as `2024-07-12T16:18:53.048`.
- `epoch`: parses numeric timestamp fields, such as `1720772378893`.
- `decolorize`: removes ANSI color codes from log data.
- `dissect`: splits log data fields.
- `gsub`: replaces log data fields.
- `join`: merges array-type fields in logs.
Expand Down Expand Up @@ -108,6 +109,21 @@ In the above example, the configuration of the `epoch` processor includes the fo
- `resolution`: Timestamp precision, supports `s`, `sec`, `second`, `ms`, `millisecond`, `milli`, `us`, `microsecond`, `micro`, `ns`, `nanosecond`, `nano`. Defaults to `ms`.
- `ignore_missing`: Ignores the case when the field is missing. Defaults to `false`. If the field is missing and this configuration is set to `false`, an exception will be thrown.

### `decolorize`

The `decolorize` processor is used to remove ANSI color codes from log data. Here's an example configuration:

```yaml
processors:
- decolorize:
fields:
- message
```

In the above example, the configuration of the `decolorize` processor includes the following fields:

- `fields`: A list of field names to be decolorized.

### `dissect`

The `dissect` processor is used to split log data fields. Here's an example configuration:
Expand Down Expand Up @@ -142,12 +158,12 @@ Similar to Logstash's dissect pattern, the dissect pattern consists of `%{key}`,

The dissect pattern supports the following modifiers:

| Modifier | Description | Example |
| ---------- | ---------------------------------------------------- | -------------------- |
| `+` | Concatenates two or more fields together | `%{+key} %{+key}` |
| Modifier | Description | Example |
| ------------ | ------------------------------------------------------ | --------------------- |
| `+` | Concatenates two or more fields together | `%{+key} %{+key}` |
| `+` and `/n` | Concatenates two or more fields in the specified order | `%{+key/2} %{+key/1}` |
| `->` | Ignores any repeating characters on the right side | `%{key1->} %{key2->}` |
| `?` | Ignores matching values | `%{?key}` |
| `->` | Ignores any repeating characters on the right side | `%{key1->} %{key2->}` |
| `?` | Ignores matching values | `%{?key}` |

#### `dissect` examples

Expand Down
Loading