diff --git a/content/en/logs/log_configuration/parsing.md b/content/en/logs/log_configuration/parsing.md index a4ac6c8d9072c..85b8bcf98b7be 100644 --- a/content/en/logs/log_configuration/parsing.md +++ b/content/en/logs/log_configuration/parsing.md @@ -423,11 +423,25 @@ Note that "id" is an integer and not a string. MyParsingRule (%{integer:user.id}|%{word:user.firstname}) connected on %{date("MM/dd/yyyy"):connect_date} ``` -**Results**: - -{{< img src="logs/processing/parsing/parsing_example_4.png" alt="Parsing example 4" style="width:80%;" >}} - -{{< img src="logs/processing/parsing/parsing_example_4_bis.png" alt="Parsing example 4 bis" style="width:80%;" >}} +**Results**:
+`%{integer:user.id}` +```json +{ + "user": { + "id": 12345 + }, + "connect_date": 1510099200000 +} +``` +`%{word:user.firstname}` +```json +{ + "user": { + "firstname": "john" + }, + "connect_date": 1510099200000 +} +``` ### Optional attribute @@ -437,6 +451,7 @@ Some logs contain values that only appear part of the time. In this case, make a ```text john 1234 connected on 11/08/2017 +john connected on 11/08/2017 ``` **Rule**: @@ -447,9 +462,28 @@ MyParsingRule %{word:user.firstname} (%{integer:user.id} )?connected on %{date(" **Note**: A rule will not match if you include a space after the first word in the optional section. -{{< img src="logs/processing/parsing/parsing_example_5.png" alt="Parsing example 5" style="width:80%;" >}} +**Result**:
+`(%{integer:user.id} )?` -{{< img src="logs/processing/parsing/parsing_example_5_bis.png" alt="Parsing example 5 bis" style="width:80%;" >}} +```json +{ + "user": { + "firstname": "john", + "id": 1234 + }, + "connect_date": 1510099200000 +} +``` + +`%{word:user.firstname} (%{integer:user.id} )?` +```json +{ + "user": { + "firstname": "john", + }, + "connect_date": 1510099200000 +} +``` ### Nested JSON @@ -467,7 +501,17 @@ Sep 06 09:13:38 vagrant program[123]: server.1 {"method":"GET", "status_code":20 parsing_rule %{date("MMM dd HH:mm:ss"):timestamp} %{word:vm} %{word:app}\[%{number:logger.thread_id}\]: %{notSpace:server} %{data::json} ``` -{{< img src="logs/processing/parsing/nested_json.png" alt="Nested JSON Parsing example" style="width:80%;" >}} +**Result**: +```json +{ + "timestamp": 1567761218000, + "vm": "vagrant", + "app": "program", + "logger": { + "thread_id": 123 + } +} +``` ### Regex @@ -483,7 +527,15 @@ john_1a2b3c4 connected on 11/08/2017 MyParsingRule %{regex("[a-z]*"):user.firstname}_%{regex("[a-zA-Z0-9]*"):user.id} .* ``` -{{< img src="logs/processing/parsing/regex_parsing.png" alt="Parsing example 6" style="width:80%;" >}} +**Result**: +```json +{ + "user": { + "firstname": "john", + "id": "1a2b3c4" + } +} +``` ### List to array @@ -501,7 +553,17 @@ Users [John, Oliver, Marc, Tom] have been added to the database myParsingRule Users %{data:users:array("[]",",")} have been added to the database ``` -{{< img src="logs/processing/parsing/array_parsing.png" alt="Parsing example 6" style="width:80%;" >}} +**Result**: +```json +{ + "users": [ + "John", + " Oliver", + " Marc", + " Tom" + ] +} +``` **Log**: diff --git a/content/en/logs/log_configuration/processors.md b/content/en/logs/log_configuration/processors.md index 1981c42b4eccb..638efe19d8af4 100644 --- a/content/en/logs/log_configuration/processors.md +++ b/content/en/logs/log_configuration/processors.md @@ -37,7 +37,7 @@ In [log configuration settings][1], you can configure processors such as the [Gr Create custom grok rules to parse the full message or a specific attribute of your raw event. As a best practice, limit your grok parser to 10 parsing rules. For more information on Grok syntax and parsing rules, see [Parsing][10]. -{{< img src="logs/log_configuration/processor/grok_parser.png" alt="Grok Parser" style="width:80%;" >}} +{{< img src="/logs/processing/processors/define_parsing_rules_syntax_suggestions.png" alt="Grok parser syntax suggestions in the UI" style="width:90%;" >}} {{< tabs >}} {{% tab "UI" %}} @@ -154,8 +154,6 @@ Use the [Datadog Log Pipeline API endpoint][1] with the following log date remap Use the status remapper processor to assign attributes as an official status to your logs. For example, add a log severity level to your logs with the status remapper. -{{< img src="logs/processing/processors/log_post_severity_bis.png" alt="Log severity after remapping" style="width:40%;" >}} - Each incoming status value is mapped as follows: * Integers from 0 to 7 map to the [Syslog severity standards][4] @@ -289,9 +287,7 @@ Use the [Datadog Log Pipeline API endpoint][1] with the following log message re ## Remapper -The remapper processor remaps any source attribute(s) or tags to another target attribute or tag. For example, remap `user` by `firstname` to target your logs in the Log Explorer: - -{{< img src="logs/processing/processors/attribute_post_remapping.png" alt="Attribute after remapping" style="width:60%;">}} +The remapper processor remaps any source attribute(s) or tags to another target attribute or tag. For example, remap `user` by `firstname` to target your logs in the Log Explorer. Constraints on the tag/attribute name are explained in the [attributes and tags documentation][5]. Some additional constraints, applied as `:` or `,`, are not allowed in the target tag/attribute name. diff --git a/static/images/logs/processing/processors/define_parsing_rules_syntax_suggestions.png b/static/images/logs/processing/processors/define_parsing_rules_syntax_suggestions.png new file mode 100644 index 0000000000000..a77fc766329cc Binary files /dev/null and b/static/images/logs/processing/processors/define_parsing_rules_syntax_suggestions.png differ