Skip to content

Commit 3ceab62

Browse files
maycmleemaycmlee
authored
[DOCS-10886] Add OP Custom Processor (#29565)
* add doc and shortcode * add nav item * update processor name * add to all use cases * Update layouts/shortcodes/observability_pipelines/processors/custom_processor.md * fix links * add instructions * apply suggestions * apply suggestions
1 parent e7af172 commit 3ceab62

File tree

94 files changed

+537
-15
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

94 files changed

+537
-15
lines changed

‎config/_default/menus/main.en.yaml

Lines changed: 20 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -5654,81 +5654,86 @@ menu:
56545654
parent: observability_pipelines_processors
56555655
identifier: observability_pipelines_processors_add_hostname
56565656
weight: 902
5657+
- name: Custom Processor
5658+
url: observability_pipelines/processors/custom_processor
5659+
parent: observability_pipelines_processors
5660+
identifier: observability_pipelines_processors_custom_processor
5661+
weight: 903
56575662
- name: Deduplicate
56585663
url: observability_pipelines/processors/dedupe
56595664
parent: observability_pipelines_processors
56605665
identifier: observability_pipelines_processors_dedupe
5661-
weight: 903
5666+
weight: 904
56625667
- name: Edit fields
56635668
url: observability_pipelines/processors/edit_fields
56645669
parent: observability_pipelines_processors
56655670
identifier: observability_pipelines_processors_edit_fields
5666-
weight: 904
5671+
weight: 905
56675672
- name: Enrichment Table
56685673
url: observability_pipelines/processors/enrichment_table
56695674
parent: observability_pipelines_processors
56705675
identifier: observability_pipelines_processors_enrichment_table
5671-
weight: 905
5676+
weight: 906
56725677
- name: Filter
56735678
url: observability_pipelines/processors/filter
56745679
parent: observability_pipelines_processors
56755680
identifier: observability_pipelines_processors_filter
5676-
weight: 906
5681+
weight: 907
56775682
- name: Generate Metrics
56785683
url: observability_pipelines/processors/generate_metrics
56795684
parent: observability_pipelines_processors
56805685
identifier: observability_pipelines_processors_generate_metrics
5681-
weight: 907
5686+
weight: 908
56825687
- name: Grok Parser
56835688
url: observability_pipelines/processors/grok_parser
56845689
parent: observability_pipelines_processors
56855690
identifier: observability_pipelines_processors_grok_parser
5686-
weight: 908
5691+
weight: 909
56875692
- name: Parse JSON
56885693
url: observability_pipelines/processors/parse_json
56895694
parent: observability_pipelines_processors
56905695
identifier: observability_pipelines_processors_parse_json
5691-
weight: 909
5696+
weight: 910
56925697
- name: Parse XML
56935698
url: observability_pipelines/processors/parse_xml
56945699
parent: observability_pipelines_processors
56955700
identifier: observability_pipelines_processors_parse_xml
5696-
weight: 910
5701+
weight: 911
56975702
- name: Quota
56985703
url: observability_pipelines/processors/quota
56995704
parent: observability_pipelines_processors
57005705
identifier: observability_pipelines_processors_quota
5701-
weight: 911
5706+
weight: 912
57025707
- name: Reduce
57035708
url: observability_pipelines/processors/reduce
57045709
parent: observability_pipelines_processors
57055710
identifier: observability_pipelines_processors_reduce
5706-
weight: 912
5711+
weight: 913
57075712
- name: Remap to OCSF
57085713
url: observability_pipelines/processors/remap_ocsf
57095714
parent: observability_pipelines_processors
57105715
identifier: observability_pipelines_processors_remap_ocsf
5711-
weight: 913
5716+
weight: 914
57125717
- name: Sample
57135718
url: observability_pipelines/processors/sample
57145719
parent: observability_pipelines_processors
57155720
identifier: observability_pipelines_processors_sample
5716-
weight: 914
5721+
weight: 915
57175722
- name: Sensitive Data Scanner
57185723
url: observability_pipelines/processors/sensitive_data_scanner
57195724
parent: observability_pipelines_processors
57205725
identifier: observability_pipelines_processors_sensitive_data_scanner
5721-
weight: 915
5726+
weight: 916
57225727
- name: Split Array
57235728
url: observability_pipelines/processors/split_array
57245729
parent: observability_pipelines_processors
57255730
identifier: observability_pipelines_processors_split_array
5726-
weight: 916
5731+
weight: 917
57275732
- name: Throttle
57285733
url: observability_pipelines/processors/throttle
57295734
parent: observability_pipelines_processors
57305735
identifier: observability_pipelines_processors_throttle
5731-
weight: 917
5736+
weight: 918
57325737
- name: Destinations
57335738
url: observability_pipelines/destinations/
57345739
parent: observability_pipelines

‎content/en/observability_pipelines/processors/custom_processor.md

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
---
2+
title: Custom Processor
3+
disable_toc: false
4+
---
5+
6+
{{< callout src="#" header="false" btn_hidden="true" >}}
7+
The Custom Processor is in Preview.
8+
{{< /callout >}}
9+
10+
{{% observability_pipelines/processors/custom_processor %}}
11+
12+
## Custom functions
13+
14+
{{< whatsnext desc="The functions are organized into the following categories:" >}}
15+
{{< nextlink href="observability_pipelines/processors/custom_processor/#array" >}}Array{{< /nextlink >}}
16+
{{< nextlink href="observability_pipelines/processors/custom_processor/#codec" >}}Codec{{< /nextlink >}}
17+
{{< nextlink href="observability_pipelines/processors/custom_processor/#convert" >}}Convert{{< /nextlink >}}
18+
{{< nextlink href="observability_pipelines/processors/custom_processor/#cryptography" >}}Cryptography{{< /nextlink >}}
19+
{{< nextlink href="observability_pipelines/processors/custom_processor/#debug" >}}Debug{{< /nextlink >}}
20+
{{< nextlink href="observability_pipelines/processors/custom_processor/#enrichment" >}}Enrichment{{< /nextlink >}}
21+
{{< nextlink href="observability_pipelines/processors/custom_processor/#ip" >}}IP{{< /nextlink >}}
22+
{{< nextlink href="observability_pipelines/processors/custom_processor/#number" >}}Number{{< /nextlink >}}
23+
{{< nextlink href="observability_pipelines/processors/custom_processor/#object" >}}Object{{< /nextlink >}}
24+
{{< nextlink href="observability_pipelines/processors/custom_processor/#parse" >}}Parse{{< /nextlink >}}
25+
{{< nextlink href="observability_pipelines/processors/custom_processor/#path" >}}Path{{< /nextlink >}}
26+
{{< nextlink href="observability_pipelines/processors/custom_processor/#random" >}}Random{{< /nextlink >}}
27+
{{< nextlink href="observability_pipelines/processors/custom_processor/#string" >}}String{{< /nextlink >}}
28+
{{< nextlink href="observability_pipelines/processors/custom_processor/#system" >}}System{{< /nextlink >}}
29+
{{< nextlink href="observability_pipelines/processors/custom_processor/#timestamp" >}}Timestamp{{< /nextlink >}}
30+
{{< nextlink href="observability_pipelines/processors/custom_processor/#type" >}}Type{{< /nextlink >}}
31+
{{< /whatsnext >}}
32+
33+
{{< vrl-functions >}}

‎content/en/observability_pipelines/set_up_pipelines/archive_logs/amazon_data_firehose.md

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -201,6 +201,11 @@ Follow the instructions for the cloud provider you are using to archive your log
201201

202202
{{% observability_pipelines/processors/add_hostname %}}
203203

204+
{{% /tab %}}
205+
{{% tab "Custom Processor" %}}
206+
207+
{{% observability_pipelines/processors/custom_processor %}}
208+
204209
{{% /tab %}}
205210
{{% tab "Dedupe" %}}
206211

‎content/en/observability_pipelines/set_up_pipelines/archive_logs/amazon_s3.md

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -200,6 +200,11 @@ Follow the instructions for the cloud provider you are using to archive your log
200200

201201
{{% observability_pipelines/processors/add_hostname %}}
202202

203+
{{% /tab %}}
204+
{{% tab "Custom Processor" %}}
205+
206+
{{% observability_pipelines/processors/custom_processor %}}
207+
203208
{{% /tab %}}
204209
{{% tab "Dedupe" %}}
205210

‎content/en/observability_pipelines/set_up_pipelines/archive_logs/datadog_agent.md

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -203,6 +203,11 @@ Follow the instructions for the cloud provider you are using to archive your log
203203

204204
{{% observability_pipelines/processors/add_hostname %}}
205205

206+
{{% /tab %}}
207+
{{% tab "Custom Processor" %}}
208+
209+
{{% observability_pipelines/processors/custom_processor %}}
210+
206211
{{% /tab %}}
207212
{{% tab "Dedupe" %}}
208213

‎content/en/observability_pipelines/set_up_pipelines/archive_logs/fluent.md

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -205,6 +205,11 @@ Follow the instructions for the cloud provider you are using to archive your log
205205

206206
{{% observability_pipelines/processors/add_hostname %}}
207207

208+
{{% /tab %}}
209+
{{% tab "Custom Processor" %}}
210+
211+
{{% observability_pipelines/processors/custom_processor %}}
212+
208213
{{% /tab %}}
209214
{{% tab "Dedupe" %}}
210215

‎content/en/observability_pipelines/set_up_pipelines/archive_logs/google_pubsub.md

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -202,6 +202,11 @@ Follow the instructions for the cloud provider you are using to archive your log
202202

203203
{{% observability_pipelines/processors/add_hostname %}}
204204

205+
{{% /tab %}}
206+
{{% tab "Custom Processor" %}}
207+
208+
{{% observability_pipelines/processors/custom_processor %}}
209+
205210
{{% /tab %}}
206211
{{% tab "Dedupe" %}}
207212

‎content/en/observability_pipelines/set_up_pipelines/archive_logs/http_client.md

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -203,6 +203,11 @@ Follow the instructions for the cloud provider you are using to archive your log
203203

204204
{{% observability_pipelines/processors/add_hostname %}}
205205

206+
{{% /tab %}}
207+
{{% tab "Custom Processor" %}}
208+
209+
{{% observability_pipelines/processors/custom_processor %}}
210+
206211
{{% /tab %}}
207212
{{% tab "Dedupe" %}}
208213

‎content/en/observability_pipelines/set_up_pipelines/archive_logs/http_server.md

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -200,6 +200,11 @@ Follow the instructions for the cloud provider you are using to archive your log
200200

201201
{{% observability_pipelines/processors/add_hostname %}}
202202

203+
{{% /tab %}}
204+
{{% tab "Custom Processor" %}}
205+
206+
{{% observability_pipelines/processors/custom_processor %}}
207+
203208
{{% /tab %}}
204209
{{% tab "Dedupe" %}}
205210

‎content/en/observability_pipelines/set_up_pipelines/archive_logs/kafka.md

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -201,6 +201,11 @@ Follow the instructions for the cloud provider you are using to archive your log
201201

202202
{{% observability_pipelines/processors/add_hostname %}}
203203

204+
{{% /tab %}}
205+
{{% tab "Custom Processor" %}}
206+
207+
{{% observability_pipelines/processors/custom_processor %}}
208+
204209
{{% /tab %}}
205210
{{% tab "Dedupe" %}}
206211

‎content/en/observability_pipelines/set_up_pipelines/archive_logs/logstash.md

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -201,6 +201,11 @@ Follow the instructions for the cloud provider you are using to archive your log
201201

202202
{{% observability_pipelines/processors/add_hostname %}}
203203

204+
{{% /tab %}}
205+
{{% tab "Custom Processor" %}}
206+
207+
{{% observability_pipelines/processors/custom_processor %}}
208+
204209
{{% /tab %}}
205210
{{% tab "Dedupe" %}}
206211

‎content/en/observability_pipelines/set_up_pipelines/archive_logs/splunk_hec.md

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -203,6 +203,11 @@ Follow the instructions for the cloud provider you are using to archive your log
203203

204204
{{% observability_pipelines/processors/add_hostname %}}
205205

206+
{{% /tab %}}
207+
{{% tab "Custom Processor" %}}
208+
209+
{{% observability_pipelines/processors/custom_processor %}}
210+
206211
{{% /tab %}}
207212
{{% tab "Dedupe" %}}
208213

‎content/en/observability_pipelines/set_up_pipelines/archive_logs/splunk_tcp.md

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -203,6 +203,11 @@ Follow the instructions for the cloud provider you are using to archive your log
203203

204204
{{% observability_pipelines/processors/add_hostname %}}
205205

206+
{{% /tab %}}
207+
{{% tab "Custom Processor" %}}
208+
209+
{{% observability_pipelines/processors/custom_processor %}}
210+
206211
{{% /tab %}}
207212
{{% tab "Dedupe" %}}
208213

‎content/en/observability_pipelines/set_up_pipelines/archive_logs/sumo_logic_hosted_collector.md

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -203,6 +203,11 @@ Follow the instructions for the cloud provider you are using to archive your log
203203

204204
{{% observability_pipelines/processors/add_hostname %}}
205205

206+
{{% /tab %}}
207+
{{% tab "Custom Processor" %}}
208+
209+
{{% observability_pipelines/processors/custom_processor %}}
210+
206211
{{% /tab %}}
207212
{{% tab "Dedupe" %}}
208213

‎content/en/observability_pipelines/set_up_pipelines/archive_logs/syslog.md

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -204,6 +204,11 @@ Follow the instructions for the cloud provider you are using to archive your log
204204

205205
{{% observability_pipelines/processors/add_hostname %}}
206206

207+
{{% /tab %}}
208+
{{% tab "Custom Processor" %}}
209+
210+
{{% observability_pipelines/processors/custom_processor %}}
211+
207212
{{% /tab %}}
208213
{{% tab "Dedupe" %}}
209214

‎content/en/observability_pipelines/set_up_pipelines/dual_ship_logs/amazon_data_firehose.md

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -154,6 +154,11 @@ To set up the destination, follow the instructions for the cloud provider you ar
154154

155155
{{% observability_pipelines/processors/add_hostname %}}
156156

157+
{{% /tab %}}
158+
{{% tab "Custom Processor" %}}
159+
160+
{{% observability_pipelines/processors/custom_processor %}}
161+
157162
{{% /tab %}}
158163
{{% tab "Dedupe" %}}
159164

‎content/en/observability_pipelines/set_up_pipelines/dual_ship_logs/amazon_s3.md

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -152,6 +152,11 @@ To set up the destination, follow the instructions for the cloud provider you ar
152152

153153
{{% observability_pipelines/processors/add_hostname %}}
154154

155+
{{% /tab %}}
156+
{{% tab "Custom Processor" %}}
157+
158+
{{% observability_pipelines/processors/custom_processor %}}
159+
155160
{{% /tab %}}
156161
{{% tab "Dedupe" %}}
157162

‎content/en/observability_pipelines/set_up_pipelines/dual_ship_logs/datadog_agent.md

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -155,6 +155,11 @@ To set up the destination, follow the instructions for the cloud provider you ar
155155

156156
{{% observability_pipelines/processors/add_hostname %}}
157157

158+
{{% /tab %}}
159+
{{% tab "Custom Processor" %}}
160+
161+
{{% observability_pipelines/processors/custom_processor %}}
162+
158163
{{% /tab %}}
159164
{{% tab "Dedupe" %}}
160165

‎content/en/observability_pipelines/set_up_pipelines/dual_ship_logs/fluent.md

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -155,6 +155,11 @@ To set up the destination, follow the instructions for the cloud provider you ar
155155

156156
{{% observability_pipelines/processors/add_hostname %}}
157157

158+
{{% /tab %}}
159+
{{% tab "Custom Processor" %}}
160+
161+
{{% observability_pipelines/processors/custom_processor %}}
162+
158163
{{% /tab %}}
159164
{{% tab "Dedupe" %}}
160165

‎content/en/observability_pipelines/set_up_pipelines/dual_ship_logs/google_pubsub.md

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -152,6 +152,11 @@ To set up the destination, follow the instructions for the cloud provider you ar
152152

153153
{{% observability_pipelines/processors/add_hostname %}}
154154

155+
{{% /tab %}}
156+
{{% tab "Custom Processor" %}}
157+
158+
{{% observability_pipelines/processors/custom_processor %}}
159+
155160
{{% /tab %}}
156161
{{% tab "Dedupe" %}}
157162

‎content/en/observability_pipelines/set_up_pipelines/dual_ship_logs/http_client.md

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -154,6 +154,11 @@ To set up the destination, follow the instructions for the cloud provider you ar
154154

155155
{{% observability_pipelines/processors/add_hostname %}}
156156

157+
{{% /tab %}}
158+
{{% tab "Custom Processor" %}}
159+
160+
{{% observability_pipelines/processors/custom_processor %}}
161+
157162
{{% /tab %}}
158163
{{% tab "Dedupe" %}}
159164

‎content/en/observability_pipelines/set_up_pipelines/dual_ship_logs/http_server.md

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -152,6 +152,11 @@ To set up the destination, follow the instructions for the cloud provider you ar
152152

153153
{{% observability_pipelines/processors/add_hostname %}}
154154

155+
{{% /tab %}}
156+
{{% tab "Custom Processor" %}}
157+
158+
{{% observability_pipelines/processors/custom_processor %}}
159+
155160
{{% /tab %}}
156161
{{% tab "Dedupe" %}}
157162

‎content/en/observability_pipelines/set_up_pipelines/dual_ship_logs/kafka.md

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -153,6 +153,11 @@ To set up the destination, follow the instructions for the cloud provider you ar
153153

154154
{{% observability_pipelines/processors/add_hostname %}}
155155

156+
{{% /tab %}}
157+
{{% tab "Custom Processor" %}}
158+
159+
{{% observability_pipelines/processors/custom_processor %}}
160+
156161
{{% /tab %}}
157162
{{% tab "Dedupe" %}}
158163

‎content/en/observability_pipelines/set_up_pipelines/dual_ship_logs/logstash.md

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -153,6 +153,11 @@ To set up the destination, follow the instructions for the cloud provider you ar
153153

154154
{{% observability_pipelines/processors/add_hostname %}}
155155

156+
{{% /tab %}}
157+
{{% tab "Custom Processor" %}}
158+
159+
{{% observability_pipelines/processors/custom_processor %}}
160+
156161
{{% /tab %}}
157162
{{% tab "Dedupe" %}}
158163

0 commit comments

Comments
 (0)