x-pack/filebeat/input/cel: improve user docs (#34831)

* include docs for state, program and regexp
* move state, program and regexp docs up
* clarify cursor number requirements

(cherry picked from commit f20ce9d)

# Conflicts:
#	x-pack/filebeat/docs/inputs/input-cel.asciidoc

CHANGELOG.next.asciidoc

@@ -154,6 +154,34 @@ automatic splitting at root level, if root level element is an array. {pull}3415
 - Add cloudflare R2 to provider list in AWS S3 input. {pull}32620[32620]
 - Add support for single string containing multiple relation-types in getRFC5988Link. {pull}32811[32811]
 - Add backup to bucket and delete functionality for the `aws-s3` input. {issue}30696[30696] {pull}33559[33559]
+- Add metrics for UDP packet processing. {pull}33870[33870]
+- Convert UDP input to v2 input. {pull}33930[33930]
+- Improve collection of risk information from Okta debug data. {issue}33677[33677] {pull}34030[34030]
+- Adding filename details from zip to response for httpjson {issue}33952[33952] {pull}34044[34044]
+- Allow user configuration of keep-alive behaviour for HTTPJSON and CEL inputs. {issue}33951[33951] {pull}34014[34014]
+- Add support for polling system UDP stats for UDP input metrics. {pull}34070[34070]
+- Add support for recognizing the log level in Elasticsearch JVM logs {pull}34159[34159]
+- Add new Entity Analytics input with Azure Active Directory support. {pull}34305[34305]
+- Added metric `sqs_lag_time` for aws-s3 input. {pull}34306[34306]
+- Add metrics for TCP packet processing. {pull}34333[34333]
+- Add metrics for unix socket packet processing. {pull}34335[34335]
+- Add beta `take over` mode for `filestream` for simple migration from `log` inputs {pull}34292[34292]
+- Add pagination support for Salesforce module. {issue}34057[34057] {pull}34065[34065]
+- Allow users to redact sensitive data from CEL input debug logs. {pull}34302[34302]
+- Added support for HTTP destination override to Google Cloud Storage input. {pull}34413[34413]
+- Added metric `sqs_messages_waiting_gauge` for aws-s3 input. {pull}34488[34488]
+- Add support for new Rabbitmq timestamp format for logs {pull}34211[34211]
+- Allow user configuration of timezone offset in Cisco ASA and FTD modules. {pull}34436[34436]
+- Allow user configuration of timezone offset in Checkpoint module. {pull}34472[34472]
+- Add support for Okta debug attributes, `risk_reasons`, `risk_behaviors` and `factor`. {issue}33677[33677] {pull}34508[34508]
+- Fill okta.request.ip_chain.* as a flattened object in Okta module. {pull}34621[34621]
+- Fixed GCS log format issues. {pull}34659[34659]
+- Add nginx.ingress_controller.upstream.ip to related.ip {issue}34645[34645] {pull}34672[34672]
+- Include NAT and firewall IPs in `related.ip` in Fortinet Firewall module. {issue}34640[34640] {pull}34673[34673]
+- Add Basic Authentication support on constructed requests to CEL input {issue}34609[34609] {pull}34689[34689]
+- Add string manipulation extensions to CEL input {issue}34610[34610] {pull}34689[34689]
+- Add unix socket log parsing for nginx ingress_controller {pull}34732[34732]
+- Improve CEL input documentation {pull}34831[34831]
 
 *Auditbeat*
 

x-pack/filebeat/docs/inputs/input-cel.asciidoc

@@ -263,6 +263,59 @@ The `cel` input supports the following configuration options plus the
 
 Duration between repeated requests. It may make additional pagination requests in response to the initial request if pagination is enabled. Default: `60s`.
 
+[[program-cel]]
+[float]
+=== `program`
+
+The CEL program that is executed each polling period. This field is required.
+
+[[state-cel]]
+[float]
+==== `state`
+
+`state` is an optional object that is passed to the CEL program on the first execution. It is available to the executing program as the `state` variable. It is made available to subsequent executions of the program during the life of input as the returned value of the previous execution, but with the `state.events` field removed. Except for the `state.cursor` field, `state` does not persist over restarts.
+
+[[cursor-cel]]
+[float]
+==== `state.cursor`
+
+The cursor is an object available as `state.cursor` where arbitrary values may be stored. Cursor state is kept between input restarts and updated after each event of a request has been published. When a cursor is used the CEL program must either create a cursor state for each event that is returned by the program, or a single cursor that reflect the cursor for completion of the full set of events.
+
+["source","yaml",subs="attributes"]
+----
+filebeat.inputs:
+# Fetch your public IP every minute and note when the last request was made.
+- type: cel
+  interval: 1m
+  resource.url: https://api.ipify.org/?format=json
+  program: |
+    bytes(get(state.url).Body).as(body, {
+        "events": [body.decode_json().with({
+            "last_requested_at": has(state.cursor) && has(state.cursor.last_requested_at) ?
+                state.cursor.last_requested_at
+            :
+                now
+        })],
+        "cursor": {"last_requested_at": now}
+    })
+----
+
+[[regexp-cel]]
+[float]
+=== `regexp`
+
+A set of named regular expressions that may be used during a CEL program's execution using the `regexp` extension library. The syntax used for the regular expressions is https://github.com/google/re2/wiki/Syntax[RE2].
+
+["source","yaml",subs="attributes"]
+----
+filebeat.inputs:
+- type: cel
+  # Define two regular expressions, 'products' and 'solutions' for use during CEL execution.
+  regexp:
+    products: '(?i)(Elasticsearch|Beats|Logstash|Kibana)'
+    solutions: '(?i)(Search|Observability|Security)'
+----
+
 [float]
 ==== `auth.basic.enabled`
 
@@ -529,6 +582,7 @@ Whether to use the host's local time rather that UTC for timestamping rotated lo
 
 This determines whether rotated logs should be gzip compressed.
 
+<<<<<<< HEAD
 [[cursor-cel]]
 [float]
 ==== `cursor`
@@ -553,6 +607,17 @@ filebeat.inputs:
         "cursor": {"last_requested_at": now}
     })
 ----
+=======
+[float]
+==== `redact.fields`
+
+This specifies fields in the `state` to be redacted prior to debug logging. Fields listed in this array will be either replaced with a `*` or deleted entirely from messages sent to debug logs.
+
+[float]
+==== `redact.delete`
+
+This specifies whether fields should be replaced with a `*` or deleted entirely from messages sent to debug logs. If delete is `true`, fields will be deleted rather than replaced.
+>>>>>>> f20ce9d599 (x-pack/filebeat/input/cel: improve user docs (#34831))
 
 [id="{beatname_lc}-input-{type}-common-options"]
 include::../../../../filebeat/docs/inputs/input-common-options.asciidoc[]