From 94808ddad989d77bd3329b0ff30596a5b02f73a4 Mon Sep 17 00:00:00 2001 From: Mike Birnstiehl Date: Wed, 12 Nov 2025 10:52:20 -0600 Subject: [PATCH 1/6] [Streams] Add note that manual pipeline config is classi streams --- .../extract/manual-pipeline-configuration.md | 4 ++++ solutions/observability/streams/streams.md | 14 +++++++------- 2 files changed, 11 insertions(+), 7 deletions(-) diff --git a/solutions/observability/streams/management/extract/manual-pipeline-configuration.md b/solutions/observability/streams/management/extract/manual-pipeline-configuration.md index 4ea782354f..2da6772a56 100644 --- a/solutions/observability/streams/management/extract/manual-pipeline-configuration.md +++ b/solutions/observability/streams/management/extract/manual-pipeline-configuration.md @@ -5,6 +5,10 @@ applies_to: --- # Manual pipeline configuration [streams-manual-pipeline-configuration] +:::{note} +The manual pipeline configuration processor is only available on [classic streams](../../streams.md#streams-classic-vs-wired). +::: + The **Manual pipeline configuration** lets you create a JSON-encoded array of ingest pipeline processors.This is helpful if you want to add more advanced processing that isn't currently available as part of the UI-based processors. Refer to the following documentation for more on manually configuring processors: diff --git a/solutions/observability/streams/streams.md b/solutions/observability/streams/streams.md index 0038f4553b..70ce11e465 100644 --- a/solutions/observability/streams/streams.md +++ b/solutions/observability/streams/streams.md @@ -8,11 +8,11 @@ applies_to: Streams provides a single, centralized UI within {{kib}} that streamlines common tasks like extracting fields, setting data retention, and routing data, so you don't need to use multiple applications or manually configure underlying {{es}} components. -## Classic vs. wired streams +## Classic vs. wired streams [streams-classic-vs-wired] Streams can operate in two modes: wired and classic. Both manage data streams in {{es}}, but differ in configuration, inheritance, and field mapping. -### Classic streams +### Classic streams [streams-classic-streams] Classic streams work with existing {{es}} data streams. Use classic streams when you want the ease of extracting fields and configuring data retention while working with data that's already being ingested into {{es}}. @@ -22,7 +22,7 @@ Classic streams: - Can follow the data retention policy set in the existing index template. - Do not support hierarchical inheritance or cascading configuration updates. -### Wired streams +### Wired streams [streams-wired-streams] ```{applies_to} stack: preview 9.2 serverless: preview @@ -37,12 +37,12 @@ Wired streams: For more information, refer to [sending data to wired streams](./wired-streams.md). -## Managed components +## Managed components [streams-managed-components] When you configure classic or wired streams through the Streams UI or [Streams API](#streams-api), {{es}}-level components like templates and pipelines are created for the stream. These components are considered *managed* and shouldn't be modified using {{es}} APIs. When managing a stream through the Streams UI or API, continue doing so whenever possible. You can still edit non-managed ingest pipelines, templates, and other components, but avoid those marked as managed or any per-data-stream mappings and settings. This behavior is similar to how Elasticsearch handles components managed by integrations. Refer to the [**Advanced** tab](./management/advanced.md) to review managed components. -## Required permissions +## Required permissions [streams-required-permissions] Streams requires the following permissions: @@ -71,7 +71,7 @@ For more information, refer to [Cluster privileges](elasticsearch://reference/el :::: -## Access Streams +## Access Streams [streams-access] Open Streams from the following places in {{kib}}: @@ -79,7 +79,7 @@ Open Streams from the following places in {{kib}}: - Open the data stream for a specific document from **Discover**. To do this, expand the details flyout for a document that's stored in a data stream, and select **Stream** or an action associated with the document's data stream. Streams will open filtered to the selected data stream. -### Streams API +### Streams API [streams-access-api] ``` yaml {applies_to} stack: preview 9.1 serverless: preview From 31f7143b1893b0a93572fed6ea86a8c0d45cae33 Mon Sep 17 00:00:00 2001 From: Mike Birnstiehl Date: Wed, 12 Nov 2025 11:00:01 -0600 Subject: [PATCH 2/6] fix link vale cleanup --- solutions/observability/streams/streams.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/solutions/observability/streams/streams.md b/solutions/observability/streams/streams.md index 70ce11e465..c7bdfc99ce 100644 --- a/solutions/observability/streams/streams.md +++ b/solutions/observability/streams/streams.md @@ -8,7 +8,7 @@ applies_to: Streams provides a single, centralized UI within {{kib}} that streamlines common tasks like extracting fields, setting data retention, and routing data, so you don't need to use multiple applications or manually configure underlying {{es}} components. -## Classic vs. wired streams [streams-classic-vs-wired] +## Classic versus wired streams [streams-classic-vs-wired] Streams can operate in two modes: wired and classic. Both manage data streams in {{es}}, but differ in configuration, inheritance, and field mapping. @@ -28,7 +28,7 @@ stack: preview 9.2 serverless: preview ``` -Wired streams data is sent directly to a single endpoint, from which you can route data into child streams based on [partitioning](./management/partitioning.md) set up manually or with the help of AI suggestions. +Wired streams send data directly to a single endpoint, from which you can route data into child streams based on [partitioning](./management/partitioning.md) set up manually or with the help of AI suggestions. Wired streams: - Allow you to organize streams in a parent-child hierarchy. @@ -48,11 +48,11 @@ Streams requires the following permissions: ::::{tab-set} -:::{tab-item} Serverless -Streams requires these Elastic Cloud Serverless roles: +:::{tab-item} {{serverless-short}} +Streams requires these {{serverless-full}} roles: - Admin: Ability to manage all Streams -- Editor/Viewer: Limited access, unable to perform all actions +- Editor/Viewer: Limited access, cannot perform all actions ::: @@ -77,9 +77,9 @@ Open Streams from the following places in {{kib}}: - Select **Streams** from the navigation menu or use the [global search field](../../../explore-analyze/find-and-organize/find-apps-and-objects.md). -- Open the data stream for a specific document from **Discover**. To do this, expand the details flyout for a document that's stored in a data stream, and select **Stream** or an action associated with the document's data stream. Streams will open filtered to the selected data stream. +- Open the data stream for a specific document from **Discover**. To do this, expand the details flyout for a document that's stored in a data stream, and select **Stream** or an action associated with the document's data stream. Streams then opens filtered to the selected data stream. -### Streams API [streams-access-api] +### Streams API [streams-api] ``` yaml {applies_to} stack: preview 9.1 serverless: preview From 61f7941e65df113a4c8dc68f438104fd6b644731 Mon Sep 17 00:00:00 2001 From: Mike Birnstiehl Date: Wed, 12 Nov 2025 12:26:25 -0600 Subject: [PATCH 3/6] fix link --- .../streams/management/extract/manual-pipeline-configuration.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/solutions/observability/streams/management/extract/manual-pipeline-configuration.md b/solutions/observability/streams/management/extract/manual-pipeline-configuration.md index 2da6772a56..8ff90650d4 100644 --- a/solutions/observability/streams/management/extract/manual-pipeline-configuration.md +++ b/solutions/observability/streams/management/extract/manual-pipeline-configuration.md @@ -15,7 +15,7 @@ Refer to the following documentation for more on manually configuring processors - [Create readable and maintainable ingest pipelines](../../../../../manage-data/ingest/transform-enrich/readable-maintainable-ingest-pipelines.md) - [Error handling in ingest pipelines](../../../../../manage-data/ingest/transform-enrich/error-handling.md) -- [Ingest processor reference][elasticsearch://reference/enrich-processor.md] +- [Ingest processor reference](elasticsearch://reference/enrich-processor.md) To manually create an array of ingest pipeline processors: From 5c6d2161e4dd9f52c46fce504065172bd3ee6012 Mon Sep 17 00:00:00 2001 From: Mike Birnstiehl <114418652+mdbirnstiehl@users.noreply.github.com> Date: Wed, 12 Nov 2025 12:37:15 -0600 Subject: [PATCH 4/6] Update solutions/observability/streams/management/extract/manual-pipeline-configuration.md Co-authored-by: Nastasha Solomon <79124755+nastasha-solomon@users.noreply.github.com> --- .../streams/management/extract/manual-pipeline-configuration.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/solutions/observability/streams/management/extract/manual-pipeline-configuration.md b/solutions/observability/streams/management/extract/manual-pipeline-configuration.md index 8ff90650d4..aa7671dd92 100644 --- a/solutions/observability/streams/management/extract/manual-pipeline-configuration.md +++ b/solutions/observability/streams/management/extract/manual-pipeline-configuration.md @@ -15,7 +15,7 @@ Refer to the following documentation for more on manually configuring processors - [Create readable and maintainable ingest pipelines](../../../../../manage-data/ingest/transform-enrich/readable-maintainable-ingest-pipelines.md) - [Error handling in ingest pipelines](../../../../../manage-data/ingest/transform-enrich/error-handling.md) -- [Ingest processor reference](elasticsearch://reference/enrich-processor.md) +- [Ingest processor reference](elasticsearch://reference/elasticsearch/enrich-processor/index.md) To manually create an array of ingest pipeline processors: From 4531fd029a5dccf23522b0c2e696b3ce0c6e5261 Mon Sep 17 00:00:00 2001 From: Mike Birnstiehl Date: Wed, 12 Nov 2025 12:40:52 -0600 Subject: [PATCH 5/6] fix link --- .../streams/management/extract/manual-pipeline-configuration.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/solutions/observability/streams/management/extract/manual-pipeline-configuration.md b/solutions/observability/streams/management/extract/manual-pipeline-configuration.md index aa7671dd92..93f7a15057 100644 --- a/solutions/observability/streams/management/extract/manual-pipeline-configuration.md +++ b/solutions/observability/streams/management/extract/manual-pipeline-configuration.md @@ -15,7 +15,7 @@ Refer to the following documentation for more on manually configuring processors - [Create readable and maintainable ingest pipelines](../../../../../manage-data/ingest/transform-enrich/readable-maintainable-ingest-pipelines.md) - [Error handling in ingest pipelines](../../../../../manage-data/ingest/transform-enrich/error-handling.md) -- [Ingest processor reference](elasticsearch://reference/elasticsearch/enrich-processor/index.md) +- [Ingest processor reference](elasticsearch://reference/elasticsearch/enrich-processor.md) To manually create an array of ingest pipeline processors: From d8c43ffbbe3e0b78974a2cb22241b268ea61d5f6 Mon Sep 17 00:00:00 2001 From: Mike Birnstiehl Date: Wed, 12 Nov 2025 12:46:20 -0600 Subject: [PATCH 6/6] REALLY fix link --- .../streams/management/extract/manual-pipeline-configuration.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/solutions/observability/streams/management/extract/manual-pipeline-configuration.md b/solutions/observability/streams/management/extract/manual-pipeline-configuration.md index 93f7a15057..55669933f2 100644 --- a/solutions/observability/streams/management/extract/manual-pipeline-configuration.md +++ b/solutions/observability/streams/management/extract/manual-pipeline-configuration.md @@ -15,7 +15,7 @@ Refer to the following documentation for more on manually configuring processors - [Create readable and maintainable ingest pipelines](../../../../../manage-data/ingest/transform-enrich/readable-maintainable-ingest-pipelines.md) - [Error handling in ingest pipelines](../../../../../manage-data/ingest/transform-enrich/error-handling.md) -- [Ingest processor reference](elasticsearch://reference/elasticsearch/enrich-processor.md) +- [Ingest processor reference](elasticsearch://reference/enrich-processor/index.md) To manually create an array of ingest pipeline processors: