From 30786c07c43169e4beb6cf444260eeed6220d444 Mon Sep 17 00:00:00 2001 From: David Kilfoyle Date: Wed, 30 Apr 2025 14:04:36 -0400 Subject: [PATCH 1/3] merge Elasticsearch ingest pipelines pages --- .../ingest-pipelines-serverless.md | 43 ------------------- manage-data/toc.yml | 1 - 2 files changed, 44 deletions(-) delete mode 100644 manage-data/ingest/transform-enrich/ingest-pipelines-serverless.md diff --git a/manage-data/ingest/transform-enrich/ingest-pipelines-serverless.md b/manage-data/ingest/transform-enrich/ingest-pipelines-serverless.md deleted file mode 100644 index f24f1c3d57..0000000000 --- a/manage-data/ingest/transform-enrich/ingest-pipelines-serverless.md +++ /dev/null @@ -1,43 +0,0 @@ ---- -mapped_pages: - - https://www.elastic.co/guide/en/serverless/current/ingest-pipelines.html -applies_to: - serverless: ga ---- - -# Elasticsearch ingest pipelines (Serverless) [ingest-pipelines] - -{{es}} ingest pipelines let you perform common transformations on your data before indexing. For example, you can use pipelines to remove fields, extract values from text, and enrich your data. - -A pipeline consists of a series of configurable tasks called processors. Each processor runs sequentially, making specific changes to incoming documents. After the processors have run, {{es}} adds the transformed documents to your data stream or index. - -:::{image} /manage-data/images/elasticsearch-reference-ingest-process.svg -:alt: Ingest pipeline diagram -::: - -## Create and manage pipelines [ingest-pipelines-create-and-manage-pipelines] - -In **{{project-settings}} → {{manage-app}} → {{ingest-pipelines-app}}**, you can: - -* View a list of your pipelines and drill down into details -* Edit or clone existing pipelines -* Delete pipelines - -:::{image} /manage-data/images/serverless-ingest-pipelines-management.png -:alt: {{ingest-pipelines-app}} -:screenshot: -::: - -To create a pipeline, click **Create pipeline → New pipeline**. For an example tutorial, see [Example: Parse logs](example-parse-logs.md). - -The **New pipeline from CSV** option lets you use a file with comma-separated values (CSV) to create an ingest pipeline that maps custom data to the Elastic Common Schema (ECS). Mapping your custom data to ECS makes the data easier to search and lets you reuse visualizations from other data sets. To get started, check [Map custom data to ECS](ecs://reference/ecs-converting.md). - - -## Test pipelines [ingest-pipelines-test-pipelines] - -Before you use a pipeline in production, you should test it using sample documents. When creating or editing a pipeline in **{{ingest-pipelines-app}}**, click **Add documents**. In the **Documents** tab, provide sample documents and click **Run the pipeline**: - -:::{image} /manage-data/images/serverless-ingest-pipelines-test.png -:alt: Test a pipeline in {{ingest-pipelines-app}} -:screenshot: -::: diff --git a/manage-data/toc.yml b/manage-data/toc.yml index 89b2b5eeb1..9275294ab4 100644 --- a/manage-data/toc.yml +++ b/manage-data/toc.yml @@ -95,7 +95,6 @@ toc: - file: ingest/upload-data-files.md - file: ingest/transform-enrich.md children: - - file: ingest/transform-enrich/ingest-pipelines-serverless.md - file: ingest/transform-enrich/ingest-pipelines.md children: - file: ingest/transform-enrich/example-parse-logs.md From a11c79a8c13831adf2b61015a597dc2bc2975fe4 Mon Sep 17 00:00:00 2001 From: David Kilfoyle Date: Wed, 30 Apr 2025 14:09:16 -0400 Subject: [PATCH 2/3] Remove link to removed page --- manage-data/ingest/transform-enrich/ingest-pipelines.md | 4 ---- 1 file changed, 4 deletions(-) diff --git a/manage-data/ingest/transform-enrich/ingest-pipelines.md b/manage-data/ingest/transform-enrich/ingest-pipelines.md index 4e506cf66a..006e4d9016 100644 --- a/manage-data/ingest/transform-enrich/ingest-pipelines.md +++ b/manage-data/ingest/transform-enrich/ingest-pipelines.md @@ -17,10 +17,6 @@ A pipeline consists of a series of configurable tasks called [processors](elasti You can create and manage ingest pipelines using {{kib}}'s **Ingest Pipelines** feature or the [ingest APIs](https://www.elastic.co/docs/api/doc/elasticsearch/group/endpoint-ingest). {{es}} stores pipelines in the [cluster state](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-cluster-state). -:::{note} -To run an {{es}} pipeline in {{serverless-full}}, refer to [{{es}} Ingest pipelines (Serverless)](./ingest-pipelines-serverless.md). -::: - ## Prerequisites [ingest-prerequisites] * Nodes with the [`ingest`](../../../deploy-manage/distributed-architecture/clusters-nodes-shards/node-roles.md#node-ingest-node) node role handle pipeline processing. To use ingest pipelines, your cluster must have at least one node with the `ingest` role. For heavy ingest loads, we recommend creating [dedicated ingest nodes](../../../deploy-manage/distributed-architecture/clusters-nodes-shards/node-roles.md#node-ingest-node). From 4e853fdfa4d6fc9e83e65eee455bdbab89e29c6c Mon Sep 17 00:00:00 2001 From: David Kilfoyle <41695641+kilfoyle@users.noreply.github.com> Date: Wed, 30 Apr 2025 14:20:52 -0400 Subject: [PATCH 3/3] Update ingest-pipelines.md --- manage-data/ingest/transform-enrich/ingest-pipelines.md | 1 + 1 file changed, 1 insertion(+) diff --git a/manage-data/ingest/transform-enrich/ingest-pipelines.md b/manage-data/ingest/transform-enrich/ingest-pipelines.md index 006e4d9016..89fc66d4e6 100644 --- a/manage-data/ingest/transform-enrich/ingest-pipelines.md +++ b/manage-data/ingest/transform-enrich/ingest-pipelines.md @@ -3,6 +3,7 @@ mapped_pages: - https://www.elastic.co/guide/en/elasticsearch/reference/current/ingest.html applies_to: stack: ga + serverless: ga --- # Elasticsearch ingest pipelines [ingest]