Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add pipeline name to ingest metadata #42106

Closed
peterpramb opened this issue May 12, 2019 · 4 comments · Fixed by #50467
Closed

Add pipeline name to ingest metadata #42106

peterpramb opened this issue May 12, 2019 · 4 comments · Fixed by #50467
Assignees
@martijnvg
Labels
:Data Management/Ingest Node Execution or management of Ingest Pipelines including GeoIP >enhancement

Comments

@peterpramb
Copy link

Describe the feature:

When using multiple chained pipelines it would be helpful to provide access to the currently running pipeline's name, something like _ingest.pipeline.

This can be used in error messages or for building up a list of traversed pipelines without the need to hardcode the current name into the pipeline itself (which likely will be forgotten to update when the pipeline is renamed).
@javanna javanna added the :Data Management/Ingest Node Execution or management of Ingest Pipelines including GeoIP label May 13, 2019
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-core-features

@peterpramb
Copy link
Author

peterpramb commented Jul 25, 2019 •
edited

Additionally, when an exception is propagated to a parent pipeline, the name of the pipeline where the exception originally occured would be useful as well (in the exception handler).

@jbaiera
Copy link
Member

jbaiera commented Jul 26, 2019

@peterpramb In the interest of keeping the issue scope limited, could you open a new issue for including the pipeline name when throwing an exception?

@peterpramb
Copy link
Author

peterpramb commented Jul 26, 2019 •
edited

Of course, sorry - created #44920.

@martijnvg martijnvg mentioned this issue Nov 12, 2019
Closed
12 tasks
@martijnvg martijnvg self-assigned this Dec 17, 2019
martijnvg added a commit to martijnvg/elasticsearch that referenced this issue Dec 23, 2019
@martijnvg
Add pipeline name to ingest metadata
37d789f 
This commit adds the name of the current pipeline to ingest metadata.
This pipeline name is accessible under the following key: '_ingest.pipeline'.

Example usage in pipeline:
PUT /_ingest/pipeline/2
{
    "processors": [
        {
            "set": {
                "field": "pipeline_name",
                "value": "{{_ingest.pipeline}}"
            }
        }
    ]
}

Closes elastic#42106
@martijnvg martijnvg mentioned this issue Dec 23, 2019
Merged
martijnvg added a commit that referenced this issue Jan 15, 2020
@martijnvg
Add pipeline name to ingest metadata (#50467)
2b2935f 
This commit adds the name of the current pipeline to ingest metadata.
This pipeline name is accessible under the following key: '_ingest.pipeline'.

Example usage in pipeline:
PUT /_ingest/pipeline/2
{
    "processors": [
        {
            "set": {
                "field": "pipeline_name",
                "value": "{{_ingest.pipeline}}"
            }
        }
    ]
}

Closes #42106
martijnvg added a commit to martijnvg/elasticsearch that referenced this issue Jan 15, 2020
@martijnvg
Add pipeline name to ingest metadata (elastic#50467)
ff491de 
This commit adds the name of the current pipeline to ingest metadata.
This pipeline name is accessible under the following key: '_ingest.pipeline'.

Example usage in pipeline:
PUT /_ingest/pipeline/2
{
    "processors": [
        {
            "set": {
                "field": "pipeline_name",
                "value": "{{_ingest.pipeline}}"
            }
        }
    ]
}

Closes elastic#42106
@martijnvg martijnvg mentioned this issue Jan 15, 2020
Merged
martijnvg added a commit that referenced this issue Jan 16, 2020
@martijnvg
Backport: Add pipeline name to ingest metadata (#51050)
02dfd71 
Backport: #50467

This commit adds the name of the current pipeline to ingest metadata.
This pipeline name is accessible under the following key: '_ingest.pipeline'.

Example usage in pipeline:
PUT /_ingest/pipeline/2
{
    "processors": [
        {
            "set": {
                "field": "pipeline_name",
                "value": "{{_ingest.pipeline}}"
            }
        }
    ]
}

Closes #42106
SivagurunathanV pushed a commit to SivagurunathanV/elasticsearch that referenced this issue Jan 23, 2020
@martijnvg @SivagurunathanV
Add pipeline name to ingest metadata (elastic#50467)
ee02062 
This commit adds the name of the current pipeline to ingest metadata.
This pipeline name is accessible under the following key: '_ingest.pipeline'.

Example usage in pipeline:
PUT /_ingest/pipeline/2
{
    "processors": [
        {
            "set": {
                "field": "pipeline_name",
                "value": "{{_ingest.pipeline}}"
            }
        }
    ]
}

Closes elastic#42106
@codebrain codebrain mentioned this issue Apr 1, 2020
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@martijnvg martijnvg

Labels
:Data Management/Ingest Node Execution or management of Ingest Pipelines including GeoIP >enhancement
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Add pipeline name to ingest metadata martijnvg/elasticsearch
5 participants
@martijnvg @javanna @jbaiera @peterpramb @elasticmachine