Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[sei] Add checks to avoid overriding event.original if present #8269

Merged
merged 103 commits into from
Oct 24, 2023
Merged

[sei] Add checks to avoid overriding event.original if present #8269

merged 103 commits into from
Oct 24, 2023

Conversation

marc-gr
Copy link
Contributor

@marc-gr marc-gr commented Oct 23, 2023 •
edited
Loading

Proposed commit message

Add checks to avoid overriding event.original if present.
On some setups (ie Logstash sends data to ES) event.original might be already present in the event body. Currently most of our integrations do not have a null check for it before trying to set it, leading to errors on these circumstances. By adding the required checks we improve handling in this setups.

Checklist

  • I have reviewed tips for building integrations and this pull request is aligned with them.
  • I have verified that all data streams collect metrics or logs.
  • I have added an entry to my package's changelog.yml file.
  • I have verified that Kibana version constraints are current according to guidelines.

Related issues

marc-gr added a commit to marc-gr/integrations that referenced this pull request Oct 23, 2023
@marc-gr
Improve 'event.original' check to avoid errors if set.
eaaf34b 
        [git-generate]
        #!/bin/bash

        cd /Users/marcguasch/go/src/github.com/elastic/integrations/packages/1password
        glob="./data_stream/**/elasticsearch/ingest_pipeline/default.yml"
        files=$(echo $glob | tr " " "\n")
        for file in $files; do
            if grep -q "field: event.original" $file; then
                yq e -i '(.processors | .[] | select(.rename.target_field == "event.original")) *= {"rename":{"field": "message", "target_field": "event.original", "ignore_missing": true, "if": "ctx.event?.original == null"}}' $file
                yq e -i '(.processors | .[] | select(.set.field == "event.original" and .set.copy_from != "_temp_.result._raw")) = {"rename":{"field": "message", "target_field": "event.original", "ignore_missing": true, "if": "ctx.event?.original == null"}}' $file
            fi
        done
        if ! git diff --quiet --exit-code .; then
            elastic-package changelog add --next minor --type enhancement --link elastic#8269 --description "Improve 'event.original' check to avoid errors if set."
            elastic-package check
        fi
marc-gr added a commit to marc-gr/integrations that referenced this pull request Oct 23, 2023
@marc-gr
Improve 'event.original' check to avoid errors if set.
a04bcac 
        [git-generate]
        #!/bin/bash

        cd /Users/marcguasch/go/src/github.com/elastic/integrations/packages/akamai
        glob="./data_stream/**/elasticsearch/ingest_pipeline/default.yml"
        files=$(echo $glob | tr " " "\n")
        for file in $files; do
            if grep -q "field: event.original" $file; then
                yq e -i '(.processors | .[] | select(.rename.target_field == "event.original")) *= {"rename":{"field": "message", "target_field": "event.original", "ignore_missing": true, "if": "ctx.event?.original == null"}}' $file
                yq e -i '(.processors | .[] | select(.set.field == "event.original" and .set.copy_from != "_temp_.result._raw")) = {"rename":{"field": "message", "target_field": "event.original", "ignore_missing": true, "if": "ctx.event?.original == null"}}' $file
            fi
        done
        if ! git diff --quiet --exit-code .; then
            elastic-package changelog add --next minor --type enhancement --link elastic#8269 --description "Improve 'event.original' check to avoid errors if set."
            elastic-package check
        fi
marc-gr added a commit to marc-gr/integrations that referenced this pull request Oct 23, 2023
@marc-gr
Improve 'event.original' check to avoid errors if set.
35cb08b 
        [git-generate]
        #!/bin/bash

        cd /Users/marcguasch/go/src/github.com/elastic/integrations/packages/amazon_security_lake
        glob="./data_stream/**/elasticsearch/ingest_pipeline/default.yml"
        files=$(echo $glob | tr " " "\n")
        for file in $files; do
            if grep -q "field: event.original" $file; then
                yq e -i '(.processors | .[] | select(.rename.target_field == "event.original")) *= {"rename":{"field": "message", "target_field": "event.original", "ignore_missing": true, "if": "ctx.event?.original == null"}}' $file
                yq e -i '(.processors | .[] | select(.set.field == "event.original" and .set.copy_from != "_temp_.result._raw")) = {"rename":{"field": "message", "target_field": "event.original", "ignore_missing": true, "if": "ctx.event?.original == null"}}' $file
            fi
        done
        if ! git diff --quiet --exit-code .; then
            elastic-package changelog add --next minor --type enhancement --link elastic#8269 --description "Improve 'event.original' check to avoid errors if set."
            elastic-package check
        fi
marc-gr added a commit to marc-gr/integrations that referenced this pull request Oct 23, 2023
@marc-gr
Improve 'event.original' check to avoid errors if set.
8e83321 
        [git-generate]
        #!/bin/bash

        cd /Users/marcguasch/go/src/github.com/elastic/integrations/packages/arista_ngfw
        glob="./data_stream/**/elasticsearch/ingest_pipeline/default.yml"
        files=$(echo $glob | tr " " "\n")
        for file in $files; do
            if grep -q "field: event.original" $file; then
                yq e -i '(.processors | .[] | select(.rename.target_field == "event.original")) *= {"rename":{"field": "message", "target_field": "event.original", "ignore_missing": true, "if": "ctx.event?.original == null"}}' $file
                yq e -i '(.processors | .[] | select(.set.field == "event.original" and .set.copy_from != "_temp_.result._raw")) = {"rename":{"field": "message", "target_field": "event.original", "ignore_missing": true, "if": "ctx.event?.original == null"}}' $file
            fi
        done
        if ! git diff --quiet --exit-code .; then
            elastic-package changelog add --next minor --type enhancement --link elastic#8269 --description "Improve 'event.original' check to avoid errors if set."
            elastic-package check
        fi
marc-gr added a commit to marc-gr/integrations that referenced this pull request Oct 23, 2023
@marc-gr
Improve 'event.original' check to avoid errors if set.
1730798 
        [git-generate]
        #!/bin/bash

        cd /Users/marcguasch/go/src/github.com/elastic/integrations/packages/atlassian_bitbucket
        glob="./data_stream/**/elasticsearch/ingest_pipeline/default.yml"
        files=$(echo $glob | tr " " "\n")
        for file in $files; do
            if grep -q "field: event.original" $file; then
                yq e -i '(.processors | .[] | select(.rename.target_field == "event.original")) *= {"rename":{"field": "message", "target_field": "event.original", "ignore_missing": true, "if": "ctx.event?.original == null"}}' $file
                yq e -i '(.processors | .[] | select(.set.field == "event.original" and .set.copy_from != "_temp_.result._raw")) = {"rename":{"field": "message", "target_field": "event.original", "ignore_missing": true, "if": "ctx.event?.original == null"}}' $file
            fi
        done
        if ! git diff --quiet --exit-code .; then
            elastic-package changelog add --next minor --type enhancement --link elastic#8269 --description "Improve 'event.original' check to avoid errors if set."
            elastic-package check
        fi
marc-gr added a commit to marc-gr/integrations that referenced this pull request Oct 23, 2023
@marc-gr
Improve 'event.original' check to avoid errors if set.
cb8db33 
        [git-generate]
        #!/bin/bash

        cd /Users/marcguasch/go/src/github.com/elastic/integrations/packages/atlassian_confluence
        glob="./data_stream/**/elasticsearch/ingest_pipeline/default.yml"
        files=$(echo $glob | tr " " "\n")
        for file in $files; do
            if grep -q "field: event.original" $file; then
                yq e -i '(.processors | .[] | select(.rename.target_field == "event.original")) *= {"rename":{"field": "message", "target_field": "event.original", "ignore_missing": true, "if": "ctx.event?.original == null"}}' $file
                yq e -i '(.processors | .[] | select(.set.field == "event.original" and .set.copy_from != "_temp_.result._raw")) = {"rename":{"field": "message", "target_field": "event.original", "ignore_missing": true, "if": "ctx.event?.original == null"}}' $file
            fi
        done
        if ! git diff --quiet --exit-code .; then
            elastic-package changelog add --next minor --type enhancement --link elastic#8269 --description "Improve 'event.original' check to avoid errors if set."
            elastic-package check
        fi
marc-gr added a commit to marc-gr/integrations that referenced this pull request Oct 23, 2023
@marc-gr
Improve 'event.original' check to avoid errors if set.
3b2e386 
        [git-generate]
        #!/bin/bash

        cd /Users/marcguasch/go/src/github.com/elastic/integrations/packages/atlassian_jira
        glob="./data_stream/**/elasticsearch/ingest_pipeline/default.yml"
        files=$(echo $glob | tr " " "\n")
        for file in $files; do
            if grep -q "field: event.original" $file; then
                yq e -i '(.processors | .[] | select(.rename.target_field == "event.original")) *= {"rename":{"field": "message", "target_field": "event.original", "ignore_missing": true, "if": "ctx.event?.original == null"}}' $file
                yq e -i '(.processors | .[] | select(.set.field == "event.original" and .set.copy_from != "_temp_.result._raw")) = {"rename":{"field": "message", "target_field": "event.original", "ignore_missing": true, "if": "ctx.event?.original == null"}}' $file
            fi
        done
        if ! git diff --quiet --exit-code .; then
            elastic-package changelog add --next minor --type enhancement --link elastic#8269 --description "Improve 'event.original' check to avoid errors if set."
            elastic-package check
        fi
marc-gr added a commit to marc-gr/integrations that referenced this pull request Oct 23, 2023
@marc-gr
Improve 'event.original' check to avoid errors if set.
119e571 
        [git-generate]
        #!/bin/bash

        cd /Users/marcguasch/go/src/github.com/elastic/integrations/packages/auditd
        glob="./data_stream/**/elasticsearch/ingest_pipeline/default.yml"
        files=$(echo $glob | tr " " "\n")
        for file in $files; do
            if grep -q "field: event.original" $file; then
                yq e -i '(.processors | .[] | select(.rename.target_field == "event.original")) *= {"rename":{"field": "message", "target_field": "event.original", "ignore_missing": true, "if": "ctx.event?.original == null"}}' $file
                yq e -i '(.processors | .[] | select(.set.field == "event.original" and .set.copy_from != "_temp_.result._raw")) = {"rename":{"field": "message", "target_field": "event.original", "ignore_missing": true, "if": "ctx.event?.original == null"}}' $file
            fi
        done
        if ! git diff --quiet --exit-code .; then
            elastic-package changelog add --next minor --type enhancement --link elastic#8269 --description "Improve 'event.original' check to avoid errors if set."
            elastic-package check
        fi
marc-gr added a commit to marc-gr/integrations that referenced this pull request Oct 23, 2023
@marc-gr
Improve 'event.original' check to avoid errors if set.
c7a3250 
        [git-generate]
        #!/bin/bash

        cd /Users/marcguasch/go/src/github.com/elastic/integrations/packages/auth0
        glob="./data_stream/**/elasticsearch/ingest_pipeline/default.yml"
        files=$(echo $glob | tr " " "\n")
        for file in $files; do
            if grep -q "field: event.original" $file; then
                yq e -i '(.processors | .[] | select(.rename.target_field == "event.original")) *= {"rename":{"field": "message", "target_field": "event.original", "ignore_missing": true, "if": "ctx.event?.original == null"}}' $file
                yq e -i '(.processors | .[] | select(.set.field == "event.original" and .set.copy_from != "_temp_.result._raw")) = {"rename":{"field": "message", "target_field": "event.original", "ignore_missing": true, "if": "ctx.event?.original == null"}}' $file
            fi
        done
        if ! git diff --quiet --exit-code .; then
            elastic-package changelog add --next minor --type enhancement --link elastic#8269 --description "Improve 'event.original' check to avoid errors if set."
            elastic-package check
        fi
marc-gr added a commit to marc-gr/integrations that referenced this pull request Oct 23, 2023
@marc-gr
Improve 'event.original' check to avoid errors if set.
0d29a8b 
        [git-generate]
        #!/bin/bash

        cd /Users/marcguasch/go/src/github.com/elastic/integrations/packages/azure_frontdoor
        glob="./data_stream/**/elasticsearch/ingest_pipeline/default.yml"
        files=$(echo $glob | tr " " "\n")
        for file in $files; do
            if grep -q "field: event.original" $file; then
                yq e -i '(.processors | .[] | select(.rename.target_field == "event.original")) *= {"rename":{"field": "message", "target_field": "event.original", "ignore_missing": true, "if": "ctx.event?.original == null"}}' $file
                yq e -i '(.processors | .[] | select(.set.field == "event.original" and .set.copy_from != "_temp_.result._raw")) = {"rename":{"field": "message", "target_field": "event.original", "ignore_missing": true, "if": "ctx.event?.original == null"}}' $file
            fi
        done
        if ! git diff --quiet --exit-code .; then
            elastic-package changelog add --next minor --type enhancement --link elastic#8269 --description "Improve 'event.original' check to avoid errors if set."
            elastic-package check
        fi
marc-gr added a commit to marc-gr/integrations that referenced this pull request Oct 23, 2023
@marc-gr
Improve 'event.original' check to avoid errors if set.
b4ef5ee 
        [git-generate]
        #!/bin/bash

        cd /Users/marcguasch/go/src/github.com/elastic/integrations/packages/barracuda
        glob="./data_stream/**/elasticsearch/ingest_pipeline/default.yml"
        files=$(echo $glob | tr " " "\n")
        for file in $files; do
            if grep -q "field: event.original" $file; then
                yq e -i '(.processors | .[] | select(.rename.target_field == "event.original")) *= {"rename":{"field": "message", "target_field": "event.original", "ignore_missing": true, "if": "ctx.event?.original == null"}}' $file
                yq e -i '(.processors | .[] | select(.set.field == "event.original" and .set.copy_from != "_temp_.result._raw")) = {"rename":{"field": "message", "target_field": "event.original", "ignore_missing": true, "if": "ctx.event?.original == null"}}' $file
            fi
        done
        if ! git diff --quiet --exit-code .; then
            elastic-package changelog add --next minor --type enhancement --link elastic#8269 --description "Improve 'event.original' check to avoid errors if set."
            elastic-package check
        fi
marc-gr added a commit to marc-gr/integrations that referenced this pull request Oct 23, 2023
@marc-gr
Improve 'event.original' check to avoid errors if set.
c1a68ea 
        [git-generate]
        #!/bin/bash

        cd /Users/marcguasch/go/src/github.com/elastic/integrations/packages/barracuda_cloudgen_firewall
        glob="./data_stream/**/elasticsearch/ingest_pipeline/default.yml"
        files=$(echo $glob | tr " " "\n")
        for file in $files; do
            if grep -q "field: event.original" $file; then
                yq e -i '(.processors | .[] | select(.rename.target_field == "event.original")) *= {"rename":{"field": "message", "target_field": "event.original", "ignore_missing": true, "if": "ctx.event?.original == null"}}' $file
                yq e -i '(.processors | .[] | select(.set.field == "event.original" and .set.copy_from != "_temp_.result._raw")) = {"rename":{"field": "message", "target_field": "event.original", "ignore_missing": true, "if": "ctx.event?.original == null"}}' $file
            fi
        done
        if ! git diff --quiet --exit-code .; then
            elastic-package changelog add --next minor --type enhancement --link elastic#8269 --description "Improve 'event.original' check to avoid errors if set."
            elastic-package check
        fi
marc-gr added a commit to marc-gr/integrations that referenced this pull request Oct 23, 2023
@marc-gr
Improve 'event.original' check to avoid errors if set.
647f460 
        [git-generate]
        #!/bin/bash

        cd /Users/marcguasch/go/src/github.com/elastic/integrations/packages/bitdefender
        glob="./data_stream/**/elasticsearch/ingest_pipeline/default.yml"
        files=$(echo $glob | tr " " "\n")
        for file in $files; do
            if grep -q "field: event.original" $file; then
                yq e -i '(.processors | .[] | select(.rename.target_field == "event.original")) *= {"rename":{"field": "message", "target_field": "event.original", "ignore_missing": true, "if": "ctx.event?.original == null"}}' $file
                yq e -i '(.processors | .[] | select(.set.field == "event.original" and .set.copy_from != "_temp_.result._raw")) = {"rename":{"field": "message", "target_field": "event.original", "ignore_missing": true, "if": "ctx.event?.original == null"}}' $file
            fi
        done
        if ! git diff --quiet --exit-code .; then
            elastic-package changelog add --next minor --type enhancement --link elastic#8269 --description "Improve 'event.original' check to avoid errors if set."
            elastic-package check
        fi
marc-gr added a commit to marc-gr/integrations that referenced this pull request Oct 23, 2023
@marc-gr
Improve 'event.original' check to avoid errors if set.
33771f5 
        [git-generate]
        #!/bin/bash

        cd /Users/marcguasch/go/src/github.com/elastic/integrations/packages/bitwarden
        glob="./data_stream/**/elasticsearch/ingest_pipeline/default.yml"
        files=$(echo $glob | tr " " "\n")
        for file in $files; do
            if grep -q "field: event.original" $file; then
                yq e -i '(.processors | .[] | select(.rename.target_field == "event.original")) *= {"rename":{"field": "message", "target_field": "event.original", "ignore_missing": true, "if": "ctx.event?.original == null"}}' $file
                yq e -i '(.processors | .[] | select(.set.field == "event.original" and .set.copy_from != "_temp_.result._raw")) = {"rename":{"field": "message", "target_field": "event.original", "ignore_missing": true, "if": "ctx.event?.original == null"}}' $file
            fi
        done
        if ! git diff --quiet --exit-code .; then
            elastic-package changelog add --next minor --type enhancement --link elastic#8269 --description "Improve 'event.original' check to avoid errors if set."
            elastic-package check
        fi
marc-gr added a commit to marc-gr/integrations that referenced this pull request Oct 23, 2023
@marc-gr
Improve 'event.original' check to avoid errors if set.
a0f5936 
        [git-generate]
        #!/bin/bash

        cd /Users/marcguasch/go/src/github.com/elastic/integrations/packages/bluecoat
        glob="./data_stream/**/elasticsearch/ingest_pipeline/default.yml"
        files=$(echo $glob | tr " " "\n")
        for file in $files; do
            if grep -q "field: event.original" $file; then
                yq e -i '(.processors | .[] | select(.rename.target_field == "event.original")) *= {"rename":{"field": "message", "target_field": "event.original", "ignore_missing": true, "if": "ctx.event?.original == null"}}' $file
                yq e -i '(.processors | .[] | select(.set.field == "event.original" and .set.copy_from != "_temp_.result._raw")) = {"rename":{"field": "message", "target_field": "event.original", "ignore_missing": true, "if": "ctx.event?.original == null"}}' $file
            fi
        done
        if ! git diff --quiet --exit-code .; then
            elastic-package changelog add --next minor --type enhancement --link elastic#8269 --description "Improve 'event.original' check to avoid errors if set."
            elastic-package check
        fi
marc-gr added a commit to marc-gr/integrations that referenced this pull request Oct 23, 2023
@marc-gr
Improve 'event.original' check to avoid errors if set.
200782a 
        [git-generate]
        #!/bin/bash

        cd /Users/marcguasch/go/src/github.com/elastic/integrations/packages/box_events
        glob="./data_stream/**/elasticsearch/ingest_pipeline/default.yml"
        files=$(echo $glob | tr " " "\n")
        for file in $files; do
            if grep -q "field: event.original" $file; then
                yq e -i '(.processors | .[] | select(.rename.target_field == "event.original")) *= {"rename":{"field": "message", "target_field": "event.original", "ignore_missing": true, "if": "ctx.event?.original == null"}}' $file
                yq e -i '(.processors | .[] | select(.set.field == "event.original" and .set.copy_from != "_temp_.result._raw")) = {"rename":{"field": "message", "target_field": "event.original", "ignore_missing": true, "if": "ctx.event?.original == null"}}' $file
            fi
        done
        if ! git diff --quiet --exit-code .; then
            elastic-package changelog add --next minor --type enhancement --link elastic#8269 --description "Improve 'event.original' check to avoid errors if set."
            elastic-package check
        fi
marc-gr added a commit to marc-gr/integrations that referenced this pull request Oct 23, 2023
@marc-gr
Improve 'event.original' check to avoid errors if set.
dc89c52 
        [git-generate]
        #!/bin/bash

        cd /Users/marcguasch/go/src/github.com/elastic/integrations/packages/carbon_black_cloud
        glob="./data_stream/**/elasticsearch/ingest_pipeline/default.yml"
        files=$(echo $glob | tr " " "\n")
        for file in $files; do
            if grep -q "field: event.original" $file; then
                yq e -i '(.processors | .[] | select(.rename.target_field == "event.original")) *= {"rename":{"field": "message", "target_field": "event.original", "ignore_missing": true, "if": "ctx.event?.original == null"}}' $file
                yq e -i '(.processors | .[] | select(.set.field == "event.original" and .set.copy_from != "_temp_.result._raw")) = {"rename":{"field": "message", "target_field": "event.original", "ignore_missing": true, "if": "ctx.event?.original == null"}}' $file
            fi
        done
        if ! git diff --quiet --exit-code .; then
            elastic-package changelog add --next minor --type enhancement --link elastic#8269 --description "Improve 'event.original' check to avoid errors if set."
            elastic-package check
        fi
marc-gr added a commit to marc-gr/integrations that referenced this pull request Oct 23, 2023
@marc-gr
Improve 'event.original' check to avoid errors if set.
a771bc4 
        [git-generate]
        #!/bin/bash

        cd /Users/marcguasch/go/src/github.com/elastic/integrations/packages/cef
        glob="./data_stream/**/elasticsearch/ingest_pipeline/default.yml"
        files=$(echo $glob | tr " " "\n")
        for file in $files; do
            if grep -q "field: event.original" $file; then
                yq e -i '(.processors | .[] | select(.rename.target_field == "event.original")) *= {"rename":{"field": "message", "target_field": "event.original", "ignore_missing": true, "if": "ctx.event?.original == null"}}' $file
                yq e -i '(.processors | .[] | select(.set.field == "event.original" and .set.copy_from != "_temp_.result._raw")) = {"rename":{"field": "message", "target_field": "event.original", "ignore_missing": true, "if": "ctx.event?.original == null"}}' $file
            fi
        done
        if ! git diff --quiet --exit-code .; then
            elastic-package changelog add --next minor --type enhancement --link elastic#8269 --description "Improve 'event.original' check to avoid errors if set."
            elastic-package check
        fi
marc-gr added a commit to marc-gr/integrations that referenced this pull request Oct 23, 2023
@marc-gr
Improve 'event.original' check to avoid errors if set.
76a6148 
        [git-generate]
        #!/bin/bash

        cd /Users/marcguasch/go/src/github.com/elastic/integrations/packages/checkpoint
        glob="./data_stream/**/elasticsearch/ingest_pipeline/default.yml"
        files=$(echo $glob | tr " " "\n")
        for file in $files; do
            if grep -q "field: event.original" $file; then
                yq e -i '(.processors | .[] | select(.rename.target_field == "event.original")) *= {"rename":{"field": "message", "target_field": "event.original", "ignore_missing": true, "if": "ctx.event?.original == null"}}' $file
                yq e -i '(.processors | .[] | select(.set.field == "event.original" and .set.copy_from != "_temp_.result._raw")) = {"rename":{"field": "message", "target_field": "event.original", "ignore_missing": true, "if": "ctx.event?.original == null"}}' $file
            fi
        done
        if ! git diff --quiet --exit-code .; then
            elastic-package changelog add --next minor --type enhancement --link elastic#8269 --description "Improve 'event.original' check to avoid errors if set."
            elastic-package check
        fi
marc-gr added a commit to marc-gr/integrations that referenced this pull request Oct 23, 2023
@marc-gr
Improve 'event.original' check to avoid errors if set.
089e44d 
        [git-generate]
        #!/bin/bash

        cd /Users/marcguasch/go/src/github.com/elastic/integrations/packages/cisco_aironet
        glob="./data_stream/**/elasticsearch/ingest_pipeline/default.yml"
        files=$(echo $glob | tr " " "\n")
        for file in $files; do
            if grep -q "field: event.original" $file; then
                yq e -i '(.processors | .[] | select(.rename.target_field == "event.original")) *= {"rename":{"field": "message", "target_field": "event.original", "ignore_missing": true, "if": "ctx.event?.original == null"}}' $file
                yq e -i '(.processors | .[] | select(.set.field == "event.original" and .set.copy_from != "_temp_.result._raw")) = {"rename":{"field": "message", "target_field": "event.original", "ignore_missing": true, "if": "ctx.event?.original == null"}}' $file
            fi
        done
        if ! git diff --quiet --exit-code .; then
            elastic-package changelog add --next minor --type enhancement --link elastic#8269 --description "Improve 'event.original' check to avoid errors if set."
            elastic-package check
        fi
marc-gr added a commit to marc-gr/integrations that referenced this pull request Oct 23, 2023
@marc-gr
Improve 'event.original' check to avoid errors if set.
e7a5ed4 
        [git-generate]
        #!/bin/bash

        cd /Users/marcguasch/go/src/github.com/elastic/integrations/packages/cisco_asa
        glob="./data_stream/**/elasticsearch/ingest_pipeline/default.yml"
        files=$(echo $glob | tr " " "\n")
        for file in $files; do
            if grep -q "field: event.original" $file; then
                yq e -i '(.processors | .[] | select(.rename.target_field == "event.original")) *= {"rename":{"field": "message", "target_field": "event.original", "ignore_missing": true, "if": "ctx.event?.original == null"}}' $file
                yq e -i '(.processors | .[] | select(.set.field == "event.original" and .set.copy_from != "_temp_.result._raw")) = {"rename":{"field": "message", "target_field": "event.original", "ignore_missing": true, "if": "ctx.event?.original == null"}}' $file
            fi
        done
        if ! git diff --quiet --exit-code .; then
            elastic-package changelog add --next minor --type enhancement --link elastic#8269 --description "Improve 'event.original' check to avoid errors if set."
            elastic-package check
        fi
marc-gr added a commit to marc-gr/integrations that referenced this pull request Oct 23, 2023
@marc-gr
Improve 'event.original' check to avoid errors if set.
d7fcee7 
        [git-generate]
        #!/bin/bash

        cd /Users/marcguasch/go/src/github.com/elastic/integrations/packages/cisco_duo
        glob="./data_stream/**/elasticsearch/ingest_pipeline/default.yml"
        files=$(echo $glob | tr " " "\n")
        for file in $files; do
            if grep -q "field: event.original" $file; then
                yq e -i '(.processors | .[] | select(.rename.target_field == "event.original")) *= {"rename":{"field": "message", "target_field": "event.original", "ignore_missing": true, "if": "ctx.event?.original == null"}}' $file
                yq e -i '(.processors | .[] | select(.set.field == "event.original" and .set.copy_from != "_temp_.result._raw")) = {"rename":{"field": "message", "target_field": "event.original", "ignore_missing": true, "if": "ctx.event?.original == null"}}' $file
            fi
        done
        if ! git diff --quiet --exit-code .; then
            elastic-package changelog add --next minor --type enhancement --link elastic#8269 --description "Improve 'event.original' check to avoid errors if set."
            elastic-package check
        fi
marc-gr added a commit to marc-gr/integrations that referenced this pull request Oct 23, 2023
@marc-gr
Improve 'event.original' check to avoid errors if set.
1e16065 
        [git-generate]
        #!/bin/bash

        cd /Users/marcguasch/go/src/github.com/elastic/integrations/packages/cisco_ftd
        glob="./data_stream/**/elasticsearch/ingest_pipeline/default.yml"
        files=$(echo $glob | tr " " "\n")
        for file in $files; do
            if grep -q "field: event.original" $file; then
                yq e -i '(.processors | .[] | select(.rename.target_field == "event.original")) *= {"rename":{"field": "message", "target_field": "event.original", "ignore_missing": true, "if": "ctx.event?.original == null"}}' $file
                yq e -i '(.processors | .[] | select(.set.field == "event.original" and .set.copy_from != "_temp_.result._raw")) = {"rename":{"field": "message", "target_field": "event.original", "ignore_missing": true, "if": "ctx.event?.original == null"}}' $file
            fi
        done
        if ! git diff --quiet --exit-code .; then
            elastic-package changelog add --next minor --type enhancement --link elastic#8269 --description "Improve 'event.original' check to avoid errors if set."
            elastic-package check
        fi
marc-gr added a commit to marc-gr/integrations that referenced this pull request Oct 23, 2023
@marc-gr
Improve 'event.original' check to avoid errors if set.
e1f5a08 
        [git-generate]
        #!/bin/bash

        cd /Users/marcguasch/go/src/github.com/elastic/integrations/packages/cisco_ios
        glob="./data_stream/**/elasticsearch/ingest_pipeline/default.yml"
        files=$(echo $glob | tr " " "\n")
        for file in $files; do
            if grep -q "field: event.original" $file; then
                yq e -i '(.processors | .[] | select(.rename.target_field == "event.original")) *= {"rename":{"field": "message", "target_field": "event.original", "ignore_missing": true, "if": "ctx.event?.original == null"}}' $file
                yq e -i '(.processors | .[] | select(.set.field == "event.original" and .set.copy_from != "_temp_.result._raw")) = {"rename":{"field": "message", "target_field": "event.original", "ignore_missing": true, "if": "ctx.event?.original == null"}}' $file
            fi
        done
        if ! git diff --quiet --exit-code .; then
            elastic-package changelog add --next minor --type enhancement --link elastic#8269 --description "Improve 'event.original' check to avoid errors if set."
            elastic-package check
        fi
marc-gr added a commit to marc-gr/integrations that referenced this pull request Oct 23, 2023
@marc-gr
Improve 'event.original' check to avoid errors if set.
ce6220e 
        [git-generate]
        #!/bin/bash

        cd /Users/marcguasch/go/src/github.com/elastic/integrations/packages/cisco_ise
        glob="./data_stream/**/elasticsearch/ingest_pipeline/default.yml"
        files=$(echo $glob | tr " " "\n")
        for file in $files; do
            if grep -q "field: event.original" $file; then
                yq e -i '(.processors | .[] | select(.rename.target_field == "event.original")) *= {"rename":{"field": "message", "target_field": "event.original", "ignore_missing": true, "if": "ctx.event?.original == null"}}' $file
                yq e -i '(.processors | .[] | select(.set.field == "event.original" and .set.copy_from != "_temp_.result._raw")) = {"rename":{"field": "message", "target_field": "event.original", "ignore_missing": true, "if": "ctx.event?.original == null"}}' $file
            fi
        done
        if ! git diff --quiet --exit-code .; then
            elastic-package changelog add --next minor --type enhancement --link elastic#8269 --description "Improve 'event.original' check to avoid errors if set."
            elastic-package check
        fi
marc-gr added a commit to marc-gr/integrations that referenced this pull request Oct 23, 2023
@marc-gr
Improve 'event.original' check to avoid errors if set.
f4c31b1 
        [git-generate]
        #!/bin/bash

        cd /Users/marcguasch/go/src/github.com/elastic/integrations/packages/cisco_meraki
        glob="./data_stream/**/elasticsearch/ingest_pipeline/default.yml"
        files=$(echo $glob | tr " " "\n")
        for file in $files; do
            if grep -q "field: event.original" $file; then
                yq e -i '(.processors | .[] | select(.rename.target_field == "event.original")) *= {"rename":{"field": "message", "target_field": "event.original", "ignore_missing": true, "if": "ctx.event?.original == null"}}' $file
                yq e -i '(.processors | .[] | select(.set.field == "event.original" and .set.copy_from != "_temp_.result._raw")) = {"rename":{"field": "message", "target_field": "event.original", "ignore_missing": true, "if": "ctx.event?.original == null"}}' $file
            fi
        done
        if ! git diff --quiet --exit-code .; then
            elastic-package changelog add --next minor --type enhancement --link elastic#8269 --description "Improve 'event.original' check to avoid errors if set."
            elastic-package check
        fi
marc-gr added a commit to marc-gr/integrations that referenced this pull request Oct 23, 2023
@marc-gr
Improve 'event.original' check to avoid errors if set.
894931e 
        [git-generate]
        #!/bin/bash

        cd /Users/marcguasch/go/src/github.com/elastic/integrations/packages/cisco_nexus
        glob="./data_stream/**/elasticsearch/ingest_pipeline/default.yml"
        files=$(echo $glob | tr " " "\n")
        for file in $files; do
            if grep -q "field: event.original" $file; then
                yq e -i '(.processors | .[] | select(.rename.target_field == "event.original")) *= {"rename":{"field": "message", "target_field": "event.original", "ignore_missing": true, "if": "ctx.event?.original == null"}}' $file
                yq e -i '(.processors | .[] | select(.set.field == "event.original" and .set.copy_from != "_temp_.result._raw")) = {"rename":{"field": "message", "target_field": "event.original", "ignore_missing": true, "if": "ctx.event?.original == null"}}' $file
            fi
        done
        if ! git diff --quiet --exit-code .; then
            elastic-package changelog add --next minor --type enhancement --link elastic#8269 --description "Improve 'event.original' check to avoid errors if set."
            elastic-package check
        fi
@elasticmachine
Copy link

Package vectra_detect - 1.5.0 containing this change is available at https://epr.elastic.co/search?package=vectra_detect

@elasticmachine
Copy link

Package zeek - 2.20.0 containing this change is available at https://epr.elastic.co/search?package=zeek

@elasticmachine
Copy link

Package zerofox - 1.20.0 containing this change is available at https://epr.elastic.co/search?package=zerofox

@elasticmachine
Copy link

Package zeronetworks - 1.9.0 containing this change is available at https://epr.elastic.co/search?package=zeronetworks

@elasticmachine
Copy link

Package zscaler_zia - 2.16.0 containing this change is available at https://epr.elastic.co/search?package=zscaler_zia

@elasticmachine
Copy link

Package zscaler_zpa - 1.15.0 containing this change is available at https://epr.elastic.co/search?package=zscaler_zpa

@janvi-elastic janvi-elastic mentioned this pull request Oct 25, 2023
Merged
15 tasks
@taylor-swanson taylor-swanson mentioned this pull request Mar 4, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@kcreddy kcreddy kcreddy approved these changes

Assignees
No one assigned
Labels
enhancement New feature or request Integration:akamai Akamai Integration:amazon_security_lake Amazon Security Lake Integration:arista_ngfw Arista NG Firewall Integration:atlassian_bitbucket Atlassian Bitbucket Integration:atlassian_confluence Atlassian Confluence Integration:atlassian_jira Atlassian Jira Integration:auditd Auditd Logs Integration:barracuda_cloudgen_firewall Barracuda CloudGen Firewall Logs Integration:barracuda Barracuda Web Application Firewall Integration:bitdefender BitDefender Integration:bitwarden Bitwarden Integration:box_events Box Events Integration:carbon_black_cloud VMware Carbon Black Cloud Integration:checkpoint Check Point Integration:cisco_aironet Cisco Aironet Integration:cisco_asa Cisco ASA Integration:cisco_duo Cisco Duo Integration:cisco_ftd Cisco FTD Integration:cisco_ise Cisco ISE Integration:cisco_meraki Cisco Meraki Integration:cisco_nexus Cisco Nexus Integration:1password 1Password
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[SEI] Add check for event.original rename/set to avoid issues with Logstash events
5 participants
@marc-gr @elasticmachine @efd6 @kcreddy @andrewkroh