Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[syslog processor] Fix 'when' conditions #36767

Merged

Conversation

andrewkroh
Copy link
Member

@andrewkroh andrewkroh commented Oct 5, 2023

Proposed commit message

Fix the ability to use when conditions with the syslog processor. Previously the config would be rejected.

Fixes #36762

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Related issues

Manual test

---

filebeat.inputs:
  - type: cel
    publisher_pipeline:
      disable_host: true
    interval: 1m
    resource:
      url: not_used_1
    redact:
      fields: ~
    program: |
      {
          'want_more': false,
          'events': [
              {
                'message': '<30>1 2023-09-28T12:10:12.175599+02:00 test.lab.com systemd 153589 - - Stopped target',
              },
              {
                'message': 'Not syslog.',
              }
          ],
      }
    processors:
      # Before:
      # {"log.level":"error","@timestamp":"2023-10-05T12:14:57.275-0400","log.logger":"input.cel","log.origin":{"file.name":"compat/compat.go","file.line":132},"message":"Input 'cel' failed with: input.go:130: input  CA8C6DD2BACAD24 failed (id= CA8C6DD2BACAD24)\n\tunexpected when option in filebeat.inputs.0.processors.0.syslog","service.name":"filebeat","id":" CA8C6DD2BACAD24","ecs.version":"1.6.0"}
      #
      # After:
      # {
      #  "@timestamp": "2023-09-28T10:10:12.175Z",
      #  "@metadata": {
      #    "beat": "filebeat",
      #    "type": "_doc",
      #    "version": "8.12.0"
      #  },
      #  "agent": {
      #    "id": "886265c2-046c-4069-a564-15f7baea5ab4",
      #    "name": "mac16-m1",
      #    "type": "filebeat",
      #    "version": "8.12.0",
      #    "ephemeral_id": "007fe335-ce32-48ea-a834-d27e2b839210"
      #  },
      #  "message": "Stopped target",
      #  "input": {
      #    "type": "cel"
      #  },
      #  "log": {
      #    "syslog": {
      #      "priority": 30,
      #      "facility": {
      #        "code": 3,
      #        "name": "system"
      #      },
      #      "severity": {
      #        "code": 6,
      #        "name": "Informational"
      #      },
      #      "appname": "systemd",
      #      "procid": "153589",
      #      "hostname": "test.lab.com",
      #      "version": "1"
      #    }
      #  },
      #  "ecs": {
      #    "version": "8.0.0"
      #  }
      #}
      #{
      #  "@timestamp": "2023-10-05T16:16:09.714Z",
      #  "@metadata": {
      #    "beat": "filebeat",
      #    "type": "_doc",
      #    "version": "8.12.0"
      #  },
      #  "message": "Not syslog.",
      #  "input": {
      #    "type": "cel"
      #  },
      #  "agent": {
      #    "ephemeral_id": "007fe335-ce32-48ea-a834-d27e2b839210",
      #    "id": "886265c2-046c-4069-a564-15f7baea5ab4",
      #    "name": "mac16-m1",
      #    "type": "filebeat",
      #    "version": "8.12.0"
      #  },
      #  "ecs": {
      #    "version": "8.0.0"
      #  }
      #}
      - syslog:
          when.regexp.message: '^<\d+>\d '
          field: message

output.console.pretty: true

Fix the ability to use `when` conditions with the
syslog processor.

Fixes elastic#36762
@elasticmachine
Copy link
Collaborator

Pinging @elastic/security-external-integrations (Team:Security-External Integrations)

@botelastic botelastic bot added needs_team Indicates that the issue/PR needs a Team:* label and removed needs_team Indicates that the issue/PR needs a Team:* label labels Oct 5, 2023
@andrewkroh andrewkroh added the Team:Elastic-Agent-Data-Plane Label for the Agent Data Plane team label Oct 5, 2023
@mergify
Copy link
Contributor

mergify bot commented Oct 5, 2023

This pull request does not have a backport label.
If this is a bug or security fix, could you label this PR @andrewkroh? 🙏.
For such, you'll need to label your PR with:

  • The upcoming major version of the Elastic Stack
  • The upcoming minor version of the Elastic Stack (if you're not pushing a breaking change)

To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

  • backport-v8./d.0 is the label to automatically backport to the 8./d branch. /d is the digit

@andrewkroh andrewkroh added backport-v8.10.0 Automated backport with mergify backport-v8.11.0 Automated backport with mergify labels Oct 5, 2023
@andrewkroh andrewkroh enabled auto-merge (squash) October 5, 2023 17:39
@elasticmachine
Copy link
Collaborator

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview preview

Expand to view the summary

Build stats

  • Duration: 80 min 19 sec

❕ Flaky test report

No test was executed to be analysed.

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

  • /test : Re-trigger the build.

  • /package : Generate the packages and run the E2E tests.

  • /beats-tester : Run the installation tests with beats-tester.

  • run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

@andrewkroh andrewkroh merged commit c9d17a7 into elastic:main Oct 5, 2023
87 checks passed
mergify bot pushed a commit that referenced this pull request Oct 5, 2023
Fix the ability to use `when` conditions with the
syslog processor.

Fixes #36762

(cherry picked from commit c9d17a7)
mergify bot pushed a commit that referenced this pull request Oct 5, 2023
Fix the ability to use `when` conditions with the
syslog processor.

Fixes #36762

(cherry picked from commit c9d17a7)
andrewkroh pushed a commit that referenced this pull request Oct 5, 2023
)

Fix the ability to use `when` conditions with the
syslog processor.

Fixes #36762

(cherry picked from commit c9d17a7)
andrewkroh added a commit that referenced this pull request Oct 5, 2023
Fix the ability to use `when` conditions with the
syslog processor.

Fixes #36762

(cherry picked from commit c9d17a7)

Co-authored-by: Andrew Kroh <andrew.kroh@elastic.co>
Scholar-Li pushed a commit to Scholar-Li/beats that referenced this pull request Feb 5, 2024
Fix the ability to use `when` conditions with the
syslog processor.

Fixes elastic#36762
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport-v8.10.0 Automated backport with mergify backport-v8.11.0 Automated backport with mergify bug :Processors Team:Elastic-Agent-Data-Plane Label for the Agent Data Plane team
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[syslog processor] Cannot use a 'when' condition
3 participants