Fix adding multiple processors in cloudfront logs and syslog

[kaiyan-sheng]

What does this PR do?

This PR is to fix #4395 and #4396 by considering adding multiple processors instead of only one...

Checklist

• I have reviewed tips for building integrations and this pull request is aligned with them.
• I have verified that all data streams collect metrics or logs.
• I have added an entry to my package's changelog.yml file.
• I have verified that Kibana version constraints are current according to guidelines.

Screenshots

With this PR, adding processors should look like this:

or you can add an empty line in between processors:

The new agent policy looks like this:

inputs:
  - id: aws-s3-cloudfront-57fe3240-fd2e-4759-929d-28332e0939b7
    name: aws-2
    revision: 1
    type: aws-s3
    use_output: default
    meta:
      package:
        name: aws
        version: 1.24.5
    data_stream:
      namespace: default
    streams:
      - id: aws-s3-aws.cloudfront_logs-57fe3240-fd2e-4759-929d-28332e0939b7
        data_stream:
          dataset: aws.cloudfront_logs
          type: logs
        queue_url: test
        max_number_of_messages: 5
        access_key_id: a
        secret_access_key: b
        session_token: c
        tags:
          - forwarded
          - aws-cloudfront
        publisher_pipeline.disable_host: true
        processors:
          - drop_event:
              when:
                regexp:
                  message: ^#.*
          - drop_event:
              when:
                not:
                  regexp:
                    message: .*text/html.*
          - rename:
              field: test
              target_field: new_test

and when there is no processor added, here is the agent policy:

inputs:
  - id: aws-s3-cloudfront-57fe3240-fd2e-4759-929d-28332e0939b7
    name: aws-2
    revision: 2
    type: aws-s3
    use_output: default
    meta:
      package:
        name: aws
        version: 1.24.5
    data_stream:
      namespace: default
    streams:
      - id: aws-s3-aws.cloudfront_logs-57fe3240-fd2e-4759-929d-28332e0939b7
        data_stream:
          dataset: aws.cloudfront_logs
          type: logs
        access_key_id: a
        session_token: c
        queue_url: test
        secret_access_key: b
        processors:
          - drop_event:
              when:
                regexp:
                  message: ^#.*
        max_number_of_messages: 5
        tags:
          - forwarded
          - aws-cloudfront
        publisher_pipeline.disable_host: true

For syslog, I tested with the same processors and here is what the agent policy looks like:

inputs:
  - id: logfile-system-ff6096a4-102c-461e-a844-d1b9c80e6034
    name: system-3
    revision: 2
    type: logfile
    use_output: default
    meta:
      package:
        name: system
        version: 1.20.3
    data_stream:
      namespace: default
    streams:
      - id: logfile-system.syslog-ff6096a4-102c-461e-a844-d1b9c80e6034
        data_stream:
          dataset: system.syslog
          type: logs
        exclude_files:
          - .gz$
        ignore_older: 72h
        paths:
          - /var/log/messages*
          - /var/log/syslog*
        multiline:
          pattern: ^\s
          match: after
        processors:
          - add_locale: null
          - drop_event:
              when:
                not:
                  regexp:
                    message: .*text/html.*
          - rename:
              field: test
              target_field: new_test

This PR also added a default pro

cessor as an example:

[elasticmachine]

🚀 Benchmarks report

Package aws 👍(9) 💚(0) 💔(4)

Expand to view

Data stream	Previous EPS	New EPS	Diff (%)	Result
ec2_logs	50000	27777.78	-22222.22 (-44.44%)	💔
elb_logs	5555.56	4081.63	-1473.93 (-26.53%)	💔
route53_public_logs	12820.51	7407.41	-5413.1 (-42.22%)	💔
route53_resolver_logs	6896.55	5347.59	-1548.96 (-22.46%)	💔

To see the full report comment with /test benchmark fullreport

[elasticmachine]

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

• Start Time: 2022-10-11T20:50:31.792+0000

• Duration: 38 min 37 sec

Test stats 🧪

Test	Results
Failed	0
Passed	408
Skipped	2
Total	410

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

• /test : Re-trigger the build.

[elasticmachine]

🌐 Coverage report

Name	Metrics % (covered/total)	Diff
Packages	100.0% (16/16)	💚
Files	94.444% (17/18)	👎 -3.06
Classes	94.444% (17/18)	👎 -3.06
Methods	78.438% (251/320)	👎 -11.895
Lines	96.733% (7906/8173)	👍 5.097
Conditionals	100.0% (0/0)	💚

[kaiyan-sheng]

@elasticsatch Thanks for the suggestion. I also added a default processor (commented out) as an example. Please see the last screenshot in PR description. This should help hopefully!

[elasticsatch]

@kaiyan-sheng Awesome! The change looks all good to me. Thanks for your prompt actions!

[tommyers-elastic]

LGTM