[aws_logs] Add system tests for awss3 input

[bhapas]

Add system tests for awss3 input in aws_logs integration

What does this PR do?

Adds system test by using terraform service deployer and using the SQS queue_url generated in terraform outputs.

Created test after the implementation of elastic/elastic-package#1272

Checklist

• I have reviewed tips for building integrations and this pull request is aligned with them.
  - [ ] I have verified that all data streams collect metrics or logs.
  - [ ] I have added an entry to my package's changelog.yml file.
  - [ ] I have verified that Kibana version constraints are current according to guidelines.

How to test this PR locally

• Setup AWS credentials and add AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY to the env vars.
• Navigate to packages/aws_logs
• Spin up elastic stack using elastic-package stack up -d -v
• Run tests using elastic-package test system -v
• Take down elastic stack elastic-package stack down -v

[elasticmachine]

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

• Start Time: 2023-07-25T06:56:15.966+0000

• Duration: 17 min 14 sec

Test stats 🧪

Test	Results
Failed	0
Passed	3
Skipped	0
Total	3

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

• /test : Re-trigger the build.

[elasticmachine]

🌐 Coverage report

Name	Metrics % (covered/total)	Diff
Packages	100.0% (0/0)	💚
Files	100.0% (0/0)	💚
Classes	100.0% (0/0)	💚
Methods	100.0% (3/3)	💚 4.938
Lines	100.0% (0/0)	💚 20.154
Conditionals	100.0% (0/0)	💚

[elasticmachine]

Pinging @elastic/security-external-integrations (Team:Security-External Integrations)

[kaiyan-sheng]

@bhapas Thank you so much for working on this! Quick question, are we testing logs ingestion using aws-s3 input here? Are we sending any sample log into S3 bucket and expect a notification message generated in SQS queue and read by the aws-s3 input?

[bhapas]

@bhapas Thank you so much for working on this! Quick question, are we testing logs ingestion using aws-s3 input here? Are we sending any sample log into S3 bucket and expect a notification message generated in SQS queue and read by the aws-s3 input?

@kaiyan-sheng Yes.. So , we have an S3 and SQS and when some object is Put into S3 , an SQS Notification is configured and the integration reads the logs through SQS queue_url. The queue_url is dynamically generated by terraform and is read through the system test config using handlebars template like queue_url: '{{TF_OUTPUT_queue_url}}'.

[kaiyan-sheng]

LGTM