[AWS] Change default value for dataset name on cloudwatch logs

[P1llus]

What does this PR do?

This PR is a reaction to #5467 (comment).

When the default name of dataset is wrong, the related ingest pipelines and other assets are not working out of the box.

While we wait to convert to "input type" packages, the best course of action is to ensure the default name is correct.

This might be seen as a breaking change, and I will let it be up to the owner of the package to decide on this, any users with using the default value (since the option was hidden at the bottom of the page under advanced options, that is likely), will suddenly send data to a new datastream.

Checklist

• I have reviewed tips for building integrations and this pull request is aligned with them.
• I have verified that all data streams collect metrics or logs.
• I have added an entry to my package's changelog.yml file.
• I have verified that Kibana version constraints are current according to guidelines.

Related issues

• Relates AWS Cloudwatch is indexing data in the wrong data stream #5467

[elasticmachine]

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

• Start Time: 2023-03-10T08:22:56.445+0000

• Duration: 60 min 11 sec

Test stats 🧪

Test	Results
Failed	0
Passed	188
Skipped	4
Total	192

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

• /test : Re-trigger the build.

[elasticmachine]

🌐 Coverage report

Name	Metrics % (covered/total)	Diff
Packages	100.0% (15/15)	💚
Files	93.75% (15/16)	👎 -2.823
Classes	93.75% (15/16)	👎 -2.823
Methods	86.131% (236/274)	👎 -5.114
Lines	85.925% (7387/8597)	👎 -5.461
Conditionals	100.0% (0/0)	💚

[dmathieu]

This does indeed seems like a breaking change.

[aspacca]

@P1llus I'd take the occasion to remove the deprecate s3 input since we are introducing a breaking change. the major version should be bumped.

do we have any migration path?
is it possible to show some kind of changelog/breaking change notice to the users before they install the package, when notified about the new major version?

if so I'd add there the fact that if they are upgrading from a previous minor and they didn't customised already the dataset they must do that setting it to generic as it was before the new major

[P1llus]

@P1llus I'd take the occasion to remove the deprecate s3 input since we are introducing a breaking change. the major version should be bumped.

do we have any migration path? is it possible to show some kind of changelog/breaking change notice to the users before they install the package, when notified about the new major version?

if so I'd add there the fact that if they are upgrading from a previous minor and they didn't customised already the dataset they must do that setting it to generic as it was before the new major

Yeah that is the biggest hurdle right now, since we can't inform the user of the breaking change.
Another option is to simply leave it like this for now @aspacca @dmathieu, and wait for the new input type package, which will have a better user experience when it comes to changing dataset names.

I can bump the major version, remove the S3 input and create a changelog entry for it, newer versions of the stack will have a better way of displaying the changelog when upgrading, so maybe that is sufficient? We can bump the minimum stack version to 8.6+ or so?

[botelastic]

Hi! We just realized that we haven't looked into this PR in a while. We're sorry! We're labeling this issue as Stale to make it hit our filters and make sure we get back to it as soon as possible. In the meantime, it'd be extremely helpful if you could take a look at it as well and confirm its relevance. A simple comment with a nice emoji will be enough :+1. Thank you for your contribution!

[botelastic]

Hi! This PR has been stale for a while and we're going to close it as part of our cleanup procedure. We appreciate your contribution and would like to apologize if we have not been able to review it, due to the current heavy load of the team. Feel free to re-open this PR if you think it should stay open and is worth rebasing. Thank you for your contribution!

[leandrojmp]

Hello @P1llus is this issue still planned to be applied? I didn't seen this PR before and opened a similar PR #8632 changing the dataset name.

I opened it based on this issue #5155 I reported in January after speding a couple of hours trying to understand why an ingest pipeline wasn't working.

If this isn't going to be changed, then the documentation should be changed to inform that the integration ingest pipeline will not be used, as this can lead to confusion for the users.