Move ELB lightweight module config into integration

[kaiyan-sheng]

What does this PR do?

This PR is to move lightweight module configuration from metricbeat into integrations. Also this PR adds collection to HealthyHostCount and UnHealthyHostCount for ApplicationELB.

Checklist

• I have reviewed tips for building integrations and this pull request is aligned with them.
• I have verified that all data streams collect metrics or logs.
• I have added an entry to my package's changelog.yml file.
• I have verified that Kibana version constraints are current according to guidelines.

[elasticmachine]

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

• Start Time: 2022-06-16T15:31:12.698+0000

• Duration: 28 min 53 sec

Test stats 🧪

Test	Results
Failed	0
Passed	328
Skipped	0
Total	328

🤖 GitHub comments

To re-run your PR in the CI, just comment with:

• /test : Re-trigger the build.

[elasticmachine]

🌐 Coverage report

Name	Metrics % (covered/total)	Diff
Packages	100.0% (11/11)	💚
Files	91.667% (11/12)	👎 -4.901
Classes	91.667% (11/12)	👎 -4.901
Methods	82.439% (169/205)	👎 -6.47
Lines	92.673% (2087/2252)	👍 2.728
Conditionals	100.0% (0/0)	💚

[tommyers-elastic]

this LGTM - but i don't really know what i'm looking at yet 😜 - would you mind walking me through the changes @kaiyan-sheng ?

also - whats the story with testing these kinds of changes? i've seen a couple or PRs already which appear to be copy-pasted configurations - is there any way to guarantee there are no typos or copy-paste errors?

thanks!

[kaiyan-sheng]

@tommyers-elastic Yes definitely! When we first came up with lightweight modules, we have the configuration files in Beats, such as https://github.com/elastic/beats/blob/main/x-pack/metricbeat/module/aws/elb/manifest.yml. When we migrated all the metricsets into the AWS package as individual integrations, the migration script we use migrated the light module directly. This means the configuration for collecting ELB metrics is still in the Beats repo here instead of in the Integrations repo. When we find a bug, for example, this bug in ELB with missing HealthyHostCount and UnHealthyHostCount for ApplicationELB, in order to fix it, we have to make the change in Beats repo instead of directly here. Fixing it in Beats repo means we have to wait for the next minor release in order for users to pick up the bug fix, which might take a long time. That's the reason why we are doing the migration to move lightweight module configs into the Integrations repo.

For testing: great question! This is not very reliable right now. What we have is a check static test: Verify sample_event.json – aws.elb_metrics in CI that checks if the fields in sample_event.json event are all documented under fields folder. If not CI would fail. The reason why I say it's not very reliable is because the sample_event.json is generated by the PR contributor manually and most of the time, the event documented in sample_event.json only includes some of the fields, not all.

[aspacca]

👍

[aspacca]

please, @kaiyan-sheng , see my comment about the integration version

[tommyers-elastic]

thanks for this explanation @kaiyan-sheng - it's super helpful