Init commit of linux system tests

[fearful-symmetry]

What does this PR do?

This adds system tests to (most) of the linux data streams, which should help in the future with catching low-hanging bugs.

However, we may want to look into expanding the test architecture somewhat, and I'd like input from @mtojek before this gets merged. The system integration is a bit odd, in that 90% of the code can be covered by pointed at a test file, without any external services, containers, etc. If we want to expand the system tests used in linux and system further, we need some kind of system that populates a test directory inside the agent container with a handful of files that the data streams can read from. I struggled to hack up a solution with what we have now, but decided it wasn't quite worth the ugliness. The system also seems to require test container service...so I had to make a dummy one.

Checklist

• I have reviewed tips for building integrations and this pull request is aligned with them.
• I have verified that all data streams collect metrics or logs.
• I have added an entry to my package's changelog.yml file.

How to test this PR locally

• Pull down PR
• run elastic-package -v test system in the linux directory

[elasticmachine]

Pinging @elastic/integrations (Team:Integrations)

[elasticmachine]

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

• Start Time: 2021-07-26T21:40:02.026+0000

• Duration: 17 min 49 sec

• Commit: 8657407

Test stats 🧪

Test	Results
Failed	0
Passed	25
Skipped	0
Total	25

Trends 🧪

[exekias]

we need some kind of system that populates a test directory inside the agent container with a handful of files that the data streams can read from

Could you elaborate on this? Are you talking about /proc&/sys folders, or is this something else?

[jsoriano]

we need some kind of system that populates a test directory inside the agent container with a handful of files that the data streams can read from

For system tests of metrics so far I think that we are only checking that metrics are collected without errors, we are not checking their values. For this case it may be also enough with checking that it collects any data, so the actual /proc and /sys filesystems available in the agent container would serve.

For logs you can use ${SERVICE_LOGS_DIR}.

[fearful-symmetry]

Could you elaborate on this? Are you talking about /proc&/sys folders, or is this something else?
@exekias

Yes, along with a few other things. Depending on the OS config, some procfs files (such as RAID) won't contain any useful information for the metricset to parse. Some, such as cgroup files, won't exist at all inside a container. I spent some time hacking at the container orchestration that exists now to do that, but it was awfully ugly and I decided not to. At minimum, it would be nice to omit the "auxillary" container that elastic-package test system seems to require, as it's not actually needed here.

[exekias]

For system tests of metrics so far I think that we are only checking that metrics are collected without errors, we are not checking their values. For this case it may be also enough with checking that it collects any data, so the actual /proc and /sys filesystems available in the agent container would serve.

I'm very much onboard with this for the initial batch of tests (this PR). It won't be testing the full package but many parts of it are still valid.

On top of that (maybe later), we could also have a testdata folder mounted into the agent, the same way we have one for logs, right? This way you could mock a raid file structure and configure the agent to retrieve it, for example.

@mtojek @jsoriano any thoughts on the need for a dummy container when there is no need for it? I'm not particularly concerned with its need by I see @fearful-symmetry point

[mtojek]

This is a specific use case where you don't need to deploy anything. Temporarily you can use a lightweight/micro/dummy image.

Definitely not a blocker, but could be considered as a feature. Please open an issue in elastic-package for this.

[mtojek]

As you introduce the dependency on ECS, could you please review and adjust also other data streams? I see there are potential candidates, not sure about others:

https://github.com/fearful-symmetry/integrations/blob/linux-system-tests/packages/linux/data_stream/users/fields/ecs.yml
https://github.com/fearful-symmetry/integrations/blob/linux-system-tests/packages/linux/data_stream/socket/fields/ecs.yml

[fearful-symmetry]

Alright, added the ECS fields to the other data streasm. @mtojek / @exekias I agree this isn't a blocker, just wanted to make the shortcomings known here. I'll open an issue on elastic-package.

[mtojek]

LGTM, but I left one comment regarding error.message to double-check.

[mtojek]

Usually we don't add error.message fields. Is there any reason why you added it here? I mean, are there any problems with system tests? If so, then they should be fixed first.

[fearful-symmetry]

Ah, that was probably just me copying the ECS fields from other data streams. I can remove it.