Reduce false negatives with assert.hit_count

[andrewkroh]

When the assert.hit_count condition is satisfied the test completes. But if more data is ingested after the condition is satisfied then the test will not catch it leading to a false negative result.

To avoid this problem the test case runner could perform a one or more additional hit count checks after some delay to help ensure that the test fails when more data is ingested than is expected.

[marc-gr]

A possible approach could be to wait until the number of events are stable after a period of time to consider the run "finished", we do something similar in benchmarks https://github.com/elastic/elastic-package/blob/main/internal/benchrunner/runners/system/runner.go#L615-L642