Skip to content

Conversation

@mrodm
Copy link
Contributor

@mrodm mrodm commented Jan 29, 2024

Currently, "could not find hits in <data_stream> data stream" error is not reported as part of the xUnit report.

For instance, in this build https://buildkite.com/elastic/integrations/builds/8409#018d4368-0d0c-4ddd-a1b9-1f6c3d4b6753 , no hits were found and elastic-package returned an error without completing the test run. Moreover, there is no jUnit report (XML file)
because of that failure:

Error: error running package system tests: could not complete test run: could not find hits in logs-aws.vpcflow-ep data stream

https://buildkite.com/elastic/integrations/builds/8409#018d4368-0d0c-4ddd-a1b9-1f6c3d4b6753/6-10235

This PR adds this specific error as a ErrTestCaseFailed so it gets added into the system xUnit report. As a side-effect, elastic-package now allows to run the other tests even if for one data stream there are no hits found.

Report example for mysql package:

<?xml version="1.0" encoding="UTF-8"?>
<testsuites>
  <testsuite name="system" tests="10" failures="2">
    <!--test suite for system tests-->
    <testcase name="system test: default (variant: mysql_8_0_35)" classname="mysql.slowlog" time="38.373606196"></testcase>
    <testcase name="system test: default (variant: percona_8_0_36)" classname="mysql.slowlog" time="39.211554004"></testcase>
    <testcase name="system test: default (variant: mysql_8_0_35)" classname="mysql.status" time="41.172481079"></testcase>
    <testcase name="system test: default (variant: percona_8_0_36)" classname="mysql.status" time="41.041845086"></testcase>
    <testcase name="system test: default (variant: mysql_8_0_35)" classname="mysql.error" time="639.829253568">
      <failure>could not find hits in logs-mysql.error-ep data stream</failure>
    </testcase>
    <testcase name="system test: default (variant: percona_8_0_36)" classname="mysql.error" time="636.446832929">
      <failure>could not find hits in logs-mysql.error-ep data stream</failure>
    </testcase>
    <testcase name="system test: default (variant: mysql_8_0_35)" classname="mysql.galera_status" time="39.484423072"></testcase>
    <testcase name="system test: default (variant: percona_8_0_36)" classname="mysql.galera_status" time="40.702275199"></testcase>
    <testcase name="system test: default (variant: mysql_8_0_35)" classname="mysql.performance" time="39.012759805"></testcase>
    <testcase name="system test: default (variant: percona_8_0_36)" classname="mysql.performance" time="38.714038382"></testcase>
  </testsuite>
</testsuites>

How to test locally

elastic-package stack up -v -d

# Go to `mysql` package in integrations repository
cd /path/integrations/repo/packages/mysql

elastic-package test system -v --report-format xUnit --report-output file

@mrodm mrodm requested a review from a team January 29, 2024 11:43
@mrodm mrodm self-assigned this Jan 29, 2024
@elasticmachine
Copy link
Collaborator

💚 Build Succeeded

cc @mrodm

@mrodm mrodm merged commit 8b4aaa0 into elastic:main Jan 29, 2024
@mrodm mrodm deleted the report_no_hits_as_test_error branch January 29, 2024 15:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants