Skip to content

Ensure that required commands are run for test benchmark packages #1327

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Jun 29, 2023
Merged

Ensure that required commands are run for test benchmark packages #1327

Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
107363b
Add PACKAGE_UNDER_TEST for benchmark tests
mrodm Jun 28, 2023
57c4ac1
Check package to test in benchmark in every loop
mrodm Jun 28, 2023
65f88fe
Add debug
mrodm Jun 28, 2023
d601d47
Revert "Add debug"
mrodm Jun 28, 2023
f5a280a
Commented out system_benchmark to not run benchmark
mrodm Jun 29, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 10 additions & 7 deletions scripts/test-check-packages.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -69,23 +69,26 @@ for d in test/packages/${PACKAGE_TEST_TYPE:-other}/${PACKAGE_UNDER_TEST:-*}/; do
elastic-package install -v

if [ "${PACKAGE_TEST_TYPE:-other}" == "benchmarks" ]; then
if [ "${PACKAGE_UNDER_TEST:-*}" == "pipeline_benchmark" ]; then
# It is not used PACKAGE_UNDER_TEST, so all benchmark packages are run in the same loop
package_to_test=$(basename ${d})
if [ "${package_to_test}" == "pipeline_benchmark" ]; then
rm -rf "${OLDPWD}/build/benchmark-results"
elastic-package benchmark pipeline -v --report-format xUnit --report-output file --fail-on-missing

rm -rf "${OLDPWD}/build/benchmark-results-old"
mv "${OLDPWD}/build/benchmark-results" "${OLDPWD}/build/benchmark-results-old"

elastic-package benchmark pipeline -v --report-format json --report-output file --fail-on-missing

elastic-package report --fail-on-missing benchmark \
--new ${OLDPWD}/build/benchmark-results \
--old ${OLDPWD}/build/benchmark-results-old \
--threshold 1 --report-output-path="${OLDPWD}/build/benchreport"
fi
if [ "${PACKAGE_UNDER_TEST:-*}" == "system_benchmark" ]; then
elastic-package benchmark system --benchmark logs-benchmark -v --defer-cleanup 1s
fi
# FIXME: running system benchmark in package "system_benchmark" fails with panic
# if [ "${package_to_test}" == "system_benchmark" ]; then
# elastic-package benchmark system --benchmark logs-benchmark -v --defer-cleanup 1s
# fi
else
# defer-cleanup is set to a short period to verify that the option is available
elastic-package test -v --report-format xUnit --report-output file --defer-cleanup 1s --test-coverage
Expand Down