Skip to content

Commit

Permalink
ci/deqp-runner: compress results.csv before uploading it to GitLab
Browse files Browse the repository at this point in the history
The results.csv file can get ridiculously big for a vkcts run (up to
135MB). Given how seldomly this file is being used, and the fact that
it cannot be viewed directly in gitlab's artifact page anyway.

Let's compress the file using zstd, and enjoy a ~95% reduction in size
at the cost of probably less than 1 second of compression time on even
the slowest of the devices in CI (which would use sharing), and about
150ms on the CI machines in the Valve farm.

Suggested-by: Daniel Stone <daniels@collabora.com>
Acked-by: David Heidelberg <david.heidelberg@collabora.com>
Signed-off-by: Martin Roukala (né Peres) <martin.roukala@mupuf.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20669>
  • Loading branch information
mupuf authored and Marge Bot committed Jan 13, 2023
1 parent f19f119 commit d110299
Showing 1 changed file with 5 additions and 0 deletions.
5 changes: 5 additions & 0 deletions .gitlab-ci/deqp-runner.sh
Original file line number Diff line number Diff line change
Expand Up @@ -230,6 +230,11 @@ if [ -n "$FLAKES_CHANNEL" ]; then
--branch-title "${CI_MERGE_REQUEST_TITLE:-$CI_COMMIT_TITLE}"
fi

# Compress results.csv to save on bandwidth during the upload of artifacts to
# GitLab. This reduces the size in a VKCTS run from 135 to 7.6MB, and takes
# 0.17s on a Ryzen 5950X (16 threads, 0.95s when limited to 1 thread).
zstd --rm -T0 -8qc $RESULTS/results.csv -o $RESULTS/results.csv.zst

echo -e "\e[0Ksection_end:$(date +%s):test_post_process\r\e[0K"

exit $DEQP_EXITCODE

0 comments on commit d110299

Please sign in to comment.