[FLINK-24875][ci] Split E2E job #17781
Conversation
The cache task fails if the cache directory does not exist, which can happen if you get a cache miss and no test creates said directory.
|
Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community Automated ChecksLast check on commit 953a8f9 (Fri Nov 12 13:13:54 UTC 2021) Warnings:
Mention the bot in a comment to re-run the automated checks. Review Progress
Please see the Pull Request Review Guide for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commandsThe @flinkbot bot supports the following commands:
|
| # the cache task does not create directories on a cache miss, and can later fail when trying to tar the directory if the test haven't created it | ||
| # this may for example happen if a given directory is only used by a subset of tests, which are run in a different 'group' | ||
| - bash: | | ||
| mkdir -p $(MAVEN_CACHE_FOLDER) |
There was a problem hiding this comment.
Why didn't we need these in the previous version in jobs-template.yml?
There was a problem hiding this comment.
This came up in the release sync today. It's actually connected to FLINK-24801, isn't it? If yes, I'd suggest linking the two tickets and updating the commit message of this change.
There was a problem hiding this comment.
It's not related to FLINK-24801.
The particular case that this covers are the java e2e tests only being run in one of the groups; hence E2E_CACHE_FOLDER is never created in the run of the other group, failing tar down the line.
There was a problem hiding this comment.
I see - thanks for clarification.
The E2E job is now split into 2 jobs, each running roughly half of all tests. This remedies a number of issues, like lack of disk space, timeouts or long feedback cycles.