Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Work around merge failures in subsequent CI attempts #882

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

kbattocchi
Copy link
Collaborator

The migration to the v4 version of the artifact API has somewhat broken rerunning tests for a PR in some scenarios because now artifacts are scoped to the attempt, but it's possible that we won't generate every type of artifact on every attempt (e.g. if all of our notebook tests passes in attempt 1, we won't try to run them again in attempt 2, see, e.g. https://github.com/py-why/EconML/actions/runs/8940985090/attempts/2 for a concrete example).

Unfortunately, the new merge artifact action's API doesn't provide a graceful way to handle this at the moment, so we can just set the step's continue-on-error property to true to ignore failures (ideally we wouldn't have to assume that this is the only reason the job can fail, but I don't see any obvious alternative).

Note that this isn't exactly a blocker, because our Verify CI checks job doesn't look for failures in the merge jobs anyway, but it's potentially confusing (IMO) that we can have red Xs on some jobs for the attempt and yet still merge the PR and so it would be better to have the merge jobs not fail in this case. (There is some other semantic weirdness with the fact that artifacts are per-attempt, like the fact that we want the final coverage report to ideally include just the last (successful) run for each test configuration, but instead we'll get one separate coverage report per attempt that includes only the tests that were run in that attempt; again, there does not seem to be a clean workaround at the moment).

Signed-off-by: Keith Battocchi <kebatt@microsoft.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant