Skip to content

[microsoft/release-branch.go1.17] Add "make" retry capability, with 5 in Windows CI#299

Merged
dagood merged 2 commits into
microsoft/release-branch.go1.17from
dev/dagood/retry-1.17
Dec 6, 2021
Merged

[microsoft/release-branch.go1.17] Add "make" retry capability, with 5 in Windows CI#299
dagood merged 2 commits into
microsoft/release-branch.go1.17from
dev/dagood/retry-1.17

Conversation

@dagood
Copy link
Copy Markdown
Member

@dagood dagood commented Dec 3, 2021

Add retry on the make.bat calls in CI to mitigate flakiness:

Submitting directly to 1.17 because this flakiness has only been seen here. This is nice: we can mitigate it while avoiding masking other potential flakiness issues in microsoft/main and the other release branches.

In this test build, we got up to three attempts in one job before success, and all 20 jobs succeeded building: https://dev.azure.com/dnceng/internal/_build/results?buildId=1496055&view=results

@dagood dagood marked this pull request as ready for review December 3, 2021 20:19
@dagood dagood requested a review from a team as a code owner December 3, 2021 20:19
Comment thread eng/_core/cmd/build/build.go
@dagood dagood requested review from jaredpar and qmuntal December 6, 2021 20:21
- template: ../steps/checkout-windows-task.yml
- pwsh: |
Write-Host "Increasing max build retries to mitigate 'Access denied' flakiness during EXE copying on Windows."
Write-Host "##vso[task.setvariable variable=GO_MAKE_MAX_RETRY_ATTEMPTS]5"
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given that we are outputting to stderr now we should be explicit on the failOnStderr property

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Never mind that is fmt.ErrorF not printing to stderr 😦

@dagood dagood merged commit 8523c1f into microsoft/release-branch.go1.17 Dec 6, 2021
@dagood dagood deleted the dev/dagood/retry-1.17 branch December 6, 2021 23:01
dagood added a commit that referenced this pull request Jan 27, 2022
… in Windows CI (#299)

* Add "make" retry capability, with 5 in Windows CI

* Report error if GO_MAKE_MAX_RETRY_ATTEMPTS is not an int
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants