feat(ci): avoid push to Azure if branch has been updated #2048
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
+1 from me |
robertylewis
changed the title
|
I have no idea why this is failing? The new test works when it's performed before the build stage, but not after. See the difference between the last two commits. |
|
@robertylewis Do you remember the magic number of 60 minutes? I think you need to use the credentials for @leanprover-community-bot. |
I think I blocked it out of my memory... |
gebner
reviewed
Feb 25, 2020
|
GitHub is struggling right now but I think the fixes made it here. |
robertylewis
added
the
awaiting-review
semorrison
reviewed
Feb 26, 2020
semorrison
added
ready-to-merge
semorrison
approved these changes
Feb 27, 2020
mergify bot
pushed a commit
that referenced
this pull request
Apr 4, 2020
PR #2048 changed the CI so that olean caches are not pushed to Azure if a later commit on the same branch occurs. The reasoning was that it was unlikely that anyone would fetch those caches. That's changed as of #2278, since `fetch_olean_cache.sh` now looks for caches from commits earlier in the history. This means that currently, we can observe the following wasteful sequence of events in several PRs (e.g. #2258): 1. commit A gets pushed to a certain branch and CI_A starts. 2. While CI_A is still running the `leanpkg build` step, commit B is pushed and CI_B starts. 3. CI_A finishes `leanpkg build` but doesn't upload its cache to Azure because of #2048 4. Now commit C is pushed and can't use the results of CI_A 5. CI_B finishes `leanpkg build` but doesn't upload its cache 6. Commit D is pushed and can't use the results of CI_A or CI_B ... (This can keep happening as long as the next commit arrives before the most recent CI uploads to Azure). I also cleaned up some stuff that was left over from when we ran the build on both "pr" and "push" events.
anrddh
pushed a commit
to anrddh/mathlib
that referenced
this pull request
May 15, 2020
…community#2048) * avoid push to Azure if branch has been updated * changes to git config in deploy_nightly.sh break git fetch * I don't understand why this is different than on my own repo * artifact upload breaks fetch, I guess? * factor out git config * fix env variable * adjustments * removed repo check Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
anrddh
pushed a commit
to anrddh/mathlib
that referenced
this pull request
May 15, 2020
PR leanprover-community#2048 changed the CI so that olean caches are not pushed to Azure if a later commit on the same branch occurs. The reasoning was that it was unlikely that anyone would fetch those caches. That's changed as of leanprover-community#2278, since `fetch_olean_cache.sh` now looks for caches from commits earlier in the history. This means that currently, we can observe the following wasteful sequence of events in several PRs (e.g. leanprover-community#2258): 1. commit A gets pushed to a certain branch and CI_A starts. 2. While CI_A is still running the `leanpkg build` step, commit B is pushed and CI_B starts. 3. CI_A finishes `leanpkg build` but doesn't upload its cache to Azure because of leanprover-community#2048 4. Now commit C is pushed and can't use the results of CI_A 5. CI_B finishes `leanpkg build` but doesn't upload its cache 6. Commit D is pushed and can't use the results of CI_A or CI_B ... (This can keep happening as long as the next commit arrives before the most recent CI uploads to Azure). I also cleaned up some stuff that was left over from when we ran the build on both "pr" and "push" events.
anrddh
pushed a commit
to anrddh/mathlib
that referenced
this pull request
May 16, 2020
…community#2048) * avoid push to Azure if branch has been updated * changes to git config in deploy_nightly.sh break git fetch * I don't understand why this is different than on my own repo * artifact upload breaks fetch, I guess? * factor out git config * fix env variable * adjustments * removed repo check Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
anrddh
pushed a commit
to anrddh/mathlib
that referenced
this pull request
May 16, 2020
PR leanprover-community#2048 changed the CI so that olean caches are not pushed to Azure if a later commit on the same branch occurs. The reasoning was that it was unlikely that anyone would fetch those caches. That's changed as of leanprover-community#2278, since `fetch_olean_cache.sh` now looks for caches from commits earlier in the history. This means that currently, we can observe the following wasteful sequence of events in several PRs (e.g. leanprover-community#2258): 1. commit A gets pushed to a certain branch and CI_A starts. 2. While CI_A is still running the `leanpkg build` step, commit B is pushed and CI_B starts. 3. CI_A finishes `leanpkg build` but doesn't upload its cache to Azure because of leanprover-community#2048 4. Now commit C is pushed and can't use the results of CI_A 5. CI_B finishes `leanpkg build` but doesn't upload its cache 6. Commit D is pushed and can't use the results of CI_A or CI_B ... (This can keep happening as long as the next commit arrives before the most recent CI uploads to Azure). I also cleaned up some stuff that was left over from when we ran the build on both "pr" and "push" events.
cipher1024
pushed a commit
to cipher1024/mathlib
that referenced
this pull request
Mar 15, 2022
PR leanprover-community#2048 changed the CI so that olean caches are not pushed to Azure if a later commit on the same branch occurs. The reasoning was that it was unlikely that anyone would fetch those caches. That's changed as of leanprover-community#2278, since `fetch_olean_cache.sh` now looks for caches from commits earlier in the history. This means that currently, we can observe the following wasteful sequence of events in several PRs (e.g. leanprover-community#2258): 1. commit A gets pushed to a certain branch and CI_A starts. 2. While CI_A is still running the `leanpkg build` step, commit B is pushed and CI_B starts. 3. CI_A finishes `leanpkg build` but doesn't upload its cache to Azure because of leanprover-community#2048 4. Now commit C is pushed and can't use the results of CI_A 5. CI_B finishes `leanpkg build` but doesn't upload its cache 6. Commit D is pushed and can't use the results of CI_A or CI_B ... (This can keep happening as long as the next commit arrives before the most recent CI uploads to Azure). I also cleaned up some stuff that was left over from when we ran the build on both "pr" and "push" events.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
On non-master builds, it seems unnecessary to push an olean archive if another commit is added to the branch before the build finishes.
A common situation in an open PR is to add a sequence of commits very quickly, e.g. accepting review suggestions. It seems very unlikely anyone would check out one of the commits in the middle.
TO CONTRIBUTORS:
Make sure you have:
If this PR is related to a discussion on Zulip, please include a link in the discussion.
For reviewers: code review check list