Optimize check #418
Merged
Optimize check #418
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The check container doesn't make any use of an actual checkout, but on large repositories the time to checkout after fetching can be 90% or more of the overall time to clone. Because the --bare option prevents the --branch option from being setting the refspec in the config we have to do it as a follow-up step. Instead of relying solely on the config having the refspec set correctly, we can also explicitly specify the branch when fetching if one is set. Signed-off-by: Ryann Graham <r.m.graham@gmail.com>
The check operation doesn't need any blobs. The most detail is ever needs about commits is the paths they touched, which only requires commit and tree objects, not the blobs. On large repositories, especially with lots of history, this can dramatically reduce the time to clone and fetch while also dramatically reducing the server-side resources used to serve up those large git repositories. Signed-off-by: Ryann Graham <r.m.graham@gmail.com>
taylorsilva
approved these changes
Mar 16, 2024
|
@rmg we went through a similar pain with big repos. HUGE thanks! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
YMMV, but with a large monorepo we saw:
Details
check: use bare clone
The check container doesn't make any use of an actual checkout, but on
large repositories the time to checkout after fetching can be 90% or
more of the overall time to clone.
Because the
--bareoption prevents the--branchoption from beingsetting the refspec in the config we have to do it as a follow-up step.
Instead of relying solely on the config having the refspec set
correctly, we can also explicitly specify the branch when fetching if
one is set.
check: use blobless partial clones
The check operation doesn't need any blobs. The most detail is ever
needs about commits is the paths they touched, which only requires
commit and tree objects, not the blobs.
On large repositories, especially with lots of history, this can
dramatically reduce the time to clone and fetch while also dramatically
reducing the server-side resources used to serve up those large git
repositories.