tc-build: Improvements around '--shallow-clone' #83
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
nathanchance
requested review from
nickdesaulniers and
stephenhines
as code owners
As it stands now, '--shallow-clone' does not allow switching to a branch other than master. This is because '--depth=1' implies '--single-branch' so we only fetch the default (master). To support switching branches with '--shallow-clone', we add '--no-single-branch' if the user specifies a branch other than master. There is one snag though: this does not allow a user to switch branches if they clone with '--shallow-clone' because '--[no-]single-branch' is a cloning time decision. There will be a future patch that improves the error reporting as well as help option to let users know that unless they are seriously bandwidth restricted, they should be doing a full clone. Closes: ClangBuiltLinux#81 Signed-off-by: Nathan Chancellor <natechancellor@gmail.com>
Make it clear that this is an option that needs to be used with caution: 1. It is hard to flip around on branches due to an implied '--single-branch' during cloning by default. 2. It is hard to checkout to just a SHA1 (like when using '--use-good-revision' or tracking down an issue in the middle of master). This is really an option designed for CIs where a one off clone is needed for speed purposes. Manually managing the repo and taking the script's heuristic out of it is usually a better option. While we're here, alphabetize the position of '--shallow-clone'. Signed-off-by: Nathan Chancellor <natechancellor@gmail.com>
GitHub does not support cloning straight from an arbitrary SHA1. Forbid using these two options together as they will not work. Signed-off-by: Nathan Chancellor <natechancellor@gmail.com>
Our update workflow is: 1. Fetch the origin remote. 2. Checkout the ref requested by the user. 3. See if the ref is a branch. 4. Pull if it is. Both full and shallow clones will fail at step 2 when the ref does not exist. However, a ref might exist for a full clone and not for a shallow clone and the error message might be confusing. For example, if a user uses '--shallow-clone' with no '--branch' argument, they will only get access to the default branch (master) because of the implicit '--single-branch'. If they were to run the script with '-b release/10.x' after that, the checkout would fail even though they have technically supplied what would be a valid ref. We could just live with this failure since the user did shoot themselves in the foot by not following the advice in the '--shallow-clone' help text. However, for a better UX, error before attempting a checkout when .git/shallow is present and git show-branch errors on the ref. This allows us to tell the user to either: 1. Remove the 'llvm-project' folder and re-run the script with '--shallow-clone' + '-b <branch>' so that '--no-single-branch' is passed. 2. Run 'git -C llvm-project fetch --unshallow origin' to get all of the history. 3. Just do the management themselves and pass in '--no-update'. If the repo is unshallow, the 'git checkout' error is sufficient enough to explain what went wrong. This also adds a small note to the '--branch' help text that explains '--shallow-clone' does not allow a commit hash to be passed to it because GitHub does not support '--depth=1' from arbitrary commits. Checking if a ref is a commit hash programmatically has its flaws so it is not explicitly checked for. Signed-off-by: Nathan Chancellor <natechancellor@gmail.com>
nathanchance
force-pushed
the
shallow-clone-with-branch
branch
from
1bbdfac
to
2b95696
Compare
|
What to do with "tc-build: Check ref exists before checking out when repo is shallow" patch? |
|
I tried (sorry partly German): |
kdrag0n
approved these changes
Apr 15, 2020
|
Just curious and bandwith limitation does not allow me to test... What happens when a branch does not exist? Can I do a shallow-clone on ...and get the same result like doing step #2 alone? Is there an option for |
It fails after fetching master.
No. The script will tell you its impossible for it to do and that its time for manual intervention.
No, that makes little sense in what is supposed to be a build script. |
|
Thanks you @msfjarvis and @nathanchance for making this happen! We are very close to get a Green-IT award. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Addresses #81 (cc @dileks) and some other things that I have thought of while investigating it.
I've written fairly detailed commit messages for each individual change. Some hammering on this would be appreciated as I think I have covered every corner case I can think of but I might have missed something.
@kdrag0n, please make sure this won't break your automated workflow.