Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Investigate use of --depth 1 for git cloning #217

Closed
sindresorhus opened this issue Jan 27, 2013 · 14 comments
Closed

Investigate use of --depth 1 for git cloning #217

sindresorhus opened this issue Jan 27, 2013 · 14 comments

Comments

@sindresorhus
Copy link
Contributor

Would make it faster to clone, but not sure how it works with tags. Needs some investigation.

@mehcode
Copy link

mehcode commented Jan 27, 2013

This should work nicely:

git clone URL -b TAG --depth 1

@wibblymat
Copy link
Member

As mentioned above, you can replace the code that currently does a clone followed by a checkout of the correct branch with the single git clone URL -b TAG --depth 1 mentioned above.

Alternatively, when you do a git clone --depth 1 without specifying the tag you might get all of the objects needed for a particular tag, in which case you just git checkout TAG. But then again, you might not, so you actually have to do a git fetch TAG first to make sure.

So, currently if a repo is cloned it is added to the cache, then a copy is made in the components folder where a git checkout TAG is done. Then if you need it again you can go to the cached version. So I would recommend the following logic:

  1. If the repo we want is not cached, do a git clone --depth 1 -b TAG into the cache.
  2. If the repo in the cache does not contain the tag we need, do a git fetch TAG
  3. Copy the repo from the cache and git checkout TAG

If it is decided that caching isn't so important any more then you can just clone straight into the components folder. From some testing and reading around it seems that --depth 1 doesn't really affect download sizes by all that much so you probably still want caching.

@satazor
Copy link
Member

satazor commented May 29, 2013

Being used in the rewrite!

@satazor satazor closed this as completed May 29, 2013
@sindresorhus
Copy link
Contributor Author

yay

@dylang
Copy link
Contributor

dylang commented Jul 11, 2013

This is causing problems for me in the rewrite branch. Repos on our Github Enterprise server never get checked out.

Example:

$ git clone https://github.enterprise/dylan-greene/repo.git -b 0.0.1 --depth 1
Cloning into 'repo'...

# An now we wait forever....

This is what happens in bower rewrite branch as well.

My local git version: git version 1.8.3.2

Unfortunately because the purpose of our Github Enterprise server is to keep it out of the public I'm not provide access for testing but I'm happy to try anything out.

@satazor
Copy link
Member

satazor commented Jul 11, 2013

@dylang that's strange. We will need some extra information to understand the real reason. Indeed, accessing a repository within your GitHub enterprise server would be useful. Would you be able to cooperate with me, perhaps giving me access to a new dummy repository of your server? If so, feel free to send me an email so we can start the debug process.

@dylang
Copy link
Contributor

dylang commented Jul 11, 2013

@satazor I wish i could do that but our Github Enterprise instance is behind our firewall.

I'll send some emails today to our admins and Git experts and see if they have any ideas.

@dylang
Copy link
Contributor

dylang commented Jul 11, 2013

@satazor I have a potential workaround:

-v added for verbose output.

This waits forever:

❯ git clone https://github.enterprise/dylan-greene/repo.git -b 0.0.1 --depth 1 -v
Cloning into 'repo'...
POST git-upload-pack (157 bytes)
^C

This works:

❯ git clone git://github.enterprise/dylan-greene/repo.git -b 0.0.1 --depth 1 -v
Cloning into 'repo'...
remote: Counting objects: 10, done.
remote: Compressing objects: 100% (9/9), done.
remote: Total 10 (delta 2), reused 7 (delta 1)
Receiving objects: 100% (10/10), done.
Resolving deltas: 100% (2/2), done.
Checking connectivity... done
Note: checking out 'cfd0818a6620886eb4860985606c3b78d18d0013'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:

  git checkout -b new_branch_name

The difference is changing the protocol from https: to git:. I'm not sure why this makes the difference.

@satazor
Copy link
Member

satazor commented Jul 11, 2013

You could try setting the inverse of this answer on stackoverflow: http://stackoverflow.com/questions/15669091/bower-install-using-only-https

I've tried to git clone https://github.com/IndigoUnited/automaton.git -b 0.1.4 --depth 1 -v and works but takes SOOO LONG. I think it would eventually work for you too, but your repository might be much bigger.

I would suggest using the git protocol as is much faster than anything else.

@dylang
Copy link
Contributor

dylang commented Jul 11, 2013

@satazor I have different experience using https vs git on Github:

These both took the same amount of time for me (1 second):
git clone https://github.com/IndigoUnited/automaton.git -b 0.1.4 --depth 1 -v
git clone git://github.com/IndigoUnited/automaton.git -b 0.1.4 --depth 1 -v .

Here's a test using a repo with a large history ad git: is slower.

❯ git clone git://github.com/angular/angular.js angular-git --depth 1 -b v1.1.5
...cut...
previous command took 6 seconds

❯ git clone https://github.com/angular/angular.js angular-https --depth 1 -b v1.1.5
...cut...
previous command took 2 seconds

❯ git clone git://github.com/angular/angular.js angular-git --depth 1 -b v1.1.5
...cut...
previous command took 6 seconds

I did it twice in case there was some caching going on.

Note: I'm in the US and on a very fast connection if that matters.

@satazor
Copy link
Member

satazor commented Jul 12, 2013

Strange. Even the git documentation says that the git protocol is internally optimised to speedup fetching. Did my suggestion worked?

@dylang
Copy link
Contributor

dylang commented Jul 12, 2013

@satazor I would trust the git documentation over one random test I happened to do.

The suggestion you posted might work but it will be easier for us to just standardize on git:// urls when using Github Enterprise.

@tapwork
Copy link

tapwork commented Sep 17, 2014

having the same issue here
git clone --depth 1 -v git@example.com:ios/therepo.git ~/Desktop/test
and
git clone --depth 1 -v git://example.com/ios/therepo.git ~/Desktop/test
works perfectly
but
git clone --depth 1 -v https://example.com/ios/therepo.git ~/Desktop/test
stucks at POST git-upload-pack (222 bytes)

@sheerun
Copy link
Contributor

sheerun commented Sep 17, 2014

newest bower disabled shallow cloning for non-github hosts

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

No branches or pull requests

7 participants