New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cloning Git Pod repo is taking too long (git --depth 1 problem) #2481

Closed
martincik opened this Issue Sep 17, 2014 · 28 comments

Comments

Projects
None yet
@martincik
Contributor

martincik commented Sep 17, 2014

Hey guys,

I'm working with new CocoaPods 0.34.0.rc2 and git version '1.9.3 (Apple Git-50)', Xcode 6.

My Podfile with line like this:

    pod 'AdobeMobileLibrary', git: "https://secret/AdobeMobileLibrary.git"

Is resulting in git clone with params '--single-branch --depth 1'. Where the '--depth 1' is causing the taking too long problem.

When I remove the '--depth 1' it works perfectly fine and is finished in 10 seconds.

Can somebody confirm the same issue?

@yoshiki

This comment has been minimized.

Show comment
Hide comment
@yoshiki

yoshiki Sep 17, 2014

+1
I have the same issue with CocoaPods 0.34.0.rc2 and git version 1.9.3 (Apple Git-50).

yoshiki commented Sep 17, 2014

+1
I have the same issue with CocoaPods 0.34.0.rc2 and git version 1.9.3 (Apple Git-50).

@martincik

This comment has been minimized.

Show comment
Hide comment
@martincik

martincik Sep 17, 2014

Contributor

To fix this temporally I've removed from 'cocoapod-downloader' gem '--depth 1' parameter in git.rb line 63.

Contributor

martincik commented Sep 17, 2014

To fix this temporally I've removed from 'cocoapod-downloader' gem '--depth 1' parameter in git.rb line 63.

@fabiopelosin

This comment has been minimized.

Show comment
Hide comment
@fabiopelosin

fabiopelosin Sep 17, 2014

Member

I don't see anithing special in the man pages of git 1.9.3: https://github.com/git/git/blob/v1.9.3/Documentation/git-clone.txt#L182-L184

Is the issue present only with this repo?

Member

fabiopelosin commented Sep 17, 2014

I don't see anithing special in the man pages of git 1.9.3: https://github.com/git/git/blob/v1.9.3/Documentation/git-clone.txt#L182-L184

Is the issue present only with this repo?

@martincik

This comment has been minimized.

Show comment
Hide comment
@martincik

martincik Sep 17, 2014

Contributor

Doesn't happen for Github repos. We're using Github Enterprise and that's maybe the issue. I'm guessing that if the remote is not updated (meaning able to provide support for --depth 1) than it's hanging in indefinite loop (or taking just too long to wait for it). Otherwise I have no clue where is the problem. :(

Contributor

martincik commented Sep 17, 2014

Doesn't happen for Github repos. We're using Github Enterprise and that's maybe the issue. I'm guessing that if the remote is not updated (meaning able to provide support for --depth 1) than it's hanging in indefinite loop (or taking just too long to wait for it). Otherwise I have no clue where is the problem. :(

@yoshiki

This comment has been minimized.

Show comment
Hide comment
@yoshiki

yoshiki Sep 17, 2014

I tried to review the details, there seems to be a problem with Github Enterprise.
I was able to clone it if I use the git protocol. :(

yoshiki commented Sep 17, 2014

I tried to review the details, there seems to be a problem with Github Enterprise.
I was able to clone it if I use the git protocol. :(

@fabiopelosin

This comment has been minimized.

Show comment
Hide comment
@fabiopelosin

fabiopelosin Sep 17, 2014

Member

Some context:

In #886, I was specifically adding support for GH Enterprise because I assumed this was a problem with GHE's specific git server implementation. It seems like this isn't correct and that it would be beneficial to allow a configuration option to disable the shallow clone, in case your git server doesn't allow it.

Member

fabiopelosin commented Sep 17, 2014

Some context:

In #886, I was specifically adding support for GH Enterprise because I assumed this was a problem with GHE's specific git server implementation. It seems like this isn't correct and that it would be beneficial to allow a configuration option to disable the shallow clone, in case your git server doesn't allow it.

@sheerun

This comment has been minimized.

Show comment
Hide comment
@sheerun

sheerun Sep 17, 2014

newest bower disabled shallow cloning for non-github hosts

sheerun commented Sep 17, 2014

newest bower disabled shallow cloning for non-github hosts

@tapwork

This comment has been minimized.

Show comment
Hide comment
@tapwork

tapwork Sep 17, 2014

having the same issue
see my comment here bower/bower#217 (comment)

tapwork commented Sep 17, 2014

having the same issue
see my comment here bower/bower#217 (comment)

@fabiopelosin

This comment has been minimized.

Show comment
Hide comment
@fabiopelosin

fabiopelosin Sep 17, 2014

Member

So a white list of host supporting the depth argument?

@CocoaPods/Core

Member

fabiopelosin commented Sep 17, 2014

So a white list of host supporting the depth argument?

@CocoaPods/Core

@alloy

This comment has been minimized.

Show comment
Hide comment
@alloy

alloy Sep 17, 2014

Member

@fabiopelosin From quick reading, it appears that the exact cause is not know yet, so working around it before then sounds like a bad idea. Maybe I’m missing it in any of the linked issues, if so, please describe the actual technical reason here on this ticket.

Member

alloy commented Sep 17, 2014

@fabiopelosin From quick reading, it appears that the exact cause is not know yet, so working around it before then sounds like a bad idea. Maybe I’m missing it in any of the linked issues, if so, please describe the actual technical reason here on this ticket.

@kylef kylef added this to the 0.34.0 milestone Sep 17, 2014

@segiddins

This comment has been minimized.

Show comment
Hide comment
@segiddins

segiddins Sep 19, 2014

Member

If anyone can share a publicly accessible repo that exhibits this problem, I'll do my best to track it down.

Member

segiddins commented Sep 19, 2014

If anyone can share a publicly accessible repo that exhibits this problem, I'll do my best to track it down.

@martincik

This comment has been minimized.

Show comment
Hide comment
@martincik

martincik Sep 22, 2014

Contributor

Because it's failing mainly for Github Enterprise and those are private repos by design, I don't think (IMHO) you can simulate this with public repo. :(

Contributor

martincik commented Sep 22, 2014

Because it's failing mainly for Github Enterprise and those are private repos by design, I don't think (IMHO) you can simulate this with public repo. :(

@alloy

This comment has been minimized.

Show comment
Hide comment
@alloy

alloy Sep 22, 2014

Member

@martincik Seeing as you can at least reproduce this issue, can you spend some time to look into a fix that would still download as little as possible?

If you are unable to find a fix at all, then please file a ticket with GitHub and share that with us here and we’ll look into a workaround such as an option to disable it.

Member

alloy commented Sep 22, 2014

@martincik Seeing as you can at least reproduce this issue, can you spend some time to look into a fix that would still download as little as possible?

If you are unable to find a fix at all, then please file a ticket with GitHub and share that with us here and we’ll look into a workaround such as an option to disable it.

@sheerun

This comment has been minimized.

Show comment
Hide comment
@sheerun

sheerun Sep 22, 2014

Maybe someone can invite @segiddins to sample private github enterprise repository?

sheerun commented Sep 22, 2014

Maybe someone can invite @segiddins to sample private github enterprise repository?

@martincik

This comment has been minimized.

Show comment
Hide comment
@martincik

martincik Sep 22, 2014

Contributor

Can you help me out with the patch here: CocoaPods/cocoapods-downloader#31

Thank you. Ladislav

Contributor

martincik commented Sep 22, 2014

Can you help me out with the patch here: CocoaPods/cocoapods-downloader#31

Thank you. Ladislav

@segiddins

This comment has been minimized.

Show comment
Hide comment
@segiddins

segiddins Sep 22, 2014

Member

@martincik I'm hesitant to merge a patch like that, since it likely will result in slowing clones of many repos, likely unnecessarily.

Member

segiddins commented Sep 22, 2014

@martincik I'm hesitant to merge a patch like that, since it likely will result in slowing clones of many repos, likely unnecessarily.

@martincik

This comment has been minimized.

Show comment
Hide comment
@martincik

martincik Sep 23, 2014

Contributor

Roger that. Any other way to do it? I'm willing to write the patch and learn from it, but don't know which way to go. Suggestions?

Contributor

martincik commented Sep 23, 2014

Roger that. Any other way to do it? I'm willing to write the patch and learn from it, but don't know which way to go. Suggestions?

@segiddins

This comment has been minimized.

Show comment
Hide comment
@segiddins

segiddins Sep 23, 2014

Member

@martincik I couldn't tell you, since I have yet to see a repo where this is the issue.

Member

segiddins commented Sep 23, 2014

@martincik I couldn't tell you, since I have yet to see a repo where this is the issue.

@alloy

This comment has been minimized.

Show comment
Hide comment
@alloy

alloy Sep 23, 2014

Member

Alas, this should probably be an option somehow. Either CLI or Podfile. I’m on the fence about adding it to the Podfile, although that does seem to be what makes the most sense, as it will be the same for all people using those repos.

Member

alloy commented Sep 23, 2014

Alas, this should probably be an option somehow. Either CLI or Podfile. I’m on the fence about adding it to the Podfile, although that does seem to be what makes the most sense, as it will be the same for all people using those repos.

@martincik

This comment has been minimized.

Show comment
Hide comment
@martincik

martincik Sep 23, 2014

Contributor

We solved it by using 'git://' instead of 'https://' and it works perfectly. It's good enough for us. Thanks guys for trying to help. ;)

Contributor

martincik commented Sep 23, 2014

We solved it by using 'git://' instead of 'https://' and it works perfectly. It's good enough for us. Thanks guys for trying to help. ;)

@alloy

This comment has been minimized.

Show comment
Hide comment
@alloy

alloy Sep 23, 2014

Member

@martincik Oh! Well it’s great to know that there’s a difference there, whatever the technical reason is exactly. Thanks for that info!

Member

alloy commented Sep 23, 2014

@martincik Oh! Well it’s great to know that there’s a difference there, whatever the technical reason is exactly. Thanks for that info!

@alloy alloy closed this Sep 23, 2014

@neonichu

This comment has been minimized.

Show comment
Hide comment
@neonichu

neonichu Sep 23, 2014

Member

@martincik I was just wondering if maybe GHE does not support Git's Smart HTTP transport and that's why using git:// makes a difference here. Would be nice if you could ask their support, that might allow us to simulate this.

Member

neonichu commented Sep 23, 2014

@martincik I was just wondering if maybe GHE does not support Git's Smart HTTP transport and that's why using git:// makes a difference here. Would be nice if you could ask their support, that might allow us to simulate this.

@akisute

This comment has been minimized.

Show comment
Hide comment
@akisute

akisute Oct 1, 2014

@martincik @neonichu I have personally asked GH:E support and you're right, they have an issue over http(s) connections. Here's a reply:

The problem you are experiencing is a known issue which will be addressed in the next feature release of GitHub Enterprise.
In the mean time, you can workaround the issue by using SSH URIs for your repositories as the issue only affects cloning over HTTP(S).

So I guess we don't have to do some workaround patch to the CocoaPods itself.

akisute commented Oct 1, 2014

@martincik @neonichu I have personally asked GH:E support and you're right, they have an issue over http(s) connections. Here's a reply:

The problem you are experiencing is a known issue which will be addressed in the next feature release of GitHub Enterprise.
In the mean time, you can workaround the issue by using SSH URIs for your repositories as the issue only affects cloning over HTTP(S).

So I guess we don't have to do some workaround patch to the CocoaPods itself.

@martincik

This comment has been minimized.

Show comment
Hide comment
@martincik

martincik Oct 1, 2014

Contributor

@akisute Thanks for coming back to us with the reply. :) 👍

Contributor

martincik commented Oct 1, 2014

@akisute Thanks for coming back to us with the reply. :) 👍

@x2on

This comment has been minimized.

Show comment
Hide comment
@x2on

x2on Oct 27, 2014

@akisute @neonichu @martincik
The problem is that not only Github has this issue.
Google Code has also the problem! Could you have a look at this?

x2on commented Oct 27, 2014

@akisute @neonichu @martincik
The problem is that not only Github has this issue.
Google Code has also the problem! Could you have a look at this?

@x2on

This comment has been minimized.

Show comment
Hide comment
@sheerun

This comment has been minimized.

Show comment
Hide comment
@sheerun

sheerun Oct 28, 2014

In bower we'll whitelist hosts that support shallow clones. For now it's only GitHub. In next minor release we'll allow for adding hosts to whitelist.

sheerun commented Oct 28, 2014

In bower we'll whitelist hosts that support shallow clones. For now it's only GitHub. In next minor release we'll allow for adding hosts to whitelist.

@x2on

This comment has been minimized.

Show comment
Hide comment
@x2on

x2on Oct 28, 2014

in general the new implementation fails more often then the old one in our ci environment.

x2on commented Oct 28, 2014

in general the new implementation fails more often then the old one in our ci environment.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment