New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

using a git project with a submodule #289

Closed
chrisdrackett opened this Issue May 25, 2011 · 12 comments

Comments

Projects
None yet
6 participants
@chrisdrackett

chrisdrackett commented May 25, 2011

We have a git project that has a submodule. At this point the git project is installed correctly, but the submodule is not updated. It would be nice if pip would grab submodules when installing a git project.

@carljm

This comment has been minimized.

Show comment
Hide comment
@carljm

carljm May 26, 2011

Contributor

My initial thought is that this is too VCS-specific to have support written into pip. In general, we try to support the lowest common denominator across VCSes but not get into VCS-specific details.

If another maintainer has another opinion, I don't feel strongly about it.

Contributor

carljm commented May 26, 2011

My initial thought is that this is too VCS-specific to have support written into pip. In general, we try to support the lowest common denominator across VCSes but not get into VCS-specific details.

If another maintainer has another opinion, I don't feel strongly about it.

@jezdez

This comment has been minimized.

Show comment
Hide comment
@jezdez

jezdez May 26, 2011

Contributor

Yeah, I feel the same about that. In case you're using fabric or similar you can easily script the submodule update in the src/<package> dir.

Contributor

jezdez commented May 26, 2011

Yeah, I feel the same about that. In case you're using fabric or similar you can easily script the submodule update in the src/<package> dir.

@jezdez jezdez closed this May 26, 2011

@chrisdrackett

This comment has been minimized.

Show comment
Hide comment
@chrisdrackett

chrisdrackett Jun 3, 2011

what about adding the --recursive flag to clone? that way it'll get the submodule without adding a lot of overhead

chrisdrackett commented Jun 3, 2011

what about adding the --recursive flag to clone? that way it'll get the submodule without adding a lot of overhead

@pnasrat

This comment has been minimized.

Show comment
Hide comment
@pnasrat

pnasrat Jun 3, 2011

Contributor

In my opinion if the repository requires a user to do extra stuff to make it work that should be encapsulated in setup.py.

recursive may be an option to consider, but we'd probably need to do client version checks - it's been available since

http://git.kernel.org/?p=git/git.git;a=commit;h=adc542353199bdc0b9b1ed5d03cf881b7efaff6e

Contributor

pnasrat commented Jun 3, 2011

In my opinion if the repository requires a user to do extra stuff to make it work that should be encapsulated in setup.py.

recursive may be an option to consider, but we'd probably need to do client version checks - it's been available since

http://git.kernel.org/?p=git/git.git;a=commit;h=adc542353199bdc0b9b1ed5d03cf881b7efaff6e

@hltbra

This comment has been minimized.

Show comment
Hide comment
@hltbra

hltbra Jun 6, 2011

Member

In my day-to-day work it is very usual to have submodules in projects, and I think we should consider --recursive or alike when cloning a git repo.

A project like Pinax could be made of some git submodules, and that's why I think we should support submodules inititialization (even if it is needed to call update --init).

Member

hltbra commented Jun 6, 2011

In my day-to-day work it is very usual to have submodules in projects, and I think we should consider --recursive or alike when cloning a git repo.

A project like Pinax could be made of some git submodules, and that's why I think we should support submodules inititialization (even if it is needed to call update --init).

@chrisdrackett

This comment has been minimized.

Show comment
Hide comment
@chrisdrackett

chrisdrackett Jun 6, 2011

Yeah, I've been setting up some new developers and this is the number 1 issue that causes problems. I suppose that we could move the process into setup.py, but it just seems to me that it should be part of pip vs. us having to edit many setup.py files.

chrisdrackett commented Jun 6, 2011

Yeah, I've been setting up some new developers and this is the number 1 issue that causes problems. I suppose that we could move the process into setup.py, but it just seems to me that it should be part of pip vs. us having to edit many setup.py files.

@carljm

This comment has been minimized.

Show comment
Hide comment
@carljm

carljm Jun 6, 2011

Contributor

I still don't think this is pip's responsibility to a degree that we ought to add additional complication to the git backend for it; the git backend is troublesome enough as-is. If we could just add --recursive to the clone command and be done, I'd be open to that, but if --recursive was just introduced in 2009 I'm not sure we can just drop support for git versions without it. And adding version checks gets back into "more complications" territory; then we have packages installing differently depending on which version of git the user has installed.

@chrisdrackett - I'm not sure I understand why you'd have to edit "many setup.py files" - do you have many Python packages whose repo uses submodules (if so, that seems questionable to me; I'd look at re-thinking that)? Or are you imagining that you'd have to handle it in the setup.py for any package dependent on the one that uses a submodule? I think you should be able to just handle it in the setup.py for the one package using a submodule; have setup.py check if its inside a git repo, and if so run the appropriate submodule update before running an "install" or "develop".

Contributor

carljm commented Jun 6, 2011

I still don't think this is pip's responsibility to a degree that we ought to add additional complication to the git backend for it; the git backend is troublesome enough as-is. If we could just add --recursive to the clone command and be done, I'd be open to that, but if --recursive was just introduced in 2009 I'm not sure we can just drop support for git versions without it. And adding version checks gets back into "more complications" territory; then we have packages installing differently depending on which version of git the user has installed.

@chrisdrackett - I'm not sure I understand why you'd have to edit "many setup.py files" - do you have many Python packages whose repo uses submodules (if so, that seems questionable to me; I'd look at re-thinking that)? Or are you imagining that you'd have to handle it in the setup.py for any package dependent on the one that uses a submodule? I think you should be able to just handle it in the setup.py for the one package using a submodule; have setup.py check if its inside a git repo, and if so run the appropriate submodule update before running an "install" or "develop".

@chrisdrackett

This comment has been minimized.

Show comment
Hide comment
@chrisdrackett

chrisdrackett Jun 6, 2011

Ok, so the big example for me at the moment is the javascript framework we're working on has a wrapper app built in django. The framework itself (a javascript repo) is a submodule of the django/python helper app. I plan on doing the same thing with common CSS/sass files that live in a submodule, etc.

So the problem isn't so much with python requirements, those are handled fine by pip, its the non-python files that cause issues.

I'm guessing that this is more rare of an issue for me as a "technical designer" but its a hard point for the developers I'm working with (and deployment) regardless.

chrisdrackett commented Jun 6, 2011

Ok, so the big example for me at the moment is the javascript framework we're working on has a wrapper app built in django. The framework itself (a javascript repo) is a submodule of the django/python helper app. I plan on doing the same thing with common CSS/sass files that live in a submodule, etc.

So the problem isn't so much with python requirements, those are handled fine by pip, its the non-python files that cause issues.

I'm guessing that this is more rare of an issue for me as a "technical designer" but its a hard point for the developers I'm working with (and deployment) regardless.

@kmmbvnr

This comment has been minimized.

Show comment
Hide comment
@kmmbvnr

kmmbvnr Mar 13, 2012

So, what is the correct solution, if submodules will not be handled by pip? Embedding VCS-specific logic into setup.py scripts seems really bad idea.

kmmbvnr commented Mar 13, 2012

So, what is the correct solution, if submodules will not be handled by pip? Embedding VCS-specific logic into setup.py scripts seems really bad idea.

@kmmbvnr

This comment has been minimized.

Show comment
Hide comment
@kmmbvnr

kmmbvnr Mar 13, 2012

If someone stucked with git submodules problem, there is ugly solution:

In setup.py

class build_with_submodules(build):
    def run(self):
        if path.exists('.git'):
            check_call(['git', 'submodule', 'init'])
            check_call(['git', 'submodule', 'update'])
        build.run(self)

setup(
    cmdclass={"build": build_with_submodules},
    ....

kmmbvnr commented Mar 13, 2012

If someone stucked with git submodules problem, there is ugly solution:

In setup.py

class build_with_submodules(build):
    def run(self):
        if path.exists('.git'):
            check_call(['git', 'submodule', 'init'])
            check_call(['git', 'submodule', 'update'])
        build.run(self)

setup(
    cmdclass={"build": build_with_submodules},
    ....
@chrisdrackett

This comment has been minimized.

Show comment
Hide comment
@chrisdrackett

chrisdrackett Mar 23, 2012

this is still a terrible pain point for us, especially as we're not deploying to heroku

chrisdrackett commented Mar 23, 2012

this is still a terrible pain point for us, especially as we're not deploying to heroku

This was referenced May 28, 2012

@carljm

This comment has been minimized.

Show comment
Hide comment
@carljm

carljm Jun 14, 2012

Contributor

This was fixed in merging #577.

Contributor

carljm commented Jun 14, 2012

This was fixed in merging #577.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment