Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for Git repositories with submodules #4581

Merged
merged 1 commit into from Mar 4, 2019

Conversation

@hsyl20
Copy link
Contributor

commented Feb 11, 2019

Add support for Git submodules (recursive or not) in Pantry. Useful when a git repository containing submodules is used as a Stack "extra-deps".

A test has been added: it tries to stack setup a package that has an "extra-deps" on a package contained in a sub-submodule of a Git repo.

Also export the function withRepo that I use to implement #4567.

  • Any changes that could be relevant to users have been recorded in the ChangeLog.md
  • The documentation has been updated, if necessary: not necessary

@hsyl20 hsyl20 force-pushed the hsyl20:hsyl20-git-submodules branch 2 times, most recently from e95ff8a to 0c49c9e Feb 11, 2019

@dbaynard

This comment has been minimized.

Copy link
Contributor

commented Feb 11, 2019

Hi @hsyl20,

Thanks for the PR. I misread 'submodules' as 'subdirectories' the first time.

Am I correct to say: currently pantry doesn't support git submodules; this change introduces that support? If not, please correct me.

Also, would you please give a brief description of how your change handles recursive submodules.

There are a few issues at the moment concerning pantry and git (#4579, #4580) so I suggest @qrilka take a look at this PR at the same time as them.

Thanks!

@hsyl20

This comment has been minimized.

Copy link
Contributor Author

commented Feb 11, 2019

Hi @dbaynard,

Am I correct to say: currently pantry doesn't support git submodules; this change introduces that support?

Yes exactly. Pantry uses git archive to create a tarball containing the repository files. But git archive doesn't include files from submodules.

Also, would you please give a brief description of how your change handles recursive submodules.

I use git submodule foreach --recursive to execute some commands in each submodule of the top-level repository recursively:

  • git archive --prefix=path/from/toplevel/ bar.tar to create an archive containing the files of the submodule prefixed with the relative path to the top-level repository's root directory
  • tar -Af bar.tar path/to/toplevel/archive.tar to append the submodule archive to the top-level one
@hsyl20 hsyl20 referenced this pull request Feb 12, 2019
@qrilka
Copy link
Contributor

left a comment

Overall it looks good but we need a passing integration test

@hsyl20 hsyl20 force-pushed the hsyl20:hsyl20-git-submodules branch 2 times, most recently from 767a952 to f556c95 Feb 19, 2019

@qrilka

This comment has been minimized.

Copy link
Contributor

commented Feb 19, 2019

@hsyl20 it still fails for me, now with a different error:

       Running: echo 'extra-deps:' >> stack.yaml
       Running: echo "- git: $PWD/../tmpRepo" >> stack.yaml
       Running: echo "  commit: $(cd ../tmpRepo && git rev-parse HEAD)" >> stack.yaml
       Running: echo '  subdir: sub/sub/pkg' >> stack.yaml
       Running: /home/qrilka/ws/h/stack/.stack-work/install/x86_64-linux-tinfo6/custom-snapshot-for-building-stack-with-ghc-8.2.2-M5bKQSIGuB0R/8.2.2/bin/stack setup
       Cloning 6eb1e7d2f7f8c950a8238faeb69ee32057c6f87c from /tmp/stack-integration-git-submodules13863/tmpPackage/../tmpRepo
       No cabal file found for Repo from /tmp/stack-integration-git-submodules13863/tmpPackage/../tmpRepo, commit 6eb1e7d2f7f8c950a8238faeb69ee32057c6f87c in subdir sub/sub/pkg
       Main.hs: Exited with exit code: ExitFailure 1
       CallStack (from HasCallStack):
         error, called at /home/qrilka/ws/h/stack/test/integration/lib/StackTest.hs:52:34 in main:StackTest
         stack, called at /home/qrilka/ws/h/stack/test/integration/tests/git-submodules/Main.hs:41:7 in main:Main
       )
Add support for repositories with submodules
`git archive` doesn't include files from git submodules in the generated
archive. This patch uses `git submodule foreach` to explicitly generate
an archive for each submodule and to append it to the main one with
`tar` command.

We also export `withRepo` which can be useful independently of repo
archive creation.

@hsyl20 hsyl20 force-pushed the hsyl20:hsyl20-git-submodules branch from f556c95 to b24fa32 Feb 19, 2019

@hsyl20

This comment has been minimized.

Copy link
Contributor Author

commented Feb 19, 2019

@qrilka I have pushed a new version of the test that doesn't use a relative path ("..") in the git repo path. I am not sure it fixes the issue you get though.

Could it be that you don't have the "tar" program in your PATH? Can you show the verbose output of stack?

@qrilka

qrilka approved these changes Feb 19, 2019

@qrilka

This comment has been minimized.

Copy link
Contributor

commented Feb 19, 2019

@hsyl20 it looks like I just forgot to rebuild local Stack executable before running the test. Probably we need to have some way to do that automatically but that's outside of this PR for sure :)

@dbaynard dbaynard merged commit 8c90b99 into commercialhaskell:master Mar 4, 2019

8 checks passed

commercialhaskell.stack Build #20190219.7 succeeded
Details
commercialhaskell.stack (Linux nightly) Linux nightly succeeded
Details
commercialhaskell.stack (Linux pedantic) Linux pedantic succeeded
Details
commercialhaskell.stack (Linux stack-def) Linux stack-def succeeded
Details
commercialhaskell.stack (Linux stack-lts-12) Linux stack-lts-12 succeeded
Details
commercialhaskell.stack (Linux style) Linux style succeeded
Details
commercialhaskell.stack (Windows stack) Windows stack succeeded
Details
commercialhaskell.stack (macOS stack-def) macOS stack-def succeeded
Details
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.