Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for Git repositories with submodules #4581

Merged
merged 1 commit into from Mar 4, 2019

Conversation

@hsyl20
Copy link
Contributor

@hsyl20 hsyl20 commented Feb 11, 2019

Add support for Git submodules (recursive or not) in Pantry. Useful when a git repository containing submodules is used as a Stack "extra-deps".

A test has been added: it tries to stack setup a package that has an "extra-deps" on a package contained in a sub-submodule of a Git repo.

Also export the function withRepo that I use to implement #4567.

  • Any changes that could be relevant to users have been recorded in the ChangeLog.md
  • The documentation has been updated, if necessary: not necessary
@hsyl20 hsyl20 force-pushed the hsyl20-git-submodules branch 2 times, most recently from e95ff8a to 0c49c9e Feb 11, 2019
@dbaynard
Copy link
Contributor

@dbaynard dbaynard commented Feb 11, 2019

Hi @hsyl20,

Thanks for the PR. I misread 'submodules' as 'subdirectories' the first time.

Am I correct to say: currently pantry doesn't support git submodules; this change introduces that support? If not, please correct me.

Also, would you please give a brief description of how your change handles recursive submodules.

There are a few issues at the moment concerning pantry and git (#4579, #4580) so I suggest @qrilka take a look at this PR at the same time as them.

Thanks!

Loading

@hsyl20
Copy link
Contributor Author

@hsyl20 hsyl20 commented Feb 11, 2019

Hi @dbaynard,

Am I correct to say: currently pantry doesn't support git submodules; this change introduces that support?

Yes exactly. Pantry uses git archive to create a tarball containing the repository files. But git archive doesn't include files from submodules.

Also, would you please give a brief description of how your change handles recursive submodules.

I use git submodule foreach --recursive to execute some commands in each submodule of the top-level repository recursively:

  • git archive --prefix=path/from/toplevel/ bar.tar to create an archive containing the files of the submodule prefixed with the relative path to the top-level repository's root directory
  • tar -Af bar.tar path/to/toplevel/archive.tar to append the submodule archive to the top-level one

Loading

@hsyl20 hsyl20 mentioned this pull request Feb 12, 2019
Copy link
Contributor

@qrilka qrilka left a comment

Overall it looks good but we need a passing integration test

Loading

test/integration/tests/git-submodules/Main.hs Show resolved Hide resolved
Loading
@hsyl20 hsyl20 force-pushed the hsyl20-git-submodules branch 2 times, most recently from 767a952 to f556c95 Feb 19, 2019
@qrilka
Copy link
Contributor

@qrilka qrilka commented Feb 19, 2019

@hsyl20 it still fails for me, now with a different error:

       Running: echo 'extra-deps:' >> stack.yaml
       Running: echo "- git: $PWD/../tmpRepo" >> stack.yaml
       Running: echo "  commit: $(cd ../tmpRepo && git rev-parse HEAD)" >> stack.yaml
       Running: echo '  subdir: sub/sub/pkg' >> stack.yaml
       Running: /home/qrilka/ws/h/stack/.stack-work/install/x86_64-linux-tinfo6/custom-snapshot-for-building-stack-with-ghc-8.2.2-M5bKQSIGuB0R/8.2.2/bin/stack setup
       Cloning 6eb1e7d2f7f8c950a8238faeb69ee32057c6f87c from /tmp/stack-integration-git-submodules13863/tmpPackage/../tmpRepo
       No cabal file found for Repo from /tmp/stack-integration-git-submodules13863/tmpPackage/../tmpRepo, commit 6eb1e7d2f7f8c950a8238faeb69ee32057c6f87c in subdir sub/sub/pkg
       Main.hs: Exited with exit code: ExitFailure 1
       CallStack (from HasCallStack):
         error, called at /home/qrilka/ws/h/stack/test/integration/lib/StackTest.hs:52:34 in main:StackTest
         stack, called at /home/qrilka/ws/h/stack/test/integration/tests/git-submodules/Main.hs:41:7 in main:Main
       )

Loading

`git archive` doesn't include files from git submodules in the generated
archive. This patch uses `git submodule foreach` to explicitly generate
an archive for each submodule and to append it to the main one with
`tar` command.

We also export `withRepo` which can be useful independently of repo
archive creation.
@hsyl20 hsyl20 force-pushed the hsyl20-git-submodules branch from f556c95 to b24fa32 Feb 19, 2019
@hsyl20
Copy link
Contributor Author

@hsyl20 hsyl20 commented Feb 19, 2019

@qrilka I have pushed a new version of the test that doesn't use a relative path ("..") in the git repo path. I am not sure it fixes the issue you get though.

Could it be that you don't have the "tar" program in your PATH? Can you show the verbose output of stack?

Loading

qrilka
qrilka approved these changes Feb 19, 2019
@qrilka
Copy link
Contributor

@qrilka qrilka commented Feb 19, 2019

@hsyl20 it looks like I just forgot to rebuild local Stack executable before running the test. Probably we need to have some way to do that automatically but that's outside of this PR for sure :)

Loading

@dbaynard dbaynard merged commit 8c90b99 into commercialhaskell:master Mar 4, 2019
8 checks passed
Loading
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

3 participants