Support subdirectories with remote tarball deps in cabal.project

[brandon-leapyear]

Related to: #2189

Currently, one can add

source-repository-package
  type: git
  location: git://...
  tag: ...

to use a git repo to download a project to use as a dependency. I would like to explicitly specify a URL to a tarball that can be unzipped and used.

This is useful because when making a PR from a forked repo, GitHub adds the commit to the original repo's set of commits. For example, if I fork user1/my-repo to user2/my-repo, add commit A to the user2/my-repo fork, and make a PR, GitHub allows downloading the commit from https://github.com/user1/my-repo/archive/A.tar.gz. However, cloning the original user1/my-repo repo and doing git checkout A does not work.

One could workaround this with

packages:
  https://github.com/owner/repo/commit.tar.gz

which works in most use-cases, but doesn't work if the Cabal project is within a subdirectory of the archive (e.g. a monorepo)

This is already supported in Stack: https://docs.haskellstack.org/en/stable/yaml_configuration/#extra-deps

[gbaz]

Out of curiosity, why not simply point at the forked repo instead?

[brandon-leapyear]

✨ This is an old work account. Please reference @brandonchinn178 for all future communication ✨

---

Because after the PR is merged, the author is free to delete the forked repo. I know for myself, I always delete the forked repo after my PR is merged

[phadej]

I would like to explicitly specify a URL to a tarball that can be unzipped and used.

You can say packages: https://.../mypackage.tar.gz, though that package should be sdist tarball. If we could tell cabal-install which directory to pick up inside tarball (instead of looking for the only pkg-id one), we would get something less adhoc.

[brandon-leapyear]

✨ This is an old work account. Please reference @brandonchinn178 for all future communication ✨

---

Would that do the equivalent of source-repository-package? In stack, packages means "the packages in this project", not third party dependencies

[phadej]

URL packages are not local packages, so e.g. cabal test doesn't run their tests etc. source-repository-package is another way to add package, which is also non-local.

[brandon-leapyear]

✨ This is an old work account. Please reference @brandonchinn178 for all future communication ✨

---

makes sense. so conceptually speaking, why add a source-repository-package stanza instead of just allowing git://... URLs in the packages field? (with some syntax to specify the commit)

And there's still no way to specify a subdirectory within a tarball using this packages: method, right?

[phadej]

git://... URLs in the packages field? (with some syntax to specify the commit)

Yes, that would make sense (though wiring things might be painful with current project file handling code).

And there's still no way to specify a subdirectory within a tarball using this packages: method, right?

No. Nobody needed it. (For example I sdist and either add to the repository, upload to a server or make candidate releases. EDIT: vendoring tarballs into the project's repository works very nicely. Once checked out, they are there).

[brandon-leapyear]

✨ This is an old work account. Please reference @brandonchinn178 for all future communication ✨

---

Ok, so I did confirm this works, unless I need to access a subdirectory, so I'll update the issue to be specifically about supporting subdirs for remote tarballs.

[brandon-leapyear]

✨ This is an old work account. Please reference @brandonchinn178 for all future communication ✨

---

You can say packages: https://.../mypackage.tar.gz, though that package should be sdist tarball.

Ah so this doesn't quite work with GitHub's archiving then. It would be nice if there could be some way of doing:

1. Specifying a remote tarball
2. Specifying a subdirectory in the tarball
3. Supporting both sdist format (all files within another subdirectory named <name>-<version>) and plain format (the .cabal file is at the root of the tarball)

[Mikolaj]

Sounds sensible. Any volunteers for any part?

[codygman]

This would be useful to workaround full cloning large repos like Amazonka that don't yet have a hackage release such as this stack user did.