Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Usage of alternative package indices #4137

Closed
snoyberg opened this issue Jul 7, 2018 · 7 comments

Comments

Projects
4 participants
@snoyberg
Copy link
Contributor

commented Jul 7, 2018

Stack—like cabal-install—has support for specifying alternative package indices, where the default package index is Hackage, or more specifically, FP Complete's S3-backed mirror. Providing alternative package indices is useful for two completely different use cases in my experience:

  1. Providing for packages that are not on Hackage
  2. Providing for a local mirror of Hackage, such as one behind a corporate firewall

There's no issue at all with the second use case. However, there are some major issues with the first use case:

  • It breaks the meaning of snapshots. foo-1.2.3 means "foo version 1.2.3 as uploaded on Hackage." With an alternative package index, that could mean something quite different.
  • Stack already provides a more direct way of providing non-Hackage packages: via tarball URLs or Git repos in either stack.yaml or custom snapshot files.

I would like to move in the direction of more reliably checking package hashes to ensure that build plan you're getting actually matches what a snapshot references. This may have an impact on how we deal with package indices. So I'm reaching out now to find out: is anyone using alternative package indices in ways they would be worried would be broken by changes here?

@snoyberg snoyberg added this to the P1: Must milestone Jul 7, 2018

@centromere

This comment has been minimized.

Copy link

commented Jul 7, 2018

We use stack in production and would not be impacted by your changes.

@DanBurton

This comment has been minimized.

Copy link
Contributor

commented Jul 7, 2018

I would like to move in the direction of more reliably checking package hashes to ensure that build plan you're getting actually matches what a snapshot references.

I was under the impression that it already does this. Specifically, I thought that since stackage snapshots specify hashes, that these were used in order to select the desired revision of a package. Is this behavior disabled by the use of alternative package indices?

@DanBurton

This comment has been minimized.

Copy link
Contributor

commented Jul 7, 2018

I see a 3rd potential use case, which would be something similar to HEAD.hackage, facilitating the development and testing of packages against GHC HEAD (or alpha/rc ghc versions) and corresponding pre-release versions of libraries.

I am unaware of anyone currently using stack's alternative package index in this way, but I'd like for it to be something that people start doing.

@snoyberg

This comment has been minimized.

Copy link
Contributor Author

commented Jul 7, 2018

That's actually an example of exactly the kind of use case I'd like to avoid: there's no way to ensure reproducibility of builds. I'd rather see a tool that creates custom snapshots that use a specific set of packages from an overlay.

@DanBurton

This comment has been minimized.

Copy link
Contributor

commented Jul 8, 2018

Good point. I agree 100%

So in such a scenario, the custom snapshot based on an overlay would still need some way to reference things that aren't available on the usual hackage package index. As long as there is some sort of "download by hash", it doesn't really matter where it comes from.

@snoyberg snoyberg added this to Needs review in Pantry Aug 7, 2018

@snoyberg snoyberg closed this in 7aecad8 Aug 26, 2018

Pantry automation moved this from Not merged to Done Aug 26, 2018

vvv added a commit to vvv/stack that referenced this issue Sep 9, 2018

doc: Don't break `issue #` links
When a link is wrapped and new line starts with '#', Markdown renderers
that line as a header and the result looks ugly.

| [issue
| commercialhaskell#4137](commercialhaskell#4137)

vvv added a commit to vvv/stack that referenced this issue Sep 9, 2018

doc: Don't break `issue #` links
When a link is wrapped and new line starts with '#', Markdown renders
that line as a header and the result looks ugly.

| [issue
| commercialhaskell#4137](commercialhaskell#4137)
@gridaphobe

This comment has been minimized.

Copy link

commented Mar 27, 2019

Sorry to be so late to the discussion here...

At Bloomberg we have an internal mirror of Hackage that we also publish internal packages to, i.e. we do both (1) and (2). This is very convenient because we have a single place to download packages and browse documentation, we can have internal haddocks cross-link to public packages, etc.

I understand the concern about breaking snapshots due to name clashes, and don't have a problem with checking hashes more aggressively. If, somehow, we got to a point where foo-1.2.3 pointed to a different package in our internal mirror, I would want stack to complain! But we've been careful about package names and haven't had any issues so far, so it would be unfortunate if stack made changes that broke the broader workflow of having an internal Hackage mirror that also contains private packages.

That being said, it's not clear to me whether stack has actually made any changes that would break our workflow. We configure stack with a single package index, our internal mirror, and use the newer 01-index. Package revisions have been an issue for us, the mirror tool doesn't seem to mirror them correctly, but I'd also like to fix that if possible.

@snoyberg

This comment has been minimized.

Copy link
Contributor Author

commented Mar 28, 2019

As described, I think the changes to Stack should work without problem for the use case you describe.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.