Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Extensible snapshots #3249

Merged
merged 72 commits into from Jul 13, 2017
Merged

Extensible snapshots #3249

merged 72 commits into from Jul 13, 2017

Conversation

@snoyberg
Copy link
Contributor

@snoyberg snoyberg commented Jul 5, 2017

This is a significant change touching large parts of the code base and relating to many issues. I'm going to do a write-up of what ended up getting covered here (I'll link to it from this issue when ready), as well as try to collect issues that I believe are related.

At the time of opening this PR, all tests and integration tests pass on my OS X machine, and I'm currently using this branch for my main development. Nonetheless, there are still some outstanding issues, such as a Store hash mismatch on GHC 7.10, a few FIXMEs scattered through the PR that I'd like to improve upon. Also, additional real world testing is needed.

There's not really any kind of logical grouping of these commits that I could rebase to. The best I could do is split off the first two commits if desired and squash the rest down into one mega-commit. If people prefer that, let me know.

EDIT Here's a work in progress write-up about the new feature: https://fpcomplete-site-extensible-snapshots.review.gitlab.fpcomplete.com/blog/2017/07/stacks-new-extensible-snapshots

Related issues:

  • Build-tools not detected when not using Stackage snapshot #595
  • Implicit snapshots (aka: Unify extra-deps and extensible snapshots) #1265
  • Stack cannot find happy during build #3178
  • stack builds happy again and again #3151
  • Failure to locate binary while installing cabal-helper #3202
  • Unify behaviour of flags and ghc-options #849
  • Better error messages for "invariant violated: multiple results when describing installed package" #1408
snoyberg added 30 commits Jun 28, 2017
This came in automatically at the inception of Stack, by copying data
types from stackage-curator. In reality, we don't need all of this
information within Stack. As we move towards extensible snapshots, we
want to make sure we have the bare minimum of information to allow a
shared data type for parsing all kinds of snapshots.

Additionally, we probably want to move away from depending on extra
information present in the build plan, in case there are mistakes in it.
Because (1) we don't necessarily trust it (may be wrong upstream, or
using a different GHC install locally), and (2) custom snapshots don't
want to provide this.

This patch is not complete on its own. In particular, it's possible
(such as with the script command) to try to load up the global package
information before GHC is installed. Next step is to have a proper
separation between a resolved and unresolved build plan.
This is still a WIP, with lots of commented out code and the naming
still up in the air. But it compiles.
Not yet complete; ultimate goal would be to unify a huge amount of the
logic between these two modules so that the shadowing behavior and
similar is identical between "within snapshots" and "local on top of
snapshots".
@dkasak
Copy link
Contributor

@dkasak dkasak commented Jul 5, 2017

Just to clarify, when you mention in the write-up that

In my ideal world:

  • The packages key would accept a list of filepaths

do you mean the packages key of stack.yaml, but not the packages key of custom snapshots?

Loading

@snoyberg
Copy link
Contributor Author

@snoyberg snoyberg commented Jul 5, 2017

I didn't, but I do now after reading through the previous issues. I'll be pushing a commit for that (and updating the writeup).

EDIT Both the code and the writeup have been updated.

Loading

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

2 participants