Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

--only-dependencies instead of --only-snapshot #387

Closed
gregwebs opened this issue Jun 23, 2015 · 12 comments
Closed

--only-dependencies instead of --only-snapshot #387

gregwebs opened this issue Jun 23, 2015 · 12 comments
Milestone

Comments

@gregwebs
Copy link
Contributor

@gregwebs gregwebs commented Jun 23, 2015

As a stack user, I don't know what the terminology "snapshot" means. Also, I am not sure if it means only the resolver or if extra-deps are included. The behavior that I would like is for extra-deps to be included (well at least the ones that are from hackage tip, maybe not a local directory) so it is more similar to cabal install --only-dependencies.

@gregwebs gregwebs changed the title better name for --only-snapshot --only-dependencies instead of --only-snapshot Jun 23, 2015
@gregwebs
Copy link
Contributor Author

@gregwebs gregwebs commented Jun 23, 2015

--only-snapshot ignores extra-deps that are just packages from hackage instead of the resolver. For my continuous integration build I need a --only-dependencies flag that install all dependencies. So I am changing the ticket name.

@snoyberg
Copy link
Contributor

@snoyberg snoyberg commented Jun 24, 2015

See the discussion in #310 to see how we got to where we are now.

@gregwebs
Copy link
Contributor Author

@gregwebs gregwebs commented Jun 24, 2015

#310 seems to say that --only-dep is used for ci builds. This ticket is to take the concept to its logical conclusion of installing extra deps

@snoyberg
Copy link
Contributor

@snoyberg snoyberg commented Jun 24, 2015

Sorry, I just realized that some of that discussion happened outside of the issue. The basic idea is: there are weird corner cases that come into play if we try to install only dependencies, specifically because of cycles between dependencies and local packages. Consider working on the wai package: some of the dependencies (like wai-logger) depend on non-dependencies (wai). Do we install wai and wai-logger, or not? The obvious cut-off was to just install whatever would go into the snapshot database and drop anything local/custom.

That's not to say that there isn't value in doing something better, I'm just explaining how we got to the current place. If someone wants to put in the time to figure out a better solution, that would be good. But it doesn't seem like this is high-value enough to be worth it (unless you have use cases I'm not anticipating).

@snoyberg snoyberg added this to the Later improvements milestone Jun 24, 2015
@gregwebs
Copy link
Contributor Author

@gregwebs gregwebs commented Jun 24, 2015

For wai-logger, sure you cannot install that one ahead of time. But that seems like a rare corner case (that should be handled properly). The usual case is just wanting packages from the head of hackage instead of the LTS. And for some projects, that could mean a lot or packages.

For us it is a big deal because even though we only have a few packages, it slows down every CI build. We need a step to cache as much as possible, and obviously we don't want to cache the actual build.

However, I think I can work around this for now by just manually installing the extra-deps.

@gregwebs
Copy link
Contributor Author

@gregwebs gregwebs commented Jun 24, 2015

Here is my current workaround to install everything in extra-deps:

# install extra-deps from stack.yaml
# This is hacky, there is probably a better way
# look for the leading '-' in the yaml file and the ending '-' with a version number
stack install $(cat stack.yaml | grep -e '^\s*-.*-[0-9]' | awk '{print $2}' | xargs)
@zimbatm
Copy link

@zimbatm zimbatm commented Jul 6, 2015

In my scenario the project source files aren't available during the build step. Basically I want to build all the dependencies except the top-level ones.

stack build --only-snapshot
# Install extra-deps
grep -e '^\s*-.*-[0-9]' stack.yaml | awk '{print $2}' | xargs stack build
# Install github dependencies
grep 'git: git@github.com:' stack.yaml | sed -e 's/.*:.*\/\(.*\).git.*/\1/g' | xargs stack build

Instead of --only-snapshot having a --excluded <list-of-packages> would also work for me.

@gregwebs
Copy link
Contributor Author

@gregwebs gregwebs commented Jul 17, 2015

Another related issue for me is that the extra dependencies are installed into the .stack-work directory: this makes them difficult to cache between CI builds. Perhaps more "nix-like" package management can solve this issue, although I could also solve it with a custom snapshot.

@gregwebs
Copy link
Contributor Author

@gregwebs gregwebs commented Jul 25, 2015

We are solving this issue by creating a custom snapshot. That is a good enough solution for me. Perhaps this workflow should just be documented when the next release is made.

@snoyberg
Copy link
Contributor

@snoyberg snoyberg commented Aug 9, 2015

See plan laid out at: #651 (comment)

@snoyberg snoyberg closed this Aug 9, 2015
snoyberg added a commit that referenced this issue Aug 11, 2015
@snoyberg
Copy link
Contributor

@snoyberg snoyberg commented Aug 11, 2015

Implemented for #651, please review PR #753.

@gregwebs
Copy link
Contributor Author

@gregwebs gregwebs commented Aug 12, 2015

This is working great for us! We are also going to create a custom snapshot now and watch our build time plummet :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
3 participants