New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add --test and --bench options to build command #651

Closed
snoyberg opened this Issue Jul 21, 2015 · 9 comments

Comments

Projects
None yet
6 participants
@snoyberg
Contributor

snoyberg commented Jul 21, 2015

Motivation: you could write something like the following:

stack build --test --bench --haddock --copy-bins

This just makes the commands far more composable than they are right now. We should not remove or deprecate stack test and stack bench IMO: they are useful shortcuts for common cases.

@snoyberg snoyberg added this to the 0.3.0.0 milestone Jul 21, 2015

@borsboom

This comment has been minimized.

Contributor

borsboom commented Jul 29, 2015

What do you think about an --exec option to execute a command after the build (particularly useful in conjunction with --file-watch)? It's a slightly different case than the others because stack exec isn't one of the build commands (unlike stack test etc), and currently uses the execve system call, so the stack build --exec option would probably have to follow a different code path than stack exec.

@snoyberg

This comment has been minimized.

Contributor

snoyberg commented Jul 29, 2015

I'm in favor of it, though we will probably need to bikeshed the name to
death.

On Wed, Jul 29, 2015, 8:51 AM Emanuel Borsboom notifications@github.com
wrote:

What do you think about an --exec option to execute a command after the
build (particularly useful in conjunction with --file-watch)? It's a
slightly different case than the others because stack exec isn't one of
the build commands (unlike stack test etc), and currently uses the execve
system call, so the stack build --exec option would probably have to
follow a different code path than stack exec.


Reply to this email directly or view it on GitHub
#651 (comment)
.

@snoyberg snoyberg modified the milestones: 0.2.0.0, 0.3.0.0 Aug 9, 2015

@snoyberg snoyberg self-assigned this Aug 9, 2015

@snoyberg

This comment has been minimized.

Contributor

snoyberg commented Aug 9, 2015

I'd like to write up clearly my plans for overhauling how targets, "wanted"s,
and components will be handled. The goals of this overhaul are to make
tests/benchmarks more modular, make error reporting of bad arguments more
reliable, and make it possible to do things like running a single test suite. This procedure should address the following issues:

  1. stack test is a synonym for stack build --test. Similarly, stack bench is a synonym for stack build --bench. At this point: --copy-bins, --haddock, --test, and --bench are composable with each other.
  2. When running stack build, you provide a list of targets. Targets can be in one of the following formats:
    • package[:comptype]:compname, e.g. bar:exe:bar-exe1. This will specify as a target the given component of a local package (fails for any non-local package). If the package name is not available locally, this will be an error. comptype can be lib, exe, test, and bench. When using lib, :compname must not be used.
    • package:compname, e.g. bar:bar-exe1. Same as above, but determines the comptype from the compname. This will fail if the same compname is used by two different comptypes in a package (e.g., both a executable and test suite named foobar).
    • :compname, e.g. :bar-exe1. Same as above, but also determines the package name. Will fail for any ambiguity.
    • package, e.g., foo. Fails for non-local package. For a local package, adds all of the library and executables components available in the given package. If --test is passed in, all test suites are also added. If --bench is passed in, all benchmarks are also added.
    • directory, e.g., ./subdir/bar. Finds all of the locals that are in that directory a subdirectory therefore, gets their package names, and then passes each of those as a package
    • package-identifier, e.g., foo-1.2.3: pull that exact version from the package index.
      • If a package by that name is in the locals:
        • Same version: same as passing in foo as a target
        • Different version: results in an error
      • If that package is in extra-deps:
        • Same version: same as passing in foo as a target
        • Different version: results in an error
      • If that package is in the snapshot:
        • Same version: same as passing in foo as a target
        • Different version: implicitly adds foo-1.2.3 to extra-deps and then the same as passing in foo as a target
  3. If no targets are specified on the command line, same as passing in the package names of all locals.
  4. After parsing input, check for any duplicate package references on the command line, e.g. stack build foo foo-1.2.3. If two different arguments refer to the same package name, errors out.
  5. If no targets exist after this parsing, errors out.
  6. Take all of these targets (a.k.a. wanteds) and track down their dependencies. For each package depended on, mark its library and all executable components as dependencies. Only mark targets as dependencies if they are depended on by other dependencies, not other targets, so that --only-dependencies has the correct behavior.
  7. If --only-dependencies is specified, ignore any packages/components that are not marked as being a dependency.
  8. Construct a list of actions to perform with dependencies between each, and execute them in parallel.
  9. When building test suite and benchmark components, also run them unless --no-run-tests or --no-run-benchmarks (respectively) is passed in.
  10. If any builds fail or test suites fail, error out.
  11. If --copy-bins is on: copy all generated executables to the local bin path
  12. Run any --exec foo commands

Note: above doesn't explicitly take --haddock into account, assuming it will just follow from the existing codebase.

Pinging @borsboom @chrisdone

@chrisdone

This comment has been minimized.

Member

chrisdone commented Aug 9, 2015

Sounds good to me. I think that perhaps the stack ghci and stack ide can piggy back onto this target parsing, first for ensuring it builds and second for choosing which targets and components to load.

If --only-dependencies is specified, ignore any packages/components that are not marked as being a dependency.

Presumably this will take into account when you do stack build a b and a depends on b, that it should probably build just the dependencies of a and b.

package:comptype[:compname], e.g. bar:exe:bar-exe1. This will specify as a target the given component of a local package (fails for any non-local package). If the package name is not available locally, this will be an error. comptype can be lib, exe, test, and bench. When using lib, :compname must not be used.

As a would-be-nice enhancement for the command-line implementation of this, we can possibly provide completions via the optparse API. That'd make it easier to write stack foo TAB and get stack foo: with lib, exe, bench listed then stack foo:exe: TAB etc.

@snoyberg

This comment has been minimized.

Contributor

snoyberg commented Aug 9, 2015

Presumably this will take into account when you do stack build a b and a depends on b, that it should probably build just the dependencies of a and b.

Good point, and not reflected above. The heuristic above should be "mark something as a dependency if it's a dependency of a dependency, or if it's a dependency of a target and not a target itself." EDIT I updated the comment above to include this.

Providing autocompletion would be nice, but I don't intend to include it in the initial implementation.

snoyberg added a commit that referenced this issue Aug 10, 2015

snoyberg added a commit that referenced this issue Aug 10, 2015

snoyberg added a commit that referenced this issue Aug 11, 2015

@snoyberg

This comment has been minimized.

Contributor

snoyberg commented Aug 11, 2015

I've opened up PR #753 with these changes, closing this issue so we can discuss on the PR.

@snoyberg snoyberg closed this Aug 11, 2015

@snoyberg snoyberg removed the in progress label Aug 11, 2015

@WillSewell

This comment has been minimized.

Contributor

WillSewell commented Nov 23, 2015

When I run stack test mypackage:mytestssuite1 only mytestsuite1 is run, and no other tests suites are. But when I ruck stack bench mypackage:mybenchmarks1, all other benchmarks are also built and run. Am I misunderstanding something?

Stack version 0.1.6.0

@rvion

This comment has been minimized.

Contributor

rvion commented Nov 23, 2015

@WillSewell Could you open a new issue a reference this one ?

@mgsloan

This comment has been minimized.

Collaborator

mgsloan commented Nov 23, 2015

@WillSewell Also, please try with version 0.1.8.0, since a lot has changed since 0.1.6.0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment