New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Run test suites without rebuilding executables #398

Closed
lostman opened this Issue Jun 24, 2015 · 6 comments

Comments

Projects
None yet
2 participants
@lostman

lostman commented Jun 24, 2015

Running stack test also builds all executables. Using fast-tags package as an example :

❯ stack test        
fast-tags-1.1.0: configure (test)
Configuring fast-tags-1.1.0...
fast-tags-1.1.0: build (test)
Building fast-tags-1.1.0...
Preprocessing library fast-tags-1.1.0...
In-place registering fast-tags-1.1.0...
Preprocessing executable 'fast-tags' for fast-tags-1.1.0...
Linking .stack-work/dist/x86_64-osx/Cabal-1.18.1.5/build/fast-tags/fast-tags ...
Preprocessing test suite 'test-fast-tags' for fast-tags-1.1.0...
Linking .stack-work/dist/x86_64-osx/Cabal-1.18.1.5/build/test-fast-tags/test-fast-tags ...
fast-tags-1.1.0: test (suite: test-fast-tags)
tests
  tokenize
    "a::b->c":                                                                OK (0.02s)
(...)

I can simulate the desired behavior by defining a testonly flag and marking all executables with if flag(testonly) buildable: False. With executables:

time stack test
stack test  5.79s user 1.60s system 127% cpu 5.777 total

And without:

time stack test --flag fast-tags:test
stack test --flag fast-tags:test  4.73s user 1.45s system 130% cpu 4.724 total

These are incremental, not clean, builds. Skipping executables makes running tests slightly faster. This is particularly useful when there are many executables and only a single test-suite.

@snoyberg

This comment has been minimized.

Contributor

snoyberg commented Jun 24, 2015

I don't believe runghc Setup configure has an option to disable executable building unfortunately, though it would be nice. Instead, you could try using components so force only some things to build, though I haven't tested this, e.g. stack test packagename:test:testname.

@lostman

This comment has been minimized.

lostman commented Jun 24, 2015

@snoyberg I tried this. It is equivalent to stack test. Moreover, if there are many test suites picking a specific one doesn't work:

❯ stack test fast-tags:test-fast-tags
fast-tags-1.1.0: build (test)
Building fast-tags-1.1.0...
Preprocessing library fast-tags-1.1.0...
In-place registering fast-tags-1.1.0...
Preprocessing executable 'fast-tags' for fast-tags-1.1.0...
Linking .stack-work/dist/x86_64-osx/Cabal-1.18.1.5/build/fast-tags/fast-tags ...
Preprocessing test suite 'test-fast-tags' for fast-tags-1.1.0...
Linking
.stack-work/dist/x86_64-osx/Cabal-1.18.1.5/build/test-fast-tags/test-fast-tags
...
Preprocessing test suite 'test-fast-tags2' for fast-tags-1.1.0...
Linking
.stack-work/dist/x86_64-osx/Cabal-1.18.1.5/build/test-fast-tags2/test-fast-tags2
...
fast-tags-1.1.0: test (suite: test-fast-tags)
tests
  tokenize
    "a::b->c":                                                                OK
(...)
All 250 tests passed (0.06s)
fast-tags-1.1.0: test (suite: test-fast-tags2)
tests
  tokenize
    "a::b->c":                                                                OK
(...)
All 250 tests passed (0.05s)

Here I defined a duplicate test-fast-tags2 test suite.

@lostman

This comment has been minimized.

lostman commented Jun 24, 2015

Also, with cabal it is possible to do something like this:

cabal install --only-dependencies --enable-tests
cabal build my-test-suite
./dist/.../my-test-suite

Not ideal, but does the trick. I don't think there's a way to do the same with stack:

❯ stack build fast-tags:test-fast-tags                      
fast-tags-1.1.0: build
Setup.hs: Cannot build the test suite 'test-fast-tags' because test suites are
not enabled. Run configure with the flag --enable-tests

snoyberg added a commit that referenced this issue Jun 24, 2015

@snoyberg snoyberg added this to the 0.2.0.0 milestone Jun 24, 2015

@snoyberg

This comment has been minimized.

Contributor

snoyberg commented Jun 24, 2015

I confirmed that in fact it was not working previously. I've just pushed a commit that should address it. Can you give it a shot?

@lostman

This comment has been minimized.

lostman commented Jun 25, 2015

👍

Works great. Thanks!

Only the first test suite:

(master ✗) fast-tags stack test fast-tags:test-fast-tags
fast-tags-1.1.0: configure (test)
Configuring fast-tags-1.1.0...
fast-tags-1.1.0: build (test)
Preprocessing library fast-tags-1.1.0...
In-place registering fast-tags-1.1.0...
Preprocessing test suite 'test-fast-tags' for fast-tags-1.1.0...
Linking .stack-work/dist/x86_64-linux/Cabal-1.18.1.5/build/test-fast-tags/test-fast-tags ...
fast-tags-1.1.0: test (suite: test-fast-tags)
tests
(...)

Only the second:

(master ✗) fast-tags stack test fast-tags:test-fast-tags2
fast-tags-1.1.0: configure (test)
Configuring fast-tags-1.1.0...
fast-tags-1.1.0: build (test)
Preprocessing library fast-tags-1.1.0...
In-place registering fast-tags-1.1.0...
Preprocessing test suite 'test-fast-tags2' for fast-tags-1.1.0...
Linking .stack-work/dist/x86_64-linux/Cabal-1.18.1.5/build/test-fast-tags2/test-fast-tags2 ...
fast-tags-1.1.0: test (suite: test-fast-tags2)
tests
(...)

And both:

(master ✗) fast-tags stack test
fast-tags-1.1.0: configure (test)
Configuring fast-tags-1.1.0...
fast-tags-1.1.0: build (test)
Preprocessing library fast-tags-1.1.0...
In-place registering fast-tags-1.1.0...
Preprocessing test suite 'test-fast-tags' for fast-tags-1.1.0...
Linking .stack-work/dist/x86_64-linux/Cabal-1.18.1.5/build/test-fast-tags/test-fast-tags ...
Preprocessing test suite 'test-fast-tags2' for fast-tags-1.1.0...
Linking .stack-work/dist/x86_64-linux/Cabal-1.18.1.5/build/test-fast-tags2/test-fast-tags2 ...
fast-tags-1.1.0: test (suite: test-fast-tags)
tests
(...)
@snoyberg

This comment has been minimized.

Contributor

snoyberg commented Jun 25, 2015

Cool, thanks for reporting and confirming!

@snoyberg snoyberg closed this Jun 25, 2015

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment