line numbers for failures #4

Closed
gregwebs opened this Issue Mar 1, 2011 · 9 comments

Projects

None yet

3 participants

@gregwebs
hspec member

Given the descriptive nature of hspec, this is a less pressing concern. Right now one can look at the string for the failure, and find that in the source. However this process could be automated to having hspec report a line number.

HTF provides line numbers by running a pre-processor that replaces assertions with assertions with line numbers. Perhaps that could be used directly. Otherwise, a simpler pre-processor for hspec could could just give line numbers for the 'it' string without figuring out the exact test failure line.

http://hackage.haskell.org/packages/archive/HTF/0.7.0.0/doc/html/Test-Framework-Tutorial.html

@gregwebs
hspec member

here is another idea for line numbers- use template haskell.
This should be possible given that this package seems to somehow reflect on the existing source file:
http://hackage.haskell.org/package/test-framework-th

Instead of replacing functions, the template haskell would record line number information that the runner would match up with failed tests.

main = $(hspecWithLineNos specs)
@trystan
hspec member

I was thinking of having an external tool (also called hspec) that would take a file as input and run any specs in there. Since it's looking at the file it could report file number or even the example code itself. This could be a halfway solution until we do something like that.

@gregwebs
hspec member

Now that cabal test is being released, I think that is going to be the preferred way to test. Mostly because you can exactly specify a test build, but hspec runtests.hs can have a dependency failure. So if the tool can somehow integrate with cabal test it will continue to be useful in the future. But I don't know how this is possible to do and insert line numbers.

@sol
hspec member
sol commented Sep 6, 2012

@gregwebs I think this should ultimately be addressed by GHC (see e.g. my proposal for source locations).

Would it be fine with you to close this as "won't fix"?

@gregwebs
hspec member

ok. rather than your proposal I think Haskell needs real stack traces for other purposes also. We should probably provide a simple $assert function.

@gregwebs gregwebs closed this Sep 6, 2012
@gregwebs
hspec member

I am adding these functions to the file-location package: http://hackage.haskell.org/package/file-location-0.4.8/docs/Test-FileLocation.html

@sol
hspec member
sol commented Jul 3, 2015

👍

I still hope we get source locations into GHC soon.

@sol
hspec member

@gregwebs we have support for source locations on Hackage now.

@gregwebs
hspec member

fantastic! One more reason I will quickly be upgrading to 7.10.2

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment