The issue is, that if you use hspec instead of hspecX in a cabal test suite, cabal test will not report failing tests.
This is not a theoretical issue, at least http-types does it wrong.
So I'd suggest to rename hspecX to hspec, and keep hspecX as an alias. In the long run I'd tend to first deprecate and later remove hspecX.
@trystan @gregwebs Are you aware of any use cases for hspec? If I understand correctly, it would theoretically allow you to re-create the report from an evaluate spec (possibly with a different formatter). But we have no code to support this; and the user could always just reevaluate the spec to achieve the same thing. Am I missing something?
BTW: From a discussion with @markwright (related to yesodweb/yesod@9f4c92f) I realized that the steps required to update a non-monadic spec for hspec-1.0 can in some cases be less obvious than I thought. I made some changes that will give better error messages (basically I newtype-wrapped UnevaluatedSpec). I'll push a 1.1 release that includes that change on Hackage today.
I created the hspecX function, and that is the only one that I have ever used.
The hspec function is used by Specs.hs because it needs to test that the test runners and formatters work as expected without exiting. See https://github.com/hspec/hspec/blob/master/Specs.hs#L116 and https://github.com/hspec/hspec/blob/master/Specs.hs#L116
Specs.hs used to also use hspec to update the README file so the documentation was always up to date with the latest specs of hspec, but that doesn't seem to be the case anymore.
If you do rename hspecX to hspec, I suggest renaming the current hspec to something to the effect of hspecButDon'tExitTheProcess since I think there should be a way to get evaluated specs without exiting the process.
perhaps hspec -> hspecEvaluate
@trystan Yes, the README does not contain the spec anymore. I just run the spec, pipe the output to a file, wrap it with a pre-tag, apply some semi-automated processing, and feed it to hakyll, which then generates a colorized HTML representation. But I'd like to get this automated (#66).
Ok, we need something like hspec for hspec's own spec. I'm still puzzled whether hspecX and hspecB would be sufficient for the public interface? Can we come up with a use case where the user wants to traverse the result tree?
I don't have a specific use case in mind but rather a general guide that I follow for APIs. One extreme viewpoint is to expose the building blocks so the user can do whatever they want and only restrict it when necessary (like for security reasons). The opposite extreme is to expose as little data, structure, or behavior as possible and open it up only when there is a clear reason too. Both have advocates but I prefer the more open extreme - which explains some of the early structure of hspec.
I haven't used haskell or hspec in a long time though so I don't have a strong opinion on this. I'm mostly explaining why it is the way it is.
@trystan Thanks for the explanation, that really makes sense. And I think it's indeed nice if the user can tinker with the internals, if he wants to. And I just realized that this approach is very consistent with your mission statement for hspec.
What about the following approach, we have a public API, that we try to keep stable over time; everything else is exposed from Test.Hspec.Internal.*. I'd then even expose e.g. Test.Hspec.Monadic.SpecM, that way the user may extend the monadic API in some creative ways (or at least experiment with it). And we still have an easy way to tell, how changes affect existing users.
Make `hspec` an alias of `hspecX` (close #71)
The old functionality is still available by using `hHspec stdout`.
@gregwebs hspecX exits with ExitSuccess. This has the disadvantage, that tests may be silently ignored, when used in the wrong way.
main = do
-- spec2 will never be run, and the user may not notice it
Would it be fine to only exit with exitFailure, if an example fails, and do nothing if all examples passed?
yes, good change
@gregwebs Ok, I'll change that for hspec. hspecX will keep the current behavior (it's deprecated anyway).
`hspec` does not exit on success anymore (close #71)