Make Test.Hspec.hspec an alias of Test.Hspec.hspecX #71

Closed
sol opened this Issue May 7, 2012 · 9 comments

3 participants

@sol
hspec member
sol commented May 7, 2012

The issue is, that if you use hspec instead of hspecX in a cabal test suite, cabal test will not report failing tests.

This is not a theoretical issue, at least http-types does it wrong.

So I'd suggest to rename hspecX to hspec, and keep hspecX as an alias. In the long run I'd tend to first deprecate and later remove hspecX.

@trystan @gregwebs Are you aware of any use cases for hspec? If I understand correctly, it would theoretically allow you to re-create the report from an evaluate spec (possibly with a different formatter). But we have no code to support this; and the user could always just reevaluate the spec to achieve the same thing. Am I missing something?

BTW: From a discussion with @markwright (related to yesodweb/yesod@9f4c92f) I realized that the steps required to update a non-monadic spec for hspec-1.0 can in some cases be less obvious than I thought. I made some changes that will give better error messages (basically I newtype-wrapped UnevaluatedSpec). I'll push a 1.1 release that includes that change on Hackage today.

@sol sol referenced this issue in aristidb/http-types May 7, 2012
Closed

Cabal test suite misses failing tests #20

@gregwebs
hspec member

I created the hspecX function, and that is the only one that I have ever used.

@trystan
hspec member

The hspec function is used by Specs.hs because it needs to test that the test runners and formatters work as expected without exiting. See https://github.com/hspec/hspec/blob/master/Specs.hs#L116 and https://github.com/hspec/hspec/blob/master/Specs.hs#L116

Specs.hs used to also use hspec to update the README file so the documentation was always up to date with the latest specs of hspec, but that doesn't seem to be the case anymore.

If you do rename hspecX to hspec, I suggest renaming the current hspec to something to the effect of hspecButDon'tExitTheProcess since I think there should be a way to get evaluated specs without exiting the process.

@gregwebs
hspec member

perhaps hspec -> hspecEvaluate

@sol
hspec member
sol commented May 7, 2012

@trystan Yes, the README does not contain the spec anymore. I just run the spec, pipe the output to a file, wrap it with a pre-tag, apply some semi-automated processing, and feed it to hakyll, which then generates a colorized HTML representation. But I'd like to get this automated (#66).

Ok, we need something like hspec for hspec's own spec. I'm still puzzled whether hspecX and hspecB would be sufficient for the public interface? Can we come up with a use case where the user wants to traverse the result tree?

@trystan
hspec member

I don't have a specific use case in mind but rather a general guide that I follow for APIs. One extreme viewpoint is to expose the building blocks so the user can do whatever they want and only restrict it when necessary (like for security reasons). The opposite extreme is to expose as little data, structure, or behavior as possible and open it up only when there is a clear reason too. Both have advocates but I prefer the more open extreme - which explains some of the early structure of hspec.

I haven't used haskell or hspec in a long time though so I don't have a strong opinion on this. I'm mostly explaining why it is the way it is.

@sol
hspec member
sol commented May 8, 2012

@trystan Thanks for the explanation, that really makes sense. And I think it's indeed nice if the user can tinker with the internals, if he wants to. And I just realized that this approach is very consistent with your mission statement for hspec.

What about the following approach, we have a public API, that we try to keep stable over time; everything else is exposed from Test.Hspec.Internal.*. I'd then even expose e.g. Test.Hspec.Monadic.SpecM, that way the user may extend the monadic API in some creative ways (or at least experiment with it). And we still have an easy way to tell, how changes affect existing users.

@sol sol added a commit that referenced this issue May 27, 2012
@sol sol Make `hspec` an alias of `hspecX` (close #71)
The old functionality is still available by using `hHspec stdout`.
4f35a24
@sol sol added a commit that closed this issue May 27, 2012
@sol sol Make `hspec` an alias of `hspecX` (close #71)
The old functionality is still available by using `hHspec stdout`.
4f35a24
@sol sol closed this in 4f35a24 May 27, 2012
@sol sol reopened this May 27, 2012
@sol
hspec member

@gregwebs hspecX exits with ExitSuccess. This has the disadvantage, that tests may be silently ignored, when used in the wrong way.

E.g.

main = do
  hspecX spec1
  -- spec2 will never be run, and the user may not notice it
  hspecX spec2

Would it be fine to only exit with exitFailure, if an example fails, and do nothing if all examples passed?

@gregwebs
hspec member

yes, good change

@sol
hspec member

@gregwebs Ok, I'll change that for hspec. hspecX will keep the current behavior (it's deprecated anyway).

@sol sol closed this in 080cd1f May 27, 2012
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment