Deprecation of MustMatchers makes specifications sound optional

[haraldme]

Since scalatest 2.0, MustMatchers has been deprecated.

By deprecating MustMatchers and making people switch over from "must" to "should", the language in specifications becomes softer. For specifications, soft language is bad. Specification statements no longer read like assertions, but rather like something that the test framework might optionally attempt to assert.

I've tried to find information on the reason for the deprecation, beyond the rather fuzzy "we want to make the verb available to be used for a different purpose" in scalatest's documentation. Searching the mailing list and issue database left me none the wiser... but
I did find a twitter thread where Bill Venners had responded, saying that he planned to replace "must" with something that returned a result rather than performed an assertion. In that thread, "will" was suggested as a possible (unused) word for this new result-returning verb: https://twitter.com/viktorklang/status/412992663236124672

That left me somewhat hopeful... but I now see that MustMatchers remain deprecated in the upcoming 2.1.0 release, so I'm filing this issue.

(Luckily, I've discovered that org.scalatest's package object has a type alias for MustMatchers, and that has not (yet) been deprecated, so I can still have my must-using code compile without warnings.)

[bvenners]

Hi Herald,

Sorry about the MustMatchers transition pain. I was really worried about deprecating that one. Fear not, though, must will return, but when it returns, it will return (a result, instead of throwing an exception). At least that's the plan.

One thing I could have done instead was just let must mean two different things depending upon which trait was mixed in. I was (and am) worried that would be error prone. I actually never thought of "will" until Viktor mentioned it, and I thought about it, but I don't like the sound of it. We could still allow must to be used in both ways, but I want to stick to my guns for now.

I was unaware we'd left the type alias undeprecated, so that's good. I'll leave it that way so you and others can keep using it to avoid the deprecation warnings. One thing I have done, which I do for all deprecated things, is I have left all the tests in place to ensure MustMatchers continues to work as well as plain old Matchers. When must returns, I'd like to see if you, Viktor, etc., perhaps actually like writing tests with pure assertions. If so that will solve the problem. If not, I will listen and try to come up with some solution.

One of the other reasons I did this was to follow my own advice on minimizing redundancy. I talk about it in he last talk listed on this page:

http://www.scalatest.org/videos

I even mention in that talk that in retrospect I felt I should have picked one of either should or must.

Lastly, I wanted to explore what you mean by "the specification". I tend to think of the specification as the text part of the test, not the code of the test. As the text, you can still use must. You can write:

"A stack" when {
  "empty" must {
     "be empty" in {
         // ...
      }
    }
}

And so on. When you run it you'll see something like:

StackSpec:
A Stack
  when empty
    must be empty

And so on. That's the specification to me. The code is something extra that validates that the software under test is adhering to the specification. Should in that sense merely means that the code should do something, and if it doesn't, throw an exception to fail the test. Should is used to form assertions that run when you "execute" the specification, but they aren't the specification themselves. Anyway, that's the way I have seen it, and that's influenced the way ScalaTest has come out. I'm very interested in hearing your way of thinking about it.

[haraldme]

Regarding my use of the word "specification" in this bug report: Mea culpa. I ought to have written "assertion". However, I find it nicely symmetric if one can use the "must" verb both in the specification text and in the assertions that "execute" it.

As for the yet-to-be-seen non-exception-throwing must: I am still to be convinced that having "should" and "must" have such semantically different behaviour (one will cause program flow to be affected by assertion failure, while the other will not) is a less confusing situation than having "must"'s behaviour depend on what trait one mixes in.

If the deprecation of MustMatchers keeps going according to plan, and the package object's type alias is left undeprecated, won't that mean that some users might not see any deprecation warnings at all until MustMatchers (in some future release) is entirely removed?

[bvenners]

Hi Harald,

Actually that's a good point about not seeing a deprecation warning. I hadn't thought of that. It wasn't intentional that we left that "hole" in the first place. I do want to have a release in which there is no must before I reuse it for pure. So at some point we'll need to add the deprecation warning there.

Just to be clear, org.scalatest.matchers.MustMatchers will definitely continue to be deprecated and eventually removed from ScalaTest. What I have considered as a possibility after I heard from Viktor is that we could deploy a separate JAR file that offers MustMatchers in a different package, for folks who want ye olde MustMatchers that throw TestFailedException. I too like consistency, which is another of my simplicity guidelines, and I quite liked the look of the test class that Victor pointed me to where "must" was used both in the assertions and the specification. I actually proposed this to the Akka folks, but never heard back. What I have found is that most people don't seem to care whether they are writing should or must. But we could add it back as something like org.scalatest.thrown.MustMatchers, and just offer that in a separate scalatest-mustmatchers.jar.

What my plan is for pure is that pure assertions will be enabled at compile time by the presence of an implicit. That implicit will be provided in pure traits, but not in traditional traits. So if you try to use a pure assertion in a traditional style trait, you will get a compiler error. That should reduce most of the error proneness of them. You can always use a traditional assertion in a pure style. Throwing an exception will always fail the test in the usual way. I may deploy pure as a separate, add-on library to minimize the choices in the main library and make it more obvious what approach is recommended as the "default." Especially if we have traditional MustMatchers offered as a separate add-on library, I think this would make sense.

Bill

[bvenners]

Hi Herald,

In the end I decided to just follow Viktor's suggestion. We'll bring MustMatchers back, but in a new package. It will return in ScalaTest 2.1.0 as org.scalatest.MustMatchers. It's original name, org.scalatest.matchers.MustMatchers, will continue to be deprecated, but will as of 2.1.0 suggest people use org.scalatest.MustMatchers instead of org.scalatest.Matchers. After the completion of the deprecation cycle, org.scalatest.matchers.MustMatchers (along with org.scalatest.matchers.ShouldMatchers) will be removed from ScalaTest (about a year from now, probably).

When the day comes that we add pure matchers to ScalaTest, if that day comes, it will likely go in as org.scalatest.PureMatchers, and the verb will be "will" (as proposed by Viktor Klang) instead of "should" or "must". Thus "must" will always either result in () or throw a TestFailedException; "will" will return a result whose type is yet to be determined, and will never throw an exception.

2.1.0 is finished except for this final change. Had my laptop with Java 6 on it not died recently I would have already released it. Since it is not out yet we'll add this and then release a 2.1.0-RC3 as soon as possible, and release as 2.1.0 final soon thereafter.

[haraldme]

Excellent!

I'm looking forward to 2.1.0 (which, according to this plan, will not cause any MustMatchers transition pain at all for our code base, as we've already switched to org.scalatest.MustMatchers to avoid the deprecation warnings in 2.0).

[haraldme]

As 2.1.0 now is out the door with an un-deprecated org.scalatest.MustMatchers, I'm closing this.