Fix crash and new features in unittest

[jdmansour]

The following unit test crashes currently:

import unittest

suite "ShowsBug":
  test "Compare with nil variable":
    let str: string = nil
    check str == "hello"

Traceback (most recent call last)
unittest.nim(267)        test_bug
SIGSEGV: Illegal storage access. (Attempt to read from nil?)

The reason is that the unittest module just checks wheter string($str) compiles, and then prints out $str even if str is nil. I've added a check for nil and it doesn't crash anymore.

Maybe long term it should use something like repr instead of $, which handles nil and also adds quotation marks. I didn't use it here because it also prepends the memory address, which I didn't want in the test output.

[jdmansour]

I've added a few simple improvements to the unittest module. Seems GitHub added them to my bugfix pull request, hope that is OK.

• Print out values of nested dot expressions on failure:

test_feature.nim(17,23): Check failed: c.color.name == "red"
c.color.name was blue

• Same with bracket expressions

test_feature.nim(18,26): Check failed: c.passengers[0] == "Anna"
c.passengers[0] was Anne

• Suppress constants (enum members) in output, like the last line here:

test_feature.nim(22,15): Check failed: meth == httpPOST
meth was httpGET
httpPOST was httpPOST

• Allow let inside of check blocks. I found this really nice if you tested a list of invariants, and want to factor out a variable in the middle.

I've tried to find any unintended side effects, but so far these changes seem OK and backwards-compatible.

Here is a test case (the first two tests fail on purpose, the last doesn't compile without patch):

import unittest

type
  Car = object
    color: Color
    passengers: seq[string] not nil

  Color = object
    name: string not nil

  HttpMethod = enum
    httpGET, httpPOST

suite "ShowsFeatures":
  test "Shows fields on failure":
    let c = Car(color:Color(name:"blue"), passengers: @["Anne", "Bob"])
    check c.color.name == "red"
    check c.passengers[0] == "Anna"

  test "Supresses constants on failure":
    let meth = httpGET
    check meth == httpPOST

  test "Allow let in check":
    let x = 3
    check:
      x == 3
      let square = x*x
      square == 9

Here is the output with patch

test_feature.nim(18,23): Check failed: c.color.name == "red"
c.color.name was blue
test_feature.nim(19,26): Check failed: c.passengers[0] == "Anna"
c.passengers[0] was Anne
[FAILED] Shows fields
test_feature.nim(23,15): Check failed: meth == httpPOST
meth was httpGET
[FAILED] Supresses constants
[OK] Allow let in check

[Araq]

Use a case exp[i] here please.

[Araq]

Please also add the test case. The testing framework is not really documented, but you can look at over 500 examples to figure it out. :-)

[jdmansour]

Thanks for the review. I also just found out that there is a builtin isNil, so I should probably change that.

Once I figure out how to write a test that passes if it actually fails but doesn't crash, I'll add the test.

[dom96]

This is really nice, thank you for your contribution!

For the future, it's helpful to create a new branch for each PR. That way you can work on other things in another branch, and not worry about an older PR including your new commits.

[andreaferretti]

@jdmansour Did you have the time to fix the issues here?

[Araq]

No feedback. Closing.