TestCheckers/All_slice_mismatch_with_DeepEqual fails with "go test -v"

[anthonyfok]

Greetings!

In the process of upgrading the Debian packaging of quicktest (https://tracker.debian.org/pkg/golang-github-frankban-quicktest) to v1.4.0, I came across the following test error:

    --- FAIL: TestCheckers/All_slice_mismatch_with_DeepEqual (0.00s)
        quicktest_test.go:621: prefix:
            got  "\nerror:\n  mismatch at index 1\nerror:\n  values are not deep equal\nfirst mismatched element:\n  []string{\"a\", \"c\"}\ndiff (-got +want):\n  \u00a0\u00a0[]string{\n  \u00a0\u00a0\t\"a\",\n  -\u00a0\t\"c\",\n  +\u00a0\t\"b\",\n  \u00a0\u00a0}\nstack:\n  /build/golang-github-frankban-quicktest-1.4.0/obj-x86_64-linux-gnu/src/github.com/frankban/quicktest/checker_test.go:2167\n    ok := c.Check(test.got, checker, test.args...)\n"
            want "\nerror:\n  mismatch at index 1\nerror:\n  values are not deep equal\ndiff (-got +want):\n  \u00a0\u00a0[]string{\n  \u00a0\u00a0\t\"a\",\n  -\u00a0\t\"c\",\n  +\u00a0\t\"b\",\n  \u00a0\u00a0}\nstack:\n"
            -------------------- got --------------------
            
            error:
              mismatch at index 1
            error:
              values are not deep equal
            first mismatched element:
              []string{"a", "c"}
            diff (-got +want):
                []string{
                        "a",
              -         "c",
              +         "b",
                }
            stack:
              /build/golang-github-frankban-quicktest-1.4.0/obj-x86_64-linux-gnu/src/github.com/frankban/quicktest/checker_test.go:2167
                ok := c.Check(test.got, checker, test.args...)
            
            -------------------- want -------------------
            
            error:
              mismatch at index 1
            error:
              values are not deep equal
            diff (-got +want):
                []string{
                        "a",
              -         "c",
              +         "b",
                }
            stack:
            
            ---------------------------------------------

and yet the Travis CI says all is fine.

It turns out that Debian tools are using go test -v whereas .travis.yml runs go test without the -v flag. Indeed, this is the very first time that I see that a go test result can change depending on whether it is in verbose mode or not. Interesting.

I could easily patch the Debian package to work around this issue, though I wonder if there is any easy way to allow TestCheckers/All_slice_mismatch_with_DeepEqual to run fine regardless of verbose or not? My quick attempt at setting verbose: false failed as it seems that the verbose setting is not passed down from qt.All() into nested checkers (based on my very limited understanding.)

Many thanks!

Anthony

[frankban]

Thanks for reporting! You described the issue exactly. Since the output of checkers can change based on whether verbosity is set or not, we need to patch the internal function used for reporting verbosity.

[frankban]

The fix is now released as v1.4.1