Spurious output when running unit tests

[elmart]

When I run unit tests, I get the following:

●●●●●●●●●E94: No matching buffer for _test_
E94: No matching buffer for _test_
E94: No matching buffer for _test_●
E94: No matching buffer for test●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
200 successes / 0 failures / 0 errors / 0 pending : 0.0 seconds

So, tests are ok, but there's that spurious output ('E94: No matching...') in the middle of them.
I think this is because those tests are testing error cases behaving as expected. But in those cases, vim, in addition to returning correct error code, also emits error output (through EMSG macros) to stdout, which ends in the test results.
So, I think we should capture vim's stdout while running tests for this to show nice.
I'll have a look to see if I can do it.
Suggestions are welcome in the meantime.

---

Want to back this issue? Place a bounty on it! We accept bounties via Bountysource.

[tarruda]

#1016 fixes this, though it still needs to be updated(I sent that before the moonscript drop)

[justinmk]

I still don't think we need mocking. Anything that has undesirable side-effects can be an integration test or functional test. For this particular cosmetic issue, capturing stdout could be a reasonable solution.

[elmart]

@justinmk You're right that this is a cosmetic issue, but an important one, I think. Having a nice test output is one of those things that have multiplicative effects on people's productivity and happiness (I read some other thing in that sense yesterday, somewhere else, and I agree).

Regarding mocking, I don't see so many problems with it, as long as it's well documented/explained what developers are supposed to do, and warned against abusing it.

In any case, some way of capturing stdout during test execution should be provided, IMHO.

So, should I close this issue, or are we going to use it to discuss the mocking thing?
In that case I will change the title to "To mock or not to mock? That is... the question." ;-)

[lslah]

I am also unhappy with the current unit test output. Just in case it helps anyone, a simple workaround is to redirect stderr to /dev/null, since the EMSG macros use stderr whereas busted uses stdout.

$ make unittest 2> /dev/null