[Breaking] error should not emit expected/actual diags

[rhendric]

Unlike throws, where one can argue that the error thrown is an actual that should be compared to an expected error provided to throws, the error provided to error is not compared to anything. It should therefore not appear in an expected/actual pair, where downstream TAP reporters can pick it up and diff it against undefined—I can't think of a case where such output would be helpful.

[ljharb]

This seems like a potential breaking change. Is the current output causing problems for you with a downstream tap reporter?

[ljharb]

where is this field being read?

[rhendric]

Here, I believe:

https://github.com/substack/tape/blob/a1e8f7e21cc7831cf5a857b00627b5dbcdd68aea/lib/results.js#L164

[rhendric]

Forgive me, that was imprecise. What I should have said was that field is read off of the opts object here:
https://github.com/substack/tape/blob/a1e8f7e21cc7831cf5a857b00627b5dbcdd68aea/lib/test.js#L224
and that line populates the error field on the 'result' event, which itself is read in the location I linked above.

[ljharb]

Personally i do find this useful - it makes it clear what value is expected to make it pass.

[rhendric]

IMO, there isn't really an expected value, though... I think of error as like fail, where you call it to report that something went wrong, and the expected behavior is that it not get called at all. expected: undefined doesn't add any additional information on top of operator: error. If you find it useful anyway, hey, you do you, but I don't really understand why.

[rhendric]

Hi, thanks for getting back to me so quickly!

I've tested this with both tap-diff and tap-difflet; both seem to assume that if there are expected and actual fields in the diagnostic info, they ought to be diffed and displayed.

Here is a quick and dirty test.js:

const tap = require(process.argv[2]);

tap.test('example test producing error', t => {
    t.error(new Error('this is a message'));
    t.end();
});

tap.test('example test producing unexpected result', t => {
    t.equals('bad', 'good');
    t.end();
});

node test.js tape | tap-difflet

node test.js tape | tap-diff

In both cases, the message of the error is both reported as the reason for the test failure and diffed against the text ‘undefined’ (tap-difflet's handling of this looks especially bad). I think the message alone is sufficient; after all, I figure that error is going to be used in cases where the test knows it's in a bad state, and thus there isn't really an ‘expected’ value—the expectation is that the test won't run that branch.

For comparison, here's how node-tap fares:

node test.js tap | tap-difflet

(node test.js tap | tap-diff produces a honkin' ugly stack trace, so I'll spare you that. 😄)

[rhendric]

Thanks!