Handle missing "pass" event for subtests #188
Conversation
Sometimes no "pass" event is present for subtests in the JSON output, even though the output indicates the subtests passed (with "--- PASS:") and the parent test had a pass event. In these cases, gotestsum can output spurious failure messages for the subtests. Handle this by treating a pass event for a parent test as implying that the subtests also passed.
| p.addEndTestEvent(event, tc) | ||
| } | ||
|
|
||
| func (p *Package) addEndTestEvent(event TestEvent, tc TestCase) { |
There was a problem hiding this comment.
gocyclo forced this change on me. I'm personally not a fan of gocyclo and how it can force small fixes to be bundled with refactors, but hey, it's not my codebase.
|
@coryb pointed out to me that the root cause actually looks like So this PR might help with the symptom, but since there appears to be a deeper issue with bad data coming from the JSON, I'm not sure it makes sense to move forward with this. |
|
Hi @aaronlehmann , thank you for the PR! Yes this issue sounds familiar. I think #141 has some investigation and a link to the upstream Go issue (golang/go#38063). From that discussion it seems like the issue is more common with parallel tests. As you found in the code there is already some logic to handle other Which version of Go are you using when you see these bugs? I believe the misattributing lines was a serious regression in 1.14.x , and was eventually fixed in 1.15.x. There are a few closed issues tagged with |
|
I'm seeing this with Go 1.14.7. I suspect my problem is related to tests writing directly to stdout/stderr. That sounds like unsupported behavior. Unfortunately it will be a bit painful to fix. I found this upstream issue that looks related: golang/go#38050 It was ultimately fixed by tweaking the test output, rather than adjusting the parser: golang/go@2ba9d45 ...so it's not a huge surprise that tests writing directly to stdout/stderr might cause the parser to get confused in a similar way. |
|
I believe the specific issue I'm hitting is golang/go#40771. It sounds like a fix is included in Go 1.14.10 (and 1.15.3, and 1.16). So upgrading Go will likely fix the issue for me. I'll close this since it looks like upgrading Go to get the bug fix is a better path forward. |
Sometimes no "pass" event is present for subtests in the JSON output,
even though the output indicates the subtests passed (with "--- PASS:")
and the parent test had a pass event. In these cases, gotestsum can
output spurious failure messages for the subtests. Handle this by
treating a pass event for a parent test as implying that the subtests
also passed.