testing: failure messages should be easier to find #47080
$ go version go version devel go1.17-dc00dc6c6b Thu Jun 10 12:41:37 2021 +0000 linux/amd64
I've spent a lot of time poring over CI test output trying to find exactly why a given test has failed.
There is currently no easy way to distinguish between info-level and error-level messages which makes
This is made harder by the fact that some failure messages are generated by calling
Perhaps we could include an extra action type in the JSON test output that represents a call to
Perhaps the line we could also include a stack trace of the failure, making it easier for tooling to point directly to the offending code.
This doesn't help with regular non-JSON test output, but that's perhaps hard to fix without breaking people's expected test messages, though we could add a flag to enable an explicit "test failed here" output line.
The text was updated successfully, but these errors were encountered:
Adding the JSON test output line sounds like a good idea to me. I empathize with having to skim through huge test output files, though those are almost never JSON output directly.
Worth noting that many CI systems support special output syntax for cases like these, such as https://docs.github.com/en/actions/reference/workflow-commands-for-github-actions#setting-an-error-message. I think they also tend to mix in some regex-based heuristics to automatically pick up errors with positions from common tools.
I wonder why we'd need a flag to opt into the "test failed here" information; that sounds generally useful, and shouldn't take much space.