Skip to content

testing: structured output for test attributes #43936

Closed
@marwan-at-work

Description

@marwan-at-work

Test2json, more particularly go test -json, has been quite a pleasant discovery. It allows for programs to analyze go tests and create their own formatted output.

For example, using GitHub Actions' formatting capabilities, I was able to better format go tests to look more user friendly when running in the UI:

Before:

Screen Shot 2021-01-26 at 5 22 08 PM

After:

Screen Shot 2021-01-26 at 5 21 58 PM

With that said, there are still some missing features that would allow programs to better understand the JSON output of a test.

Proposal

It would be great if Go Tests can attach metadata to be included in the JSON output of a test2json run.

Something along these lines:

func TestFoo(t *testing.T) {
  t.Log("Foo")
  // outputs: {"action": "output", "output": "foo_test.go:12 Foo\n"}
  t.WithMetadata(map[string]string{"requestID": "123"}).Errorf("Foo failed")
  // outputs: {"action": "output", "output": "Foo failed", "metadata": {"requestID": "123"}}
}

Benefits:

This allows for a few highly beneficial use cases:

  1. If a test fails, then the program that's analyzing the failed test's json can receive metadata about why it failed: such as requestID, userID etc and then provide the user helpful links to logs and queries.
  2. A test can provide source-code information about where things failed. Because right now test2json cannot distinguish between when a user called t.Fatal(...) or t.Log(...) which makes sense as t.Fatal just calls t.Log -- but the user can include metadata so we know exactly where the error occurred and use CI capabilities such as Actions' error command to set the file and line number to be displayed in the UI.

Alternative solutions:

Include directives in the output string that the json-parsing program can analyze to see if there's metadata. But this solution is very fragile and prone to error.

Thanks!

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    Status

    Accepted

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions