proposal: testing: allow examples the use of testing.T/B/F for test framework examples

[hugelgupf]

Proposal Details

Today, examples are tests presented alongside godoc as a "func main()" typically, as shown in https://go.dev/blog/examples, and is executable like that.

I'm trying to write examples for a test framework I created (https://github.com/hugelgupf/vmtest) that requires the use of testing.TB. To create an executable example in pkg.go.dev, examples have to be able to take testing.T or testing.B as an argument.

IMO, the existing Example "API" should remain as is. If an example is given a testing.T/B/F argument, though, it should be assumed to require it and be presented that way in the documentation as well, e.g.

func ExampleFunctionname(t *testing.T) {
  t.Logf("foobar")
  // Output: foobar
}

Should appear alongside Functionname in documentation as an executable snippet like this:

package main_test

import (
  "testing"

  "packagepath"
)

func TestExample(t *testing.T) {
    t.Logf("foobar")
}

[earthboundkid]

I wrote a test framework and had this same problem. I usually use example driven development, but it was sort of hard to do that for a test framework. I had to make a dummy testing.TB, and it didn't always work very well. For example, testing.TB doesn't have T.Run, so I just stub that with code that compiles but doesn't execute.

[ianlancetaylor]

This seems fairly special purpose. Examples don't have to have outputs. Should we also permit example functions to take a slog.Logger argument? It seems hard to know when to stop.

[earthboundkid]

I think the proposal should be instead to make it easier to make a testing.T/B/F for test purposes. So the code would look like

func ExampleWhatever() {
    result := testing.RunTest(func(t *testing.T) {
        // do stuff with *testing.T
    })
    fmt.Println("Test failed was", result.Failed, "because", result.Output)
    // Output: 
    // Test failed was true because error: bad potatoes
}

This could also be used in a TestFunction to test a testing framework more easily.