proposal: testing/test: Add {T,B}.RawLogf methods

[akalin-keybase]

Currently, {T,B}.Logf calls into c.log, which calls decorate on the argument (which prefixes with the file/line, etc.) before appending it to c.output. A library that wants to override this decoration has to resort to printing out \r as a hack, e.g. https://github.com/stretchr/testify/blob/master/assert/assertions.go#L207 .

What if T and B exposed an additional method RawLogf, which wouldn't call decorate on the argument before appending it to c.output? Then the above hack wouldn't be necessary.

[bradfitz]

I don't think we want to grow the testing.TB interface or the testing package API much at this point.

One option that might be less invasive is to introduce a magic Logf value that prevents decoration. E.g.:

     t.Logf("some value = %v", val, testing.NoDecorate)

Where NoDecorate is:

    var NoDecorate = noDecorate{}

type noDecorate struct{}

... so users can't replace it, supports ==, and fits nicely in an interface{} for Logf args.

[bradfitz]

/cc @robpike @adg for proposal.

[adg]

One issue I see with your proposal @bradfitz is that vet would need to be made aware of the special value. Not a show-stopper but it smells funny.

@akalin-keybase does the assertion library dislike the decoration because the file/line would always be in the assertion library?

[akalin-keybase]

@adg yeah. I've run into this myself, too, where I want to redirect some logger to use T.Logf, but that logger already comes with file/line info, so I don't want the decoration there.

[adg]

Considering what I wrote above, I think RawLogf and RawLog are better than the suggested magic constant. What do you think @bradfitz?

[bradfitz]

I still don't like it. I'd prefer something extensible that also lends itself towards resolving the report-file/line-N-stack-frames-up issue. (forget its number and on poor connection)

We did make the testing.TB interface have a private method so people can't implement it without embedding a T or B, so we can extend it, but I'd rather not.

[cespare]

@bradfitz is probably referring to #4899

[bradfitz]

Indeed, it'd be nice to solve #4899 in the process of this one, if we solve either.

If we had testing.NoDecorate, you could imagine something similar like testing.FramesUp(2) to change the decoration to point up two frames. Or say combine them... testing.Decorate(-1) and testing.Decorate(2) ... negative one for off, >= 0 for how many frames up.

[adg]

Could the problem that prompted this proposal be solved by #2981?

If so, I suggest closing this issue in favor of that one, which has been accepted.

[akalin-keybase]

If I'm reading the proposal right, I don't think so. In particular, it looks like the entire log for a test is a single field in the Result object:

{
    "Package": "example.com/foobar",
    "Result": {
        "Name": "TestBar",
        "State": "PASS",
        "T": 1000000,
        "Log": "some test output\n"
    }
}

So I think that field would look like: "file.go:100 some test output\n..." normally, and "file.go:100\rotherfile.go:200 some test output\n..." for logs that want to override the file/line info. Ideally, it would just be "otherfile.go:200 some test output\n...".

[mdwhatcott]

Either solution proposed here so far would be wonderful. I'm the author of GoConvey, gunit, and assertions and have long been frustrated that there wasn't a nice way to squelch/tweak the decorations added by the testing package.

[adg]

The proposal in #2981 is coming along nicely (blocked on me) so I think we should decline this in favor of that one.

[mdwhatcott]

@adg - I don't see how the work being done on #2981 resolves this particular issue. Don't get me wrong, I'm really excited to have a -json flag, but this issue is all about providing a way to omit the decoration provided by the testing package in order to have more control over the message conveyed to the user in the test output within the currently running test process.

[cespare]

@adg the creator of this issue already explained in #14128 (comment) why #2981 doesn't solve this problem :\

This is about the testing framework controlling output in a way that can only be overridden with hacks and half-measures. This proposal is one specific idea for addressing this issue (which is more thoroughly discussed in #4899).

[adg]

Sorry, I skimmed that comment when I was revising this issue.

But the proposal does cover this use case. Output and test logs are emitted as a stream JSON objects as the tests execute. That should be sufficient to implement any kind of UI layer on top of the JSON output. I'm happy to reopen this proposal if that's not the case.

[cespare]

@adg #2981 only helps for this use case if the caller wishes to change from being some functions called from normal Go tests (and invoked with go test) to being a post-processor built on top of go test -json.

[mdwhatcott]

Yeah, it's all about being able to modify the output of go test -v (without -json) in the running process to stdout. That's different than building a tool/ui that reads the JSON stdout of go test -json and interpret/adapt it.

[bradfitz]

We're having another proposal review meeting now. Nobody is excited about this. If people want to reform or recolor the output, they can post-process the output with a wrapper tool, especially with the JSON output. But we don't want everybody to start picking their subjectively favorite colors (literally) for the test output. Consistency is nice.

Sorry, I'm going to close this.