testing: add t.Helper to make file:line results more useful

[eaigner]

Currently the caller level of T's decorate function is fixed

    _, file, line, ok := runtime.Caller(3) // decorate + log + public function.

When wrapping common assertions in a code block, it would be useful to be able to modify
the caller level for correct log output.

[bradfitz]

Comment 1:

Rob and I were discussing possibilities, like:
t.Frames(5).Errorf("...")
But probably after Go1.1.

Labels changed: added priority-later, removed priority-triage, go1.1maybe.

[rsc]

Comment 2:

[The time for maybe has passed.]

[mxk]

Comment 3:

This is a hack, but it works (except when it doesn't, which is in deferred function
calls):
package demo_test
import (
    "fmt"
    "runtime"
    "testing"
)
func up(frame int, s string) string {
    _, origFile, origLine, _ := runtime.Caller(1)
    _, frameFile, frameLine, _ := runtime.Caller(frame + 1)
    if origFile != frameFile {
        return s // Deferred call after a panic or runtime.Goexit()
    }
    erase := []byte("\b\b\b")
    for ; origLine > 9; origLine /= 10 {
        erase = append(erase, '\b')
    }
    return fmt.Sprintf("%s%d: %s", erase, frameLine, s)
}
func assert(t *testing.T) {
    t.Logf("wrong line number")
    t.Logf(up(1, "right line number"))
}
func TestDemo(t *testing.T) {
    assert(t)
}

[ianlancetaylor]

Comment 4:

Labels changed: added go1.2maybe.

[rsc]

Comment 5:

Status changed to Accepted.

[remyoudompheng]

Comment 6:

Issue #6024 has been merged into this issue.

[davecheney]

Comment 7:

I made a start on this, https://golang.org/cl/12405043/. It is not ready for
submission, but certainly ready for feedback.

Owner changed to @davecheney.

Status changed to Started.

[mxk]

Comment 8:

What about a simpler solution that prints out all line numbers within the same _test.go
file? I found that my hack actually makes it more difficult to figure out the true cause
of the error. In some cases it's better to have the location in the Test function, while
in others you want to know where in the helper function the test failed. Something like
"my_test.go:63:591: a != b" makes it very easy to track down the problem by giving you
all the relevant line numbers without dumping a full stack trace.

[davecheney]

Comment 9:

@max, yes, this is what I am proposing
t.Error("something") // logs the file and line of where t.Error is called
t.Frames(1).Error("something") // logs the file and line of the caller of the function
that called Error.

[mxk]

Comment 10:

Right, but I'm suggesting not having the Frames function. You could change the decorate
function instead to look through all stack frames starting at 3 and concatenate all line
numbers that are in the current _test.go file. If t.Error("something") is called in a
Test function, then the output stays as it is right now. But if it's called from a
helper function, then you get two or more line numbers, starting in the Test function
and working your way through all the helpers.

[davecheney]

Comment 11:

No thank you.
1. how do you defined a helper function ? I can't think of a way short of making them
end with helper or something.
2. this isn't want is spec'd by brad in comment #1
3. That isn't want Erik or I wanted in the original request or issue #6024.

[mxk]

Comment 12:

So for issue #6024, instead of "tt_test.go:8: an error" you would get "tt_test.go:16:8:
an error" without any changes to your code. No special naming is necessary. If chk is in
the current _test.go file, which you can easily figure out in decorate, then append its
line number.
In my experience, having both line numbers is frequently useful in tracking down the
problem, especially if your chk function tests for multiple error conditions. For an
example, take a look at how I use the "cl" function in my SQLite test code:
http://code.google.com/p/go-sqlite/source/browse/go1/sqlite3/sqlite3_test.go

[robpike]

Comment 13:

Still not convinced this is a good idea. Deferring to 1.3.

Labels changed: added go1.3maybe, removed go1.2maybe.

[robpike]

Comment 14:

Labels changed: removed go1.3maybe.

[rsc]

Comment 15:

Labels changed: added go1.3maybe.