testing: test.timeout uses panic(), returning an exitcode of 0

[mordyovits]

Please answer these questions before submitting your issue. Thanks!

What version of Go are you using (go version)?

go version go1.8.1 linux/amd64

What operating system and processor architecture are you using (go env)?

GOARCH="amd64"
GOBIN=""
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/movits/gocode"
GORACE=""
GOROOT="/home/movits/go"
GOTOOLDIR="/home/movits/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build095751258=/tmp/go-build"
CXX="g++"
CGO_ENABLED="1"
PKG_CONFIG="pkg-config"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"

What did you do?

if you set a test.timeout on the cmdline when using go test, and you hit the timeout, the testing code calls panic to stop and print a stack trace:

go/src/testing/testing.go

Line 1148 in bd74fd3

panic(fmt.Sprintf("test timed out after %v", *timeout))

However, this causes the execution to exit with an exitcode of 0, unlike all other failure scenarios.

What did you expect to see?

An exitcode of 1. As with this comment:

go/src/testing/testing.go

Line 907 in bd74fd3

// Run runs the tests. It returns an exit code to pass to os.Exit.

// Run runs the tests. It returns an exit code to pass to os.Exit.

IOW, failure usually means a non-zero exitcode--except in the case of timeout. (Or perhaps other panic()s too?)

What did you see instead?

And exitcode of 0.

[mvdan]

Could you provide a small, self-contained reproducer for this? I just tried to reproduce this with:

package bug

import (
        "testing"
        "time"
)

func TestBug(t *testing.T) {
        time.Sleep(time.Minute)
}

$ go test -timeout=1s
panic: test timed out after 1s

goroutine 17 [running]:
testing.startAlarm.func1()
        /home/mvdan/tip/src/testing/testing.go:1146 +0xf9
created by time.goFunc
        /home/mvdan/tip/src/time/sleep.go:170 +0x44

goroutine 1 [chan receive]:
testing.(*T).Run(0xc42007c000, 0x5342b6, 0x7, 0x53c6b0, 0x468b26)
[...]
exit status 2
FAIL    _/home/mvdan/bug        1.004s
$ echo $?
1

This is on both tip and 1.8.3.

[mordyovits]

You're right, my test harness had a logic bug.