forked from golang/go
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
testing: fix many test2json inaccuracies
Test2json is parsing the output stream from the test, which includes package testing's own framing lines intermingled with other output, in particular any output printed via fmt.Printf, println, and so on. We have had recurring problems with unexpected partial output lines causing a framing line to be missed. A recent talk at GopherCon gave an example of an integration test involving Docker that happened to print \r-terminated lines instead of \n-terminated lines in some configurations, which in turn broke test2json badly. (https://www.gophercon.com/agenda/session/944259) There are also a variety of open reported issues with similar problems, which this CL also addresses. The general approach is to add a new testing flag -test.v=json that means to print additional output to help test2json. And then test2json takes advantage of that output. Among the fixes: - Identify testing framing more reliably, using ^V (golang#23036, golang#26325, golang#43683, GopherCon talk) - Test that output with \r\n endings is handled correctly (golang#43683, golang#34286) - Use === RUN in fuzz tests (golang#52636, golang#48132) - Add === RUN lines to note benchmark starts (golang#27764, golang#49505) - Print subtest --- PASS/FAIL lines as they happen (golang#29811) - Add === NAME lines to emit more test change events, such as when a subtest stops and the parent continues running. - Fix event shown in overall test failure (golang#27568) - Avoid interleaving of writes to os.Stdout and os.Stderr (golang#33419) Fixes golang#23036. Fixes golang#26325. Fixes golang#27568. Fixes golang#27764. Fixes golang#29811. Fixes golang#33419. Fixes golang#34286. Fixes golang#43683. Fixes golang#49505. Fixes golang#52636. Change-Id: Id4207b746a20693f92e52d68c6e4a7f8c41cc7c6 Reviewed-on: https://go-review.googlesource.com/c/go/+/443596 Auto-Submit: Russ Cox <rsc@golang.org> Run-TryBot: Russ Cox <rsc@golang.org> Reviewed-by: Bryan Mills <bcmills@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
- Loading branch information
1 parent
e5e2628
commit 2955168
Showing
26 changed files
with
890 additions
and
101 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
20 changes: 10 additions & 10 deletions
20
src/cmd/go/testdata/script/test_chatty_parallel_success.txt
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
4 changes: 2 additions & 2 deletions
4
src/cmd/go/testdata/script/test_chatty_parallel_success_run.txt
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,48 @@ | ||
| go test -json | ||
|
|
||
| stdout '"Action":"output","Package":"p","Output":"M1"}' | ||
| stdout '"Action":"output","Package":"p","Test":"Test","Output":"=== RUN Test\\n"}' | ||
| stdout '"Action":"output","Package":"p","Test":"Test","Output":"T1"}' | ||
| stdout '"Action":"output","Package":"p","Test":"Test/Sub1","Output":"=== RUN Test/Sub1\\n"}' | ||
| stdout '"Action":"output","Package":"p","Test":"Test/Sub1","Output":"Sub1 x_test.go:19: SubLog1\\n"}' | ||
| stdout '"Action":"output","Package":"p","Test":"Test/Sub1","Output":"Sub2"}' | ||
| stdout '"Action":"output","Package":"p","Test":"Test/Sub1","Output":"--- PASS: Test/Sub1 \([\d.]+s\)\\n"}' | ||
| stdout '"Action":"pass","Package":"p","Test":"Test/Sub1","Elapsed"' | ||
| stdout '"Action":"output","Package":"p","Test":"Test/Sub3","Output":"foo bar"}' | ||
| stdout '"Action":"output","Package":"p","Test":"Test/Sub3","Output":"baz\\n"}' | ||
| stdout '"Action":"output","Package":"p","Test":"Test","Output":"T2"}' | ||
| stdout '"Action":"output","Package":"p","Test":"Test","Output":"--- PASS: Test \([\d.]+s\)\\n"}' | ||
| stdout '"Action":"pass","Package":"p","Test":"Test","Elapsed"' | ||
| stdout '"Action":"output","Package":"p","Output":"M2ok \\tp\\t[\d.]+s\\n"}' | ||
| stdout '"Action":"pass","Package":"p","Elapsed"' | ||
|
|
||
| -- go.mod -- | ||
| module p | ||
|
|
||
| -- x_test.go -- | ||
| package p | ||
|
|
||
| import ( | ||
| "os" | ||
| "testing" | ||
| ) | ||
|
|
||
| func TestMain(m *testing.M) { | ||
| print("M1") | ||
| code := m.Run() | ||
| print("M2") | ||
| os.Exit(code) | ||
| } | ||
|
|
||
| func Test(t *testing.T) { | ||
| print("T1") | ||
| t.Run("Sub1", func(t *testing.T) { | ||
| print("Sub1") | ||
| t.Log("SubLog1") | ||
| print("Sub2") | ||
| }) | ||
| t.Run("Sub3", func(t *testing.T) { | ||
| print("\x16foo bar\x16baz\n") | ||
| }) | ||
| print("T2") | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,19 @@ | ||
| ! go test -json -timeout=1ms | ||
|
|
||
| stdout '"Action":"output","Package":"p","Output":"FAIL\\tp\\t' | ||
| stdout '"Action":"fail","Package":"p","Elapsed":' | ||
|
|
||
| -- go.mod -- | ||
| module p | ||
|
|
||
| -- x_test.go -- | ||
| package p | ||
|
|
||
| import ( | ||
| "testing" | ||
| "time" | ||
| ) | ||
|
|
||
| func Test(t *testing.T) { | ||
| time.Sleep(1*time.Hour) | ||
| } |
Oops, something went wrong.