Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Review ultra-verbose go tool dist test logs for unexpected skips. #65

Open
dagood opened this issue Apr 26, 2021 · 0 comments
Open

Review ultra-verbose go tool dist test logs for unexpected skips. #65

dagood opened this issue Apr 26, 2021 · 0 comments

Comments

@dagood
Copy link
Member

dagood commented Apr 26, 2021

Broken off from original issue tracking full coverage vs. upstream: #1

Something we can do to be a little more sure our tests are running properly is to look at the verbose logs, with a modification to make them more verbose and show SKIPs:

Here's what I did to make tests show skips:

diff --git a/src/cmd/dist/test.go b/src/cmd/dist/test.go
index 0c8e2c56bc..1248d9145b 100644
--- a/src/cmd/dist/test.go
+++ b/src/cmd/dist/test.go
@@ -271,7 +271,7 @@ func short() string {
 // defaults as later arguments in the command line.
 func (t *tester) goTest() []string {
 	return []string{
-		"go", "test", "-short=" + short(), "-count=1", t.tags(), t.runFlag(""),
+		"go", "test", "-v", "-short=" + short(), "-count=1", t.tags(), t.runFlag(""),
 	}
 }
 
@@ -350,6 +350,7 @@ func (t *tester) registerStdTest(pkg string, useG3 bool) {
 			}
 			args := []string{
 				"test",
+				"-v",
 				"-short=" + short(),
 				t.tags(),
 				t.timeout(timeoutSec),
@@ -388,6 +389,7 @@ func (t *tester) registerRaceBenchTest(pkg string) {
 			ranGoBench = true
 			args := []string{
 				"test",
+				"-v",
 				"-short=" + short(),
 				"-race",
 				t.timeout(1200), // longer timeout for race with benchmarks

~500 results for 'SKIP': https://gist.github.com/dagood/2bc08e37da295c2022b9196572d378a4
Results with 1 line of context (skip reason): https://gist.github.com/dagood/8105bad6aff4803794a4ed42c265a4fc

500 lines isn't small, but it isn't impossible. There are duplicate skip reasons, which helps.

Unfortunately, there is no baseline from upstream to check against. We need to reason out whether each one is a problem.

This is based on a golang-devs thread: https://groups.google.com/g/golang-dev/c/PNzwZXOe7bQ/m/M43Gl9mVDAAJ.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant