New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x/tools/go/packages: TestJSON may be non-deterministic/flaky #27594

Open
dmitshur opened this Issue Sep 10, 2018 · 2 comments

Comments

Projects
None yet
4 participants
@dmitshur
Member

dmitshur commented Sep 10, 2018

I can't reproduce this locally on darwin/amd64, but I've seen TestJSON fail many times with the same error message on various less common environments:

--- FAIL: TestJSON (0.67s)
	packages_test.go:1205: wrong JSON: got <<{
			"ID": "a",
			"Name": "a",
			"PkgPath": "a",
			"GoFiles": [
				"a.go"
			],
			"CompiledGoFiles": [
				"a.go"
			]
		}
		{
			"ID": "b",
			"Name": "b",
			"PkgPath": "b",
			"GoFiles": [
				"b.go"
			],
			"CompiledGoFiles": [
				"b.go"
			],
			"Imports": {
				"a": "a"
			}
		}
		{
			"ID": "d",
			"Name": "d",
			"PkgPath": "d",
			"GoFiles": [
				"d.go"
			],
			"CompiledGoFiles": [
				"d.go"
			],
			"Imports": {
				"b": "b"
			}
		}
		{
			"ID": "c",
			"Name": "c",
			"PkgPath": "c",
			"GoFiles": [
				"c.go"
			],
			"CompiledGoFiles": [
				"c.go"
			],
			"Imports": {
				"b": "b"
			}
		}
		>>, want <<{
			"ID": "a",
			"Name": "a",
			"PkgPath": "a",
			"GoFiles": [
				"a.go"
			],
			"CompiledGoFiles": [
				"a.go"
			]
		}
		{
			"ID": "b",
			"Name": "b",
			"PkgPath": "b",
			"GoFiles": [
				"b.go"
			],
			"CompiledGoFiles": [
				"b.go"
			],
			"Imports": {
				"a": "a"
			}
		}
		{
			"ID": "c",
			"Name": "c",
			"PkgPath": "c",
			"GoFiles": [
				"c.go"
			],
			"CompiledGoFiles": [
				"c.go"
			],
			"Imports": {
				"b": "b"
			}
		}
		{
			"ID": "d",
			"Name": "d",
			"PkgPath": "d",
			"GoFiles": [
				"d.go"
			],
			"CompiledGoFiles": [
				"d.go"
			],
			"Imports": {
				"b": "b"
			}
		}
		>>
	packages_test.go:1244: Package 2 has ID "d" want "c"
	packages_test.go:1247: Package "d" has Name "d" want "c"
	packages_test.go:1244: Package 3 has ID "c" want "d"
	packages_test.go:1247: Package "c" has Name "c" want "d"
FAIL
FAIL	golang.org/x/tools/go/packages	21.562s

For example, it failed just now on freebsd-amd64-10_3 builder for Go 1.10. See https://build.golang.org/log/b5fd57cad07f4cb3e1e3c9e1654a21fc23d7c025.

The diff in JSON is simply that the last two packages c and d are swapped.

Looking at the source code of TestJSON, packages c and d both depend on b only, so unless go/packages API is meant to guarantee a stable package visiting order, d being visited before c seems valid. Perhaps on those unusual environments, filesystem directory listing is not deterministic, causing d sometimes to be visited before c.

/cc @ianthehat @matloob

@gopherbot gopherbot added this to the Unreleased milestone Sep 10, 2018

@dmitshur

This comment has been minimized.

Show comment
Hide comment
@dmitshur

dmitshur Sep 11, 2018

Member

It happened again on darwin-amd64-10_11 builder on go1.10 branch:

https://build.golang.org/log/c5f6c6b6e6863169c92564c528adbb0cbc163b23

So it's not just FreeBSD that's affected. Just providing another data point on the issue.

Member

dmitshur commented Sep 11, 2018

It happened again on darwin-amd64-10_11 builder on go1.10 branch:

https://build.golang.org/log/c5f6c6b6e6863169c92564c528adbb0cbc163b23

So it's not just FreeBSD that's affected. Just providing another data point on the issue.

@dominikh

This comment has been minimized.

Show comment
Hide comment
@dominikh

dominikh Sep 11, 2018

Member

Another data point: I've run into this issue locally as well, Go 1.10.3 on Linux. I've only hit that bug once so far, after dozens of times of running the tests (as part of my normal work, not specifically testing for this issue)

Member

dominikh commented Sep 11, 2018

Another data point: I've run into this issue locally as well, Go 1.10.3 on Linux. I've only hit that bug once so far, after dozens of times of running the tests (as part of my normal work, not specifically testing for this issue)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment