Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x/build/cmd/coordinator: test packages in nested modules (in multi-module repositories) #32528

Open
dmitshur opened this issue Jun 10, 2019 · 4 comments

Comments

Projects
None yet
2 participants
@dmitshur
Copy link
Member

commented Jun 10, 2019

Right now, the trybots and post-submit builders assume there's only one module in each subrepository, and run go test [-short] [-race] golang.org/x/subrepo/... inside the subrepo root directory:

https://github.com/golang/build/blob/a3d123a17ca979f921ade4800222d1b8586fff87/cmd/coordinator/coordinator.go#L2443-L2459

That means modules not at the root are not covered.

We have some subrepos that have nested modules (e.g., x/net/h2demo, x/build/maintner/cmd/maintserve), and we may have more in the future. They need to be tested.

/cc @bradfitz @bcmills @ianthehat

Somewhat related to #30233, because the golang.org/x/subrepo/... import path pattern matches different packages depending on whether operating in GOPATH mode or module mode.

@dmitshur

This comment has been minimized.

Copy link
Member Author

commented Jun 10, 2019

@bcmills Do you know how this was solved in the main go repository, which also has more than one module now? Is it all abstracted away in go tool dist test, or was there some logic added to builders?

@bcmills

This comment has been minimized.

Copy link
Member

commented Jun 10, 2019

go test golang.org/x/subrepo/...

That might not be quite as bad as it seems. If the root module has require directives for the nested modules, then the path wildcard will also cover them.

That said, we should probably have the builders look for go.mod files throughout each repo and run go test ./... (or, better still, go test golang.org/x/subrepo/path/to/module/... from outside the module) explicitly for each module.

@bcmills

This comment has been minimized.

Copy link
Member

commented Jun 10, 2019

The modules in the main repo are indeed abstracted away by go tool dist test.

@dmitshur

This comment has been minimized.

Copy link
Member Author

commented Jun 13, 2019

Here's how I expect this can be implemented.

runSubrepoTests would need to be modified to:

  1. Determine if the selected configuration (Go version + whether or not subrepo has go.mod file) results in building in module mode or GOPATH mode by default.
    • This can likely be done by running go env GOMOD and seeing if that's set to non-empty value, or something along those lines (but need to get this exactly right and confirm it works for all cases).
  2. Similarly to how findDeps finds deps in all packages in the subrepo, need to just walk the dir tree and find all the places with go.mod files, and save those in a list.
  3. For each dir where a go.mod file was found (in addition to the root), run go test [-short] [-race] {{.ModulePath}}/... like it's currently done once for the root.
  4. Test that everything works and deploy to prod.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.