Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x/build: automated testing of popular, open source Go projects #38803

Open
mdempsky opened this issue May 1, 2020 · 4 comments
Open

x/build: automated testing of popular, open source Go projects #38803

mdempsky opened this issue May 1, 2020 · 4 comments

Comments

@mdempsky
Copy link
Member

@mdempsky mdempsky commented May 1, 2020

Basic idea is that we'd automatically build and test a large, representative sampling of open source Go projects, to help catch regressions during the development cycle that aren't caught by tests in the Go repo itself.

Some thoughts:

If running tests is a concern for resource/security concerns, even just verifying that projects still build would be useful data sometimes. E.g., cmd/cgo changes have a high frequency of breaking builds.

I think ~nightly testing would be adequate, but even weekly would probably be useful.

/cc @dmitshur

@mdempsky mdempsky added this to the Backlog milestone May 1, 2020
@gopherbot gopherbot added the Builders label May 1, 2020
@mdempsky
Copy link
Member Author

@mdempsky mdempsky commented May 2, 2020

The thing that got me thinking of this was the Coq proof assistant project offers a CI service where projects that commit to certain maintenance responsibilities will received automated CI testing and Coq developer support to deal with upstream changes: https://github.com/coq/coq/blob/master/dev/ci/README-users.md

Of course, circumstances for Go are somewhat different. In Go we have to deal with flaky tests, and we also have stronger backwards compatibility guarantees (I think). But I think the basic principles of being more proactive in watching for downstream breakage is still applicable to Go.

@dmitshur
Copy link
Member

@dmitshur dmitshur commented May 2, 2020

Thanks for filing this @mdempsky.

/cc @cagedmantis @toothrot @andybons

@cherrymui
Copy link
Contributor

@cherrymui cherrymui commented May 3, 2020

I don't remember whether @dr2chase 's daily bent run includes tests.

cc @dr2chase

@dr2chase
Copy link
Contributor

@dr2chase dr2chase commented May 4, 2020

My daily run does not include tests, but it is capable of tests, and I ran them for a recent CL.

bent doesn't do a good job (yet) of quantifying test results -- yet another possible intern project, I suppose, though it is a small one (and bent is internally somewhat cruddy).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
5 participants
You can’t perform that action at this time.