Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x/build: use rr-project.org for running Go builds/tests on Linux? #45219

Open
mdempsky opened this issue Mar 24, 2021 · 2 comments
Open

x/build: use rr-project.org for running Go builds/tests on Linux? #45219

mdempsky opened this issue Mar 24, 2021 · 2 comments

Comments

@mdempsky
Copy link
Member

@mdempsky mdempsky commented Mar 24, 2021

We have a lot of intermittent failures on the builders, which are hard to reproduce. Evidently the Firefox project had similar struggles, so they built https://rr-project.org/, which allows recording and replaying program executions on linux/x86. Perhaps we should look into using it for our builders too?

Quickly skimming build.golang.org, it's not clear to me if there are enough flaky failures on linux/x86 to warrant the ~20% slowdown (according to their website). It seems like most flakiness is on non-linux or non-x86, unfortunately, and rr doesn't currently support any other platforms.

Maybe the regabi builder could benefit from it at least. It seems to have had some flakiness a couple weeks ago.

/cc @aclements @bcmills

@dmitshur
Copy link
Contributor

@dmitshur dmitshur commented Mar 25, 2021

@tobiaskohlbau
Copy link

@tobiaskohlbau tobiaskohlbau commented Apr 4, 2021

I'm not sure wether this is from any help. But I've stumbled upon this issue and the description from rr-project reminded me a lot about a technique I've heard before. @mdempsky stated that most flakey tests are on non Linux. Maybe it's from interest that MS has a similar technique called Time travel debugging. I've no experience with this but maybe it's worth evaluating if TTD can help with some none Linux flakiness.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
4 participants