Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x/build/env: run more tests in js-wasm trybot mode #29636

Open
bradfitz opened this issue Jan 9, 2019 · 4 comments

Comments

@bradfitz
Copy link
Member

commented Jan 9, 2019

See #29632 wherein the build breaks due to js-wasm skipping too many tests on trybots.

We could just shard out the js-wasm trybot builder wider and enable more tests.

@gopherbot gopherbot added this to the Unreleased milestone Jan 9, 2019
@gopherbot gopherbot added the Builders label Jan 9, 2019
@dmitshur

This comment has been minimized.

Copy link
Member

commented Jan 9, 2019

This sounds like a good idea, in order to catch issues like this in the future during trybot runs.

@bradfitz The js-wasm builder currently has numTryTestHelpers set to 4, which seems quite close to most other builders. What do you think would be a good number to set it to in order to be able js-wasm trybot to run all dist tests? Any suggestions on how to evaluate how that number correlates to the trybot execution time?

@bradfitz

This comment has been minimized.

Copy link
Member Author

commented Jan 9, 2019

Any suggestions on how to evaluate how that number correlates to the trybot execution time?

Write some SQL to query BigQuery. All the test timing data is in there for every build (both trybots and post-submit builds).

@dmitshur

This comment has been minimized.

Copy link
Member

commented Jan 9, 2019

Ok. I'll try to look for when a numTryTestHelpers value has changed and see how it affected corresponding trybot/post-submit times.

For js-wasm trybots specifically, I can't predict how the execution time will change when its ShouldRunDistTest policy is modified to not skip tests, other than to make that change and see the effect.

Edit: Perhaps I can get a sense for it by running the old and new tests on my local machine, and see the relative timing difference.

@bradfitz

This comment has been minimized.

Copy link
Member Author

commented Jan 9, 2019

I wouldn't even try to correlate it with numTryTestHelpers historical values.

You should be able to pick a goal time (5 or 6 minutes), subtract empirical make.bash time, and then that's the amount of time to you have to run all the tests. So find the sum of all the test durations for js-wasm and divide, and that's the shard count you want.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.