New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Run benchmarks periodically in PyTorch CI #123

Closed

wconstab opened this issue Sep 23, 2020 · 0 comments

Assignees

Projects

Benchmark Issues

Contributor

wconstab commented Sep 23, 2020

new workflow on pytorch side
- install pytorch, torchvision, torchtext
  - can't use conda like benchmark CI becuase we want access to more fine grained versions of pytorch than nightly
  - @malfet can help with how to access build artifacts from any particular git version of pytorch repo in last X days
  - install those
- run benchmarks
- upload results

bisect job would build on top by

get previous and current periodic result
if regression suspected, then
- do binary search over all commits between A, B
  - for each, get the corresponding pytorch artifacts, install them, run measurement

wconstab assigned nikithamalgifb

wconstab added this to Feature Requests in Benchmark Issues

wconstab closed this as completed

Benchmark Issues automation moved this from Feature Requests to Done

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment