Add the runscript of PR test

[xuzhao9]

Add the script to run the benchmark assuming the pytorch packages have been installed

[wconstab]

this seems like it duplicates a lot of what is already done in other scripts. can another script be deleted now that this is added, or what is the difference?

[xuzhao9]

this seems like it duplicates a lot of what is already done in other scripts. can another script be deleted now that this is added, or what is the difference?

Could you please point which script is the duplicate? In .github/scripts directory, the run-devbig* scripts are for running local abtesting on devbig machine (internal). The run-bisection.sh script is for running the bisection job. This script is for running abtesting on AWS machine for the PR CI.

It is essentially the run-v0.sh on the 0.1 branch. But I would like the PR CI to be able to run all models available on the master branch, so I need to introduce this script to master.

[wconstab]

hmm, what about these scripts?
https://github.com/pytorch/benchmark/blob/master/scripts/run_bench_and_upload.sh
https://github.com/pytorch/benchmark/blob/master/scripts/run_sweep_existing_envs.sh

it's possible the above are no longer used, and then I am OK with adding your new one and deleting the old ones.

I also wonder if you have a plan for which scripts go in .github/scripts and which ones go in top-level scripts/. If it is not well organized, maybe just consider collapsing to use one folder.

[xuzhao9]

hmm, what about these scripts?
https://github.com/pytorch/benchmark/blob/master/scripts/run_bench_and_upload.sh
https://github.com/pytorch/benchmark/blob/master/scripts/run_sweep_existing_envs.sh

it's possible the above are no longer used, and then I am OK with adding your new one and deleting the old ones.

I also wonder if you have a plan for which scripts go in .github/scripts and which ones go in top-level scripts/. If it is not well organized, maybe just consider collapsing to use one folder.

The run_sweep_existing_envs.sh is no longer used and it is safe to remove it.
The run_bench_and_upload.sh is still used by the Circle CI test "run_benchmarks". It is slightly different from this script, because in Circle CI test we only want to test the correctness, not actual performance numbers (don't have cpu isolation, nvidia frequency set, etc).
Currently, scripts for Circle CI tests go in top-level scripts/, and scripts for Github Actions go to .github/scripts.