automatic run test benchmark with docker file

[jinsong-mao]

Hi

there is docker file to build the docker image, but no script to run the benchmark test within docker image automatically, I tried many ways to run the test_bench.py in docker container build with [torchbench-nightly.dockerfile],
looks there is no good docker run command to run it,

here is my command:

this command sometimes bad to work, some strange issue, such as "/runner should be mount to simpleDir", could u please provide the command flow/script to run benchmark test through docker?

many thanks.

[jinsong-mao]

strange to get this feedback:

when direct run with command:
docker run -it --gpus all ghcr.io/pytorch/torchbench:latest

thanks

[xuzhao9]

Here is an example of running benchmark with docker: https://github.com/pytorch/benchmark/blob/main/.github/workflows/pr-a10g.yml#L56

[jinsong-mao]

@xuzhao9 thanks, with this workflow file, I can modify the dockerfile to access volume mounting of my user id and runner, I can run this flow smoothly now.

there is another question of running test_bench.py, how can I get all of the output performance metrics to a file? looks --benchmark-autosave not help.

thanks

[xuzhao9]

@jinsong-mao Here is our example script to run test_bench.py: https://github.com/pytorch/benchmark/blob/main/.github/scripts/run.sh#L48

[jinsong-mao]

@xuzhao9 hi xuzhao, if I want to run the test_bench.py in fp16, how should i add configs? thanks

[jinsong-mao]

Hi @xuzhao9 , after running the tests and checking the output json file, it's very strange that the GPU clock frequency is only 765,

I am running test on A100, am I correct? how to explain it?

Thanks

[xuzhao9]

@jinsong-mao test_bench.py does not support FP16 as a global config. However, some models are running at fp16 precision by default, so some of the numbers you get from test_bench.py are already at FP16.

[xuzhao9]

@jinsong-mao To minimize latency variation, we are pinning the A100 GPU frequency at 1215,1410: https://github.com/pytorch/benchmark/blob/main/.github/workflows/userbenchmark-a100.yml#L33

For NVIDIA T4 GPU, we pin the frequency at 5001,900: https://github.com/pytorch/benchmark/blob/v2.0/torchbenchmark/score/configs/v2/config-v2.env#L16C15-L16C23

Sorry I am not sure why the GPU clock frequency is at 765 in your case. Please check your environment.

[FindHao]

The GPU clock frequency may not be correct in this log. I remember it was obtained before or after the testing. Since GPUs automatically adjust their frequency, the frequency may return to the default 765 for A100 when GPUs are idle. But when GPUs are busy on testing, their frequency is as high as 1410 if you check with nvidia-smi -q -d CLOCK.

[FindHao]

You can use sudo nvidia-smi -lgc 1410,1410 to boost the frequency to 1410MHz for A100. But the default frequency when GPUs are idle is still 765.