simple-gpt usage and requirements

[jinsong-mao]

Hi folks，

many thanks for this great work, I am at the very beginning stage of using torchbench, I want to try simple_gpt, looks it's different from nanogpt, I can't run it successfully based on the instructions of https://github.com/pytorch/benchmark/blob/main/README.md, BTW, there is one command: #1867, but this command looks needs to be run on pytorch repo, I am confused here.
1, what's the difference between torchbench and pytorch/benchmark?
2, what's the necessary's step/pre-request to run simple_gpt model?

looking forward to get feedback from you,

thanks again.

[xuzhao9]

See my answers below:

1. pytorch/benchmark is used by pytorch/pytorch in its CI workflows. The PyTorch team uses the torchbench.py as a customized benchmark driver to implement their own features.
2. To run the simple_gpt model:
   a. Install torch, torchvision, and torchaudio nightly packages
   b. git clone the pytorch/pytorch repo (https://github.com/pytorch/pytorch) and cd into it.
   b. Run the command python benchmarks/dynamo/torchbench.py -d cuda --output-directory=benchmark_logs --output=performance.csv --inference --performance --timing --print-memory --multiprocess --nothing --only simple_gpt

[jinsong-mao]

thanks Xu zhao, it's much clear.
BTW, regarding to run simple_gpt, is multi_gpu the basic requirement or can we run it with single gpu?

Best Regrads

[xuzhao9]

cc the model owner on more details @xmfan

[xmfan]

You can run simple_gpt with single gpu using the command in the linked PR, provided your GPU has enough memory. You can ignore the warning message:

WARNING:common:The use multiprocess flag is set but there are <= 1 devices available.

Note that this version of simple_gpt does not contain any model weights, nor the code to load them

[jinsong-mao]

got it, thanks, @xuzhao9 @xmfan