New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add script to run benchmarks on Modal #311

Merged

karpathy merged 1 commit into karpathy:master from leloykun:fc--modal-script

Apr 30, 2024

Contributor

leloykun commented Apr 30, 2024

This PR adds a script to run the benchmarks on the Modal platform. This is useful for folks who do not have access to expensive GPUs locally.

To run the benchmark for the attention forward pass on an A100 GPU with 80GB of memory, you can run the following command:

GPU_MEM=80 modal run benchmark_on_modal.py --compile-command "nvcc -O3 --use_fast_math attention_forward.cu -o attention_forward -lcublas" --run-command "./attention_forward 1"


          add modal script

b239b67

karpathy merged commit 179b159 into karpathy:master

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment