An implementation of karpathy/nanoGPT in C++, optimized for Apple GPU using the Metal API. It targets only the inference and uses weights exported from Pytorch.

Demo:

NanoGPT.Screen.Recording.Demo.mov

Build and run commands:

mkdir build && cd build
cmake -DCMAKE_PREFIX_PATH=/path/to/libtorch -DMETAL_CPP_DIR=/path/to/metal-cpp .. -G Xcode && cmake --build . -j
./Release/nanoGPT "Prompt"