Skip to content

Commit

Permalink
How to setup the cuda env for windows (#20444)
Browse files Browse the repository at this point in the history
  • Loading branch information
baijumeswani committed Apr 24, 2024
1 parent faf0809 commit 5e549a5
Showing 1 changed file with 24 additions and 0 deletions.
24 changes: 24 additions & 0 deletions docs/genai/howto/setup-cuda-env.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
---
title: Setup CUDA env
description: Instructions to setup the CUDA environtment to run onnxruntime-genai-cuda
has_children: false
parent: How to
grand_parent: Generative AI (Preview)
nav_order: 4
---

# Setup the CUDA Environment
{: .no_toc }

* TOC placeholder
{:toc}

## Install the CUDA Toolkit

On a CUDA capable machine, install the CUDA toolkit. onnxruntime-genai-cuda is built and packaged with CUDA-11.8.

The CUDA toolkit can be downloaded from the [CUDA Toolkit Archive](https://developer.nvidia.com/cuda-toolkit-archive).

## Special Instructions on Windows

After CUDA toolkit installation completed on windows, ensure that the `CUDA_PATH` system environment variable has been set to the path where the toolkit was installed. This variable will be used when importing the onnxruntime_genai python module on windows. Unset or incorrectly set `CUDA_PATH` variable may lead to a `DLL load failed while importing onnxruntime_genai`.

0 comments on commit 5e549a5

Please sign in to comment.