Getting 'Found no NVIDIA driver on your system ' error.

[defatoraj]

Hello!,

Getting the "Found no NVIDIA driver on your system. Please check that you have an NVIDIA GPU and installed a driver from http://www.nvidia.com/Download/index.aspx" error when trying to execute the below code.

from llmlingua import PromptCompressor
llm_lingua = PromptCompressor()

[iofu728]

Hi @defatoraj, by default, using LLMLingua requires NVIDIA CUDA to be enabled. You can switch to CPU mode with the following settings.

from llmlingua import PromptCompressor
llm_lingua = PromptCompressor(device_map="cpu")

[defatoraj]

Hi, i apologize for the late response. It is still not working for me.
Executed:
from llmlingua import PromptCompressor
llm_lingua = PromptCompressor(device_map="cpu")
And got this error:
Canceled future for execute_request message before replies were done
The Kernel crashed while executing code in the the current cell or a previous cell. Please review the code in the cell(s) to identify a possible cause of the failure. Click here for more info. View Jupyter log for further details.

[scottlilo]

Prompt compression (https://github.com/Quickpath/BigDataUniversalRemote/issues/2857) Doesnt seem to be viable for our current design pod

It adds about 3 GB to our docker image, and runs EXTREMELY slow on CPU (It was designed to be GPU accelerated)

Even with small prompts, it times out and does not respond when run on CPU

I believe to be practical, we would need an EC2 with GPU acceleration