Add Intel Arc iGPU support (Meteor Lake)

[sulliwane]

Feature request

After reading this blogpost https://huggingface.co/blog/phi2-intel-meteor-lake, I tried the instructions. My goal is to run phi2 100% locally.

I successfully executed the python file to apply 4-bit quantization to the 2.7-billion Microsoft Phi-2 model, and output a 1.7G openvino_model.bin model file.

I thought, let's run Huggingface text-generation-inference service locally + huggingface chat-ui locally and point chat-ui to tgi 127.0.0.1:8080/generate_stream.

But when trying to run TGI on my Intel® Core™ Ultra 7 155H, it says this: docker: Error response from daemon: could not select device driver "" with capabilities: [[gpu]].

docker run --rm --network none --gpus all -p 8080:8080 -v /home/victor/llm-ai/huggingface/phi-2-openvino:/data --pull always ghcr.io/huggingface/text-generation-inference:latest --model-id /data/openvino_model.bin

Status: Image is up to date for ghcr.io/huggingface/text-generation-inference:latest
docker: Error response from daemon: could not select device driver "" with capabilities: [[gpu]].

I guess TGI docker image is missing support for Intel Arc iGPU?

If that's the case, how did this guy shot this video?

Motivation

Leverage Intel Arc iGPU on Meteor Lake

Your contribution

Happy to try any solution on my hardware