[Bug] AgentQnA docker compose deployment not updated with vllm

[yongfengdu]

Priority

P2-High

OS type

Ubuntu

Hardware type

Xeon-GNR

Installation method

• Pull docker images from hub.docker.com
• Build docker images from source
• Other

Deploy method

• Docker
• Docker Compose
• Kubernetes Helm Charts
• Kubernetes GMC
• Other

Running nodes

Single Node

What's the version?

latest

Description

1. deployment/docker_compose/intel/hpu/gaudi directory is still using tgi as llm inference server, although the README always switched to vllm.
2. vllm start options should contain "--enable-auto-tool-choice --tool-call-parser ${model_parser}" according to readme here https://github.com/opea-project/GenAIComps/tree/main/comps/agent/src#12-llm-engine and the vLLM guides(https://docs.vllm.ai/en/latest/features/tool_calling.html), but the test scripts and AgentQnA readme (https://github.com/opea-project/GenAIExamples/tree/main/AgentQnA#deploy-with-docker) doesn't include it. At least they should be consistent even if the additional options doesn't matter.

Reproduce steps

Inconsistent docs.

Raw log

Attachments

No response

[yinghu5]

Hi @yongfengdu
Thank you a lot for the report. The AgentQnA sample was updated with vllm #1656 recently. Please feel free to validate them and reopen if any issues. thank you!