[Bug] MultimodalQnA with the Milvus vector dataset gets a "no video segments retrieved error" #1858
Description
Priority
P2-High
OS type
Ubuntu
Hardware type
Xeon-other (Please let us know in description)
Installation method
- Pull docker images from hub.docker.com
- Build docker images from source
- Other
- N/A
Deploy method
- Docker
- Docker Compose
- Kubernetes Helm Charts
- Kubernetes GMC
- Other
- N/A
Running nodes
Single Node
What's the version?
main and the v1.3rc branch
Description
I followed the MultimodalQnA README instructions to launch the example with the Milvus vector dataset backend.
I ingested a variety of images, videos, PDF files, and audio files and then submitted a query. It doesn't seem to matter what is ingested or what is asked, I am getting an error "There is no video segments retrieved given the query!" from both the UI and curl commands.
Reproduce steps
Tested on CLX
curl http://${host_ip}:${MEGA_SERVICE_PORT}/v1/multimodalqna \
-H "Content-Type: application/json" \
-X POST \
-d '{"messages": "What is the revenue of Nike in 2023?"}'
{"id":"chatcmpl-ZXb4gFT6gAu84MAuYUTH7n","object":"chat.completion","created":1745271813,"model":"multimodalqna","choices":[{"index":0,"message":{"role":"assistant","content":"There is no video segments retrieved given the query!","audio":null},"finish_reason":"stop","metadata":null}],"usage":{"prompt_tokens":0,"total_tokens":0,"completion_tokens":0}}
Raw log
lvm logs
/usr/local/lib/python3.11/site-packages/pydantic/_internal/_fields.py:160: UserWarning: Field "model_name_or_path" has conflict with protected namespace "model_".
You may be able to resolve this warning by setting `model_config['protected_namespaces'] = ()`.
warnings.warn(
[2025-04-21 22:00:53,283] [ ERROR] - opea_llava - Health check failed: ('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer'))
[2025-04-21 22:00:53,283] [ ERROR] - opea_llava - OpeaLlavaLvm health check failed.
[2025-04-21 22:00:53,284] [ INFO] - Base service - CORS is enabled.
[2025-04-21 22:00:53,285] [ INFO] - Base service - Setting up HTTP server
[2025-04-21 22:00:53,286] [ INFO] - Base service - Uvicorn server setup on port 9399
INFO: Waiting for application startup.
INFO: Application startup complete.
INFO: Uvicorn running on http://0.0.0.0:9399 (Press CTRL+C to quit)
[2025-04-21 22:00:53,297] [ INFO] - Base service - HTTP server setup successful
[2025-04-21 22:00:53,300] [ INFO] - opea_lvm_microservice - OPEA LVM Microservice is starting....
INFO: 172.18.0.1:45222 - "POST /v1/lvm HTTP/1.1" 200 OK
[2025-04-21 22:17:51,558] [ ERROR] - opea_lvm_microservice - Error during lvm invocation: 500: There is no video segments retrieved given the query!
INFO: 172.18.0.1:35314 - "POST /v1/lvm HTTP/1.1" 500 Internal Server ErrorAttachments
No response