Error using rerank twice

[knaga35]

LocalAI version:

• v2.23.0-cublas-cuda12

Environment, CPU architecture, OS, and Version:

• Ubuntu 22.04 x86_64

Describe the bug

• When using rerank twice on a chatbot created with Dify, an error may occur without response.
  -- The first time, it is simply reranked in the knowledge base.
  -- The second time, it is reranked in the Multi-path Retrieval mode. (It seems to be occurred the error.)

To Reproduce

• Dify chatbot application

1. Create knowledge
   　Retrieval Setting : Vector Search
   　Rerank Model : BAAI/bge-reranker-v2-m3

2. Create a chatbot application
   　Add the knowledge base in the context.
   　　Retrieval Setting : Rerank Model
   　　Rerank Model : BAAI/bge-reranker-v2-m3

3. Enter content in the box to start debugging the Chatbot
   　There is no problem depending on the content of the question, but the following error may occur.

Expected behavior

• Some kind of response will be returned.

Logs

4:12AM INF Success ip=172.18.0.10 latency=25.650528ms method=POST status=200 url=/v1/rerank
4:12AM ERR Server error error="rpc error: code = Unknown desc = Exception calling application: 'Result' object has no attribute 'results'" ip=172.18.0.10 latency=12.944511ms method=POST status=500 url=/v1/rerank

Additional context

[kyleli666]

Very similar issue here with v2.26.0-cublas-cuda12-ffmpeg.
I use bge-m3 for embedding and then bge-reranker-v2-m3 for reranking, and no matter with Dify's OpenAI-Compatible plugin v0.0.9 or v0.0.10 or LocalAI plugin v0.0.3 to connect Local-ai, I got this error.
I always get this error with some certain input, but not with other inputs.

Model config in helm

modelsConfigs:
  bge-m3.yaml: |
    name: bge-m3
    backend: sentencetransformers
    embeddings: true
    parameters:
      model: BAAI/bge-m3
    context_size: 8192
    cuda: true
    f16: true
  bge-reranker-v2-m3.yaml: |
    name: bge-reranker-v2-m3
    backend: rerankers
    parameters:
      model: BAAI/bge-reranker-v2-m3
    context_size: 8192
    cuda: true
    f16: true

local-ai log

9:45AM INF Success ip=10.4.10.76 latency=27.542686ms method=POST status=200 url=/v1/embeddings
9:45AM DBG context local model name not found, setting to the first model first model name=bge-reranker-v2-m3
9:45AM DBG guessDefaultsFromFile: not a GGUF file filePath=/models/BAAI/bge-reranker-v2-m3
9:45AM DBG JINA Rerank Request recieved model=bge-reranker-v2-m3
9:45AM DBG Model already loaded in memory: bge-reranker-v2-m3
9:45AM DBG Checking model availability (bge-reranker-v2-m3)
9:45AM DBG Model 'bge-reranker-v2-m3' already loaded
9:45AM INF Success ip=10.4.10.76 latency=107.110832ms method=POST status=200 url=/v1/rerank
9:45AM DBG context local model name not found, setting to the first model first model name=bge-reranker-v2-m3
9:45AM DBG guessDefaultsFromFile: not a GGUF file filePath=/models/BAAI/bge-reranker-v2-m3
9:45AM DBG JINA Rerank Request recieved model=bge-reranker-v2-m3
9:45AM DBG Model already loaded in memory: bge-reranker-v2-m3
9:45AM DBG Checking model availability (bge-reranker-v2-m3)
9:45AM DBG Model 'bge-reranker-v2-m3' already loaded
9:45AM ERR Server error error="rpc error: code = Unknown desc = Exception calling application: 'Result' object has no attribute 'results'" ip=10.4.10.76 latency=20.541676ms method=POST status=500 url=/v1/rerank

[kyleli666]

Hi @knaga35 I somehow solved this issue by deleting our existing knowledge base and then create a new one.
Maybe there is something corrupted when upgrading Dify. Just FYI, you could also try.

[knaga35]

@kyleli666
Thank you for your reply.
I will try it.

[github-actions]

This issue is stale because it has been open 90 days with no activity. Remove stale label or comment or this will be closed in 5 days.

[github-actions]

This issue was closed because it has been stalled for 5 days with no activity.