-
-
Notifications
You must be signed in to change notification settings - Fork 3.1k
Closed
Labels
Description
LocalAI version:
v3.7.0 (9ecfdc5) docker
Environment, CPU architecture, OS, and Version:
Linux cpu docker
Describe the bug
llama-cpp backend ignores top_n in reranker request
To Reproduce
$ curl -X POST http://localhost:8080/v1/rerank -H "Content-Type: application/json" -d '{
"query": "What is the capital of France?",
"documents": [
"Paris is the capital of France.",
"The Eiffel Tower is in Paris.",
"London is the capital of England."
],
"model": "jina-reranker-v1-base-en",
"top_n":1
}'
{"model":"jina-reranker-v1-base-en","usage":{"total_tokens":58,"prompt_tokens":58},
"results":[
{"index":0,"document":{"text":"Paris is the capital of France."},"relevance_score":0.11916422843933105},
{"index":1,"document":{"text":"The Eiffel Tower is in Paris."},"relevance_score":0.029489612206816673},
{"index":2,"document":{"text":"London is the capital of England."},"relevance_score":0.03191265091300011}
]}
Expected behavior
Expecting top_n elements output.
{"results":[{"index":0,"document":{"text":"Paris is the capital of France."},"relevance_score":0.11916422843933105]}
Refs
discussion
Python backend fix: #7025 #6700