Cohere rerank on Bedrock - `api_version` missing in the sdk

[Othmane796]

SDK Version (required)
Provide the version you are using. To get the version, run the following python snippet

import cohere

print(cohere.__version__) # 5.6.1

version is : 5.13.1

Describe the bug
A clear and concise description of what the bug is.

I'm unable to call the cohere rerank model through the AWS Bedrock client.
Going through the cohere v2 client it works fine:

co_cohere = cohere.ClientV2()
top_n = 3
query = "What is the capital of the United States?"
docs = ["text1", .."textN"]

response = co_cohere.rerank(
    model="rerank-v3.5",
    query=query,
    documents=docs,
    top_n=top_n,
)
# works

However following the documentation here and also reading the code in this sdk If I run :

co_bedrock = cohere.BedrockClient(aws_region="eu-central-1")

response_bedrock = co_bedrock.rerank(
    model="cohere.rerank-v3-5:0",
    query=query,
    documents=docs,
    top_n=top_n, 
)

I get the following error:

BadRequestError: status_code: 400, body: {'meta': {'billed_units': {'input_tokens': -1.0, 'output_tokens': -1.0}, 'tokens': {'input_tokens': -1.0, 'output_tokens': -1.0}}, 'message': 'Malformed input request: #/api_version: 1 is not greater or equal to 2, please reformat your input and try again.'}

I tried adding 'api_version' as parameter in both the .rerank() and .BedrockClient() functions but it doesn't work.

Good news is I'm able to go through the aws boto3 sdk and make it work with this:

import boto3
import json
bedrock_client = boto3.client(service_name="bedrock-runtime", region_name="eu-central-1")
json_params = {
    "query": query,
    "documents": docs,
    "top_n": top_n,
    "api_version": 2
}

result_bedrock = bedrock_client.invoke_model(
    modelId="cohere.rerank-v3-5:0",
    body=json.dumps(json_params),
    contentType="application/json",
    accept="application/json",
)
raw = result_bedrock["body"].read() # works

Question: Is what I did the expected solution ? What's the recommended way to use cohere though AWS Bedrock ?

Screenshots
If applicable, add screenshots to help explain your problem.

[mkozakov]

hey @Othmane796, bedrock currently only supports API V2, which means the recommended way to use rerank is with the V2 client:

co_bedrock = cohere.BedrockClientV2()

under the hood, that client actually injects api_version=2 into the request body.

we will:

• update cohere.BedrockClient to throw a clearer error when used with rerank
• swap our documentation to use BedrockClientV2() as soon as Chat V2 and Embed V2 become available on bedrock as well

[Othmane796]

Awesome! thanks @mkozakov !

Out of curiosity is it normal that the outputs are exactly the same when going through bedrock or cohere directly? Rerun both multiple times and output seems deterministic for each though:

I'm using the examples from the doc though

top_n = 3
query = "What is the capital of the United States?"
docs = [
    "Carson City is the capital city of the American state of Nevada.",
    "The Commonwealth of the Northern Mariana Islands is a group of islands in the Pacific Ocean. Its capital is Saipan.",
    "Capitalization or capitalisation in English grammar is the use of a capital letter at the start of a word. English usage varies from capitalization in other languages.",
    "Washington, D.C. (also known as simply Washington or D.C., and officially as the District of Columbia) is the capital of the United States. It is a federal district.",
    "Capital punishment has existed in the United States since beforethe United States was a country. As of 2017, capital punishment is legal in 30 of the 50 states.",
]

response = co_cohere.rerank(
    model="rerank-v3.5",
    query=query,
    documents=docs,
    top_n=top_n,
)
for res in response.results:
    print(res.index, res.relevance_score)
    
 #output   
3 0.8742601
0 0.17292508
4 0.10793502

When the model is hosted in bedrock:

co_bedrock = cohere.BedrockClientV2(aws_region="eu-central-1")
response_bedrock = co_bedrock.rerank(
   model="cohere.rerank-v3-5:0",
   query=query,
   documents=docs,
   top_n=top_n,
)
for res in response_bedrock.results:
   print(res.index, res.relevance_score)

# output
3 0.8745175
0 0.17288318
4 0.10782225