[BUG]: Invocation of model ID deepseek.r1-v1:0 with on-demand throughput isn’t supported. Retry your request with the ID or ARN of an inference profile that contains this model.

[NameMeLeo]

How are you running AnythingLLM?

Docker (local)

What happened?

I got the below output when I am using bedrock with deepseek, its seems that bedrock required a inferenceConfig as below.

Error msg: Invocation of model ID deepseek.r1-v1:0 with on-demand throughput isn’t supported. Retry your request with the ID or ARN of an inference profile that contains this model.

aws sample api query:

{
  "modelId": "deepseek.r1-v1:0",
  "contentType": "application/json",
  "accept": "application/json",
  "body": {
    "inferenceConfig": {
      "max_tokens": 512
    },
    "messages": [
      {
        "role": "user",
        "content": "this is where you place your input text"
      }
    ]
  }
}

Are there known steps to reproduce?

set model ID to deepseek.r1-v1:0 in LLM provider page

No response

[NameMeLeo]

another error from bedrock when using r1 with model id us.deepseek.r1-v1:0

AWSBedrock:streaming - could not stream chat. Unsupported content block type(s): { "reasoningContent": { "text": "Okay" } }