Connection for use sagemaker Endpoint

[theja0473]

Validations

• I believe this is a way to improve. I'll try to join the Continue Discord for questions
• I'm not able to find an open issue that requests the same enhancement

Problem

Currently Ollama or any self hosted services only allowed my suggestion lets connect to end point of Sagemaker and serve Copilot use case

Solution

No response

[sestinj]

Hi @theja0473 can you share more about what API format you expect for the Sagemaker endpoint?

[mikhail-khodorovskiy]

We are also interested in hosting the model in Sagemaker. The format of the API is:

https://runtime.sagemaker.us-west-2.amazonaws.com/endpoints/my-model-id/invocations

For boto, the documentation of how to invoke the endpoint: https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/sagemaker-runtime/client/invoke_endpoint.html

The credentials can be provided the same way as you have it implemented for bedrock: https://docs.continue.dev/reference/Model%20Providers/bedrock

[tobiasbartsch]

I am also very interested in hosting a model in Sagemaker. I think the main thing to add (as @mikhail-khodorovskiy pointed out) is credentials handling identical to how Bedrock is done.

[mikhail-khodorovskiy]

We actually were able to import the model we want into Bedrock where the model id but it still does not work: #1749

[sestinj]

Hey all! Wanted to check in on this: we had a number of PRs made for Bedrock custom models as well as a dedicated SageMaker provider:

• https://docs.continue.dev/customize/model-providers/more/sagemaker
• https://docs.continue.dev/customize/model-providers/bedrock#custom-imported-models
  Do these address the problems here?

[github-actions]

This issue hasn't been updated in 90 days and will be closed after an additional 10 days without activity. If it's still important, please leave a comment and share any new information that would help us address the issue.

[github-actions]

This issue was closed because it wasn't updated for 10 days after being marked stale. If it's still important, please reopen + comment and we'll gladly take another look!