Enhancement: Amazon Bedrock #2441
Replies: 6 comments 2 replies
-
I would recommend using aws4fetch, or the official node package if size doesn't matter. https://docs.aws.amazon.com/bedrock/latest/APIReference/welcome.html https://docs.aws.amazon.com/bedrock/latest/userguide/api-methods-run-inference.html |
Beta Was this translation helpful? Give feedback.
-
https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_InvokeModel.html Looks relatively easy to use. :) |
Beta Was this translation helpful? Give feedback.
-
fetch("https://bedrock-runtime.us-east-1.amazonaws.com/model/anthropic.claude-v2/invoke-with-response-stream", {
"headers": {
"accept": "*/*",
"accept-language": "en-US,en-CA;q=0.9,en;q=0.8",
"amz-sdk-invocation-id": "removed-removed-removed-removed-removed",
"amz-sdk-request": "attempt=1; max=0",
"authorization": "AWS4-HMAC-SHA256 Credential=removed/20231104/us-east-1/bedrock/aws4_request, SignedHeaders=amz-sdk-invocation-id;amz-sdk-request;content-length;content-type;host;x-amz-content-sha256;x-amz-date;x-amz-security-token;x-amz-user-agent;x-amzn-bedrock-accept, Signature=removed",
"content-type": "application/json",
"sec-ch-ua": "\"Not_A Brand\";v=\"8\", \"Chromium\";v=\"120\", \"Google Chrome\";v=\"120\"",
"sec-ch-ua-mobile": "?0",
"sec-ch-ua-platform": "\"macOS\"",
"sec-fetch-dest": "empty",
"sec-fetch-mode": "cors",
"sec-fetch-site": "cross-site",
"x-amz-content-sha256": "removed",
"x-amz-date": "20231104T162611Z",
"x-amz-security-token": "removed",
"x-amz-user-agent": "aws-sdk-js/1.0.0 os/macOS/10.15.7 lang/js md/browser/Chrome_120.0.0.0 api/bedrock_runtime/1.0.0 Bedrock",
"x-amzn-bedrock-accept": "*/*"
},
"referrer": "https://us-east-1.console.aws.amazon.com/",
"referrerPolicy": "strict-origin-when-cross-origin",
"body": "{\"prompt\":\"\\n\\nHuman: Tell me about your day.\\n\\nAssistant:\",\"max_tokens_to_sample\":2048,\"temperature\":1,\"top_k\":250,\"top_p\":0.999,\"stop_sequences\":[\"\\n\\nHuman:\"],\"anthropic_version\":\"bedrock-2023-05-31\"}",
"method": "POST",
"mode": "cors",
"credentials": "include"
}); |
Beta Was this translation helpful? Give feedback.
-
LangChain supports Bedrock too. https://js.langchain.com/docs/integrations/chat/bedrock There's now Llama 2 13B support in Bedrock! https://aws.amazon.com/blogs/aws/amazon-bedrock-now-provides-access-to-llama-2-chat-13b-model/ |
Beta Was this translation helpful? Give feedback.
-
I can see this being useful for SDXL but makes sense to use LiteLLM for Bedrock access. |
Beta Was this translation helpful? Give feedback.
-
@danny-avila Can you reconsider your decision now that AWS bedrock supports Claude Opus? |
Beta Was this translation helpful? Give feedback.
-
Contact Details
No response
What features would you like to see added?
Getting access to Anthropic's APIs via Amazon Bedrock is much easier. In the time it took me to write this ticket, Amazon already approved access to Claude 2 on my company AWS account. 🙂
More details
Pricing seems to be exactly the same.
https://www-files.anthropic.com/production/images/model_pricing_july2023.pdf
https://aws.amazon.com/bedrock/pricing/
Which components are impacted by your request?
No response
Pictures
Code of Conduct
Beta Was this translation helpful? Give feedback.
All reactions