[Feature Idea/Request] Add Amazon Bedrock as a provider in Kong AI Proxy

[kandakji]

Hi,

I'm a Data & AI Solutions Architect at AWS. Enterprise customers are adopting LLM/FM gateways to facilitate project Time-to-market and to unify and standardize monitoring, operations, security, and app integration.

Amazon Bedrock is one of the most popular services on AWS that enterprise customers, like Intuit, Adidas, Booking.com, Nasdaq and Salesforce, use to access models from 7 providers (AI21 Labs, Anthropic, Cohere, Meta, Mistral AI, Stability AI, and Amazon) as a completely serverless fully-managed service.

We (fellow solutions architects and myself) would be happy to contribute to integrating Amazon Bedrock within Kong's AI Proxy to facilitate adoption of this plugin for customers and users using Amazon Bedrock on AWS (in case Bedrock is not within your roadmap already).

[hanshuebner]

@tysoekong Is Amazon Bedrock something that is on our roadmap already?

[subnetmarco]

Yes, we would like to have AWS Bedrock. If you would like to contribute, we can offer guidance. Is it still something that you would still like to do? @kandakji

[kandakji]

Hi Marco, yes, I'd be happy to start working on the contribution with some colleagues by the beginning of next week. What kind of guidance can you provide us ?

[tysoekong]

Hey @kandakji - I think we are good to get working on this! Any help is always amazing.

We can meet ASAP and I can show you all how the interface works? It should be simple as:

1. A new file to handle chosen Bedrock inference formats (just a "sub-class" type Lua file that implements the required functions)
2. Plug that interface into the config options so that users to activate it

[tysoekong]

I predict only... 1 hour call, to go into it, alternatively I can just email you the interface docs itself and read in your own time?

[shaneutt]

🖐️ I'd like to hang out in that call too, to check things out and chat.

[Tieske]

me too 😄

[henriquelucasdf]

Hey folks, how's it going?

Firstly, I'd like to congratulate and thank the entire team involved with the AI Proxy plugin, it's a fantastic feature.

I'd like to ask if there's any news regarding integration with Amazon Bedrock? Is there any place where I can follow the development or try to contribute?

[subnetmarco]

It's currently prioritized and in the works, will ship in 3.8 in August but most likely you will be able to compile a beta by end of June.

[subnetmarco]

The AWS Bedrock PR has been opened, needs to be reviewed and then it will be merged: #13054

We also added support for Google Gemini by the way, which also needs to be reviewed before being merged: #12948