Added support for stability API for bedrock

[gauravrele87]

Summary

This PR adds support for stability AI in bedrock. Implementation is going to return the image in base64 format as text string.

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Ignored Deployment

Name	Status	Preview	Comments	Updated (UTC)
langchain	⬜️ Ignored (Inspect)			Jul 7, 2023 9:26pm

[baskaryan]

just let us know when it's ready for review @gauravrele87!

[gauravrele87]

@baskaryan Its ready for review

[efriis]

Hey @gauravrele87 ! How do you hope to use this? I believe you're calling an image generation model and getting a base64-encoded image back, which doesn't fit our LLM (text-to-text) abstraction particularly well.

Would a tool that generates images from the bedrock-stability endpoint be better?

[3coins]

@efriis
Thanks for reviewing this PR. This implementation returns a base64 image as text output from the LLM class. We are expecting most users would want to write the image to filesystem for later access.

from base64 import b64decode
img = b64decode(base64_data)
with open('image.png', 'wb') as f:
    f.write(img)

Would a tool that generates images from the bedrock-stability endpoint be better?

Most tools I see so far are generating text output. Is there an example of a similar tool that generates non-text output? I am also envisioning using LLM class directly in the tool to call the Bedrock service, so having this code here will help. Let me know if you have a better way to handle this.

[3coins]

@baskaryan @efriis
Let @gauravrele87 or me know if we can do anything to move this forward, or if there is a better way to implemented this. Thanks for your help.

[efriis]

Hey @3coins @gauravrele87 ! Let's skip merging this into the LLM class because it doesn't make sense in the LLM abstraction.

We haven't put much thought into image generation abstractions yet, so in the meantime, would you mind implementing it in a similar fashion to langchain/tools/steamship_image_generation?

It would be amazing if you could factor out shared logic between these Bedrock APIs into a shared BedrockAPIWrapper class that both the new langchain.tools.bedrock_image_generation.BedrockStabilityTool and existing/refactored langchain.llms.bedrock.Bedrock inherit from. Feel free to keep it in langchain.llms.bedrock or put it in langchain.utilities.bedrock - up to you!

[3coins]

@efriis
Thanks for the guidance on this change. We will submit another PR with those updates, as it will require refactor for the Bedrock LLM.

[baskaryan]

@3coins ok to close this PR?

[leo-gan]

Closing