Serve Query Responses from DynamoDB Conversation History

[thejustin]

Can you provide an option to serve responses only from the DynamoDB Conversation History? This way a user can cost optimize their usage of the ml.g5.24xlarge instance?

[bigadsoleiman]

Hi @thejustin checkout the new released version to achieve this.

You can now integrate custom or fake LLMs. Without being forced to deploy a model on sagemaker. You can potentially have no LLM at all and still consult the conversation history and UI.

For a custom/fake LLM you will need to write your own adapter like this one and simply return your preferred LLM object in get_llm method.