Implement semantic cache

[Rtoribiog]

That's a feature that already exists in langchain and will be beneficial to save costs. The idea will be to ported from phyton to c#

https://github.com/zilliztech/GPTCache

https://python.langchain.com/docs/integrations/llms/llm_caching/

https://www.mongodb.com/developer/products/atlas/advanced-rag-langchain-mongodb/

Be able to save all the queries together with llm responses in some database and try to fetch it first from there, if not then call the LLM, and have some parameters to invalidate or update cache from time to time.

[madsbolaris]

@markwallace-microsoft, going to assign this to @dmytrostruk since this is one of the samples he's working on using filters.

[madsbolaris]

Also assigning @eavanvalkenburg since he was working on filters for Python

[dmytrostruk]

Example for Semantic Caching in .NET is here: https://github.com/microsoft/semantic-kernel/blob/main/dotnet/samples/Concepts/Caching/SemanticCachingWithFilters.cs

[Rtoribiog]

Nice, we are going to take a look. Thanks

[moonbox3]

Removing the .Net tag, since the sample exists. Leaving python as we haven't completed this yet.