[Platform] Introduce CachedPlatform

[Guikingone]

Q	A
Bug fix?	no
New feature?	yes
Docs?	yes
Issues	Somehow related to #337
License	MIT

Hi 👋🏻

This PR aim to introduce a caching layer for Ollama platform (like OpenAI, Anthropic and more already does).

[chr-hertel]

Let's zoom a bit out here, for two reasons:

1. doesn't ollama caching already?
2. if we want to have it user land, why only ollama?

[Guikingone]

doesn't ollama caching already?

Ollama does a "context caching" and/or a K/V caching, it stores the X latest messages for the model window (or pending tokens to speed TTFT), it's not a cache that returns the generated response if the request already exist.

if we want to have it user land, why only Ollama?

Well, because that's the one that I use the most and the easiest to implement first but we can integrate it for every platform if that's the question, we just need to use the API contract, both Anthropic and OpenAI already does it natively 🤔

If the question is: Could we implement it at the platform layer for every platform without relying on API calls, well, that's not a big deal to be honest and we could easily integrate it 🙂

[chr-hertel]

there are left overs in the bundle config of Ollama and the OllamaResultConverter - see previous comments - those changes are not relevant anymore, right?

[Guikingone]

IMHO, the token usage must be added in a separate PR, it requires to update existing platforms to send the informations in the result.

Out of the current scope for this PR IMO.

[Guikingone]

@junaidbinfarooq I created an issue for the TokenUsage DTO 🙂

[OskarStark]

After my comments

[OskarStark]

Thank you @Guikingone.