AI缓存 基于语义化缓存如何区分模型

[asinglestep]

1. 一个供应商有多个模型，需要对每个模型都开启一个策略吗？
2. 使用流式输出时，模型返回的第一个chunk

{"id":"0bd488ca-1e8d-9873-a523-2c53331c3088","object":"chat.completion.chunk","created":1780995015,"model":"hy3-preview","choices":[{"index":0,"delta":{"role":"assistant"}}]}

结果无法写到缓存中，日志报错如下

handle non last chunk failed, error: [handleStreamChunk] processSSEMessage failed, error: [processSSEMessage] unable to extract content from message; cache content is nil: data: {"id":"0bd488ca-1e8d-9873-a523-2c53331c3088","object":"chat.completion.chunk","created":1780995015,"model":"hy3-preview","choices":[{"index":0,"delta":{"role":"assistant"}}]}

[CH3CHO]

1. 你可以在全局或者域名粒度开启缓存策略。如果你自己手写 YAML 的话，也可以在大模型服务粒度开启策略
2. 这个 chunk 没有 message，可能有点问题。

[asinglestep]

混元模型流式输出时，第一个chunk并不会返回content，这种要怎么处理？

data: {"id": "REPLACED_ID", "object": "chat.completion.chunk", "created": 1779958293, "model": "hy3-preview", "choices": [{"index": 0, "delta": {"role": "assistant"}}]}

data: {"id": "REPLACED_ID", "object": "chat.completion.chunk", "created": 1779958293, "model": "hy3-preview", "choices": [{"index": 0, "delta": {"content": "你好"}}]}

data: {"id": "REPLACED_ID", "object": "chat.completion.chunk", "created": 1779958293, "model": "hy3-preview", "choices": [{"index": 0, "delta": {"content": "帮你的吗"}}]}

data: {"id": "REPLACED_ID", "object": "chat.completion.chunk", "created": 1779958293, "model": "hy3-preview", "choices": [{"index": 0, "delta": {"content": " 😊"}, "finish_reason": "stop"}]}

data: {"id": "REPLACED_ID", "object": "chat.completion.chunk", "created": 1779958293, "model": "hy3-preview", "choices": [], "usage": {"prompt_tokens": 16, "completion_tokens": 11, "total_tokens": 27, "prompt_tokens_details": {"cached_tokens": 0}, "completion_tokens_details": {"reasoning_tokens": 0}}}

data: [DONE]

[CH3CHO]

提个 issue 吧，详细介绍一下我们怎么重现这个问题，看看能否在 ai-cache 里兼容这个场景。

[asinglestep]

提交了：#3953