How to pass additional data while calling InvokeStreamingAsync() method?

[ddramireddy]

Hi Team,

We have implementation of our own Chat Completion service. When we call a native prompt based function, using SK's InvokeStreamingAsync() method, this in turn is calling our custom implementation of GetStreamingChatCompletionAsync() API.

Here, I would like to pass some additional data to GetStreamingChatCompletionAsync() method. One approach can be Kernel's data. But, kernel being singleton object, it can cause multi-threading issues.

Is there any solution for this issue?

We are calling InvokeStreamingAsync as below:
var stream = function.InvokeStreamingAsync(kernel, invocationContext?.Arguments, cancellationToken: cancellationToken).ConfigureAwait(false);
                    
Signature of : GetStreamingChatCompletionsAsync

public async IAsyncEnumerable<StreamingChatMessageContent> GetStreamingChatCompletionsAsync(
            ChatHistory chat,
            Kernel kernel,
            OpenAIPromptExecutionSettings? requestSettings = null,
            CancellationToken cancellationToken = default)

[stephentoub]

But, kernel being singleton object,

It's not a singleton unless you're creating/registering it as such. The AddKernel method registers it as transient, for reasons like this.

[ddramireddy]

Thank you. This helps.