How to write streamable Native Function?

[LefanTan]

Hi. I'm looking to use InvokeStreamingAsync on a Native Function plugin. What's the best way to go about this?

I currently have a native function like this

    [KernelFunction]
    [Description("Get a chat response")]
    public async Task<ChatMessage> ChatAsync(

Ideally, I can use IAsyncEnumerable as the return type?

[dmytrostruk]

@LefanTan Yes, you can use IAsyncEnumerable as the return type in your native function, and you can use InvokeStreamingAsync to call this function. Here is test example with this functionality:

[Fact]
public async Task StreamingWithNativeFunction()
{
    // Arrange 
    async IAsyncEnumerable<int> TestMethod()
    {
        yield return 1;
        yield return 2;
        yield return 3;
    }

    var function = KernelFunctionFactory.CreateFromMethod(TestMethod);

    var kernel = Kernel.CreateBuilder().Build();

    // Act 
    var result = new List<int>();
    await foreach (var item in kernel.InvokeStreamingAsync<int>(function))
    {
        result.Add(item);
    }

    // Assert 
    Assert.Equal([1, 2, 3], result);
}

[dmytrostruk]

@LefanTan Here is another example when native function lives separately and marked with KernelFunction attribute, like in your example:

[KernelFunction]
public async IAsyncEnumerable<int> StreamingFunction()
{
    yield return 1;
    yield return 2;
    yield return 3;
}

[Fact]
public async Task StreamingWithNativeFunction()
{
    // Arrange
    var plugin = KernelPluginFactory.CreateFromType<Tests>("MyPlugin");
    var kernel = new Kernel(plugins: [plugin]);

    // Act 
    var result = new List<int>();
    await foreach (var item in kernel.InvokeStreamingAsync<int>("MyPlugin", "StreamingFunction"))
    {
        result.Add(item);
    }

    // Assert 
    Assert.Equal([1, 2, 3], result);
}

[LefanTan]

Thanks, much appreciated! For anyone reading this in the future, adding the generic type for InvokeStreamingAsync<T> is the key to getting the proper response back.