Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Python: Fix SKFunction.invoke_stream_async #3986

Merged

Conversation

johnliu55-msft
Copy link
Contributor

@johnliu55-msft johnliu55-msft commented Dec 5, 2023

Motivation and Context

The invoke_stream_async method on a native SKFunction is not functional. The following sample code should work:

import asyncio

import semantic_kernel as sk
from semantic_kernel.skill_definition import sk_function


class StreamNativeSkill:
    @sk_function(name="stream")
    async def stream(self, input: str) -> None: # This should also work: -> AsyncGenerator
        for i in range(5):
            await asyncio.sleep(1)
            yield input + str(i)


async def main():
    kernel = sk.Kernel()

    skill = kernel.import_skill(StreamNativeSkill(), "stream_skill")
    native_stream_func = skill["stream"]

    async for chunk in native_stream_func.invoke_stream_async("HELLO!"):
        print(chunk)


if __name__ == "__main__":
    asyncio.run(main())

And should output:

HELLO!0
HELLO!1
HELLO!2
HELLO!3
HELLO!4

But now it only shows:

HELLO!

Description

Add four new DelegateTypes for native asynchronous generator functions for different types of parameters:

  • OutAsyncGenerator: No parameter, has yield
  • InStringOutAsyncGenerator: One str parameter, has yield
  • InContextOutAsyncGenerator: One SKContext parameter, has yield
  • InStringAndContextOutAsyncGenerator: Both str and SKContext parameters, has yield

Along with these four new types, add four new DelegateInference functions to detect the types:

  • infer_out_async_generator
  • infer_in_string_out_async_generator
  • infer_in_context_out_async_generator
  • infer_in_string_and_context_out_async_generator

And because the async generator functions needs to use inspect.isasyncgenfunction function to check instead of inspect.iscoroutinefunction, add another parameter is_asyncgenfunc to all the infer functions.

And finally, fix SKFunction._invoke_native_stream_async so that it can correctly iterating from an async generator function.

Contribution Checklist

@johnliu55-msft johnliu55-msft requested a review from a team as a code owner December 5, 2023 16:18
@shawncal shawncal added the python Pull requests for the Python Semantic Kernel label Dec 5, 2023
Copy link
Contributor

@alexchaomander alexchaomander left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good to me! Thank you for adding this!

@moonbox3 moonbox3 added this pull request to the merge queue Dec 8, 2023
Merged via the queue into microsoft:main with commit 37f2760 Dec 8, 2023
26 checks passed
@johnliu55-msft johnliu55-msft deleted the fix-skfunc-invoke-stream-async branch January 4, 2024 01:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
python Pull requests for the Python Semantic Kernel
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants