Implement Assistants Streaming

[EthanBarlo]

What:

• Bug Fix
• New Feature

Description:

This PR implements Full Assistants Streaming API support.
https://platform.openai.com/docs/api-reference/assistants-streaming/events

• ThreadMessageDelta (Including nested classes)

• ThreadRunStepDelta (Including nested classes)

• StreamedThreadRunResponseFactory (To map the events to the correct classes)

• EventStreamResponse (Modified iterator to work with the events format)

• EventStreamResponseItem (Contains the event type and the data, similar to how the api returns the data)

Related:

#357

[EthanBarlo]

I'm fairly certain this is fully feature complete with everything new within the API.
Let me know if anything's spotted that might be missing, or any updates that need to be made.

[knash94]

Nicely done @EthanBarlo!

I was working on adding support for this myself, I made great progress and then saw your PR! I've loaded this up on my project and it seems to be working perfectly apart from the issue I've flagged with the import.

It would be neat to also support streaming for the create thread and run endpoint

[EthanBarlo]

@knash94 Thanks for reviewing it and resolving the type issue.
I'm not doing any work with images, so wouldn't have spotted that one.

I've added the createAndRunStreamed function, and updated the contracts to match the new functionality.

Let me know if any more issues crop up from you're testing.

[eng-ahmad-sameer]

Hey all,
Is there anybody available to review this PR and maybe merge and release it so we can use the streaming functionality?

Thx for your efforts.

[punyflash]

@eng-ahmad-sameer, you can use it by aliasing open-ai client version on composer.json:

"repositories": [
    {
        "type": "vcs",
        "url": "https://github.com/EthanBarlo/OpenAi-php-client.git"
    }
],
"require": {
    "openai-php/client": "dev-Implement-Assistants-Streaming as 0.8.4",
}

[punyflash]

@EthanBarlo was testing out and in most cases it works perfectly. Found 2 main issues:

• Lack of types annotations for stream events. Needed to set them in place, so my ide could give me at least some hints what is going on
• The example with tool output is not working. As soon as you set new $stream variable, foreach loop ends. Was able to run it using while loop and generator instance directly:

$stream = $client->threads()->runs()->createStreamed(
    threadId: 'thread_tKFLqzRN9n7MnyKKvc1Q7868',
    parameters: [
        'assistant_id' => 'asst_gxzBkD1wkKEloYqZ410pT5pd',
    ],
);

$iterator = $stream->getIterator();

while ($iterator->valid()) {
    $item = $iterator->current();
    $iterator->next();

    // ----

    if ($item->event === 'thread.run.requires_action') {
        $stream = $client->threads()->runs()->submitToolOutputsStreamed(
            threadId: $item->data->threadId,
            runId: $item->data->id,
            parameters: [
                'tool_outputs' => [[
                    'tool_call_id' => 'call_KSg14X7kZF2WDzlPhpQ168Mj',
                    'output' => '12',
                ]],
            ]
        );
        $iterator = $stream->getIterator();
    }

    // ----
}

[spont4e]

Hey all, Is there anybody available to review this PR and maybe merge and release it so we can use the streaming functionality?

Thx for your efforts.

🥇@gehrisandro and 🥈@nunomaduro are the people

[EthanBarlo]

@punyflash Thanks for pointing out these issues

• Lack of types annotations for stream events. Needed to set them in place, so my ide could give me at least some hints what is going on

Yeah this is something worth adding, I don't actually know how to set those up.
So if you could point me to something or request the necessary changes, that'd be helpful.

• The example with tool output is not working.

Yeah that example does need to be updated,
Heres an updated example, closer to how i've been using it.
I'll probably update the example in the readme with this.

$stream = $client->threads()->runs()->createStreamed(
    threadId: 'thread_tKFLqzRN9n7MnyKKvc1Q7868',
    parameters: [
        'assistant_id' => 'asst_gxzBkD1wkKEloYqZ410pT5pd',
    ],
);

do{
    foreach($stream as $response){
        $response->event // 'thread.run.created' | 'thread.run.in_progress' | .....
        $response->data // ThreadResponse | ThreadRunResponse | ThreadRunStepResponse | ThreadRunStepDeltaResponse | ThreadMessageResponse | ThreadMessageDeltaResponse

        switch($response->event){
            case 'thread.run.created':
            case 'thread.run.queued':
            case 'thread.run.completed':
            case 'thread.run.cancelling':
                $run = $response->data;
                break;
            case 'thread.run.expired':
            case 'thread.run.cancelled':
            case 'thread.run.failed':
                $run = $response->data;
                break 3;
            case 'thread.run.requires_action':
                // Overwrite the stream with the new stream started by submitting the tool outputs
                $stream = $client->threads()->runs()->submitToolOutputsStreamed(
                    threadId: $run->threadId,
                    runId: $run->id,
                    parameters: [
                        'tool_outputs' => [
                            [
                                'tool_call_id' => 'call_KSg14X7kZF2WDzlPhpQ168Mj',
                                'output' => '12',
                            ]
                        ],
                    ]
                );
                break;
        }
    }
} while ($run->status != "completed")

[petermjr]

You can check these changes for adding faking capabilities. They can be implemented similarly in this branch
petermjr/openai-php@v0.8.4...v0.8.5

[subet]

Hi,

Thanks for the great and full-featured SDK. Is there an estimated timeline for when this feature might be integrated and when we might expect the next release?

Regards

[EthanBarlo]

@subet
Unfortunately I have no idea for a timeline on this.
In the meantime you can use the new functionality with this
#367 (comment)

Im quite confident that all the functionality has been implemented using the same style as the rest of the API.
We have at least 4 people using it on their projects without issues,
So I do think its close to being ready to go.

The next stage is for @gehrisandro and @nunomaduro to review the PR.

[gerdemann]

Two uses are missing.

[gehrisandro]

Hi @EthanBarlo

Thank you very much for your PR! I am going to review it now.

[slaven-ii]

Hi, I have been actively tracking the progress of this PR, but lately, it looks like there has been no movement after the tests were run. Is this what is blocking it or?

[Guervyl]

Hi there, I did not check the code to see the code.

My question is it cost optimize?

I'm new to open ai gpt. I don't know much about there pricing. But they say the have output cost for tokens.

The actual way this library do to fetch the response is by fetching all the messages: $messages = $client->threads()->messages()->list('...');. Does your code fetch the last message like this?

Because if the output tokens is about fetching the messages it would be bad fetching all messages again and again.

[EthanBarlo]

@Guervyl
The output tokens refers to text generations from the Ai models.
So retrieving previous messages, and old text generations does not cost anything.

[gehrisandro]

Thanks, @EthanBarlo, for your PR!

Changed some stuff to work similar to the other streaming endpoints.

Will make a new release this week.

[magarrent]

Thank you guys! Amazing work.

Please note that the current version is pointing OpenAi-beta assistants=v1, it would crash if you're using some new parameters like file_search in v2