Fully support OpenAI Assistant v2 and add better error handling

[NawarA]

Checked

• I searched existing ideas and did not find a similar one
• I added a very descriptive title
• I've clearly described the feature request and motivation for it

Feature request

Hi, love the langchain library, but it seems to be lacking support for OpenAI's newest Assistant capabilities, which makes it hard to use langchain.

For example, the OpenAI Threads API now supports streaming, but when I try to stream using the existing implementation, I just get the fully baked result back (its not actually a stream of tokens, which is my mental model of a stream).
`

  const readableStream = await agentExecutor.stream({ content, threadId });

Also, using if something fails, like perhaps the schema is rejected, OpenAI Assistant API complains that a run is active and OpenAI wont accept anymore requests from the source code, until a developer manually cancels a run or waits a non-developer set time period before the run expires.

A better approach here is either if agentExecutor crashes or rejects, it the OpenAIAssistantRunnable wrapper auto cancels the crashed run. Alternatively, the worse, is if running agentExecutor.invoke returned the runId allowing a developer to try catch and cancel the job themselves. Without this level of error handling, a dev has to wait for a failure, and then find the last run thats still active and manually cancel it. DX is much better in the scenario I'm suggesting.

  const assistantResponse = await agentExecutor.invoke({ content, threadId });

Finally, OpenAI assistant supports "truncationStrategy" and a number of other params that are hidden or not settable by developers. Though in practice, setting a truncation strategy is super important, as are the other options that devs are supported to have access to.

I'd love if langchain could fully support the 3 features above, as OpenAI Assistant + LangChain seem like an indispensible DX, if done right.

Review:

1. Making Streaming actually stream tokens, now that OpenAI Assistant supports it
2. Better error handling crashed runs (either pass back the runID or auto cancel crashed runs).
3. Allow a dev to pass any param directly to the run, such as truncation strategy

Motivation

I'm using a combo of OpenAI + LangChain, and I'm at a fork in the road, because I need streaming and better error handling. Do I stop using agentExecutor and switch to OpenAI Assistant V2 API directly or can I fully rely langchain for these basics?

I'd like to rely on LangChain, so I wrote the ticket...and I'm sure i'm not the only one wishing for these features to be supported, as I think OpenAI will double down on its assistant API

Proposal (If applicable)

See above

[mbonet]

@NawarA Slowly but steady is getting there https://github.com/langchain-ai/langchain/pull/21484/files

[darrylfoong]

@mbonet do you happen to know if it will allow us to use an existing assistant by passing in the assistant ID instead of just creating a new assistant?

[mbonet]

@darrylfoong few updates:

1. PR is ready and released
2. The documentation is not stellar so keep calm 😋

How does it work now:

1. You need to import from langchain_community

from langchain_community.agents.openai_assistant import OpenAIAssistantV2Runnable 

2. When you don't want to create a assistant everytime - which creating a assistant everytime, generally doesn't make sense), you instantiate like this:

  agent = OpenAIAssistantV2Runnable(
     assistant_id="<your_assistant_id>",
     as_agent=True
)

[mbonet]

@darrylfoong another problem I have found is that you need to have langchain_community 0.2.1 or superior, but due to other prior installations it was not getting updated/installed.

Always verify with

pip show langchain_community 

You should see

Name: langchain-community
Version: 0.2.1
Summary: Community contributed LangChain integrations.
Home-page: https://github.com/langchain-ai/langchain
Author: 
Author-email: 
License: MIT
Location: /usr/local/lib/python3.10/dist-packages
Requires: aiohttp, dataclasses-json, langchain, langchain-core, langsmith, numpy, PyYAML, requests, SQLAlchemy, tenacity ```