Langchain streaming

[tomlynchRNA]

Hello,

I'd like to use astream_events from langchain so that I can receive a stream of events as the crew runs, howeverr it's a bit tricky because crewAI isn't async.

As a workaround, I open a thread instead to run astream_events, and then push the accumulated result to a shared queue inside Agent.execute_task but it's a bit of a hack and not sure its PR worthy.

Is there a plan to implement streaming support i.e a way to pass a calback that gets the stream events, or do you have a preferred way to do this? Happy to write code+PR as I really need this in crewAI.

Cheers.

[He-Huang]

Hi @tomlynchRNA , thanks for sharing your concern. I am new to crewAI. Actually I am curious why you are using crewAI instead of langchain. There is no offense to crewAI team. But do you find any special feature that attracts you in crewAI.

[tomlynchRNA]

@He-Huang CrewAI provides some structure, prompt engineering, etc. It works, provides utility on top of langchain and I like it, so I use it.

Have you ever used a "wrapper" before? You can wipe of the veneer of politeness with your "no offence" negging. If you are invested in langchain team, that's fine. But you should go revise their code of conduct before you send your next shill comment.

[He-Huang]

@tomlynchRNA Thank you for reply.
But don't get me wrong. It's just a pure technical discussion and there is no tendency to either langchain or crewai. Given both langchain and crewai are both open-sourced and nonprofitable project for AI community, there is no motivation to degrade either one.

[remichu-ai]

Langchain is too confusing for me and lately they doesnt seems to care about open model much. Most of their examples and docs are with other paid services. This is not just me, you can google around and see many ppl having similar comment. I am no mean disregard them, they were kinda the pioneer in this area and their support of streaming is still prob the best at the moment.

[joaomdmoura]

Quick question, what are the events you would like to get back? Just full back streamign of content?

[indievish]

Would be good to at least stream back to final answer so the UX can be better - not sure if this is already possible?

[tomlynchRNA]

@joaomdmoura Currently I'm using:

• on_chat_model_stream
• on_llm_end
• on_chain_start
• on_chain_end
• on_tool_start
• on_tool_end

Ultimately, I want to stream the chunks so that I don't have to wait for complete responses before being able to send the full message in step_callback.

I take each chunk and push them to a queue for processing (goes to a separate UI). The other events like tool start/tool end are so I know definitively when a tool has started/finished being used. The events I need could change, so I would like to just get all events, and if necessary allow to pass a filter to ignore unwanted events: https://python.langchain.com/docs/expression_language/streaming/#filtering-events

[ragyabraham]

@joaomdmoura, any update on this issue???

[BahaaEldin0]

I would love to know if there is any updates in this part.
Streaming is very important for a lot of uses in production when your agents are interacting with the user directly in a real time chat environment.

[github-actions]

This issue is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.

[github-actions]

This issue was closed because it has been stalled for 5 days with no activity.

[Sarangk90]

This is important for us as well in production environment

[imkartikahuja]

This is important for us as well. Any update on this??