New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
deepcopy is extremely slow. Can we define a copy function? #21001
Comments
@Huarong could you provide benchmarking results that provide that this is an actual bottleneck? Deepcopy may be slow, but it's a bit surprising that this would be a bottleneck at all given the latency of LLMs. Are you using this with astream_log or astream_events API? |
@eyurtsev I'm using with I'm using it in langgraph whose chunk will contain a So, multiple times of calling I will provide a benchmark code later. |
@Huarong great! We're likely going to re-work how the data for astream events is piped through to remove the dependency on json patch entirely. That should resolve your issue if you're relying on the astream_events API. Nonetheless, if you are able to benchmark and show that it takes a substantial portion of the run time, let us know -- this will be good for us to know |
@eyurtsev The code to benchmark.
Profiling by pyinstrument: pyinstrument test_basic_chatbot.py
|
Checked other resources
Example Code
No particular code.
Error Message and Stack Trace (if applicable)
No response
Description
copy.deepcopy is known to be extremely slow. When we create complex agents in langgraph, it slows things down a lot.
A solution is to define a function to copy only what necessary.
langchain/libs/core/langchain_core/tracers/log_stream.py
Line 105 in 804390b
langchain/libs/core/langchain_core/tracers/log_stream.py
Line 590 in 804390b
System Info
System Information
Package Information
Packages not installed (Not Necessarily a Problem)
The following packages were not found:
The text was updated successfully, but these errors were encountered: