Callbacks Refactor [base] #3256

agola11 · 2023-04-21T00:04:25Z

No description provided.

agola11 · 2023-04-23T04:47:01Z

langchain/callbacks/manager.py

+        return _configure(cls, inheritable_callbacks, local_callbacks, verbose)
+
+
+T = TypeVar("T", CallbackManager, AsyncCallbackManager)


Can think of a better way to do this if it's ugly -- basically want to enforce with typing that typing among arguments and return value must be consistent

Makes sense to me

nfcampos · 2023-04-25T08:31:00Z

docs/ecosystem/gpt4all.md

 # There are many CallbackHandlers supported, such as
 # from langchain.callbacks.streamlit import StreamlitCallbackHandler

 callback_manager = CallbackManager([StreamingStdOutCallbackHandler()])
-model = GPT4All(model="./models/gpt4all-model.bin", n_ctx=512, n_threads=8, callback_handler=callback_handler, verbose=True)
+model = GPT4All(model="./models/gpt4all-model.bin", n_ctx=512, n_threads=8, callback_handler=callback_handler,


Was this line wrong anyway? In the previous line creating a callback_manager, then passing a callback_handler var (that I think doesn't exist) to a callback_handler arg that I also think doesn't exist

oh yeah, this was messed up by the pycharm refactoring tool

nfcampos · 2023-04-25T08:32:48Z

langchain/callbacks/base.py

+        for handler in handlers:
+            self.add_handler(handler, inherit=inherit)
+
+    def set_handler(self, handler: BaseCallbackHandler, inherit=True) -> None:


is this method useful enough to exist?

nfcampos · 2023-04-25T08:35:20Z

langchain/callbacks/base.py

+        token: str,
+        run_id: Optional[str] = None,
+        parent_run_id: Optional[str] = None,
+        **kwargs: Any,


We don't have these kwargs in js, should we?

langchain/callbacks/base.py

nfcampos · 2023-04-25T12:58:33Z

langchain/callbacks/manager.py

+            )
+        else:
+            callback_manager = inheritable_callbacks
+        callback_manager = copy.deepcopy(callback_manager)


why call deep copy directly here rather than the copy method?

Indeed, I ran into this. What's the logic behind a copy here anyway? I'd assume many callback managers—definitely mine—maintain state that cannot be copied such as file handles, network connections, asyncio.Futures etc.

I don't know this code that well, but why even copy at all? It seems somewhat antithetical to the rationale of a callback manager that everything makes a copy of it.

(Not only speaking of the copying of the callback_manager but also all handlers few lines below.)

…hanism (#3381)

vowelparrot · 2023-04-26T21:32:58Z

langchain/callbacks/manager.py

@@ -75,7 +69,8 @@ def _handle_event(
            ):
                getattr(handler, event_name)(*args, **kwargs)
        except Exception as e:
-            logging.error(f"Error in {event_name} callback: {e}")
+            # TODO: switch this to use logging


I'm not sure why we have to wait?

Couple additional tools landed today

PeadarOhAodha · 2023-04-30T10:47:29Z

langchain/callbacks/manager.py

+def tracing_enabled(
+    session_name: str = "default",
+) -> Generator[TracerSession, None, None]:
+    """Get OpenAI callback handler in a context manager."""


Incorrect docstring?

Modified langchain dependency to prevent error caused by LangChain's recent callbacks refactor ([Callbacks Refactor [base] #3256](langchain-ai/langchain#3256))

Co-authored-by: Nuno Campos <nuno@boringbits.io> Co-authored-by: Davis Chase <130488702+dev2049@users.noreply.github.com> Co-authored-by: Zander Chase <130414180+vowelparrot@users.noreply.github.com> Co-authored-by: Harrison Chase <hw.chase.17@gmail.com>

pors · 2023-05-02T17:02:05Z

@agola11 This was a breaking change. Is it documented somewhere how to migrate? In my case, I did the following:

from langchain.callbacks.base import AsyncCallbackManager

and replaced it with:

from langchain.callbacks.manager import AsyncCallbackManager

but it broke my app:

  File "/Users/mark/.pyenv/versions/ve3109_fcc/lib/python3.10/site-packages/langchain/chains/base.py", line 180, in acall
    raise e
  File "/Users/mark/.pyenv/versions/ve3109_fcc/lib/python3.10/site-packages/langchain/chains/base.py", line 174, in acall
    await self._acall(inputs, run_manager=run_manager)
  File "/Users/mark/.pyenv/versions/ve3109_fcc/lib/python3.10/site-packages/langchain/chains/llm.py", line 195, in _acall
    response = await self.agenerate([inputs], run_manager=run_manager)
  File "/Users/mark/.pyenv/versions/ve3109_fcc/lib/python3.10/site-packages/langchain/chains/llm.py", line 90, in agenerate
    return await self.llm.agenerate_prompt(
  File "/Users/mark/.pyenv/versions/ve3109_fcc/lib/python3.10/site-packages/langchain/llms/base.py", line 136, in agenerate_prompt
    return await self.agenerate(prompt_strings, stop=stop, callbacks=callbacks)
  File "/Users/mark/.pyenv/versions/ve3109_fcc/lib/python3.10/site-packages/langchain/llms/base.py", line 217, in agenerate
    callback_manager = AsyncCallbackManager.configure(
  File "/Users/mark/.pyenv/versions/ve3109_fcc/lib/python3.10/site-packages/langchain/callbacks/manager.py", line 667, in configure
    return _configure(cls, inheritable_callbacks, local_callbacks, verbose)
  File "/Users/mark/.pyenv/versions/ve3109_fcc/lib/python3.10/site-packages/langchain/callbacks/manager.py", line 701, in _configure
    callback_manager.add_handler(copy.deepcopy(handler), False)
  File "/Users/mark/.pyenv/versions/3.10.9/lib/python3.10/copy.py", line 172, in deepcopy
    y = _reconstruct(x, memo, *rv)
  File "/Users/mark/.pyenv/versions/3.10.9/lib/python3.10/copy.py", line 271, in _reconstruct
    state = deepcopy(state, memo)
  File "/Users/mark/.pyenv/versions/3.10.9/lib/python3.10/copy.py", line 146, in deepcopy
    y = copier(x, memo)
  File "/Users/mark/.pyenv/versions/3.10.9/lib/python3.10/copy.py", line 231, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/Users/mark/.pyenv/versions/3.10.9/lib/python3.10/copy.py", line 172, in deepcopy
    y = _reconstruct(x, memo, *rv)
  File "/Users/mark/.pyenv/versions/3.10.9/lib/python3.10/copy.py", line 271, in _reconstruct
    state = deepcopy(state, memo)
  File "/Users/mark/.pyenv/versions/3.10.9/lib/python3.10/copy.py", line 146, in deepcopy
    y = copier(x, memo)
  File "/Users/mark/.pyenv/versions/3.10.9/lib/python3.10/copy.py", line 231, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/Users/mark/.pyenv/versions/3.10.9/lib/python3.10/copy.py", line 146, in deepcopy
    y = copier(x, memo)
  File "/Users/mark/.pyenv/versions/3.10.9/lib/python3.10/copy.py", line 231, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/Users/mark/.pyenv/versions/3.10.9/lib/python3.10/copy.py", line 172, in deepcopy
    y = _reconstruct(x, memo, *rv)
  File "/Users/mark/.pyenv/versions/3.10.9/lib/python3.10/copy.py", line 271, in _reconstruct
    state = deepcopy(state, memo)
  File "/Users/mark/.pyenv/versions/3.10.9/lib/python3.10/copy.py", line 146, in deepcopy
    y = copier(x, memo)
  File "/Users/mark/.pyenv/versions/3.10.9/lib/python3.10/copy.py", line 231, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/Users/mark/.pyenv/versions/3.10.9/lib/python3.10/copy.py", line 172, in deepcopy
    y = _reconstruct(x, memo, *rv)
  File "/Users/mark/.pyenv/versions/3.10.9/lib/python3.10/copy.py", line 272, in _reconstruct
    if hasattr(y, '__setstate__'):
  File "/Users/mark/.pyenv/versions/ve3109_fcc/lib/python3.10/site-packages/starlette/datastructures.py", line 702, in __getattr__
    return self._state[key]
  File "/Users/mark/.pyenv/versions/ve3109_fcc/lib/python3.10/site-packages/starlette/datastructures.py", line 702, in __getattr__
    return self._state[key]
  File "/Users/mark/.pyenv/versions/ve3109_fcc/lib/python3.10/site-packages/starlette/datastructures.py", line 702, in __getattr__
    return self._state[key]
  [Previous line repeated 949 more times]
RecursionError: maximum recursion depth exceeded

Help 😁

agola11 · 2023-05-02T17:48:42Z

@pors -- can you send me a snipped of how you were using AsyncCallbackManager and what handlers you were using along with it? I'll be able to help you more

pors · 2023-05-02T17:58:35Z

@agola11 The whole project is here: https://github.com/pors/langchain-chat-websockets/blob/main/query_data.py

The relevant code is largely based on https://github.com/hwchase17/chat-langchain, so the issue may also be there.

agola11 · 2023-05-02T17:59:43Z

@pors okay thanks. Looking in a bit

agola11 · 2023-05-02T19:55:50Z

Once this lands, your issue should be resolved: #3995 -- please comment if that's not the case

pors · 2023-05-03T06:04:54Z

Thanks! It works again for me. I like the fast release schedule :)

agola11 added 2 commits April 20, 2023 16:58

callbacks changes

3cc2ce6

Merge branch 'master' into ankush/callbacks-refactor

55c7964

agola11 changed the title ~~callbacks refactor [1/n~~ callbacks refactor [1/n] Apr 21, 2023

cr

fa4a4f2

vowelparrot approved these changes Apr 21, 2023

View reviewed changes

agola11 changed the title ~~callbacks refactor [1/n]~~ Callbacks Refactor [1/n]: non abstract and fix repeated logic Apr 22, 2023

Callbacks Refactor [2/n]: refactor CallbackManager code to own file (…

675e27c

…#3341)

agola11 changed the title ~~Callbacks Refactor [1/n]: non abstract and fix repeated logic~~ Callbacks Refactor [1/n]: add callbacks mixins, Apr 23, 2023

agola11 commented Apr 23, 2023

View reviewed changes

cr

90cef7b

nfcampos reviewed Apr 25, 2023

View reviewed changes

Callbacks Refactor [2/n] update tracer to work with new callbacks mec…

4cdd19b

…hanism (#3381)

agola11 changed the title ~~Callbacks Refactor [1/n]: add callbacks mixins,~~ Callbacks Refactor [base]: add callbacks mixins, Apr 26, 2023

agola11 changed the title ~~Callbacks Refactor [base]: add callbacks mixins,~~ Callbacks Refactor [base] Apr 26, 2023

agola11 added 2 commits April 26, 2023 11:34

fix notebook and warnings

7bcdc66

write to different session

6fec15b

vowelparrot reviewed Apr 26, 2023

View reviewed changes

agola11 added 9 commits April 26, 2023 19:19

fix execution order issue

5066869

mypy

e953d2c

cr

6cd653d

mypy

8ae809a

mypy

1fc3941

cr

15c0fa5

fix llm chain

5dcb44e

fix most tests

da27d87

fix baby agi

2ed4649

sliedes mentioned this pull request Apr 28, 2023

Pandas agent: Pass callback manager to Python tool. #3646

Merged

nfcampos and others added 3 commits April 28, 2023 10:37

Nc/callbacks docs (#3717)

0e81e83

merge

eb9de30

cr

1b48ea8

vowelparrot added 2 commits April 28, 2023 18:36

Call Manager for New Tools (#3755)

20ba888

Couple additional tools landed today

Resolve merge conflicts

a038f37

vowelparrot force-pushed the ankush/callbacks-refactor branch from 03e8023 to a038f37 Compare April 29, 2023 01:55

vowelparrot and others added 6 commits April 28, 2023 19:00

Notebook Nits

9192abc

merge

35cc38f

fix docs

ebc6242

use UUID

737467a

bw compat environ variable

3839703

fix openai callback

fa1742c

PeadarOhAodha reviewed Apr 30, 2023

View reviewed changes

cr

fb78f69

agola11 merged commit d3ec00b into master Apr 30, 2023
9 checks passed

agola11 deleted the ankush/callbacks-refactor branch April 30, 2023 18:14

lightshifted added a commit to Hedronstone/agent-actors that referenced this pull request Apr 30, 2023

Update pyproject.toml

67ec783

Modified langchain dependency to prevent error caused by LangChain's recent callbacks refactor ([Callbacks Refactor [base] #3256](langchain-ai/langchain#3256))

lightshifted mentioned this pull request Apr 30, 2023

Update pyproject.toml shaman-ai/agent-actors#5

Merged

tjaffri mentioned this pull request May 1, 2023

feat: Improvements to SQLDatabaseChain for better local model support (query checker and few shot examples) #3636

Merged

sliedes mentioned this pull request May 2, 2023

Callback manager: Do not (deep)copy handler #3973

Merged

pors mentioned this pull request May 2, 2023

feat: add websocket support ajndkr/lanarky#11

Closed

tssweeney mentioned this pull request May 4, 2023

fix(integrations): update WandbTracer for new langchain release wandb/wandb#5467

Merged

1 task

shreyabhadwal mentioned this pull request May 6, 2023

Callback Manager #3966

Closed

h3ndrik mentioned this pull request May 9, 2023

Langchain isn't verbose any more [run_manager gets dropped] #4329

Closed

14 tasks

mariokostelac mentioned this pull request Jun 14, 2023

Run event handler inline if it's not an async function #6095

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Callbacks Refactor [base] #3256

Callbacks Refactor [base] #3256

agola11 commented Apr 21, 2023 •

edited

agola11 Apr 23, 2023

vowelparrot Apr 24, 2023

nfcampos Apr 25, 2023

agola11 Apr 26, 2023

nfcampos Apr 25, 2023

nfcampos Apr 25, 2023

nfcampos Apr 25, 2023

sliedes May 2, 2023 •

edited

vowelparrot Apr 26, 2023

PeadarOhAodha Apr 30, 2023

pors commented May 2, 2023

agola11 commented May 2, 2023

pors commented May 2, 2023

agola11 commented May 2, 2023

agola11 commented May 2, 2023

pors commented May 3, 2023

		return _configure(cls, inheritable_callbacks, local_callbacks, verbose)


		T = TypeVar("T", CallbackManager, AsyncCallbackManager)

Callbacks Refactor [base] #3256

Callbacks Refactor [base] #3256

Conversation

agola11 commented Apr 21, 2023 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sliedes May 2, 2023 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pors commented May 2, 2023

agola11 commented May 2, 2023

pors commented May 2, 2023

agola11 commented May 2, 2023

agola11 commented May 2, 2023

pors commented May 3, 2023

agola11 commented Apr 21, 2023 •

edited

sliedes May 2, 2023 •

edited