core[minor]: Add v2 implementation of astream events

[eyurtsev]

This PR introduces a v2 implementation of astream events that removes intermediate abstractions and fixes some issues with v1 implementation.

The v2 implementation significantly reduces relevant code that's associated with the astream events implementation together with overhead.

After this PR, the astream events implementation:

• Uses an async callback handler
• No longer relies on BaseTracer
• No longer relies on json patch

As a result of this re-write, a number of issues were discovered with the existing implementation.

Changes in V2 vs. V1

on_chat_model_end output

The outputs associated with on_chat_model_end changed depending on whether it was within a chain or not.

As a root level runnable the output was:

"data": {"output": AIMessageChunk(content="hello world!", id='some id')}

As part of a chain the output was:

            "data": {
                "output": {
                    "generations": [
                        [
                            {
                                "generation_info": None,
                                "message": AIMessageChunk(
                                    content="hello world!", id=AnyStr()
                                ),
                                "text": "hello world!",
                                "type": "ChatGenerationChunk",
                            }
                        ]
                    ],
                    "llm_output": None,
                }
            },

After this PR, we will always use the simpler representation:

"data": {"output": AIMessageChunk(content="hello world!", id='some id')}

NOTE Non chat models (i.e., regular LLMs) are still associated with the more verbose format.

Remove some _stream events

on_retriever_stream and on_tool_stream events were removed -- these were not real events, but created as an artifact of implementing on top of astream_log.

The same information is already available in the x_on_end events.

Propagating Names

Names of runnables have been updated to be more consistent

  model = GenericFakeChatModel(messages=infinite_cycle).configurable_fields(
        messages=ConfigurableField(
            id="messages",
            name="Messages",
            description="Messages return by the LLM",
        )
    )

Before:

"name": "RunnableConfigurableFields",

After:

"name": "GenericFakeChatModel",

on_retriever_end

on_retriever_end will always return output which is a list of documents (rather than a dict containing a key called "documents")

Retry events

Removed the on_retry callback handler. It was incorrectly showing that the failed function being retried has invoked on_chain_end

https://github.com/langchain-ai/langchain/pull/21638/files#diff-e512e3f84daf23029ebcceb11460f1c82056314653673e450a5831147d8cb84dL1394

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Ignored Deployment

Name	Status	Preview	Comments	Updated (UTC)
langchain	⬜️ Ignored (Inspect)	Visit Preview		May 15, 2024 2:53pm

[nfcampos]

this comment shouldn't be here?

[jacoblee93]

Typo

[eyurtsev]

@jacoblee93 / @nfcampos

on_llm_end still has a very verbose output.

Options are:

1. Keep as is
2. output only the text (but makes it impossible to pluck out metadata)
3. output only the text, and start feeding metadata into a separate key in data
4. do a hack and start outputting AIMessageChunk

[jacoblee93]

@jacoblee93 / @nfcampos

on_llm_end still has a very verbose output.

Options are:

1. Keep as is
2. output only the text (but makes it impossible to pluck out metadata)
3. output only the text, and start feeding metadata into a separate key in data
4. do a hack and start outputting AIMessageChunk

I'd opt for making as few transformations as possible, which I think would be 1 or 4?

[eyurtsev]

We can keep as (1) for now until we decide what to do. (4) Might be a bad idea since we don't use AIMessage in regular LLMs currently.