Invalid Output Parser Format for "Router Chain"

[anapple00]

System Info

langchain version: 0.0.170
python: 3.8

Who can help?

No response

Information

• The official example notebooks/scripts
• My own modified scripts

Related Components

• LLMs/Chat Models
• Embedding Models
• Prompts / Prompt Templates / Prompt Selectors
• Output Parsers
• Document Loaders
• Vector Stores / Retrievers
• Memory
• Agents / Agent Executors
• Tools / Toolkits
• Chains
• Callbacks/Tracing
• Async

Reproduction

Here I came across an issue related to the output of router chain.
When I ran the tutorial of "router chain" in langchain website, the input query is: "What is black body radiation?" and the output of LLM is:

'{
    "destination": "physics",
    "next_inputs": "What is black body radiation?"
}'

Use the class RouterOutputParser to parse the output then I got the error:

{OutputParserException}Got invalid return object. Expected markdown code snippet with JSON object, but got:
{
"destination": "physics",
"next_inputs": "What is black body radiation?"
}

When I debug step by step I found the error raised in this function: parse_json_markdown

def parse_json_markdown(text: str, expected_keys: List[str]) -> Any:
    if "```json" not in text:
        raise OutputParserException(
            f"Got invalid return object. Expected markdown code snippet with JSON "
            f"object, but got:\n{text}"
        )

    json_string = text.split("```json")[1].strip().strip("```").strip()
    try:
        json_obj = json.loads(json_string)
    except json.JSONDecodeError as e:
        raise OutputParserException(f"Got invalid JSON object. Error: {e}")
    for key in expected_keys:
        if key not in json_obj:
            raise OutputParserException(
                f"Got invalid return object. Expected key `{key}` "
                f"to be present, but got {json_obj}"
            )
    return json_obj

You can see there is no "```json" string in the output of LLM, so it will step into the "if" in the first row of this function and raise the bug.

Expected behavior

Can anyone give me some solutions? thanks.

[4kk11]

I have the same problem.
When I use "text-davinci-003" it works, but with "gpt-3.5-turbo" I get an error.

[jan-schaeffer]

Although parse_json_markdown seems to have been reworked since 0.0.170 (I'm on 0.0.191) I still get the same error:

{
 "destination": "physics",
 "next_inputs": "What is black body radiation?"
}

A: JavaScript (ES6),  72  72 bytes
Saved 1 byte thanks to @Neil
Saved 1 byte thanks to @Neil
S

Does anyone have any idea whats going wrong?

[shubham184]

+1

[simicvm]

I have the same problem.
When I use "text-davinci-003" it works, but with "gpt-3.5-turbo" I get an error.

I can confirm that this is still an issue. However, if you use gpt-4-0613, the system works fine. It might be that gpt-3.5-turbo model just doesn't follow the template well and does not output "```json"

[vowelparrot]

gpt-3.5-turbo is optimized for chat rather than schema generation, so it is less reliable. Recommend trying out the openai functions agent or use a different model.

[maifeng]

To get through the tutorial, I had to create a new class:

import json
import langchain
from typing import Any, Dict, List, Optional, Type, cast

class RouterOutputParser_simple(langchain.schema.BaseOutputParser[Dict[str, str]]):
    """Parser for output of router chain int he multi-prompt chain."""

    default_destination: str = "DEFAULT"
    next_inputs_type: Type = str
    next_inputs_inner_key: str = "input"

    def parse(self, text: str) -> Dict[str, Any]:
        try:
            expected_keys = ["destination", "next_inputs"]
            parsed = json.loads(text) ### this line is changed
            if not isinstance(parsed["destination"], str):
                raise ValueError("Expected 'destination' to be a string.")
            if not isinstance(parsed["next_inputs"], self.next_inputs_type):
                raise ValueError(
                    f"Expected 'next_inputs' to be {self.next_inputs_type}."
                )
            parsed["next_inputs"] = {self.next_inputs_inner_key: parsed["next_inputs"]}
            if (
                parsed["destination"].strip().lower()
                == self.default_destination.lower()
            ):
                parsed["destination"] = None
            else:
                parsed["destination"] = parsed["destination"].strip()
            return parsed
        except Exception as e:
            raise OutputParserException(
                f"Parsing text\n{text}\n raised following error:\n{e}"
            )

Then use this class as the output_parser

router_template = MULTI_PROMPT_ROUTER_TEMPLATE.format(
    destinations=destinations_str
)
router_prompt = PromptTemplate(
    template=router_template,
    input_variables=["input"],
    output_parser=RouterOutputParser_simple(), ### replaced RouterOutputParser()
)

router_chain = LLMRouterChain.from_llm(llm, router_prompt)

[LifeBringer]

To get through the tutorial, I had to create a new class:

import json
import langchain
from typing import Any, Dict, List, Optional, Type, cast

class RouterOutputParser_simple(langchain.schema.BaseOutputParser[Dict[str, str]]):
    """Parser for output of router chain int he multi-prompt chain."""

    default_destination: str = "DEFAULT"
    next_inputs_type: Type = str
    next_inputs_inner_key: str = "input"

    def parse(self, text: str) -> Dict[str, Any]:
        try:
            expected_keys = ["destination", "next_inputs"]
            parsed = json.loads(text) ### this line is changed
            if not isinstance(parsed["destination"], str):
                raise ValueError("Expected 'destination' to be a string.")
            if not isinstance(parsed["next_inputs"], self.next_inputs_type):
                raise ValueError(
                    f"Expected 'next_inputs' to be {self.next_inputs_type}."
                )
            parsed["next_inputs"] = {self.next_inputs_inner_key: parsed["next_inputs"]}
            if (
                parsed["destination"].strip().lower()
                == self.default_destination.lower()
            ):
                parsed["destination"] = None
            else:
                parsed["destination"] = parsed["destination"].strip()
            return parsed
        except Exception as e:
            raise OutputParserException(
                f"Parsing text\n{text}\n raised following error:\n{e}"
            )

Then use this class as the output_parser

router_template = MULTI_PROMPT_ROUTER_TEMPLATE.format(
    destinations=destinations_str
)
router_prompt = PromptTemplate(
    template=router_template,
    input_variables=["input"],
    output_parser=RouterOutputParser_simple(), ### replaced RouterOutputParser()
)

router_chain = LLMRouterChain.from_llm(llm, router_prompt)

I get an 'OutputParserException' not defined error when running: chain.run("what is 2 + 2")

[LyndonZhao]

I met the same problem.
And I wonder why this problem has not been resolved after more than one month since it occours.

[LyndonZhao]

I resolved the problem by adding an example at the end of the MULTI_PROMPT_ROUTER_TEMPLATE.
Please check it as below.

eg:

<< INPUT >>
"What is black body radiation?"
<< OUTPUT >>
```json
{{{{
"destination": "physics",
"next_inputs": "What is black body radiation?"
}}}}
```

[aaalexlit]

@LyndonZhao, Thanks!
The solution worked for me

[LyndonZhao]

@aaalexlit Welcome!

[Nghiauet]

It fixed in version langchain 0.0.222. Just upgraded langchain and it work
pip install --upgrade langchain

[Boopalanoptisol]

eg:

<< INPUT >> "What is black body radiation?" << OUTPUT >> ```json {{{{ "destination": "physics", "next_inputs": "What is black body radiation?" }}}}

Am getting the below mentioned error.
OutputParserException: Parsing text json { "destination": "response_format", "next_inputs": "What is backbody radiation?" } raised following error: Expecting value: line 1 column 1 (char 0)

[Boopalanoptisol]

I resolved the problem by adding an example at the end of the MULTI_PROMPT_ROUTER_TEMPLATE. Please check it as below.

eg:

<< INPUT >> "What is black body radiation?" << OUTPUT >> json {{{{ "destination": "physics", "next_inputs": "What is black body radiation?" }}}}
Hi @LyndonZhao

When iadded the above example

Also getting the error as
OutputParserException: Parsing text raised following error: Expecting value: line 1 column 1 (char 0)

[SimasJan]

I resolved the problem by adding an example at the end of the MULTI_PROMPT_ROUTER_TEMPLATE. Please check it as below.

eg:

<< INPUT >> "What is black body radiation?" << OUTPUT >> json {{{{ "destination": "physics", "next_inputs": "What is black body radiation?" }}}}

In addition, to suggested solution, found out that the following worked just as well.

<< OUTPUT (must include ```json at the start of the response) >>

[hueiyuan]

@SimasJan
After adjusted based on your suggestion, we found the problem still have occurred in MultiRetrievalQAChain.
Whether other solution can solve?

• Related lib version

python: 3.9
Langchain: 0.0249
Azure Open AI: gpt-3.5-turbo

[bilalahmadai]

I resolved the problem by adding an example at the end of the MULTI_PROMPT_ROUTER_TEMPLATE. Please check it below.

eg:

<< INPUT >> "What is black body radiation?" << OUTPUT >> json {{{{ "destination": "physics", "next_inputs": "What is black body radiation?" }}}}

THIS HELPS FOR ME

[mpduarte]

@hueiyuan
Having the same issue as you, did you find a way to fix this?

[hueiyuan]

@mpduarte
I do not fix this, still have this issue..

[lucarp]

This solution worked for me:

UPDATED_MULTI_ROUTER_TEMPLATE = MULTI_PROMPT_ROUTER_TEMPLATE + '\n<< OUTPUT (must end with ```) >>\n'

Then I used UPDATED_MULTI_ROUTER_TEMPLATE instead.

[giftednewbie]

I resolved the problem by adding an example at the end of the MULTI_PROMPT_ROUTER_TEMPLATE. Please check it as below.

eg:

<< INPUT >> "What is black body radiation?" << OUTPUT >> json {{{{ "destination": "physics", "next_inputs": "What is black body radiation?" }}}}

In addition, to suggested solution, found out that the following worked just as well.

<< OUTPUT (must include ```json at the start of the response) >>

Combining the above answer and the answer from @lucarp, helped in my case. Updating the << OUTPUT >> on the MULTI_PROMPT_ROUTER_TEMPLATE as below worked:

<< OUTPUT (must include ```json at the start of the response and must end with ```) >>

[dosubot]

Hi, @anapple00,

I'm helping the LangChain team manage their backlog and am marking this issue as stale. From what I understand, the issue involves an error in the output parser format for the "Router Chain" in the langchain system. It seems that the issue has been resolved in version langchain 0.0.222, and users are advised to upgrade their langchain version to resolve the problem.

Could you please confirm if this issue is still relevant to the latest version of the LangChain repository? If it is, please let the LangChain team know by commenting on the issue. Otherwise, feel free to close the issue yourself, or the issue will be automatically closed in 7 days. Thank you!