Issue: Unable to define "output_key" param of the LLM chain class for a csv agent while binding to SequentialChain class

[abhishekperambai]

Issue you'd like to raise.

Currently, I'm using SequentialChain class to combine two steps in my workflow.

Step1: I'm using the LLM through prompt to identify the intent of the question posed by the user.

Step2: I'm using the csv based agent to answer the question posed by the user based on the csv file, but my aim is to answer the question only if the intent of the question is a textual response.

Below is the code snippets I have used to create the SequentialChain

model = AzureOpenAI(temperature=0,deployment_name="",openai_api_key="",openai_api_version="",openai_api_base="", )

template = """ You will help me identify the intent with the following examples and instructions. Give your response in this format {{"Intent":"<identified intent>", "Question":"<Input question>"}} ### Instructions # Different possible intents are textResponse, BarChart, LineChart. # If the question doesn't come under any of the intents, identify it as a None intent. #### ### Examples Question: What is the total count of stores in 2022? Intent: textResponse Question: What is the split of sale amount for each sale type? Intent: BarChart Question: What is the monthly trend of sales amount in 2022? Intent: LineChart Question: {input} """

prompt = PromptTemplate( input_variables=["input"], template=template, )

chain_one = LLMChain(llm=model, prompt=prompt, output_key = "intent")

agent = create_csv_agent( AzureOpenAI(temperature=0.5,top_p = 0.5,deployment_name="",openai_api_key="",openai_api_version="",openai_api_base="",), <csv file path>, verbose=True, agent_type=AgentType.ZERO_SHOT_REACT_DESCRIPTION, handle_parsing_errors=True )

agent.agent.llm_chain.prompt.template = """ You are a friendly assistant who is warm and thoughtful and never rude or curt. Possible Intents are: textResponse, LineChart, BarChart, None You will act further only if the {intent} is textResponse, else your Final Answer will be I cannot respond to your query. If {intent} is textResponse use the python_repl_ast to answer the question. You should use the tools below to answer the question posed of you: python_repl_ast: A Python shell. Use this to execute python commands. You should use the python_repl_ast to answer the question posed of you. You are working with a pandas dataframe in Python. The name of the dataframe is df. Input to python_repl_ast should be a valid python command. Give your Final Answer in this format {{"output":"Final Answer"}} Use the following format: Question: the input question you must answer Thought: you should always think about what to do. Action: the action to take, should be one of [python_repl_ast] Action Input: the input to the action Observation: the result of the action ... (this Thought/Action/Action Input/Observation can repeat N times) Thought: I now know the final answer Final Answer: the final answer to the original input question. This is the result of print(df.head()):{df_head} Begin! Question: {input} {agent_scratchpad} """

input_var = agent.agent.llm_chain.prompt.input_variables

input_var.append("intent")

This was done to append my input variables to already pre-defined ones for the csv agent.

agent.agent.llm_chain.output_key = "FinalAnswer"

chain_two = agent

overall_chain = SequentialChain(chains=[chain_one, chain_two],input_variables=["input"], output_variables=["intent","FinalAnswer"],verbose=True)

overall_chain.run(input = "count of total stores in 2022")

Now, when I run the above code I get the following error:

validation error for SequentialChain root Expected output variables that were not found: {'FinalAnswer'}. (type=value_error)

As far as I understood the langchain documentation [(https://python.langchain.com/docs/modules/chains/foundational/sequential_chains)]

the output_key must be defined for each LLM hit for the model to tag the response to that key, hence here I have provided the output key to the agent through the llm_chain.output_key property. But still the code throws error that output variables were not found.

Is this a bug in langchain while binding the csv agents to SequentialChain class or am I missing something? Can someone please help?

Suggestion:

No response

[dosubot]

Answer generated by a 🤖

Answer

Hello,

From your code and the error message, it seems like the issue is with the output key of the second chain in your SequentialChain. The error message indicates that the SequentialChain is expecting an output variable 'FinalAnswer' which it is not able to find.

In your code, you have set the output key of the first chain (LLMChain) as 'intent' and the output key of the second chain (CSV agent) as 'FinalAnswer'. However, the output key of the CSV agent should be set in the prompt template of the CSV agent, not by setting the llm_chain.output_key property.

The output key of the CSV agent should be the key in the final dictionary that the CSV agent returns. In your case, it should be 'output' as per the prompt template of the CSV agent:

You should use the python_repl_ast to answer the question posed of you. You are working with a pandas dataframe in Python.
The name of the dataframe is `df`.
Input to python_repl_ast should be a valid python command.
Give your Final Answer in this format {{"output":"Final Answer"}}

So, you should set the output key of the CSV agent as 'output' and also include 'output' in the output variables of the SequentialChain:

overall_chain = SequentialChain(chains=[chain_one, chain_two],input_variables=["input"],
                  output_variables=["intent","output"],verbose=True)

Please try this and let me know if it resolves your issue. If not, I would be happy to assist you further.

Sources

• libs/langchain/langchain/chains/sequential.py

---

This response is meant to be useful, save you time, and share context. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

[abhishekperambai]

Answer generated by a 🤖

Answer

Hello,

From your code and the error message, it seems like the issue is with the output key of the second chain in your SequentialChain. The error message indicates that the SequentialChain is expecting an output variable 'FinalAnswer' which it is not able to find.

In your code, you have set the output key of the first chain (LLMChain) as 'intent' and the output key of the second chain (CSV agent) as 'FinalAnswer'. However, the output key of the CSV agent should be set in the prompt template of the CSV agent, not by setting the llm_chain.output_key property.

The output key of the CSV agent should be the key in the final dictionary that the CSV agent returns. In your case, it should be 'output' as per the prompt template of the CSV agent:

You should use the python_repl_ast to answer the question posed of you. You are working with a pandas dataframe in Python.
The name of the dataframe is `df`.
Input to python_repl_ast should be a valid python command.
Give your Final Answer in this format {{"output":"Final Answer"}}

So, you should set the output key of the CSV agent as 'output' and also include 'output' in the output variables of the SequentialChain:

overall_chain = SequentialChain(chains=[chain_one, chain_two],input_variables=["input"],
                  output_variables=["intent","output"],verbose=True)

Please try this and let me know if it resolves your issue. If not, I would be happy to assist you further.

Sources

• libs/langchain/langchain/chains/sequential.py

This response is meant to be useful, save you time, and share context. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

Thank you this helped.