ConversationalRetrievalChain doesn't work along with memory and return_source_documents

[fcalabrow]

Issue you'd like to raise.

I'm trying to use a ConversationalRetrievalChain along with a ConversationBufferMemory and return_source_documents set to True. The problem is that, under this setting, I get an error when I call the overall chain.

from langchain.chains import ConversationalRetrievalChain

chain = ConversationalRetrievalChain(
    retriever=retriever,
    question_generator=question_generator,
    combine_docs_chain=doc_chain,
    memory = memory,
    return_source_documents=True,
)

query = 'dummy query'
chain({"question": query})

The error message says:

File [~/.local/share/virtualenvs/qa-8mHXn5ez/lib/python3.10/site-packages/langchain/chains/base.py:354](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/my_user/~/.local/share/virtualenvs/env-8mHXn5ez/lib/python3.10/site-packages/langchain/chains/base.py:354), in Chain.prep_outputs(self, inputs, outputs, return_only_outputs)
    352 self._validate_outputs(outputs)
    353 if self.memory is not None:
--> 354     self.memory.save_context(inputs, outputs)
...
---> 28         raise ValueError(f"One output key expected, got {outputs.keys()}")
     29     output_key = list(outputs.keys())[0]
     30 else:

ValueError: One output key expected, got dict_keys(['answer', 'source_documents'])

It works if I remove either the memory or the return_source_documents parameter.

So far, the only workaround that I found out is querying the chain using an external chat history, like this:

chain({"question": query, "chat_history":"dummy chat history"})

Thank you in advance for your help.

[dosubot]

🤖

Hello,

Thank you for bringing this issue to our attention. It seems like you're encountering a problem when using ConversationalRetrievalChain with both ConversationBufferMemory and return_source_documents set to True.

The error message you're seeing is due to the fact that the ConversationBufferMemory is expecting a single output key, but it's receiving two keys: 'answer' and 'source_documents'. This is because when return_source_documents is set to True, the ConversationalRetrievalChain includes the source documents in the output.

As a workaround, you've found that you can avoid the error by providing an external chat history when querying the chain. This is because when a chat history is provided, the ConversationalRetrievalChain does not attempt to save the context to memory, thus avoiding the error.

However, this is not an ideal solution as it requires you to manage the chat history externally. We understand that this might not be the most convenient way to use the ConversationalRetrievalChain.

We have previously encountered a similar issue as reported in Is there no chain for question answer with sources and memory?. The solution suggested there was to use the ConversationalRetrievalChain with an external chat history, which is the workaround you've already discovered.

We acknowledge that this is a limitation of the current design of the ConversationalRetrievalChain and ConversationBufferMemory. We will consider this issue for future improvements to the LangChain framework. In the meantime, you might have to continue using the workaround of providing an external chat history when querying the chain.

Thank you for your understanding and patience. If you have any other questions or issues, please feel free to ask.

Sources

• Is there no chain for question answer with sources and memory?
• libs/langchain/langchain/chains/conversational_retrieval/base.py

---

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[hashim-ch]

Same issue, please help

[hashim-ch]

Nvm I found the fix, you need to define output_key='answer' inside ConversationBufferMemory

ConversationBufferMemory(memory_key="chat_history", output_key='answer', return_messages=True)

@fcalabrow

[fcalabrow]

output_key='answer'

Great, now it's working. Thanks!

[nkityd09]

I tried this with output_key='answer but get the same error

qa_chain = ConversationalRetrievalChain.from_llm(llm=llm, 
                                   chain_type="stuff", 
                                   retriever=set_retriver(retriever),
                                   memory=memory,
                                   output_key='answer',
                                   return_source_documents=True,
                                   verbose=True)

ValueError: One output key expected, got dict_keys(['answer', 'source_documents'])

When I try to remove the return_source_documents=True setting, the chain works but does not refer to the chat history when answering the follow up question.

Any insights or ETA for when this can be resolved ?

[dosubot]

Hi, @fcalabrow,

I'm helping the LangChain team manage their backlog and am marking this issue as stale. From what I understand, the issue was initially resolved by defining output_key='answer' inside ConversationBufferMemory, but it seems that the fix did not work for all users, and there are still reports of the same error. There is a need for further insights and an ETA for a complete resolution.

Could you please confirm if this issue is still relevant to the latest version of the LangChain repository? If it is, please let the LangChain team know by commenting on the issue. Otherwise, feel free to close the issue yourself, or it will be automatically closed in 7 days.

Thank you for your understanding and cooperation.