RetrievalQA & RetrievalQAWithSourcesChain chains cannot be serialized/saved or loaded

[salman1993]

VectorDBQA is being deprecated in favour of RetrievalQA & similarly, VectorDBQAWithSourcesChain is being deprecated for RetrievalQAWithSourcesChain.

Currently, VectorDBQA & VectorDBQAWithSourcesChain chains can be serialized using vec_chain.save(...) because they have _chain_type property - https://github.com/hwchase17/langchain/blob/3bd5a99b835fa320d02aa733cb0c0bc4a87724fa/langchain/chains/qa_with_sources/vector_db.py#L67

However, RetrievalQA & RetrievalQAWithSourcesChain do not have that property and raise the following error when trying to save with ret_chain.save("ret_chain.yaml"):

File [~/.pyenv/versions/3.8.15/envs/internalqa/lib/python3.8/site-packages/langchain/chains/base.py:45](https://file+.vscode-resource.vscode-cdn.net/Users/smohammed/Development/hackweek-internalqa/~/.pyenv/versions/3.8.15/envs/internalqa/lib/python3.8/site-packages/langchain/chains/base.py:45), in Chain._chain_type(self)
     [43](file:///Users/smohammed/.pyenv/versions/3.8.15/envs/internalqa/lib/python3.8/site-packages/langchain/chains/base.py?line=42) @property
     [44](file:///Users/smohammed/.pyenv/versions/3.8.15/envs/internalqa/lib/python3.8/site-packages/langchain/chains/base.py?line=43) def _chain_type(self) -> str:
---> [45](file:///Users/smohammed/.pyenv/versions/3.8.15/envs/internalqa/lib/python3.8/site-packages/langchain/chains/base.py?line=44)     raise NotImplementedError("Saving not supported for this chain type.")

NotImplementedError: Saving not supported for this chain type.

There isn't any functions to support loading RetrievalQA either, unlike the VectorQA counterparts: https://github.com/hwchase17/langchain/blob/3bd5a99b835fa320d02aa733cb0c0bc4a87724fa/langchain/chains/loading.py#L313-L356

[adamyodinsky]

Facing the same issue

[Aparnamaurya]

Facing similar issue for retrivalQA, with elasticsearch as source. Any idea how to resolve this?

[qubitam]

Same issue with pinecone as a source

[tevslin]

ditto. I can get both a conversational chain and sources back from the pinecone with bit not the retrieval counterpart. sources are being sent from the db to the agnet and sent to the llm but not showing up in the final answers (maybe a separate problem)

[adamyodinsky]

Just want to mention, this was working fine in previous versions, but I'm not sure when exactly it got broken.

[nat890194]

same issue for RetrievalQA!

[dsaks9]

@adamyodinsky are you able to point to a specific version for which this worked?

[adamyodinsky]

@adamyodinsky are you able to point to a specific version for which this worked?

@dsaks9 No, I'm sorry I can't.

[shessenauer]

I'm on Langchain v0.0.219 and still running into this issue with RetrievalQAWithSourcesChain where I'm getting an error that Saving not supported for this chain type.

Note that I have RetrievalQAWithSourcesChain defined as a tool going into another agent.
(This works fine on its own with a qa_chain(query))

qa_chain =RetrievalQAWithSourcesChain.from_chain_type(llm=llm, chain_type="stuff", retriever=pinecone_retriever.as_retriever(), verbose=True, reduce_k_below_max_tokens=True)

Here I turn it into a tool

from langchain.agents import Tool

tools = [
    Tool(
        name = "Get Knowledgebase Data",
        func=qa_chain,
        description="useful for when you need to answer questions about the.... knowledgebase about content ...."
    )]
agent_aggregator = initialize_agent(tools, llm, agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION, verbose=True)

Doing a query on agent_aggregator has the following error. I did remove this specific tool and everything works fine, so it's isolated to RetrievalQAWithSourcesChain being a tool in this other agent.

> Entering new  chain...
I should check the Nanome knowledgebase data to find information about supported VR headsets.
Action: Get Nanome Knowledgebase Data
Action Input: Supported VR headsets
---------------------------------------------------------------------------
NotImplementedError                       Traceback (most recent call last)
[<ipython-input-19-52f895e71d87>](https://localhost:8080/#) in <cell line: 1>()
----> 1 agent_aggregator("query question about the knowledge_base input")
13 frames
[/usr/local/lib/python3.10/dist-packages/langchain/chains/base.py](https://localhost:8080/#) in __call__(self, inputs, return_only_outputs, callbacks, tags, include_run_info)
    164         except (KeyboardInterrupt, Exception) as e:
    165             run_manager.on_chain_error(e)
--> 166             raise e
    167         run_manager.on_chain_end(outputs)
    168         final_outputs: Dict[str, Any] = self.prep_outputs(

[/usr/local/lib/python3.10/dist-packages/langchain/chains/base.py](https://localhost:8080/#) in __call__(self, inputs, return_only_outputs, callbacks, tags, include_run_info)
    158         try:
    159             outputs = (
--> 160                 self._call(inputs, run_manager=run_manager)
    161                 if new_arg_supported
    162                 else self._call(inputs)

[/usr/local/lib/python3.10/dist-packages/langchain/agents/agent.py](https://localhost:8080/#) in _call(self, inputs, run_manager)
    985         # We now enter the agent loop (until it returns something).
    986         while self._should_continue(iterations, time_elapsed):
--> 987             next_step_output = self._take_next_step(
    988                 name_to_tool_map,
    989                 color_mapping,

[/usr/local/lib/python3.10/dist-packages/langchain/agents/agent.py](https://localhost:8080/#) in _take_next_step(self, name_to_tool_map, color_mapping, inputs, intermediate_steps, run_manager)
    848                     tool_run_kwargs["llm_prefix"] = ""
    849                 # We then call the tool on the tool input to get an observation
--> 850                 observation = tool.run(
    851                     agent_action.tool_input,
    852                     verbose=self.verbose,

[/usr/local/lib/python3.10/dist-packages/langchain/tools/base.py](https://localhost:8080/#) in run(self, tool_input, verbose, start_color, color, callbacks, **kwargs)
    297         except (Exception, KeyboardInterrupt) as e:
    298             run_manager.on_tool_error(e)
--> 299             raise e
    300         else:
    301             run_manager.on_tool_end(

[/usr/local/lib/python3.10/dist-packages/langchain/tools/base.py](https://localhost:8080/#) in run(self, tool_input, verbose, start_color, color, callbacks, **kwargs)
    269             tool_args, tool_kwargs = self._to_args_and_kwargs(parsed_input)
    270             observation = (
--> 271                 self._run(*tool_args, run_manager=run_manager, **tool_kwargs)
    272                 if new_arg_supported
    273                 else self._run(*tool_args, **tool_kwargs)

[/usr/local/lib/python3.10/dist-packages/langchain/tools/base.py](https://localhost:8080/#) in _run(self, run_manager, *args, **kwargs)
    410     ) -> Any:
    411         """Use the tool."""
--> 412         new_argument_supported = signature(self.func).parameters.get("callbacks")
    413         return (
    414             self.func(

[/usr/lib/python3.10/inspect.py](https://localhost:8080/#) in signature(obj, follow_wrapped, globals, locals, eval_str)
   3252 def signature(obj, *, follow_wrapped=True, globals=None, locals=None, eval_str=False):
   3253     """Get a signature object for the passed callable."""
-> 3254     return Signature.from_callable(obj, follow_wrapped=follow_wrapped,
   3255                                    globals=globals, locals=locals, eval_str=eval_str)
   3256 

[/usr/lib/python3.10/inspect.py](https://localhost:8080/#) in from_callable(cls, obj, follow_wrapped, globals, locals, eval_str)
   3000                       follow_wrapped=True, globals=None, locals=None, eval_str=False):
   3001         """Constructs Signature for the given callable object."""
-> 3002         return _signature_from_callable(obj, sigcls=cls,
   3003                                         follow_wrapper_chains=follow_wrapped,
   3004                                         globals=globals, locals=locals, eval_str=eval_str)

[/usr/lib/python3.10/inspect.py](https://localhost:8080/#) in _signature_from_callable(obj, follow_wrapper_chains, skip_bound_arg, globals, locals, eval_str, sigcls)
   2465                                         globals=globals, locals=locals, eval_str=eval_str)
   2466 
-> 2467     if _signature_is_builtin(obj):
   2468         return _signature_from_builtin(sigcls, obj,
   2469                                        skip_bound_arg=skip_bound_arg)

[/usr/lib/python3.10/inspect.py](https://localhost:8080/#) in _signature_is_builtin(obj)
   1992             # Can't test 'isinstance(type)' here, as it would
   1993             # also be True for regular python classes
-> 1994             obj in (type, object))
   1995 
   1996 

/usr/local/lib/python3.10/dist-packages/pydantic/main.cpython-310-x86_64-linux-gnu.so in pydantic.main.BaseModel.__eq__()

[/usr/local/lib/python3.10/dist-packages/langchain/chains/base.py](https://localhost:8080/#) in dict(self, **kwargs)
    340             raise ValueError("Saving of memory is not yet supported.")
    341         _dict = super().dict()
--> 342         _dict["_type"] = self._chain_type
    343         return _dict
    344 

[/usr/local/lib/python3.10/dist-packages/langchain/chains/base.py](https://localhost:8080/#) in _chain_type(self)
     63     @property
     64     def _chain_type(self) -> str:
---> 65         raise NotImplementedError("Saving not supported for this chain type.")
     66 
     67     @root_validator()

NotImplementedError: Saving not supported for this chain type.

Major apologies if this is the wrong issue thread, I'm pretty sure it's related but can move it somewhere more appropriate if needed.

Any thought on what could be happening?

[fabmeyer]

@shessenauer

Having the exact same problem!

Here's my stacktrace:

`

Entering new chain...
I need to find information about electric vehicles
Action: Findall QA with Sources System
Action Input: Trends in electric vehicles I need to find information about electric vehicles
Action: Findall QA with Sources System
Action Input: Trends in electric vehicles2023-07-13 12:25:48.792 Uncaught app exception
Traceback (most recent call last):
File "/home/fabmeyer/.local/share/virtualenvs/personal-investment-assistant-Y3o7tQcK/lib/python3.9/site-packages/streamlit/runtime/scriptrunner/script_runner.py", line 552, in _run_script
exec(code, module.dict)
File "/home/fabmeyer/Dev/Python/personal-investment-assistant/personal_assistant.py", line 272, in
response = st.session_state['agent']({'input': user_input}, return_only_outputs=True)
File "/home/fabmeyer/.local/share/virtualenvs/personal-investment-assistant-Y3o7tQcK/lib/python3.9/site-packages/langchain/chains/base.py", line 243, in call
raise e
File "/home/fabmeyer/.local/share/virtualenvs/personal-investment-assistant-Y3o7tQcK/lib/python3.9/site-packages/langchain/chains/base.py", line 237, in call
self._call(inputs, run_manager=run_manager)
File "/home/fabmeyer/.local/share/virtualenvs/personal-investment-assistant-Y3o7tQcK/lib/python3.9/site-packages/langchain/agents/agent.py", line 987, in _call
next_step_output = self._take_next_step(
File "/home/fabmeyer/.local/share/virtualenvs/personal-investment-assistant-Y3o7tQcK/lib/python3.9/site-packages/langchain/agents/agent.py", line 850, in _take_next_step
observation = tool.run(
File "/home/fabmeyer/.local/share/virtualenvs/personal-investment-assistant-Y3o7tQcK/lib/python3.9/site-packages/langchain/tools/base.py", line 320, in run
raise e
File "/home/fabmeyer/.local/share/virtualenvs/personal-investment-assistant-Y3o7tQcK/lib/python3.9/site-packages/langchain/tools/base.py", line 292, in run
self._run(*tool_args, run_manager=run_manager, **tool_kwargs)
File "/home/fabmeyer/.local/share/virtualenvs/personal-investment-assistant-Y3o7tQcK/lib/python3.9/site-packages/langchain/tools/base.py", line 442, in _run
new_argument_supported = signature(self.func).parameters.get("callbacks")
File "/home/fabmeyer/.pyenv/versions/3.9.0/lib/python3.9/inspect.py", line 3118, in signature
return Signature.from_callable(obj, follow_wrapped=follow_wrapped)
File "/home/fabmeyer/.pyenv/versions/3.9.0/lib/python3.9/inspect.py", line 2867, in from_callable
return _signature_from_callable(obj, sigcls=cls,
File "/home/fabmeyer/.pyenv/versions/3.9.0/lib/python3.9/inspect.py", line 2321, in _signature_from_callable
if _signature_is_builtin(obj):
File "/home/fabmeyer/.pyenv/versions/3.9.0/lib/python3.9/inspect.py", line 1861, in _signature_is_builtin
obj in (type, object))
File "pydantic/main.py", line 911, in pydantic.main.BaseModel.eq
File "/home/fabmeyer/.local/share/virtualenvs/personal-investment-assistant-Y3o7tQcK/lib/python3.9/site-packages/langchain/chains/base.py", line 556, in dict
_dict["_type"] = self._chain_type
File "/home/fabmeyer/.local/share/virtualenvs/personal-investment-assistant-Y3o7tQcK/lib/python3.9/site-packages/langchain/chains/base.py", line 94, in _chain_type
raise NotImplementedError("Saving not supported for this chain type.")
NotImplementedError: Saving not supported for this chain type.
`

When using the Retrieval QA From Sources chain everything works fine. However I have to combine it with other tools into an agent.

See this other issue: #1844 however it didn't helped me.

I am on LangChain 0.0.231

[mikexd-dev]

Hi @fabmeyer @shessenauer I am also trying with the same approach, do you have any resolutions on the issues since your last attempt?

[skeretna]

@shessenauer @fabmeyer @mikexd-dev I have the same problem, were you able to resolve this?

[fabmeyer]

@mikexd-dev @skeretna No I was not able to solve the problem. However I decided to add a UI element to select the model/tool and "hardcode" it rather than using an agent to select the appropriate model/tool.

[skeretna]

This worked for me when I called the chain using lambda instead of directly, no idea why

So, Change from this:

from langchain.agents import Tool

tools = [
    Tool(
        name = "Get Knowledgebase Data",
        func=qa_chain,
        description="useful for when you need to answer questions about the.... knowledgebase about content ...."
    )]
agent_aggregator = initialize_agent(tools, llm, agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION, verbose=True)

to this

from langchain.agents import Tool

tools = [
    Tool(
        name = "Get Knowledgebase Data",
       func =  lambda q: qa_chain(q), #Note the lambda
        description="useful for when you need to answer questions about the.... knowledgebase about content ...."
    )]
agent_aggregator = initialize_agent(tools, llm, agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION, verbose=True)

[Iana-Kasimova-TR]

@skeretna seems it works with this change, but need modification, add custom output parser can help I think or for example use the lambda q: qa_chain(q)['answer']
and if you need to keep somewhere the info about source, just create a function with this logic and put it in the tool as function