TimeOutError unhandled

[moghadas76]

System Info

`ValueError Traceback (most recent call last)
Cell In[35], line 3
1 query = "What did the president say about the Supreme Court"
2 docs = db.similarity_search(query)
----> 3 chain.run(input_documents=docs, question=query)

File ~/.local/lib/python3.10/site-packages/langchain/chains/base.py:480, in Chain.run(self, callbacks, tags, metadata, *args, **kwargs)
475 return self(args[0], callbacks=callbacks, tags=tags, metadata=metadata)[
476 _output_key
477 ]
479 if kwargs and not args:
--> 480 return self(kwargs, callbacks=callbacks, tags=tags, metadata=metadata)[
481 _output_key
482 ]
484 if not kwargs and not args:
485 raise ValueError(
486 "run supported with either positional arguments or keyword arguments,"
487 " but none were provided."
488 )

File ~/.local/lib/python3.10/site-packages/langchain/chains/base.py:282, in Chain.call(self, inputs, return_only_outputs, callbacks, tags, metadata, include_run_info)
280 except (KeyboardInterrupt, Exception) as e:
281 run_manager.on_chain_error(e)
--> 282 raise e
...
113 if self.client.task == "text-generation":
114 # Text generation return includes the starter text.
115 text = response[0]["generated_text"][len(prompt) :]

ValueError: Error raised by inference API: Model google/flan-t5-xl time out`

This exception should be handled

Who can help?

No response

Information

• The official example notebooks/scripts
• My own modified scripts

Related Components

• LLMs/Chat Models
• Embedding Models
• Prompts / Prompt Templates / Prompt Selectors
• Output Parsers
• Document Loaders
• Vector Stores / Retrievers
• Memory
• Agents / Agent Executors
• Tools / Toolkits
• Chains
• Callbacks/Tracing
• Async

Reproduction

query = "What did the president say about the Supreme Court" docs = db.similarity_search(query) chain.run(input_documents=docs, question=query)

Expected behavior

Retry mechanism, or caching

[dosubot]

🤖

Hello,

Thank you for bringing this to our attention. It seems like you're encountering a timeout error when running the google/flan-t5-xl model in LangChain.

Currently, when an error occurs during the execution of a model in LangChain, the on_chain_error method of the run_manager object is called with the error object as an argument. After calling the on_chain_error method, the error is re-raised. This is implemented in both the synchronous __call__ method and the asynchronous acall method of the Chain class.

However, this mechanism does not specifically handle timeout errors. Your suggestion of implementing a retry mechanism or caching is a good one. We could potentially add a specific exception handling for timeout errors that would retry the operation a certain number of times before finally raising the error.

As for the timeout setting for the google/flan-t5-xl model, I wasn't able to find specific information in the LangChain repository. It's possible that the timeout is set by the inference API that is being used to run the model, rather than being set in LangChain itself.

Here's a potential workaround for now: you could try catching the ValueError in your own code and implementing a retry mechanism there. Here's a simple example of how you might do this:

import time

for i in range(5):  # Retry up to 5 times
    try:
        query = "What did the president say about the Supreme Court"
        docs = db.similarity_search(query)
        chain.run(input_documents=docs, question=query)
        break  # If the operation is successful, break the loop
    except ValueError as e:
        if "time out" in str(e):
            print(f"Timeout error occurred, retrying in {i+1} seconds...")
            time.sleep(i+1)  # Wait for i+1 seconds before retrying
        else:
            raise  # If the error is not a timeout error, re-raise it

This is just a temporary solution, and we'll look into improving the error handling in LangChain to better handle these kinds of situations. Thank you for your patience and for using LangChain.

Best,
Dosu

Sources

• libs/langchain/langchain/chains/base.py

---

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[dosubot]

Hi, @moghadas76! I'm Dosu, and I'm helping the LangChain team manage their backlog. I wanted to let you know that we are marking this issue as stale.

From what I understand, the issue you raised is about a TimeOutError that is not being handled in the code. You requested help in handling this exception and suggested a retry mechanism or caching as possible solutions. In response, I suggested implementing a retry mechanism or caching specifically for timeout errors. I also provided a temporary workaround for catching the ValueError and implementing a retry mechanism in your own code. User @devstein gave a thumbs up to this solution.

Now, we would like to know if this issue is still relevant to the latest version of the LangChain repository. If it is, please let us know by commenting on this issue. Otherwise, feel free to close the issue yourself, or it will be automatically closed in 7 days.

Thank you for your understanding and contribution to the LangChain project! Let us know if you have any further questions or concerns.