Description
Describe the issue
I am trying to create an application which takes a PDF file as input document and then answers questions based on the context of the document. The expected behavior of the overall system should be such that if the assistant agent is not able to answer the question, it should forward the query to the Web Scraper agent.
However, I am not quite sure how can i integrate it with ragproxy as it is designed to give output as UPDATE CONTEXT everytime some out of context question is asked. I tried to add system_message where it is instructed to use provided tool if out of context question is asked. But it doesn't do anything
Steps to reproduce
Following is the basic QA code.
assistant = AssistantAgent(
name="assistant",
system_message="You are a helpful assistant",
llm_config=llm_config,
description="An helpful assistant"
)
ragproxyagent = RetrieveUserProxyAgent(
name="ragproxyagent",
retrieve_config={
"task": "qa",
"docs_path": "./po2.pdf",
"get_or_create": True,
},
code_execution_config=False
)
assistant.reset()
ragproxyagent.initiate_chat(assistant, message=ragproxyagent.message_generator, problem="What is content of the pdf?")
The tool for scraping is as per https://microsoft.github.io/autogen/docs/notebooks/agentchat_webscraping_with_apify/
Screenshots and logs
No response
Additional Information
No response