[FEAT]: Support @agent from API OpenAI Compatible endpoints

[amengus87]

How are you running AnythingLLM?

Docker (local)

What happened?

Hello AnythingLLM Team 👋
Awesome product you have done!

I'm encountering an issue about requesting @agentfrom OpenAI compatible endpoint (/v1/openai/chat/completions) => Agent is not triggered

How can we use @agentfeature inside OpenAI compatible endpoints from the AnythingLLM API?

Are there known steps to reproduce?

Working (using /v1/workspace/{slug}/thread/{threadSlug}/stream-chat endpoint)

import requests

url = 'http://localhost:3001/api/v1/workspace/internet-knowledge/thread/adc71dbd-5850-4a5e-aad0-7494da39b2cc/stream-chat'

headers = {
    'accept': 'text/event-stream',
    'Authorization': 'Bearer 00000000-0000-0000-0000-000000000000',
    'Content-Type': 'application/json',
}

data = {
    "message": "@agent Extract links of http://anythingllm.com",
    "mode": "chat",
    "userId": 1
}

response = requests.post(url, headers=headers, json=data)

print(response.text)

data: {"id":"05f1ab51-9965-4457-b11a-0e9f5d902b32","type":"agentThought","thought":"@agent: Scraping the content of http://anythingllm.com/","sources":[],"attachments":[],"close":false,"error":null}

data: {"id":"05f1ab51-9965-4457-b11a-0e9f5d902b32","type":"textResponse","textResponse":"Here are some of the main links from the AnythingLLM webpage:\n\n1. AnythingLLM Cloud: Link\n2. Docs: Link\n3](https://docs.anythingllm.com/)/n3). Github: Link\n4](https://github.com/Mintplex-Labs/anything-llm)/n4). Download AnythingLLM: Link\n\nAdditionally, the webpage includes links to their social media profiles and contact pages, such as their Twitter, Discord, and a contact email address.","sources":[],"attachments":[],"close":true,"error":null}

data: {"uuid":"05f1ab51-9965-4457-b11a-0e9f5d902b32","type":"finalizeResponseStream","textResponse":"Here are some of the main links from the AnythingLLM webpage:\n\n1. AnythingLLM Cloud: Link\n2. Docs: Link\n3](https://docs.anythingllm.com/)/n3). Github: Link\n4](https://github.com/Mintplex-Labs/anything-llm)/n4). Download AnythingLLM: Link\n\nAdditionally, the webpage includes links to their social media profiles and contact pages, such as their Twitter, Discord, and a contact email address.","thoughts":["@agent: Scraping the content of http://anythingllm.com/"],"close":true,"error":false}

Not working (using /v1/openai/chat/completions endpoint)

from langchain_openai import ChatOpenAI

llm = ChatOpenAI(
    model_name="internet-knowledge",
    openai_api_key="00000000-0000-0000-0000-000000000000",
    openai_api_base="http://localhost:3001/api/v1/openai",
    default_headers={
        "Authorization": "Bearer 00000000-0000-0000-0000-000000000000",
        "Accept": "*/*",
        "Content-Type": "application/json"
    }
)

messages = [
    ("human", "@agent Extract links of http://anythingllm.com"),
]
for chunk in llm.stream(messages):
    print(chunk.content, end="")

I'm unable to extract links from specific websites like http://anythingllm.com/. You may visit the website directly and browse through it for any links or resources you need.

[timothycarambat]

Ah, yes the /v1/workspace/chat endpoint is the only endpoint that supports that functionality currently. We could expand it to this endpoint as well. Can I understand more about the use case here over needing this functionality vs just using the /workspace/chat endpoint?

We would ideally not mess with the "compatible" part of OpenAI compatible too much, but I'm no opposed to it right now

[amengus87]

Ah, yes the /v1/workspace/chat endpoint is the only endpoint that supports that functionality currently

Good to know, thank you!

Can I understand more about the use case here over needing this functionality vs just using the /workspace/chat endpoint?

We use several native applications (not made by us) in our corp that allow us to change the OpenAI base endpoint URL and we would like to provide the power of @agent from AnythingLLM to them. (We can't use the widget)

[timothycarambat]

This has been supported for several months now 👍