Closed
Description
Describe the issue
I am having issues to initiate a chat between two AssistantAgent, one being a local endpoint (using fastchat based on this page) and another being GPT 4 hosted in an Azure API. My code works perfectly fine for a conversation between two local model agents. But when I tried to get the local agent to interact with the API hosted agent, I ran into the following error. Could it be due to the difference in the api_type?
InternalServerError: <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd">
<html><head>
<meta type="copyright" content="Copyright (C) 1996-2018 The Squid Software Foundation and contributors">
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<title>ERROR: The requested URL could not be retrieved</title>
<style type="text/css"><!--
/*
* Copyright (C) 1996-2018 The Squid Software Foundation and contributors
*
* Squid software is distributed under GPLv2+ license and includes
* contributions from numerous individuals and organizations.
* Please see the COPYING and CONTRIBUTORS files for details.
*/
/*
Stylesheet for Squid Error pages
Adapted from design by Free CSS Templates
http://www.freecsstemplates.org/
Released for free under a Creative Commons Attribution 2.5 License
*/
/* Page basics */
* {
font-family: verdana, sans-serif;
}
html body {
margin: 0;
padding: 0;
background: #efefef;
font-size: 12px;
color: #1e1e1e;
}
/* Page displayed title area */
#titles {
margin-left: 15px;
padding: 10px;
padding-left: 100px;
background: url('/squid-internal-static/icons/SN.png') no-repeat left;
}
/* initial title */
#titles h1 {
color: #000000;
}
#titles h2 {
color: #000000;
}
/* special event: FTP success page titles */
#titles ftpsuccess {
background-color:#00ff00;
width:100%;
}
/* Page displayed body content area */
#content {
padding: 10px;
background: #ffffff;
}
/* General text */
p {
}
/* error brief description */
#error p {
}
/* some data which may have caused the problem */
#data {
}
/* the error message received from the system or other software */
#sysmsg {
}
pre {
}
/* special event: FTP / Gopher directory listing */
#dirmsg {
font-family: courier, monospace;
color: black;
font-size: 10pt;
}
#dirlisting {
margin-left: 2%;
margin-right: 2%;
}
#dirlisting tr.entry td.icon,td.filename,td.size,td.date {
border-bottom: groove;
}
#dirlisting td.size {
width: 50px;
text-align: right;
padding-right: 5px;
}
/* horizontal lines */
hr {
margin: 0;
}
/* page displayed footer area */
#footer {
font-size: 9px;
padding-left: 10px;
}
body
:lang(fa) { direction: rtl; font-size: 100%; font-family: Tahoma, Roya, sans-serif; float: right; }
:lang(he) { direction: rtl; }
--></style>
</head><body id=ERR_CANNOT_FORWARD>
<div id="titles">
<h1>ERROR</h1>
<h2>The requested URL could not be retrieved</h2>
</div>
<hr>
<div id="content">
<p>The following error was encountered while trying to retrieve the URL: <a href="http://0.0.0.0:8000/chat/completions">[http://0.0.0.0:8000/chat/completions</a></p](http://0.0.0.0:8000/chat/completions%3C/a%3E%3C/p)>
<blockquote id="error">
<p><b>Unable to forward this request at this time.</b></p>
</blockquote>
<p>This request could not be forwarded to the origin server or to any parent caches.</p>
<p>Some possible problems are:</p>
<ul>
<li id="network-down">An Internet connection needed to access this domains origin servers may be down.</li>
<li id="no-peer">All configured parent caches may be currently unreachable.</li>
<li id="permission-denied">The administrator may not allow this cache to make direct connections to origin servers.</li>
</ul>
<p>Your cache administrator is <a href="mailto:webmaster?subject=CacheErrorInfo%20-%20ERR_CANNOT_FORWARD&body=CacheHost%3A%20ip-10-51-93-49.eu-west-1.compute.internal%0D%0AErrPage%3A%20ERR_CANNOT_FORWARD%0D%0AErr%3A%20%5Bnone%5D%0D%0ATimeStamp%3A%20Tue,%2030%20Jul%202024%2023%3A27%3A36%20GMT%0D%0A%0D%0AClientIP%3A%2010.51.93.7%0D%0A%0D%0AHTTP%20Request%3A%0D%0APOST%20%2Fchat%2Fcompletions%20HTTP%2F1.1%0AAccept-Encoding%3A%20gzip,%20deflate,%20br%0D%0AConnection%3A%20keep-alive%0D%0AAccept%3A%20application%2Fjson%0D%0AContent-Type%3A%20application%2Fjson%0D%0AUser-Agent%3A%20OpenAI%2FPython%201.37.1%0D%0AX-Stainless-Lang%3A%20python%0D%0AX-Stainless-Package-Version%3A%201.37.1%0D%0AX-Stainless-OS%3A%20Linux%0D%0AX-Stainless-Arch%3A%20x64%0D%0AX-Stainless-Runtime%3A%20CPython%0D%0AX-Stainless-Runtime-Version%3A%203.10.6%0D%0AAuthorization%3A%20Bearer%20NULL%0D%0AX-Stainless-Async%3A%20false%0D%0AContent-Length%3A%20251%0D%0AHost%3A%200.0.0.0%3A8000%0D%0A%0D%0A%0D%0A">webmaster</a>.</p>
<br>
</div>
<hr>
<div id="footer">
<p>Generated Tue, 30 Jul 2024 23:27:36 GMT by ip-10-51-93-49.eu-west-1.compute.internal (squid/4.1)</p>
<!-- ERR_CANNOT_FORWARD -->
</div>
</body></html>
Steps to reproduce
Codes for the initialisation of the agents:
openai_config = [
{
"model": os.environ['OPENAI_GPT_DEPLOYMENT_NAME'],
"api_key": access_token,
"base_url": os.environ["OPENAI_CHAT_API_BASE"],
"api_type": "azure",
"api_version": "2024-02-01",
}
]
mistral_config = [
{
"model": "Mistral-7B-Instruct-v0.2",
"base_url": "http://0.0.0.0:8000",
"api_type": "openai",
"api_key": "NULL",
}
]
local_model = AssistantAgent(
name="mistral_chatbot",
max_consecutive_auto_reply=2,
llm_config={
"config_list": mistral_config,
"temperature": 1,
}
)
api_model = AssistantAgent(
name="gpt4",
max_consecutive_auto_reply=2,
llm_config={
"config_list": openai_config,
"temperature": 1,
}
)
chat_result = local_model.initiate_chat(
api_model, message="Hi, nice to meet you."
)
Screenshots and logs
No response
Additional Information
No response