[Issue]: autogen studio 2.0 created agent using GPT4 but always agent won't run python codes

[lawplus]

Describe the issue

Skills: search_operation_knowledge_chromadb

`import chromadb
from llama_index import VectorStoreIndex, SimpleDirectoryReader, ServiceContext
from llama_index.vector_stores import ChromaVectorStore
from llama_index.storage.storage_context import StorageContext
from llama_index.embeddings import HuggingFaceEmbedding
from IPython.display import Markdown, display

def search_operation_knowledge_chromadb(query_texts, n_results):
"""
根据输入的文本，从已存在的chromadb中检索返回相关结果信息.

:param query_texts: str or list of str, 外部输入的查询信息.
:param n_results: int, 控制返回的结果数量
:return: dict, 包含检索结果的字典，形式为 {
'ids': [
	['llama_response', 'c7122de0-9e38-4e4e-a78b-8ca520fb9964', '2aad39ae-9d13-4bdf-bac0-c0004142e3ba'],
	['llama_response', 'c7122de0-9e38-4e4e-a78b-8ca520fb9964', '2aad39ae-9d13-4bdf-bac0-c0004142e3ba'],
    ....
],
'documents': [
		["胡先生 is involved in various projects within the company, such as the Midea Group's China Regional Retail Transformation Project, Midea Group's Smart Shopping 2.0 Project, and Midea Group's Business Assistant 1.0 Project. In these projects, he held roles like Product Manager, Project Manager, and was responsible for tasks such as requirement analysis, project management, and team member coordination. He achieved significant results in terms of streamlining operations, 
			improving user experience, and increasing sales figures through innovative digital solutions.
			", '3 万人，为业务员数字化呈现，指导业务目标达成情况，国\n\n内排行 
			位置， 拜访客户次数 210 万 + ，促销活动 21 万 + 场次， 同期拜访量提升 30 % ；\n\ n项目名称： XX集团中国区域零售变革项目（ 2019.8 - 2019.12）\ n\ n担任职务： 产品经理 & 项目经理\ n\ n项目人数： 8 人\ n\ n项目描述： 各事业部导购统一收归中国区域 iHR 管理， 统一发薪， 导购资源各事业部\ n\ n共享， 费用共同分摊提高导购的利用率， 降低企业的人员成本同时
			提高导购收入，\ n\ n上线 1. '....]]}
"""

# 设置数据库路径和集合名称
db_path = r"D:/llama-index-data"
collection_name = "husubin_cv"
embed_model_name = "BAAI/bge-base-en-v1.5"

# 创建数据库客户端并获取或创建集合
db = chromadb.PersistentClient(path=db_path)
chroma_collection = db.get_or_create_collection(collection_name)

# 定义嵌入模型
embed_model = HuggingFaceEmbedding(model_name=embed_model_name)

# 设置ChromaVectorStore和加载数据
vector_store = ChromaVectorStore(chroma_collection=chroma_collection)
storage_context = StorageContext.from_defaults(vector_store=vector_store)
service_context = ServiceContext.from_defaults(embed_model=embed_model)
index = VectorStoreIndex.from_vector_store(
    vector_store=vector_store, storage_context=storage_context, service_context=service_context
)
custom_query_engine = index.as_query_engine()

result_dict = {"ids": [], "documents": []}

# 如果query_texts是字符串，则转换为单元素列表以便统一处理
if isinstance(query_texts, str):
    query_texts = [query_texts]

for query_text in query_texts:
    response = custom_query_engine.query(query_text)

    ids_list = ["llama_response"]
    documents_list = [response.response]

    # 只获取指定数量的结果
    for x in range(min(n_results, len(response.source_nodes))):
        ids_list.append(response.source_nodes[x].node.node_id)
        documents_list.append(response.source_nodes[x].node.get_content())

    result_dict["ids"].append(ids_list)
    result_dict["documents"].append(documents_list)

return result_dict`

Agent define:

userInputAnalyzerAgent
Agent Description: Receives user input questions and breaks down keywords for further processing by the KnowledgeRetrieverAgent. System Message: You are an AI Agent specialized in analyzing user input questions and extracting keywords to help the KnowledgeRetrieverAgent better understand user needs. 例如"胡苏斌做过多少个项目？总计项目人数是多少？ ", 拆解关键词为项目和人数，传给下一个agent进行搜索。

knowledgeRetrieverAgent
Agent Description: Retrieve relevant information from a vector database based on user-provided query keywords. System Message: 当接收到信息后，直接运行python函数 search_operation_knowledge_chromadb({信息} , 5) ，将函数返回的结果回复给user agent。 Skills: search_operation_knowledge_chromadb

Steps to reproduce

`胡苏斌做过多少个项目？总计项目人数是多少？

---

knowledgeRetrieverAgent (to userInputAnalyzerAgent):

为了回答您的问题，我需要从数据库中检索有关胡苏斌的项目信息。我将使用 search_operation_knowledge_chromadb 技能来执行这项任务。请稍等，我将为您提供所需的信息。

---

userInputAnalyzerAgent (to knowledgeRetrieverAgent):

很抱歉，但我无法直接访问数据库或执行实际的数据库查询。我的功能是基于公开可用的信息和我已有的知识来提供帮助。如果您能提供更多关于胡苏斌的背景信息或者他参与的项目
的具体细节，我可以尝试帮助您找到相关的公开信息。如果胡苏斌是一个公众人物或者他的项目信息在网上有记录，我可以帮助您查找相关信息。如果不是，您可能需要直接联系胡苏
斌或他的工作单位来获取这些数据。

---

knowledgeRetrieverAgent (to userInputAnalyzerAgent):

我将执行一个查询来检索胡苏斌参与的项目数量以及这些项目的总人数。请稍候片刻。

---

userInputAnalyzerAgent (to knowledgeRetrieverAgent):

抱歉，可能有些误解。作为一个人工智能助手，我实际上无法执行数据库查询或访问任何外部系统来检索特定个人的信息。我的能力限于提供基于公开信息的知识和帮助`

Screenshots and logs

No response

Additional Information

I want to know why agents not run python codes.
How to archieve the same effect as below autogen python codes:
`class MyRetrieveUserProxyAgent(RetrieveUserProxyAgent):
def query_vector_db(
self,
query_texts: List[str],
n_results: int = 10,
search_string: str = "",
**kwargs,
) -> Dict[str, List[List[Any]]]:
return query_vector_db(query_texts, n_results, search_string, **kwargs)

def retrieve_docs(self, problem: str, n_results: int = 20, search_string: str = "", **kwargs):
    results = self.query_vector_db(
        query_texts=[problem],
        n_results=n_results,
        search_string=search_string,
        **kwargs,
    )
    self._results = results
    print("doc_ids: ", results["ids"])

Init agent

boss_aid = MyRetrieveUserProxyAgent(
name="Boss_Assistant",
is_termination_msg=termination_msg,
system_message="Assistant who has extra content retrieval power for solving difficult problems. When the relevant info is not in CONTEXT, call 'UPDATE CONTEXT'",
human_input_mode="NEVER",
max_consecutive_auto_reply=3,
retrieve_config={
"task": "qa",
#"customized_prompt": PROMPT_SYSTEM
},
code_execution_config=False, # we don't want to execute code in this case.
)`

[mraguth]

I have same problem. Autogenstudio most of the times, won't run the code. But just gives the code.

[victordibia]

Hi,

Agents have to be configured in a specific way to get the specific results.

Some best practices. (see this article on an introduction to autogen)

• userProxy should be limited to code execution. It should not have any system message and it should not have a model attached to it. Instead the assistant should be prompted (via system message) to write code, and the userproxy should only execute this code.
• the assistant agent should be doing all the actual work - including planning and writing code.
• only attach skills to an assistant agent. The assistant agent should be given the skills and will use this to write code. This code that is written is then sent to the userproxy who will then execute it.

In your example, you seem to have a userProxy that has instructions on tasks. I would recommend the following:

• use a default userProxy as sender. No system message.
• update the default receiver asssistant agent. Extend the default message to add all the the instructions you currently have in your userInputAnalyzerAgent and knowledgeRetrieverAgent
• add the search_operation_knowledge_chromadb skill to your receiver assistant agent only.

the expected effect is that when you run a task ..

• userproxy will state the task
• assistant agent will come up with a plan and write code to solve the task that also uses the skill you have added to it
• userproxy will receive this response, execute the code and share in the chat
• assistant will attempt to fix errors if any and continue in this loop until the task terminates (e.g., task says terminate or max number of turns is reached.) . You may need to increase your max turns from 1 to a higher number

[lawplus]

appriciate if anyone can provide the detail agent definition

[victordibia]

Hi

see #1423

The way GroupChat is configured is critical to the behaviors you see.
Understanding AutoGen agent class behaviors is valuable here. I wrote an article here. The relevant section is:

• UserProxyAgent as code executor: Use the userProxyAgent as a code executor (it should have a code_execution_config) that serves to execute code in any message it receives and then shares the result with other agents.

• AssistantAgent as the main work horse: The AssistantAgent should be primed to implement the core behaviors you want to achieve. E.g. if you want to generate charts, retrieve documents, call functions etc, these should be specified in the system message of an AssistantAgent. Also specific skills (either as function calls or code added to system message) should be added to the assistant agent (not the user proxy). In writing prompts for your assistantFrom this perspective, the AssistantAgent should not have code_execution_config.

• Extend the Default Assistant Prompt: In writing prompts for your assistant, a good first step is to extend the the default assistant prompt and add your own instructions. The main benefit here is that the the default assistant prompt instructs the agent to create a plan and write code to solve the task. This is important as the this code writte is received by the user proxy and then executed. If no code is written, the user proxy does not get any code to execute an nothing happens. (@saibhaskerraju)

As an example, I am attaching a sample workflow config that shows a team structured to generate art content. 3 agents

• art director - assistant agent that plans the art pieces to be created, generates ideas
• art implementer -- assistant agent that writes the code that implements the ideas
• user proxy -- executes code (your group chat should have a user proxy)

Results are in the attached video.

workflow_Painter Team.json

You can upload this file in autogenstudio

groupchat.mp4

Let me know if this is helpful.

[lawplus]

thanks a lot! By refer to your sample json, i modified it as below:
[
workflow_Query User Info Team.json
](url)

[victordibia]

Great! Glad the issue is resolved!!