Task-Centric Memory

[rickyloynd-microsoft]

(EXPERIMENTAL, RESEARCH IN PROGRESS)

In 2023 AutoGen introduced Teachable Agents that users could teach new facts, preferences and skills. But teachable agents were limited in several ways: They could only be ConversableAgent subclasses, they couldn't learn a new skill unless the user stated (in a single turn) both the task and how to solve it, and they couldn't learn on their own. Task-Centric Memory overcomes these limitations, allowing users to teach arbitrary agents (or teams) more flexibly and reliably, and enabling agents to learn from their own trial-and-error experiences.

This PR is large and complex. All of the files are new, and most of the added components depend on the others to run at all. But the review process can be accelerated if approached in the following order.

1. Start with the Task-Centric Memory README.
   1. Install the memory extension locally, since it won't be in pypi until it's merged. In the agentic_memory branch, and the python/packages directory:
      • pip install -e autogen-agentchat
      • pip install -e autogen-ext[openai]
      • pip install -e autogen-ext[task-centric-memory]
   2. Run the Quickstart sample code, then immediately open the ./pagelogs/quick/0 Call Tree.html file in a browser to view the work in progress.
   3. Click through the web page links to see the details.
2. Continue through the rest of the main README to get a high-level overview of the architecture.
3. Read through the code samples README, running each of the 4 code samples while viewing their page logs.
4. Skim through the 4 code samples, along with their corresponding yaml config files:
   1. chat_with_teachable_agent.py
   2. eval_retrieval.py
   3. eval_teachability.py
   4. eval_learning_from_demonstration.py
   5. eval_self_teaching.py
5. Read task_centric_memory_controller.py, referring back to the previously generated page logs as needed. This is the most important and complex file in the PR.
6. Read the remaining core files.
   1. _task_centric_memory_bank.py
   2. _string_similarity_map.py
   3. _prompter.py
7. Read the supporting files in the utils dir.
   1. teachability.py
   2. apprentice.py
   3. grader.py
   4. page_logger.py
   5. _functions.py

[rickyloynd-microsoft]

This PR is simply too large. Please propose your changes progressively and iteratively using a separate sequence of PRs. We cannot effectively review these changes as is.

@jackgerrits, I've moved the code under autogen_ext/experimentaltask_centric_memory to convey the status and manage expectations as you suggested. And reviewer feedback has led to many improvements. At this point, do you recommend a series of PRs to review the content in a more iterative fashion?

[ekzhu]

As a next step #5542.

[victordibia]

The teachability example is pretty cool, shows how the MemoryController can be used with the an AssistantAgent via the Memory interface.

[rickyloynd-microsoft]

The teachability example is pretty cool, shows how the MemoryController can be used with the an AssistantAgent via the Memory interface.

And with very little code!
agent = AssistantAgent(model_client=client, memory=[Teachability(MemoryController(False, client))])

Discussed offline.