Agent Framework Hack

Introduction

This is a demo repo showing how to create a workflow using the Microsoft Agent Framework, as well as how to test the workflow using VCR.py.

Getting Started

1. Open the repo in a dev container. This will install all the dependencies needed to run the code and tests.

2. Deploy the infrastructure.

   poe deploy  # azd up

Run the workflow

poe run-ui

This runs the workflow using DevUI. DevUI is a sample application that provides a user interface for interacting with agents and workflows built using the Agent Framework.

The workflow can be run directly, showing the individual steps within the workflow, or the workflow can be converted into an agent, allowing it to maintain memory across multiple interactions.

The workflow can also be run as an agent via the command line using:

poe run

The workflow uses three agents, which are all Azure OpenAI Responses chat models:

• Relevance Agent - Determines the relevance of the user query. Uses Structured Outputs to return a JSON response of the query's relevance to fashion.
• Fashion Agent - Answers fashion-related queries. Uses a context provider to inject (dummy) user context into the agent instructions.
• General Agent - Returns a generic answer for non-fashion queries.

The Agent Framework allows the underlying agents to be easily swapped out for different agent types, such as Foundry Agents.

Memory (threads)

By converting the workflow into an agent before it is run, workflow message history is maintained between each invocation within a thread. As a result, the start executor (node) of the workflow receives a list of all messages in the workflow thread, allowing it to maintain memory across multiple interactions. In addition, each agent within the workflow also maintains its own memory by creating its own thread. This allows follow-up conversations with each agent.

Example:

• "What were the trends in fashion 20 years ago?" Handled by FashionAgent
• "Tell me about football." Handled by GeneralAgent
• "How do they compare to today?" Handled by FashionAgent, which has memory of the previous fashion-related query, but not the general query

YAML Workflow

The repo also contains a YAML implementation of the workflow, which can be found in src/agent_framework_hack/entities/yaml_workflow. Functionally, the YAML workflow is identical to the Python implementation. However, it cannot be converted to and run as an agent.

The YAML workflow is available in DevUI, and also can be run via the command line:

poe run --yaml

Telemetry

DevUI captures and displays telemetry traces emitted by the Agent Framework during workflow and agent execution. If the workflow is run via the command line and the environment variable APPLICATIONINSIGHTS_CONNECTION_STRING is set, telemetry will be sent to Application Insights.

Testing

As the Microsoft Agent Framework is currently unstable, it is likely that the package will be updated frequently with breaking changes. If we rely solely on mocking the framework in unit tests, we risk mocking the wrong behaviour.

The tests in this repo use VCR.py as a PoC for mocking HTTP requests. VCR.py records HTTP requests and responses, and saves them to cassette files (YAML files). For subsequent HTTP requests, if the request matches a recorded request, the recorded response is returned instead of making a real HTTP request. Otherwise, an error will be thrown.

By default, an HTTP request matches a recorded request if the following attributes are the same:

• method
• scheme
• host
• port
• path
• query

In this PoC, body was also added.

Run the tests

poe test  # uv run pytest

To add new cassette files, the --record-mode flag can be added to pytest:

poe test-update # uv run pytest --record-mode=once