Microsoft Foundry
Host your agents on Foundry (livestream series): Resources #380
Replies: 46 comments
-
|
2026/04/27: When would it be better to deploy agents as hosted agents vs. using other Azure compute resources? 📹 0:28 The number one advantage of deploying as hosted agents is the evaluation story. With hosted agents, it's much easier to set up scheduled evaluation, online/continuous evaluation, and evaluation alerts — that would be significant effort to replicate on your own compute platform. Another advantage is having everything in the Foundry UI — the playground, traces, and tools all in one location, which can make it easier for other people in your organization to interact with the agent. That said, if you're currently in production on Container Apps, you should stay there for now since hosted agents are still in public preview. You can test it out in parallel — set up performance tests and load testing to compare. Only move when you're really taking advantage of the hosted agent features like evaluation. |
Beta Was this translation helpful? Give feedback.
-
|
2026/04/27: Are the Foundry evaluations kept in our tenant? That's an important topic in Europe. 📹 3:06 Evaluation is a per-region thing, and it does create storage behind the scenes. Pamela assumed the storage stays in the region but couldn't find a definitive doc statement confirming it. For organizations that need to be very certain, the recommended approach is to use "bring your own storage" — that way you control exactly where data is stored. She noted this as a follow-up question for the product team. Links shared: |
Beta Was this translation helpful? Give feedback.
-
|
2026/04/27: Do you get more customizable orchestration with hosted agents? 📹 4:51 Yes, compared to prompt agents. Prompt agents are quite limited — you can only configure instructions and tools through the SDK. With hosted agents, you can bring your own framework (Microsoft Agent Framework, LangChain, Pydantic AI, etc.) and build complex workflows, then wrap them as an agent in the responses host server. However, it's the agent framework that provides the orchestration, not the Foundry Agent Service platform itself. The platform just hosts and runs your containerized agent code. |
Beta Was this translation helpful? Give feedback.
-
|
2026/04/27: Can you connect agents with external MCP tools using the agent platform? 📹 6:25 Yes. Pamela showed the stage 4 demo code which connects to multiple MCP tools. In this case, the agent connects to both a Foundry toolbox (providing web search and code interpreter) and a separate knowledge base retrieval tool — each as separate MCP server connections. There is currently a bug where the Foundry toolbox doesn't work with MCP tools when deployed — the team is actively hot-fixing it. In the meantime, you can connect MCP tools separately. You can add any MCP server as long as you handle the authentication (key, Entra, etc.) correctly. Links shared: |
Beta Was this translation helpful? Give feedback.
-
|
2026/04/27: How does safety play a role in using the web search tool regarding prompt injection? 📹 8:27 All calls go through a Foundry model protected by Foundry guardrails (the RAI policy system). By default, guardrails check for jailbreak detection, hate, violence, sexual content, and self-harm, with blocking set to medium level. You can increase or decrease blocking levels in the Foundry portal under Build > Guardrails. For web search specifically, the concern is indirect attacks (also called "prompt shields") — where a website contains embedded prompt injection that gets pulled into the grounded data. Direct jailbreak detection is always enabled by default, but indirect attack detection may not be. If you're doing any kind of RAG or web search grounding, you should create a custom RAI policy that explicitly enables indirect attack detection. Links shared: |
Beta Was this translation helpful? Give feedback.
-
|
2026/04/27: Is there something similar to code mode in Agent Framework, or can I use Pydantic Monty? 📹 12:43 Pydantic AI has code mode powered by Monty (a Python sandbox rewritten in Rust that only implements safe parts of Python — no file system access, etc.). There isn't a direct equivalent in Agent Framework. If you want code mode specifically, Pydantic AI is probably the best choice. The Pydantic AI maintainers are working on making it easy to deploy Pydantic AI agents with a generic Responses adapter. Some people have already deployed Pydantic AI to Foundry using the invocations protocol (the simpler HTTP endpoint approach), with AG-UI on top. Links shared: Live demo: Using Monty as a local tool in Agent Framework 📹 40:26 A community member shared that they use Monty as a tool in their Strava MCP server for data analysis. Pamela live-coded a minimal example of integrating Pydantic Monty as a local tool in an Agent Framework agent. The key points:
Pamela published the working example as a GitHub Gist. Links shared: |
Beta Was this translation helpful? Give feedback.
-
|
2026/04/27: What would be an efficient way to automatically analyze traces and logs? 📹 14:54 Several approaches:
The key is to go back and forth with the LLM when analyzing errors — don't accept the first diagnosis. Links shared: |
Beta Was this translation helpful? Give feedback.
-
|
2026/04/27: Does hosted agent support per-agent identity and is there VNet support? 📹 16:55 Per-agent identity: Yes. Each hosted agent gets its own identity. The AZD post-deploy stage handles assigning roles to the agent identity automatically. If your agent code needs to access additional Azure services directly (e.g., Azure AI Search), you need to get the agent's principal ID and assign the role yourself. You can get the principal ID using VNet support: Yes, there is VNet support documented for Foundry Agent Service. One caveat noted in the docs is that the ACR must be reachable over a public network endpoint. Pamela recommended trying it out since she hadn't personally tested this flow yet. Links shared: |
Beta Was this translation helpful? Give feedback.
-
|
2026/04/27: If we wrap a workflow as an agent and it contains other agents using tools, would tracing still capture all internal information? 📹 20:09 Yes, as long as you're using Agent Framework with If your workflows do something that Agent Framework doesn't automatically instrument, you can emit your own custom spans. The Agent Framework observability docs list everything that's automatically instrumented. Links shared: |
Beta Was this translation helpful? Give feedback.
-
|
2026/04/27: How do you pick a better model for querying a database (NL to SQL)? 📹 22:32 Pamela recommended a community-made LLM SQL Benchmark that specifically tests models on NL-to-SQL generation. According to that benchmark, Claude Sonnet, Opus, GLM, and Grok 4.1 all performed well. More generally, for any specific use case, the best approach is to set up your own evaluation scenarios with your actual data and queries, rather than relying solely on generic benchmarks. GPT-5.5 was also mentioned as worth trying — the prompting guide describes it as a fundamentally different model (not just an incremental improvement over 5.4). Links shared: |
Beta Was this translation helpful? Give feedback.
-
|
2026/04/27: Pros and cons of having each agent of a multi-agent system in its own sandbox? 📹 25:10 Pros:
Cons:
|
Beta Was this translation helpful? Give feedback.
-
|
2026/04/27: What is the pricing for hosted agents? 📹 27:14 Hosted agent billing began April 22nd during preview. You pay only for active execution — there is zero idle cost and faster startup with the new microVM platform. Pricing is:
Foundry Memory is free to use until June 1st. Links shared: |
Beta Was this translation helpful? Give feedback.
-
|
2026/04/27: Can you publish prompt agents when you have public network disabled? 📹 30:05 We weren't able to answer this during the session, but researched an answer after: Publishing prompt agents (not hosted agents) with public network access disabled is not reliably supported. The agent's messaging endpoint — an Azure Bot endpoint managed by Foundry — must be reachable by external Microsoft channels such as Teams and Copilot. When you restrict access to private endpoints only, those external services cannot reach the endpoint, so publish flows can fail (for example with HTTP 403). Check this blog post for a possible workaround: download the prepared agent package from a public Foundry project, update the names and IDs in Links shared: |
Beta Was this translation helpful? Give feedback.
-
|
2026/04/27: Which endpoint do you use to connect to an agent from an external app? 📹 30:20 For calling a hosted agent via the responses API, the endpoint format is: Links shared: |
Beta Was this translation helpful? Give feedback.
-
|
2026/04/27: What RBAC roles are typically needed for hosted agents? 📹 32:48 Generally, AZD handles assigning the necessary RBAC roles during deployment. You only need to manually assign additional roles when your agent code directly accesses other Azure services. Key roles Pamela highlighted from her Bicep templates:
If you're curious what roles are assigned to your agent's identity after a default deployment, you can get the agent's principal ID and use the Azure CLI to list all role assignments (change |
Beta Was this translation helpful? Give feedback.
-
|
2026/04/29: Announcements 📹 00:32 Pip 26.1 introduces dependency cooldowns — a new feature that accepts a relative duration (number of days) to delay accepting newly uploaded packages. UV has a similar feature called Links shared: |
Beta Was this translation helpful? Give feedback.
-
|
2026/04/30: Are there evals that score on cost and token efficiency? 📹 00:53 There are no built-in evaluators for cost/token efficiency, but you can write a custom code-based evaluator for it. Token usage usually comes back in the API response (especially with the Responses API), so as long as that data is available, you can write a Python function that pulls it out and evaluates it. If token usage isn't in the API response, it should also be in OpenTelemetry traces, which is another way to access that data. Pamela also always measures latency in her evaluations, which is straightforward since it's just elapsed time. Note that Foundry evaluations do return operational metrics (latency and tokens) for each evaluation run, but those measure the evaluation itself, not the agent run being evaluated. |
Beta Was this translation helpful? Give feedback.
-
|
2026/04/30: Are there built-in evaluators that don't depend on LLM-as-a-judge? 📹 04:32 Yes — there are textual similarity evaluators (BLEU, ROUGE, etc.) from the NLP research space that don't use LLMs. However, Pamela doesn't recommend most of them for agents producing free-flowing text responses, as they tend not to be useful in that context. The more practical approach is custom code-based evaluators. For example, Pamela has a regex-based evaluator for her RAG app that checks whether citations in the answer match the ground truth — and it's one of her most useful evaluators. Code-based evaluators are also great for measuring things like token usage and latency without needing an LLM. |
Beta Was this translation helpful? Give feedback.
-
|
2026/04/30: How can agent context (like memory) be included in quality evaluations? 📹 07:23 When we send the hosted agent's response for judging, the If memory isn't showing up in the output items, you'd need to find a way to inject it into the API response, since the evaluator can only access data that's either in the uploaded dataset or in the API response. With Foundry hosted agents using the Responses API adapter, there are constraints on what can be added to the response. |
Beta Was this translation helpful? Give feedback.
-
|
2026/04/30: In a multi-agent environment, should we have evals for each agent or just the final response? 📹 10:22 Both — start with end-to-end evaluations to get an overall picture, then break it down per agent to pinpoint where failures originate. How you evaluate depends on your multi-agent architecture:
Pamela noted that in her experience with handoff orchestration in Agent Framework, the handoffs were sometimes unpredictable, so a handoff evaluator could be very worthwhile. She'll also check with the product team about potential built-in multi-agent evaluators in the future. Would the interaction between agents (agent to sub-agent) need to be evaluated differently? 📹 27:30 Yes, it depends on your architecture. For supervisor/agents-as-tools patterns, agents are just tools and can be evaluated that way. For more flexible architectures, a custom evaluator that looks at agent-to-agent interaction is the way to go — for example, evaluating whether handoffs happened at the right time with the right criteria. |
Beta Was this translation helpful? Give feedback.
-
|
2026/04/30: What does the "Analyze Results" cluster analysis do? Is it like theme analysis? 📹 12:49 The "Analyze Results" button on evaluation runs performs a cluster analysis that groups different kinds of failures together, helping you understand patterns in what went wrong. For example, on a run with 178 samples, it found clusters like "hallucinated response" (73 samples), "required rule unavailable", and "abrupt refusal" — each with suggestions for improvement. This is different from general theme analysis because it specifically focuses on failures and how to fix them, rather than looking at all responses for general themes. It filters on failed samples and provides actionable suggestions. |
Beta Was this translation helpful? Give feedback.
-
|
2026/04/30: How can you approach streaming and online evaluation together? 📹 16:13 Generally, we don't recommend blocking live user-facing flows for evaluation calls. However, you could try a few approaches:
Links shared: |
Beta Was this translation helpful? Give feedback.
-
|
2026/04/30: What is the recommended practice for running evals after the app is live? 📹 20:50 Use a mix of scheduled evaluations at different frequencies:
The Foundry documentation defaults to daily for scheduled evals. You can set up multiple schedules with different names. Running 5 questions once an hour costs almost nothing. Are there benchmarks or cutoff scores to say an app is prod ready? 📹 22:30 There's no universal cutoff. Even 100% on an evaluation doesn't mean the app is perfect, since these are LLM-based apps with LLM judges. Internal teams at Microsoft often run evaluations on at least 800–1,000 questions before the first deploy, but they use different thresholds for pass rates. The practical approach many teams take: do your best effort with evaluations, then put the app out with appropriate UI messaging ("this is AI-generated, please give feedback"), and use continuous evaluation and user feedback to continue improving. |
Beta Was this translation helpful? Give feedback.
-
|
2026/04/30: Can images and audio be used in Foundry safety evaluations? 📹 24:15 Not yet. Multimodal safety evaluation would require image and audio models with their guardrails turned off so they can analyze potentially unsafe content. The evaluation team is considering adding multimodal safety evaluations in the future. |
Beta Was this translation helpful? Give feedback.
-
|
2026/04/30: How does Azure AI Search compare to Work IQ, and how can you evaluate Work IQ retrieval? 📹 33:40 Work IQ is an agent that wraps different retrieval mechanisms (SharePoint, emails, chats). To compare it with Azure AI Search, you'd run the same queries through both and compare results. However, it's important to understand what you're comparing:
So comparing Work IQ vs. Azure AI Search with indexed SharePoint would be the more meaningful comparison, since remote SharePoint and Work IQ use similar underlying retrieval. What is Foundry IQ exactly? 📹 36:37 Foundry IQ is Azure AI Search using the new agentic knowledge base feature, typically accessed via the MCP endpoint. The agentic knowledge base can wrap multiple sources including search indexes, indexed SharePoint, remote SharePoint, Bing, OneLake, and blob storage. |
Beta Was this translation helpful? Give feedback.
-
|
2026/04/30: How do you integrate evals with CI/CD? 📹 41:27 Foundry has a built-in GitHub Action for running evaluations in CI/CD. You specify your sample data file and evaluators in the YAML config. Important: don't run evaluations on every commit. Instead, use a trigger like a Pamela showed her existing custom CI/CD eval workflow for the RAG demo app, which uses a comment-based trigger restricted to repo owners, contributors, collaborators, and members. Another idea: use an LLM to look at a PR and decide whether evaluation should run automatically. She needs to verify that the GitHub Action works with Foundry hosted agents specifically (it may have been built for prompt agents originally). |
Beta Was this translation helpful? Give feedback.
-
|
2026/04/30: Are there roadmaps for Foundry preview features going GA? 📹 45:10 Pamela doesn't know specific timelines. Build (about a month away) is a common time for GA announcements, but she can't confirm anything. For hosted agents specifically, she'd want a longer baking period since it's a complex feature. She'll keep an eye out and share any news. |
Beta Was this translation helpful? Give feedback.
-
|
2026/04/30: Is there an agent or tool that can help create evaluations automatically in Foundry? 📹 54:39 There are a few things moving in this direction:
|
Beta Was this translation helpful? Give feedback.
-
|
2026/04/30: Announcements 📹 51:23
|
Beta Was this translation helpful? Give feedback.
-
|
One useful addition to the evaluation story would be a deployment checklist that separates final-answer quality from agent-action safety. For hosted agents with MCP tools or external tools, I would evaluate at least five layers:
The trace view is valuable because these failures often do not appear in the final answer. A final response may look correct while the trace shows an unnecessary external call, a risky argument, or a missing approval step. For teams moving from Container Apps to hosted agents, I would run the same scenario pack in both environments and compare not only latency/cost, but also trace completeness, evaluation coverage, managed identity boundaries, and evidence exportability for audit review. |
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
Join us for our 3-part live stream series on deploying your agents and workflows to Microsoft Foundry!
Register for the series
Livestreams
Tune in for the live streams from April 27th - April 30th, or watch them after:
Code samples
We will be using these two repositories:
https://github.com/Azure-Samples/foundry-hosted-agentframework-demos
https://github.com/Azure-Samples/foundry-hosted-langchain-demos
You can either open that in GitHub Codespaces or locally, and then deploy with an Azure account following the steps in the README.
Office hours
After every session, at 11:00AM PT, we'll have an office hours in the Foundry Discord. Bring your questions there, or post them in this discussion!
We're also recording the OH as unlisted videos on YouTube and uploading the Q&A as comments to this thread:
Related resources
If you are enjoying this series, you might also like:
Beta Was this translation helpful? Give feedback.
All reactions