Microsoft Foundry
Microsoft IQ Deep Dive with Python (livestream series): Resources #429
Replies: 37 comments
|
2026/07/28: Can we talk about data privacy, like anonymization and keeping PII out of the LLM? 📹 0:50 For controlling who can see what, Azure AI Search has built-in document-level access control. When you set up the index you designate which fields hold the Entra OID and the Entra group IDs, store those values on each document chunk, and then pass the user's token when you search. Only chunks that pertain to that user come back. That approach is most useful when you are indexing documents from an arbitrary source and need full control over visibility. The other option is a remote knowledge source. Work IQ requires the user's token, so it will only ever return information that user can already see, and the access control comes for free. For keeping PII away from the model entirely, one option is to avoid an LLM at certain stages. Configuring the knowledge base in minimal mode means no LLM is involved in query planning, and a plain search call involves even fewer services. But if you are building an agent that answers questions or takes actions, you have to ask whether the question can still be answered without the PII. If it can, PII detection and redaction in Azure AI Language (now part of Foundry tools) can strip sensitive values before the text reaches the model. There is also now a PII category available in Foundry guardrails. Running a local model through Foundry Local is another route, as suggested in chat: the agent runs on your own machine, and it can still reach out to a Foundry IQ knowledge base in minimal mode so fewer services see the data. Lee Stott added that Azure Local supports disconnected and sovereign scenarios with Foundry Local plus cloud hybrid deployments. It is also worth knowing the Azure OpenAI data privacy policy: prompts are not available to other customers, not available to OpenAI or other model providers, and not used to train models without your permission. That is one of the stronger privacy policies in the industry and not everyone realizes it exists. Links shared: |
|
2026/07/28: What guards exist against indirect prompt injection? 📹 5:42 Foundry guardrails apply at the model level. Default guardrails are applied to Foundry models, but you can create custom guardrails and choose which risks to detect. Jailbreak detection is always on; indirect prompt injection is the one you need to add explicitly, because that is the attack that arrives inside your data or a tool response rather than from the user. Content harms support custom block lists, and there are also protected material checks, a PII category, and, for hosted agents, network egress rules. Some of the newer categories run LLM-as-judge evaluations, which will add latency, so think carefully before running those inline. Beyond guardrails, the practical rule is that your agent can only do as much as you give it access to. If you do not want it sending email, do not give it a tool that sends email. If it needs to, require human approval, or route the action through another LLM check that asks whether this particular email should be sent. Guardrails are the first place to look because they attach to the model: set up the guardrail, assign it to the model, use that model in your agent, and every model call gets checked. Links shared: Can I apply the guardrails to LLMs that are not hosted on Foundry? 📹 17:54 Not the native Foundry guardrails — those are designed for Foundry models and Foundry Agent Service workloads only, and Foundry models always come with them applied. For any other model endpoint, use Azure AI Content Safety and policy-based controls, which offer Prompt Shields and custom detection and can be applied to virtually any model by placing them in your application or agent orchestration layer. For mixed estates spanning OpenAI, Anthropic, and local models, treat guardrails as a runtime architecture pattern rather than a model feature, so the same safety controls apply regardless of where the model is hosted. |
|
2026/07/28: Are knowledge sources and the services that touch the data located in our tenant, or can we control the country/region? 📹 8:39 Region residency depends on the feature. Connected models live in whatever region you deploy them to, and most other components are region-restricted, but each Azure AI Search feature has its own behavior, so check that feature's documentation. The region support documentation shows which features are available where. Remote knowledge sources depend on the individual source — Web IQ, for example, is a global service and may have to stay global architecturally. For strict residency requirements, validate three things separately: the region of the knowledge sources (SharePoint, Blob, OneLake), the region of the Azure AI Search service backing the knowledge base, and the documented residency commitments for Foundry IQ indexing, retrieval, reasoning, and enrichment. |
|
2026/07/28: Should I add my relational database or my raw files as the data source for a document chat experience? 📹 10:27 Context from the developer asking the question: They are building an HR document management app already uses Azure Content Understanding to detect document type and extract key fields into a relational database outside Azure. They're wondering whether to index both the raw files and the extracted data. It depends on the questions users ask, and you could add both. One approach is two tools: one that queries the extracted structured data and one that queries the original documents. Or make them two knowledge sources in a single knowledge base. The extracted data is attractive because it is already clean. Another approach is to fold the extracted data into the search index as metadata fields on each chunk, so a result returns both the text chunk and its metadata — page number, section heading, date. This is sometimes called content stuffing or content expansion, and it gives the LLM more to work with. Whichever direction you pick, start experimenting and set up evaluations with something like 50 representative questions. Do not validate a solution against three sample questions, because it will work great for exactly those three and you will have no idea what happens beyond them. |
|
2026/07/28: Does Foundry IQ handle ASPX-based SharePoint content well? 📹 12:52 There are multiple SharePoint options. The indexed SharePoint knowledge source has Azure AI Search index your SharePoint content, which makes a copy. The advantage of a copy is control: you can customize ingestion, and you can still enforce the document-level permissions described earlier. There is also a remote SharePoint option, but it is essentially the Copilot indexer. Its documentation does mention indexing modern ASPX site pages, but if Copilot was already failing to retrieve reliably from those pages, this path will hit the same retrieval quality issues, because ingestion and retrieval are both handled by Copilot rather than Azure AI Search. It is also still in preview, and it is unclear how much long-term staying power it has now that Work IQ exists. The two forward-looking recommendations are Work IQ or indexing it yourself with Azure AI Search. You would need to build a custom skill for ingesting the ASPX files at this point, however. Stay tuned for improvements in the future. |
|
2026/07/28: Are RAG data permissions checked at query time, or cached? If someone is removed from an Entra group, is there a delay? 📹 16:34 Security trimming is cached, per a confirmation from the Foundry IQ team during the session. So if someone is removed from a group, there is some caching delay before that takes effect for RAG queries. The exact cache duration was still being confirmed. |
|
2026/07/28: If we add Work IQ as a knowledge source, what does that cost on top of Work IQ itself? 📹 20:52 To understand the pricing, let's break the pipeline into stages. Query planning uses an LLM, so that cost lands in your LLM budget, not your search budget. The call out to the remote source itself does not appear to carry a separate Foundry IQ charge. The cost that comes out of your search budget is agentic reasoning — the result merging and reranking stage — because that model lives in the search service. So for a remote knowledge source, search is handled by the remote source, query planning is an LLM cost, and result merging is the Azure AI Search cost. The pricing page confirms that separate charges are incurred for remote knowledge sources like SharePoint and web, with agentic retrieval billed on reranking tokens. Semantic ranker charges apply only to indexes. There is also now a serverless tier for Azure AI Search, announced at Build. It is in preview, so it is not practical to rely on yet, but once it ships properly it should substantially reduce search service pricing. |
|
2026/07/28: Do you have experience with Dataverse and using FetchXML or OData queries to connect to models? 📹 24:39 No, Pamela was not personally familiar with Microsoft Dataverse. Dataverse is the database layer for Power Platform, and it does have a Dataverse MCP server, which means you can connect it to an agent. It also shows up in the Foundry tool catalog, so you can add Dataverse to a Foundry toolbox. Authentication requires bringing your own client ID. For Power Platform and Dataverse-specific questions, the Microsoft 365 & Power Platform Community is a better venue. |
|
2026/07/28: What role do knowledge graphs and ontologies play in retrieving knowledge or reasoning in Foundry IQ? 📹 26:48 Foundry IQ currently has no particular knowledge graph integration. The team has been looking at and working with GraphRAG for a long time, because a graph answers questions that chunk-based search cannot — "what are the top five themes in this data?" is impossible to answer from retrieved chunks. GraphRAG builds semantic clusters and searches across them. The hard part is productionizing it: GraphRAG is token-hungry, slow, and expensive, and making it performant enough to ship is still an open problem. If you want to approximate it yourself, you can precompute the graph and store it as multiple indexes in one knowledge base rather than computing on demand:
Then a knowledge base spanning those sources can answer both detailed and zoomed-out questions while staying performant. As always, the right data layout depends on the questions you actually get. Note that Fabric IQ has a feature called graph, but that is about connecting relational data rather than working with unstructured content. |
|
2026/07/28: How do I filter documents indexed in AI Search based on user roles? 📹 31:46 Context from chat: the approach in question was adding a custom That approach is right. Azure AI Search does have built-in support for filtering based on Entra IDs and groups, but not for Entra roles, so you can implement it yourself with a filter. Add a roles field to each document, construct the filter yourself, and combine it with any other filters you need. |
|
2026/07/28: Can the built-in tool that Foundry agents use to connect to an AI Search index return custom fields from that index? 📹 44:09 A knowledge base should be able to return all the fields. When you create a knowledge source you specify the source data fields — things like blob path, snippet, parent ID — as well as which fields are actually searched. The source data fields are what get returned, and there is an option to request additional fields for the reference source data. The catch, demonstrated live, is the MCP server. When you use the Foundry IQ MCP server you cannot customize retrieval parameters and you cannot access the full references. It returns inline reference markers like Using the custom tool path instead gives you the references. During a live demo on the call, the custom tool path also returned |
|
2026/07/28: Is there a decision tree for when to stay inside M365 and Copilot Studio versus going to Foundry? 📹 38:44 An honest caveat first: Pamela has only tried Copilot Studio once, about three years ago, so this is not a fair head-to-head comparison. Her personal decision rule is to go where she can write the most code and have the most control, which means Foundry, particularly Foundry hosted agents. There is a whole spectrum of control:
Integrations used to be a strong argument for Copilot Studio, but Foundry now has connections to Fabric IQ, Work IQ, and Foundry IQ, a growing tool catalog (Dataverse included), toolboxes, and custom tools via OpenAPI, MCP, or A2A. The practical advice is to list the features you need, the flexibility and control you need, and how you like to develop, and see which fits. Lee Stott's advice: lead with Copilot Studio when low-code makers or business teams are building, the organization already uses Power Platform, the agent mainly lives in Teams or M365 Copilot, and governance and rapid delivery matter more than deep customization. Lead with Foundry when professional developers own the solution, the team already uses GitHub, VS Code, Azure DevOps, or agent frameworks, and you need advanced RAG, custom models, observability, evaluations, or multi-agent orchestration. Use both when the experience lives in Teams/M365 but the intelligence layer needs advanced AI engineering. |
|
2026/07/28: If information lives in Work IQ, are the retrieval methods different from retrieving through Foundry IQ? 📹 43:05 Yes. Work IQ is a remote data source, so comparing indexed SharePoint against remote Work IQ is comparing two genuinely different retrieval pipelines. The recommendation is to compare them for your data. If Work IQ is good enough, use it — with a remote data source you do not have to worry about keeping a copy fresh or reimplementing data access control. Only when it is not good enough, or is missing something you need, should you move to indexing the data yourself. |
|
2026/07/28: Where is the right boundary between Foundry IQ for grounded knowledge and your own application for tenant isolation, permissions, and action approvals? 📹 53:47 Context from chat: the developer is building an Azure-first RMM platform adding an operator chat that answers from runbooks and can help take action on endpoints, deployed into customer Azure environments. Once an agent both retrieves knowledge and takes actions, you are in lethal trifecta territory: access to private data, the ability to communicate externally, and exposure to untrusted content such as web search results. Even without the third element, access to data plus the ability to send an email is already risky. The usual recommendation is a human approval step — and the honest part is that this is something you build. Where the approval happens is your application's job. One pattern seen in production on top of Foundry IQ is a customer service setup with an inbox of items to process, ranked by priority, flagging the ones that really need human review. Building that means building an interface, or adding more LLM checks along the way that decide whether there is enough confidence to take the next step. One design habit worth adopting: whenever an agent can take an action, also give it the ability to not take the action. Let it opt out and say it does not have enough confidence right now. Links shared: |
|
2026/07/28: Announcement: MCP 2.0 ships today 📹 30:25 Today is the release of MCP 2.0, and the GitHub MCP server already has updates for the new spec version. There are MCP graduation release parties happening around the world today. Upcoming MCP coverage includes MCP Live on September 8, in-person events in San Francisco and Bengaluru, and a VS Code livestream in mid-August focused on MCP 2.0. |
|
2026/07/30: Where does Graph fit among the Foundry Toolbox tools for Fabric? 📹 29:30 Foundry does not currently expose Graph as a direct tool or dedicated MCP server. To query Fabric Graph from a Foundry agent, either add it as a data source for a Fabric data agent, or write a custom tool that can execute GQL queries. The Foundry Toolbox notebook shows how to add Fabric Data Agent to a Foundry Toolbox. |
|
2026/07/30: Can LangChain, Pydantic AI, or another agent framework use Fabric IQ? 📹 31:31 Yes. Fabric IQ capabilities are exposed through MCP servers, so another framework can connect to the server URL and supply a user token with access to the Fabric workspace. Local use is relatively straightforward; deployment is harder because the application must obtain and pass the appropriate user token. For Foundry hosted agents, framework support for the Responses protocol also matters. The Foundry documentation explains the differences between the Responses and invocation protocols. LangChain has a Foundry Responses adapter and hosted LangGraph samples, making it the more direct route discussed in the session. Pydantic AI can connect to MCP, but without a Responses adapter it may need the simpler invocations protocol, which can lose Foundry integrations that assume Responses, potentially including some publishing features. The exact unavailable features were not confirmed during the session. Links shared: |
|
2026/07/30: Is the Microsoft skills-for-fabric repository recommended for coding agents? 📹 36:25 The skills-for-fabric repository appears current and can give coding agents focused Fabric instructions. Skills can occasionally become stale or over-trigger, so install only the relevant skills and remove them if they interfere. For GitHub Copilot in VS Code, the recommended starting point is the Microsoft Fabric extension, which prompts users to install its Fabric MCP server companion and can browse or open workspace resources. What if the coding agent does not run in VS Code? 📹 42:21 The repository's skills can be installed for other compatible coding agents. For command-line workflows, Fabio is also recommended as an agent-oriented CLI for Fabric operations and deployments. |
|
2026/07/29: What are the best practices for structuring and maintaining IQ deployments in a larger environment? 📹 5:23 A deployment spanning Foundry IQ, Work IQ, and Fabric IQ is not one service. It crosses several workloads and usually needs people with expertise in Foundry, Microsoft 365, and Fabric to handle deployment and governance. Microsoft Agent 365 can help govern agents and tools by showing what data and tools an agent can access and by supporting monitoring and inspection. Work IQ itself preserves the user's existing Microsoft 365 permissions and DLP controls, so a user only receives content they are already allowed to see. The larger risk is what an agent can do after retrieval. An agent or multi-agent system that can read private organizational data and publish externally could leak that data even though Work IQ retrieved it correctly. Limit each agent to the tools it needs, pay particular attention to write or publishing tools, and use Agent 365 for governance and monitoring. |
|
2026/07/29: How should we evaluate data retrieved by Work IQ separately from the agent using it? 📹 10:25 Evaluate Work IQ by data type because retrieval difficulty varies substantially. Calendar data is relatively structured, while long email threads and Teams chats are messy, conversational, and prone to missed results or speaker misattribution. A realistic test tenant should reproduce the length, slang, multiple languages, and ambiguity of production data rather than containing only clean examples. For retrieval-specific evaluation, build a ground-truth set that maps each query to the expected source attributions. Then measure recall and precision: whether Work IQ found all expected sources, missed some, or returned irrelevant extras. Also evaluate the complete agent separately. The agent may rewrite the user's query, select the wrong Work IQ tool, or supply incorrect arguments even when retrieval itself works well. Useful agent-level measures include groundedness, relevance, tool-selection accuracy, and argument accuracy. Repeated evaluations showed a noticeable improvement in accuracy when the agent's available tools were restricted to only those it needed. The linked Foundry hosted-agent evaluation example demonstrates quality and tool-call evaluation. |
|
2026/07/29: How can we debug a declarative agent and inspect its MCP tool calls? 📹 17:34 In Microsoft 365 Copilot chat, send |
|
2026/07/29: Can one AI agent use Work IQ across different customer tenants? 📹 20:00 Technically, an external agent can call Work IQ for different tenants if it can obtain the correct delegated user access token for each target tenant. Work IQ does not support app-only tokens for its A2A, MCP, or REST APIs; every request must represent a particular user and can only retrieve what that user may access. The application surface is therefore important. A custom web interface could use a multitenant Entra application to let each customer sign in and pass their delegated token. The one-click Foundry hosted-agent publishing flow for Teams is tenant-specific and does not by itself produce an app that customers in arbitrary tenants can install. There is also an important architecture distinction: one running agent with simultaneous access to several tenants has a larger security risk than deploying the same agent separately into each tenant. The latter keeps each deployment and its Work IQ access scoped to that tenant. How can the same agent be deployed as a Teams app to multiple tenants? 📹 24:19 The portal's one-click publishing path for a Foundry hosted agent was confirmed not to support this scenario. Use the standard Microsoft 365 and Teams publishing process instead. Declarative agents and Copilot Studio agents can also be exported and deployed to multiple environments or customer tenants, with each deployment configured for its own tenant. Links shared: |
|
2026/07/29: How do Copilot Studio agents relate to Foundry agents? 📹 22:47 They are not mutually exclusive. A Copilot Studio agent can connect to other agents through A2A, including a Foundry agent, and Copilot Studio also has a dedicated option for connecting to Foundry agents. This allows a multi-agent architecture in which low-code agents use Power Automate and other Copilot Studio features while code-first agents use Foundry capabilities. |
|
2026/07/29: What security checks can catch risky agent actions before publishing? 📹 27:06 Foundry guardrails can protect the model used by an agent with checks for jailbreaks, indirect prompt injection, harmful content, protected material, PII, task adherence, and groundedness. They do not replace tool-level authorization. Start by exposing only the tools the agent needs. If it should be read-only, allow Work IQ retrieval tools such as Ask and Fetch but not action tools. When an agent does need write actions, Agent Framework middleware can intercept proposed tool calls and enforce deterministic policies, such as rejecting an email addressed to more than a permitted number of recipients. Is there a static analyzer for agent risks at build time? 📹 30:12 There is not yet a general-purpose static analyzer specifically for agent risks. You can start with more general risk checking. For example, the Microsoft Agent Framework team uses Bandit and GitHub code scanning as automated security measures for their code, though neither looks for agent-specific vulnerabilities. For an agent-focused review, a practical starting point is to create a coding-agent security skill based on the OWASP AI Agent Security Cheat Sheet, or point GitHub Copilot at that guidance and the codebase and ask it to identify relevant risks. This is not a replacement for a purpose-built analyzer, but it can provide a structured review while more specialized tooling is still being investigated. |
|
2026/07/29: How can a team get standardized, reliable Work IQ results when each user can access different data? 📹 32:00 Security takes priority over identical answers: Work IQ should not bypass source permissions merely to make every user receive the same result. Standardize access-control lists and governance for content intended to be shared broadly, while preserving legitimate differences for restricted content. Test against the messiest permitted data and account for differences such as chat volume, conversational style, and language. For more repeatable behavior, consider lower-level Work IQ tools that do not use an LLM. This removes one source of nondeterminism and gives the application more control, but may require the application to orchestrate and parallelize calls across email, Teams, and calendar itself. |
|
2026/07/29: Should an agent query SharePoint, Outlook, and Confluence through Microsoft Graph or Work IQ? 📹 43:38 Use Microsoft Graph when the application needs specific raw Microsoft 365 objects and their properties, such as individual email messages. Use Work IQ when the agent needs organizational context synthesized across emails, Teams conversations, and SharePoint documents. Work IQ can return a focused, synthesized answer instead of forcing the agent to retrieve and process a large volume of raw objects, which can reduce downstream model-token usage. Confluence is outside the Microsoft 365 data described in this answer, so it would still need its own connector or retrieval path. The choice should follow the questions the agent needs to answer rather than assuming one interface covers every source. |
|
2026/07/29: Are Work IQ MCP results truncated differently in Copilot Studio and Foundry agents? 📹 46:10 Work IQ exposes the same MCP server and response regardless of the consuming orchestrator. The speakers could not confirm whether Copilot Studio applies its own payload limits or truncation after receiving that response. A Foundry hosted agent runs the developer's code in a container, so Foundry does not impose an additional hosted-agent truncation step. The model context window remains a limit, and exceeding it without management can produce an error. Developers can add summarization or tool-call middleware, or use the Agent Framework agent harness, which includes compaction support. |
|
2026/07/29: Does every user of a shared WorkMate autopilot need a separate E5 license? 📹 49:46 No. A single WorkMate instance shared by several users needs one E5 license for that instance. If the scenario requires multiple WorkMate instances, each instance needs its own E5 license. |
|
2026/07/29: If Work IQ summarizes data, does it still enforce each user's document permissions? 📹 51:27 Yes. Work IQ evaluates access using the delegated user's token before returning content, so summarization does not expose tenant-wide data to that user. Work IQ may be capable of reaching data across the tenant, but each request only returns information the represented user is authorized to access. |
|
2026/07/29: Will Work IQ behave differently in Copilot Studio, Foundry, or another agent framework? 📹 53:01 Yes. Work IQ itself is the same service, but the surrounding agent harness, model, prompt, available tools, query decomposition, filtering, and middleware can all change reasoning quality, tool-call sequences, latency, and final answers. If tool use is disappointing, compare models and harnesses, narrow the tool list, improve the prompt, and evaluate each configuration instead of assuming the retrieval service alone determines the result. |
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Join us for our 3-part live stream series on using Microsoft IQ to ground your agents and AI apps!
Register for the series
Livestreams
Tune in for the live streams from July 28th - July 30th, or watch them after:
Code samples
We will be using this repository:
https://github.com/microsoft/iqdeepdive/
You can either open that in GitHub Codespaces or locally, and then deploy with an Azure account following the steps in the README.
Office hours
After every session, at 11:00AM PT, we'll have an office hours in the Foundry Discord. Bring your questions there, or post them in this discussion!
We're also recording the OH as unlisted videos on YouTube and uploading the Q&A as comments to this thread:
Related resources
If you are enjoying this series, you might also like:
All reactions