Skip to content

v0.9.0

Choose a tag to compare

@corticalstack corticalstack released this 22 Sep 10:08
· 67 commits to main since this release
2ab2fe5

Three new labs plus repo-wide fixes. Two labs were broken on main: the Foundry IQ knowledge bases stopped answering after Azure AI Search began rejecting APIM endpoints for knowledge base models, and the hosted Copilot SDK containers no longer started when rebuilt. Both are fixed and re-verified end to end against Azure.

Added

  • New 07-model-inference/07-02-migrate-gpt-4o-to-gpt-5.ipynb: a live migration of a GPT-4o workload to the gpt-5-mini reasoning model through the same APIM gateway as 07-01. Walks the six gotchas the model-class change introduces - sampling parameters rejected with HTTP 400, max_tokens renamed to max_completion_tokens, hidden reasoning tokens inside the output budget, the empty-answer trap, reasoning effort versus latency, and system versus developer role - then the Chat Completions to Responses shift with previous_response_id, verbosity and typed streaming events. docs/gpt-4o-to-gpt-5.1-migration-research.md carries the source research. The README already linked this lab; the notebook it pointed at was missing.
  • New 08-agents/08-11-hosted-claude-agent-sdk-agent section: the Claude counterpart to 08-10. A Claude Agent SDK container (ClaudeSDKClient behind azure-ai-agentserver-invocations) deployed as a Foundry hosted agent, with inference from a Foundry-deployed claude-sonnet-4-6 over /anthropic/v1/messages using the per-agent managed identity (CLAUDE_CODE_USE_FOUNDRY=1 + ANTHROPIC_FOUNDRY_RESOURCE). Not validated end to end: the notebook has no outputs because Claude deployments subscribe to an Anthropic Marketplace offer, which the lab subscription's tenant policy blocks (Marketplace Subscription purchase eligibility check failed) even with Claude quota available. The overview lists that as a prerequisite.
  • New 08-agents/08-10c-hosted-copilot-sdk-agent-runtime-model section: answers whether a chat UI can switch a hosted agent's model or reasoning effort mid-conversation, and what telemetry a hosted agent produces out of the box. Runtime switching belongs to the container, not the platform: an agent version is immutable, but the invocations protocol hands the request body over untouched, so main.py reads optional model / reasoning_effort fields, validates them against AZURE_AI_ALLOWED_MODELS, and calls session.set_model() on the live Copilot session, which preserves the conversation history. Verified both with a logging proxy on the outgoing /openai/v1/responses calls and hosted: all model calls matched the UI selection, gpt-5.4 used 163 reasoning tokens at high versus 26 at low, and the agent kept one version with the same image digest. Self-contained: its own Bicep provisions the resource group, Foundry account and project, capability host, registry, Application Insights and the three model deployments the UI switches between.
  • Telemetry walk-through in 08-10c separating the three layers: the platform's invoke_agent record (no code), the injected FOUNDRY_* environment variables plus the per-session container log stream and exported Python logs (no code), and the per-model-call chat <model> spans with reasoning effort and token usage (tracing.py). Notes that the platform marks an invocation successful even when the container returns HTTP 400.

Changed

  • Every notebook now declares the repo's own .venv kernel. 29 of the 51 notebooks named a foundry-samples kernel that runs the interpreter of an older foundry-nextgen checkout; the rest used mixed display names. Metadata only.
  • 08-agents/08-00-what-is-an-agent.md chapter index: added the rows missing for 08-09, 08-10 and 08-10b, then 08-11 and 08-10c. The README agents row said "eleven sub-labs" while its table listed thirteen; it now reads fifteen.
  • presentation.html is no longer tracked. The talk deck is personal presentation material rather than lab content, and nothing in the repo linked to it.
  • 13-guardrails/13-03-demo-guardrails.ipynb gained a model_refusal category: the harmful-content and protected-material prompts sit below the content filters' severity thresholds, so the request reaches the model and the model declines it. That is defence in depth rather than a guardrail block, and the notebook now says so instead of scoring it as a failure.
  • .lycheeignore excludes api.applicationinsights.io, used as an auth scope and REST endpoint in 08-10c rather than a browseable page.

Fixed

  • 10-foundry-iq: Azure AI Search now rejects APIM and custom domain endpoints in knowledge base model configurations (InvalidRequestParameter), on create and on retrieve, so arxiv-nlp-kb failed on every query and took 10-05's kb_standard tool with it. The knowledge base model now points at the core account with no API key, and a new core-search-rbac.bicep module grants the search service managed identity Cognitive Services User on aif-core-{suffix} from main.bicep. Query-time embeddings (the integrated vectorizer) and the agent's own model calls still go through APIM; both were verified working. 10-03 and 10-05 rerun end to end.
  • 08-10 and 08-10b container images failed to start when rebuilt, for two independent reasons: azure-ai-agentserver-invocations==1.0.0b4 declared azure-ai-agentserver-core>=2.0.0b4 with no upper bound and now resolves core 2.2.0b1, which removed CHAT_ISOLATION_KEY; and github-copilot-sdk>=0.2.0 now resolves 1.0.14, which removed SubprocessConfig. Pinned to azure-ai-agentserver-invocations==1.1.0 (caps core below 2.2.0) and github-copilot-sdk==0.3.0 (the version the deployed images run). Both notebooks rerun end to end on rebuilt images.
  • 13-guardrails: 13-01 left customBlocklists empty because attaching a blocklist used to break the Responses API with HTTP 500 on happy-path calls. That service bug is fixed, and the notebook had drifted from the live policy, so re-running it would have detached the blocklist, deleted the portal-added financial-data protection filters and cut the deployment from 1306K to 30K TPM. The RAI policy now attaches bank-demo-blocklist on prompt and completion, includes the financial-data filters, and the deployment cell keeps a higher existing capacity. 13-03 scores 24/24.
  • 07-model-inference/07-01: the committed outputs showed Teams Beta, Delta and Gamma failing with Connection 'core-<team>' not found. Their APIM connections had been deleted from the projects on aif-spoke-multi (the 05-04 redeploy troubleshooting cell removes them). Recreated with each team's foundry-gateway-<team> key and the notebook rerun, so all four teams answer through the gateway.