-
Notifications
You must be signed in to change notification settings - Fork 0
Web Chat Pilot
title: Web Chat Pilot description: Verified single-tenant web frontend for the staging Foundry hosted agent, with identity, flows, deployment and operating limits. ms.date: 2026-09-09
Open the web chatbot
and sign in with an approved member of Foundry Threat Assessment Pilot in
MngEnvMCAP675646.onmicrosoft.com. The tenant administrator is the initial approved member.
The web app invokes the existing staging threat-assessment-agent; there is no production
web frontend and no Teams app deployed.
Try this synthetic fixture:
Assess device ID: device-001 and list its vulnerabilities.
The pilot administrator supplied a screenshot of a successful signed-in report for
device-001 on 2026-09-08. The report includes the mocked device attributes, risk and
vulnerability data. These are synthetic fixtures, not live Air Canada Defender data.
| Link | Purpose |
|---|---|
| Chatbot | Pilot-member chat experience |
| Health | Process health, not downstream readiness |
| Web build runs | Backend tests, frontend tests and compiled artifacts |
| Source and README | Implementation and entry points |
| Agent architecture | Supervisor, specialists, model and tool isolation |
| Test trends | Hosted-agent evaluations and bounded load tests |
flowchart TB
subgraph Client[User device]
Browser[React UI and MSAL]
end
Entra[Microsoft Entra ID - approved tenant]
Browser <-->|Authorization code with PKCE| Entra
subgraph Web[Separate staging web Container App]
Static[Vite static assets]
API[FastAPI - one Uvicorn worker]
Gate[JWT, scope, group and ownership checks]
Memory[In-memory conversation store]
MI[Dedicated user-assigned managed identity]
API --> Gate --> Memory
API --> MI
end
Browser -->|Public HTTPS| Static
Browser -->|Same-origin API with user access token| API
subgraph Foundry[Existing staging Foundry project]
Hosted[Hosted Responses endpoint]
Graph[LangGraph supervisor and specialists]
Model[gpt-4o-mini]
Toolbox[security-tools toolbox]
Hosted --> Graph
Graph --> Model
Graph --> Toolbox
end
MI -->|Foundry project-scoped role| Hosted
Toolbox --> Defender[Separate staging Defender MCP app]
Toolbox --> Anomaly[Separate staging anomaly MCP app]
Hosted -->|Validated completed assistant text| API
API -->|SSE status, answer and done| Browser
The web app shares mcp-staging-mcp-env with the staging MCP apps, but has its own
container, identity and role assignments. Deploying it does not deploy the hosted agent
or either MCP server. The web app uses the staging agent's routed endpoint, not a pinned
agent version; a later staging agent release can change the behavior behind this URL.
Public HTTPS ingress is not network-private access. Static files, /healthz and
/api/config are anonymous. Every chat API request requires a valid pilot-user token.
The browser sees only public configuration and user tokens, never the backend's managed
identity token. MCP public endpoints contain synthetic fixtures and retain their existing
PoC security boundary; do not place real security telemetry there without hardening them.
sequenceDiagram
actor User as Pilot user
participant UI as Browser / MSAL
participant ID as Entra ID
participant API as FastAPI
User->>UI: Sign in with Microsoft
UI->>ID: Tenant-specific authorization code + PKCE
ID->>ID: Authenticate user and enforce enterprise app assignment
ID-->>UI: Authorization code
UI->>ID: Redeem code with verifier
ID-->>UI: Delegated Chat.Access token
UI->>API: GET /api/me with Bearer token
API->>API: Verify RS256 signature, issuer, audience and token times
API->>API: Verify tenant, authorized client, scope and pilot group
alt Authorized pilot member
API-->>UI: 200 and user object ID
else Missing, invalid or unauthorized token
API-->>UI: 401 or 403, no agent call
end
| Configuration | Value |
|---|---|
| Tenant ID | aa93b9d9-037d-4f08-a26d-783cff0e2369 |
| App registration | Foundry Threat Assessment Web Chat |
| Client ID / API audience | 9cfb9dc7-f433-47f6-826b-14bc90a817bc |
| Enterprise service principal | a1fbdafa-aed0-4f47-84e0-e7c27b646a71 |
| Pilot security group | 201b962a-8619-401e-a1f0-733bca2cd7b2 |
| Delegated scope | api://9cfb9dc7-f433-47f6-826b-14bc90a817bc/Chat.Access |
| App role assigned to group | Pilot.User |
| Allowed SPA callbacks | Deployed chatbot origin and http://localhost:8000
|
| Web managed-identity client ID | 18de54ce-cafc-43e7-8029-cc0093bfdcf4 |
| Web managed-identity principal ID | f5c5b8e9-dfa5-43ee-aae5-c45f6cc75eac |
The registration is single-tenant, issues v2 access tokens and includes security-group
claims. Enterprise app assignment is required. Admin consent grants the delegated scope,
but does not grant every tenant user access: group assignment and backend group validation
remain required. The backend checks tid, azp, scp, groups and oid as well as
signature, issuer, audience and validity times. It fails closed on group-overage tokens;
there is no Graph fallback to expand memberships.
Pilot onboarding is an administrator operation: verify the user's tenant and approval, add them as a direct member of the pilot security group, and have them sign out and back in to obtain a fresh token. Confirm app-assignment licensing and tenant policies before widening the group. Do not add individual app assignments as a bypass for the backend group check. Remove group membership to end access. Existing access tokens can remain usable until expiry; this pilot does not implement continuous access evaluation or immediate revocation.
The web identity has AcrPull on acraircanadapoc001 and Foundry User (role
53ca6127-db72-4b80-b1b0-d745d6d5456d) on the staging Foundry project. This role
is broader than an invocation-only custom role. Pilot users do
not need direct Foundry RBAC. The identity requests https://ai.azure.com/.default.
sequenceDiagram
participant UI as React chat
participant API as Authenticated FastAPI
participant Store as In-memory sessions
participant MI as Managed identity
participant Agent as Staging Responses endpoint
UI->>API: POST /api/conversations
API->>Store: Create ID owned by tenant ID + object ID
API-->>UI: 201 conversation ID
UI->>API: POST message text + Bearer token + Idempotency-Key
API->>Store: Verify owner, TTL, turn limit and not busy
Note over API,Store: Matching completed key replays the answer without another agent call
API->>MI: Acquire ai.azure.com token
API->>Agent: Full server-owned history + new user text, stream=true, store=false
API-->>UI: SSE status and keepalive while processing
Agent->>Agent: Supervisor, tools, analysis and report composition
Note over Agent: All specialists receive user/assistant conversation context
Agent-->>API: Responses stream with completed assistant output
alt Successful validated completion
API->>Store: Append successful user and assistant turn
API-->>UI: Answer then done
else Failure, incomplete stream or cancellation
API->>Store: Release busy slot, no failed turn appended
API-->>UI: Safe error with request ID when still connected
end
The browser cannot submit its own history. Conversation IDs are checked against the
signed-in tenant and user on every mutation. Unknown, expired and other-user sessions
return 404. The app strips tool receipts and runtime evidence metadata from browser
responses, accepting only completed assistant output_text. It does not independently
fact-check that text; verification of recommendations remains required.
The browser sends a UUID Idempotency-Key and retains it when retrying the same text in
the same conversation before receiving an answer. A completed key replays its stored
answer without appending a turn or invoking Foundry. Reusing it with changed text returns
409; another user still receives 404. Failed requests are not cached. Keys and answers
expire with the in-memory conversation, and restarting the service removes them. Requests
without a key and creation of new conversations are not deduplicated. This is not an
exactly-once guarantee for remote compute after cancellation or a failed stream.
Deterministic specialist lookups require explicit user fields: device ID: device-001,
account ID: jsmith, failed_logins_per_hour=10, or data_egress_mb_per_hour=900.
Each device triggers both Defender lookups. Account IDs trigger login analysis; supported
numeric metrics trigger scoring. No account is inferred from a device or assistant text.
Later user fields replace earlier fields of the same category; omitted categories retain
their earlier values. Free-form names without labels and percentage values are not supported
lookup inputs. These are read-only calls; model output only synthesizes their results.
The app requests store:false and maintains history in backend memory. This is not a
guarantee about Azure service logging, retention or compliance policies. App logs record
request IDs, outcome and duration, not prompt bodies or bearer tokens. MSAL uses browser
session storage; visible messages and conversation lists remain in React memory. There
is no transcript export, durable history, shared database or cross-device resumption.
| Limit | Current behavior |
|---|---|
| Backend topology | One replica, one worker; 0.5 vCPU and 1 GiB |
| Conversation lifetime | 3,600 seconds idle; all sessions lost on backend restart |
| Capacity | 128 total sessions, five per user, four concurrent agent calls |
| Turn and input limits | 20 successful turns; 8,000 characters per user message |
| Response limits | Bounded stream and 64,000-character completed answer |
| Progress | SSE status and 10-second keepalives, then final answer; no token deltas |
| Reset and stop | New/delete conversation; Stop cancels the frontend request and local upstream task |
Cancellation is not a guarantee that remote compute stops immediately. Browser reload loses its conversation list while server sessions remain until deletion, expiry or restart. Scaling out or adding workers would break session routing and per-process limits; first implement a shared owner-bound store and distributed concurrency control. Rolling updates can invalidate in-flight conversations. Use a maintenance notice and ask users to start fresh.
flowchart LR
Source[Reviewed source and lockfile] --> CI[GitHub-hosted backend tests and npm ci]
CI --> Artifact[Frontend artifact and test evidence]
Source --> ACR[Operator starts ACR remote Docker build]
ACR --> Test[Frontend tests and real credential startup check]
Test --> Digest[Immutable image digest]
Digest --> Bicep[Apply web-chat.bicep only]
Admin[Approved Entra administrator] --> Identity[Configure group, app, consent and callback]
Identity --> Verify[Live health, anonymous denial and pilot sign-in]
Bicep --> Verify
Verify --> Chat[Two-turn synthetic conversation]
Verify -->|Failure| Recover[Redeploy previous known-good image digest]
Web Chat Build runs on scoped pushes, pull requests and manual dispatch. Backend and frontend tests produce JUnit and categorized summary counts; frontend stream tests use Node's test runner. Node 22 and Python 3.13 run on hosted Linux. Successful runs retain compiled assets and the lockfile for 14 days. This workflow has no Azure credentials and does not deploy. Local corporate npm restrictions remain unchanged. Web frontend results are separate from the agent-evaluation trend charts.
The web build, continuous validation, both protected release entry points and trend-publishing summaries show the same clickable deployment inventory, including the chatbot and Azure resource links. Authenticated Responses API links are labeled as POST APIs, not browser experiences. A link points to an existing environment; it does not prove that a run deployed or tested it.
Use an approved Azure CLI session in the tenant and subscription, with access to ACR Tasks, Container Apps, managed identities and the relevant role assignments. The identity setup script additionally requires Entra permissions to manage apps, service principals, group assignments and delegated admin consent. These are operator permissions, not runtime roles. The pilot group must already exist. Do not run tenant-admin setup as part of ordinary builds.
Commit and review source first. From the repository root, build remotely and pin the returned digest; this avoids local npm and local ARM64 container-image compatibility problems:
$resourceGroup = 'rg-air-canada-threat-assessment-poc'
$tag = 'pilot-' + (Get-Date -Format 'yyyyMMddHHmmss')
$build = az acr build --registry acraircanadapoc001 --image "foundry-web-chat:$tag" --file apps/web-chat/Dockerfile apps/web-chat --no-logs -o json | ConvertFrom-Json
if ($LASTEXITCODE -ne 0 -or $build.status -ne 'Succeeded') { throw 'Remote build failed' }
$digest = $build.outputImages[0].digest
if ($digest -notmatch '^sha256:[a-f0-9]{64}$') { throw 'Missing image digest' }
$image = "acraircanadapoc001.azurecr.io/foundry-web-chat@$digest"
az deployment group create --name web-chat-pilot --resource-group $resourceGroup --template-file infra/web-chat.bicep --parameters image=$image location=eastus2
if ($LASTEXITCODE -ne 0) { throw 'Web deployment failed' }For initial provisioning or a changed callback, an approved administrator runs:
./scripts/setup-web-chat-identity.ps1 -RedirectUri https://foundry-threat-chat-staging.purpletree-432267ca.eastus2.azurecontainerapps.ioThe script manages the pilot app's scopes and roles; do not reuse it for an unrelated app
with the same display name. Its default localhost callback remains registered for development.
infra/web-chat.bicep references the existing staging project, registry and Container Apps
environment. Use location=eastus2 explicitly for this deployment. None of these operations
requires redeploying the hosted agent or changing production resources.
$base = 'https://foundry-threat-chat-staging.purpletree-432267ca.eastus2.azurecontainerapps.io'
Invoke-RestMethod "$base/healthz"
(Invoke-WebRequest "$base/api/me" -SkipHttpErrorCheck).StatusCode
az containerapp logs show --name foundry-threat-chat-staging --resource-group rg-air-canada-threat-assessment-poc --type console --tail 60Expect status: ok and anonymous HTTP 401. Confirm /api/config contains only tenant ID,
client ID, scope and environment. Then sign in as a pilot member, submit a synthetic fixture,
and ask a follow-up in the same conversation. Health alone does not test Entra, Foundry,
model quota, tool access or completed responses. Never include tokens or prompt bodies in
diagnostic output.
Before an update, record the current image digest. On failure, redeploy that known-good digest through the same Bicep command and repeat live checks. This is image recovery, not automatic rollback; sessions are not restored. Do not change the hosted-agent version to recover a frontend-only fault.
| Symptom | Check |
|---|---|
| Gateway timeout after deployment | Inspect console logs and replica readiness; ARM success is not proof of process health |
aiohttp package is not installed |
Async Azure Identity transport dependency was missing; corrected requirements and Docker startup check catch this |
| Sign-in blocked | Correct tenant, direct pilot membership, enterprise app assignment and registered SPA callback |
| API 401 | Missing, expired or invalid token; sign out and back in |
| API 403 | Tenant/client/scope/group mismatch or group-overage claim; do not weaken validation |
| Safe agent error | Correlate request ID, verify web identity's project role and downstream health; no raw error disclosure |
| Conversation 404/409/429 | Expiry or ownership, busy/turn limit, or pilot capacity; start fresh or close idle conversations |
On 2026-09-09, retry handling from commit f334c61 passed 25 backend tests and the browser
request-key regression. Web build 34298676791
passed the hosted test and frontend compilation workflow. ACR build chk succeeded;
revision foundry-threat-chat-staging--0000002 is healthy and active with image
acraircanadapoc001.azurecr.io/foundry-web-chat@sha256:194726db96b42cb3d0111cd3dd28e0fc044c099293532a6e2e9a4a4d88764c8c.
Live health returned OK and anonymous /api/me returned 401. The signed-in pilot user
completed a fresh assessment and recalled PILOT-73921 in a follow-up without repeating it.
An independent conversation did not contain that reference. Two completed-message retries
returned HTTP 200 and identical SSE bodies; changed text with the same key returned 409.
The three replay requests completed in 138 ms. These checks used the routed staging agent;
the full release gate remains separate evidence.
As verified on 2026-09-08:
- ACR build
chesucceeded with the real managed-identity construction check in a clean image. - Image:
acraircanadapoc001.azurecr.io/foundry-web-chat@sha256:b9fc609c040efc4cf4557bcb43dbf853c1ce68e0396aa5113b9cd4573dffdb4c. - Ready revision:
foundry-threat-chat-staging--0000001. - Public health returned OK; anonymous
/api/mereturned 401. - All 23 backend authorization/session/startup tests passed locally. Three frontend stream tests and Vite compilation passed in hosted build 34284261914.
- Pilot administrator sign-in and a synthetic
device-001report succeeded, as shown in the user's supplied browser screenshot. That hosted build predates the startup fix; the clean ACR build is the dependency-completeness evidence.
The first remote image built successfully but crashed because aiohttp was absent.
Mocked unit tests did not construct Azure Identity's real async transport. The fix adds
the dependency, a Docker build-time managed-identity construction check, and a real-client
FastAPI lifespan regression. No credential acquisition is needed for the startup regression.
Follow-up hosted verification completed on 2026-09-09 UTC (September 8 local time):
Web Chat Build 34293194567
passed on source commit 6080fd402b7d956af9fd06af30785b5d47695676, with 23 backend tests,
three frontend stream tests and successful Vite compilation. Its rendered Actions summary
was checked in the browser: all 15 expected deployment/documentation links were clickable,
including the staging chatbot, health endpoint, Azure resources, both Foundry projects and
four MCP endpoints. The local combined backend/reporting suite passed all 40 tests. This
build-only run did not deploy a new image or change the hosted agent.
No sustained web load, independent live non-member account, formal accessibility audit, immediate token revocation, durable session storage or enterprise production certification is claimed. Automated negative JWT and cross-user ownership tests cover the local access rules.
A live two-turn browser check failed: the first request supplied PILOT-4827, and the
first answer included it. The second request asked for the earlier reference without
repeating it. The agent incorrectly reported that no reference had been supplied.
Both calls completed, so this is not an authentication or transport failure.
The graph previously selected only its last message. Commit 3eb4295 changed all three
specialists to receive normalized user/assistant history, with prior turns identified as
untrusted context. Staging version 7 passed all three hosted conversation checks in
run 34294086899.
That run stopped on a separate missing Defender receipt in the conflicting-signals fixture;
it did not promote to production.
Commit 40349f0 executes explicit read-only lookup plans before model synthesis, preserves
actual tool receipts, normalizes report separators and closes model clients per invocation.
Six live repeated fixture tests passed. The browser also exposed a false model claim that
no synthetic data was used; commit e96523a supplies trusted fixture provenance and an
application-generated disclosure when tool receipts are present. The final offline suite
passed 172 tests (six opt-in live tests skipped).
Release 34298522363
passed the compatibility entry point's lint, infrastructure, staging deployment and smoke
gates. It was cancelled during evaluation because the provenance correction superseded it;
the older candidate was not promoted to production.
The corrected candidate deployed as staging version 9 in
release 34299542952.
Its smoke test and all three conversation checks passed. An authenticated browser
assessment against the updated route displayed the deterministic synthetic-data notice.
That version passed all 21 model-judge checks but failed the exact citation policy
because the report paraphrased crew-scheduling as crew scheduling. Production
was correctly skipped. Commit 4b6f8d2 retains omitted punctuated user references
in a labeled unverified section, excluding assistant-only identifiers. The offline
suite then passed 173 tests, with six opt-in live tests skipped.
Release 34300982257 evaluated staging version 10, passed all eight captures, 21 judge checks and zero policy failures, and promoted production from 34 to 35 with successful monitoring. Compatibility release 34302149559 repeated all gates successfully and retained active production version 35. These agent releases did not change the web container image.
Both release entry points now use the same protected workflow and shared deployment queue. Repeated provisioning targets existing resources; the platform can create or reuse immutable agent versions. Tool plans and completed-message replay are repeatable; model wording, latency and telemetry are not. Missing evidence is never proof of safety.
flowchart LR
Current[Verified standalone web pilot] --> Choice{Teams experience}
Choice --> Tab[Teams tab with approved manifest and Teams authentication]
Choice --> Bot[Native bot using Microsoft 365 Agents SDK]
Tab --> Gate[Preserve tenant, group and user ownership checks]
Bot --> Gate
Gate --> Backend[Authenticated service and managed-identity Foundry invocation]
A Teams tab is not a drop-in iframe: the current CSP sets frame-ancestors 'none' and the
app uses a standalone redirect flow. A tab requires an approved Teams manifest, precise
frame-origin allowlists and Teams-compatible authentication. A native chat integration
requires its own adapter, app registration, channel authentication and conversation mapping.
Neither option should forward unauthenticated activity or trust a supplied user ID.
Before broadening the pilot, decide retention and transcript handling, shared session storage, network isolation, invocation-only least privilege, rate limiting, quotas and cost alerts, user offboarding, audit requirements, accessibility, mobile support and operational ownership. Replace mocked MCP fixtures only after the real data access, privacy, security and approval model is established. Keep production promotion separate.