EntAssistant

An agentic enterprise assistant for the fictional client Acme Operations. Internal sales / support / ops staff ask operational questions in natural language — e.g. "Show me open issues for Umbrella, summarise the latest status, and suggest the next action" — and an LLM agent reasons about which tools to call, queries the operational database, and returns a grounded, auditable answer under role-based access control.

Built with Google ADK (Agent Development Kit), the MCP Toolbox for Databases as the MCP server, Gemini 2.5 Flash, Keycloak for auth/RBAC, PostgreSQL + Redis for state, and OpenTelemetry → Arize Phoenix for observability. Everything runs locally via Docker Compose.

Quickstart

cp .env.example .env            # then set GOOGLE_API_KEY (Google AI Studio key)
docker compose up --build       # builds the app, starts all 6 services,
                                # runs migrations + seed automatically
open http://localhost:8080      # chat UI — log in via Keycloak

One command. The app container applies Alembic migrations and seeds the database on start, so the stack is ready once health checks pass (~1-2 min on first build).

Demo users (realm acme, password password):

User	Role	Can
sales@acme.test	sales_user	read customers + issues
support@acme.test	support_user	read + add issue updates
admin@acme.test	admin	everything, incl. creating next actions

Other URLs: Keycloak http://localhost:8081 · Phoenix traces http://localhost:6006. A UI design showcase of every state (conversation, escalation risk, missing-information, RBAC denial, rate-limit) with sample data is at http://localhost:8080/static/demo.html.

What it does (mapped to the brief)

Requirement	Implementation
LLM agent reasons about tools	ADK LlmAgent (Gemini 2.5 Flash) — tool choice is model-driven, never hard-coded
4+ tools	search_customers, get_open_issues, get_issue_updates, create_next_action (+ add_issue_update)
≥1 MCP server	MCP Toolbox for Databases exposes the tools over /mcp; the agent consumes them via ADK MCPToolset
≥1 reusable Skill	Customer Escalation Summary — an ADK SequentialAgent with typed output
Keycloak auth + RBAC	Real realm; JWT verified via JWKS; sales_user / support_user / admin enforced at API + per tool
PostgreSQL	customers, issues, issue_updates, next_actions, app_users + seed
Redis	Cache-aside for customer lookups + per-session working context
Docker Compose	docker compose up starts app, postgres, redis, keycloak, toolbox, phoenix
Evaluation	eval/eval_set.yaml + eval/run_eval.py → eval/results/RESULTS.md
Observability	Structured JSON logs, request traces, tool-call logs, latency, error logs
OpenTelemetry + Arize Phoenix trace UI

Architecture diagram and request sequence: docs/architecture.md.

Why MCP

The five tools are declared as parameterised SQL in toolbox/tools.yaml, served by the MCP Toolbox for Databases. The agent talks to Toolbox over the standard MCP JSON-RPC endpoint (/mcp). This matters because:

• Tool definitions are decoupled from agent logic. Adding or changing a tool is a YAML edit with zero Python changes; the agent rediscovers tools over MCP.
• The same server is reusable by any MCP client (Claude Desktop, other agents), not just this app.
• Safety by construction — every tool is a parameterised statement, so there is no SQL injection surface.

The Skill: Customer Escalation Summary

A reusable, structured workflow (distinct from a one-off prompt), in app/skills/escalation.py. It is an ADK SequentialAgent:

1. gather — an LlmAgent with the read tools collects the customer profile, open issues and recent history.
2. summarise — an LlmAgent with a strict output_schema emits a typed EscalationSummary: executive summary, risk level (Low/Medium/High/Critical), recommended next action, and missing information.

SequentialAgent is @deprecated in ADK 2.5.0 (in favour of the not-yet-public Workflow, which cannot yet be an LlmAgent sub-agent), so IDEs strike it through. It is used deliberately and google-adk is pinned <3; see app/skills/escalation.py.

It is exposed to the root agent as a single tool, customer_escalation_summary. The sparse demo customer Hooli has no issues and no profile fields, so the skill reports the gaps as missing information rather than inventing data.

Auth & RBAC

Keycloak is the identity provider (realm imported from keycloak/import/acme-realm.json). The browser logs in with OIDC Authorization Code + PKCE; the app validates the bearer JWT (RS256 signature via JWKS, issuer, audience, expiry) and extracts realm roles.

RBAC is enforced in two layers:

• API layer — every /api/chat call requires a valid token.
• Tool layer (the demonstrable control) — roles are injected into ADK session state and a before_tool_callback (app/rbac.py) checks each tool against TOOL_ROLE_MAP. A denied call returns a structured permission_denied the agent explains, and no database write happens.

Tool	sales_user	support_user	admin
read tools (search_customers, get_open_issues, get_issue_updates, customer_escalation_summary)	✓	✓	✓
add_issue_update	✗	✓	✓
create_next_action	✗	✗	✓

Trade-offs

Redis vs PostgreSQL. Postgres is the durable, relational, auditable system of record - including the agent's conversation/event history (ADK DatabaseSessionService). Redis holds only ephemeral, rebuildable data: hot search_customers results (cache-aside, TTL) and per-session working context. A short TTL bounds staleness, and a Redis outage degrades to Postgres — the request still succeeds (verified: with Redis stopped, /readyz reports redis: error but /api/chat still answers).

"Runs entirely locally" vs Gemini. Every service runs in local Docker; only LLM inference calls Google (Gemini via an API key). This is a deliberate choice for reliable, native ADK tool-calling. Swapping to a fully-offline local model is a one-line change — wrap the model in ADK's LiteLlm (e.g. LiteLlm(model="ollama_chat/qwen2.5")) and add an ollama service — at the cost of weaker tool-calling reliability.

Observability

Structured JSON logs to stdout (tool-call logs, request/response traces with duration_ms and trace_id, error logs). OpenTelemetry spans are exported to Arize Phoenix (http://localhost:6006) via OpenInference's ADK instrumentation, giving per-request agent / LLM / tool traces with latency. Tracing is best-effort: if Phoenix is down, requests still succeed. Every response includes a trace_id to correlate with Phoenix.

Evaluation

10 cases across the three roles cover tool selection, grounding, RBAC (allow + deny with a verified no-mutation check), the escalation skill, sparse-data handling, and graceful handling of an unknown customer. Run against the live stack:

make eval        # POSTGRES_HOST=localhost uv run python -m eval.run_eval

Latest results (10/10): eval/results/RESULTS.md.

Testing

make check       # ruff (lint+format) + mypy (strict) + offline tests + coverage gate
make test-live   # live tests against the running stack (needs GOOGLE_API_KEY)

• Offline (pytest -m "not live") — auth, RBAC, memory, schemas, config. ~95% coverage on these unit-testable modules. No cloud, no containers; runs in CI.
• Live (-m live) — auth against real Keycloak, cache-aside against real Redis, and the full API per role. The eval set is the systematic live agent check.

The integration modules (agent, runtime, routes, caching, escalation) are verified by the live suite + the eval rather than by offline unit coverage.

Project structure

app/            FastAPI app, ADK agent, RBAC, auth, caching, memory, observability, routes
app/skills/     Customer Escalation Summary skill
toolbox/        MCP Toolbox tools.yaml (the tool surface)
keycloak/       realm export (roles, clients, demo users)
db/             SQLAlchemy models + idempotent seed
migrations/     Alembic migrations
eval/           evaluation set + runner + results
tests/          unit/ (offline) + integration/ (live)
docs/           architecture diagram + screenshots

AI tool usage

This project was built with an AI coding assistant (Claude). The workflow was test-first for the pure-logic modules (auth, RBAC, memory, schemas: red → green → refactor), and verify-as-you-go for the integration surface — every claim in this README was checked against a running system (real Keycloak tokens, real MCP tool calls, real Phoenix traces, a real browser login), not assumed. API surfaces for the newer libraries (ADK 2.5, MCP Toolbox, Phoenix) were introspected against the installed packages before use.

Path to production (deliberately out of scope)

The bar here is a complete, hardened prototype — not operational scale. Out of scope, and how they'd be added:

• Secrets → external manager (Vault / cloud KMS) instead of .env.
• TLS → terminate at an ingress / reverse proxy; Keycloak in production hostname mode.
• created_by integrity → bind it server-side to the verified token via Toolbox authenticated parameters instead of the agent supplying it.
• Scale/HA → k8s + horizontal replicas; managed Postgres/Redis; a migration job instead of the app entrypoint.
• CI/CD → the included GitHub Actions runs lint + type + tests + docker build; extend to deploy.

Further enhancements

Natural next steps that extend the existing architecture (ADK skills + tools, the MCP Toolbox boundary, the eval harness, Phoenix tracing) rather than bolting on new systems — each is a new skill, a tools.yaml entry, an eval upgrade, or a new surface over traces we already emit:

• More reusable Skills — Root-cause analysis. A second structured skill (an ADK SequentialAgent / Workflow, like Customer Escalation Summary) that scans historical ticketing data and product logs to diagnose the underlying system failure behind a cluster of issues — turning "many tickets" into one root cause, not just another summary.
• More tools — Automated CRM sync. A write-capable tool declared in toolbox/tools.yaml and gated by the same before_tool_callback RBAC (admin-scoped) that pushes profile updates, case notes and status changes back to the CRM so records stay consistent across platforms in one step — added with zero agent-code change, which is the whole point of the MCP boundary.
• Continuous, rubric-based evaluation. Grow eval/ from pass/fail into a scored rubric and close the loop: an optimiser that detects regressions and proposes concrete prompt/tool changes (a self-improving loop) instead of leaving every regression to be chased by hand.
• Voice I/O. ADK's streaming / multimodal support makes speech in (dictate a question) and out (spoken answers) an additive channel alongside typing — useful for hands-busy ops and support staff already on a call.
• Agent Trajectories. Promote the full agent trajectory (the ordered tool calls, arguments and intermediate state — already captured as Phoenix / OpenInference spans) into a first-class explainability view and score trajectories in eval: deeper "show your work" than the per-reply receipt strip already gives.

Built from official, well-trodden integrations, with every graded requirement verified against a running system and captured (eval results, Phoenix traces, screenshots). LLM tool-selection has inherent non-determinism (mitigated by crisp tool descriptions + low temperature + tolerant eval scoring), and the live eval depends on a valid GOOGLE_API_KEY.

License

Released under the MIT License.