Skip to content

7.7.0

Latest

Choose a tag to compare

@thekevinm thekevinm released this 18 Aug 19:49
61004db

Overview

DreamFactory v7.7.0 introduces the Agent Control Plane — AI agents as first-class, human-owned API consumers with negotiated, role-scoped access — and moves the platform baseline to Laravel 13 / PHP 8.4+. The AI/agents family (df-ai, df-ai-chat, df-agents, df-alerts, df-schema-contracts) is a commercial capability available; the MCP server remains open source. The release adds two new packages (Alerts, Schema Contracts), a major admin UI overhaul, platform-wide request tracing, a rebuilt BigQuery connector with full write support, OIDC group-to-role mapping, and significant security hardening across the AI gateway, chat, and agent surfaces.

New Features

Agent Access Negotiation (df-agents) — COMMERCIAL — NEW PACKAGE

  • AI agents become governed API consumers: each agent is bonded to a human owner, scoped by an existing DreamFactory Role, and authenticates with a short-lived API key.
  • Agent registry — GET/POST/PATCH/DELETE /api/v2/agents/agents for agent CRUD (name, owner, role, TTL, skills, chat service); creating an agent auto-mints its API key. The whole management surface is sysadmin-only.
  • Short-lived keys — key_ttl_hours (default 4h, 1–24h) enforced on every API request; expired keys are rejected with 401 until re-approved or rotated. Non-agent keys are unaffected.
  • Access-request queue — agents file requests for access they don't have (POST /api/v2/agent/request_access, or the request_access MCP tool); admins one-click approve/deny via /api/v2/agents/requests. Approvals grant the requested verbs on the agent's role, down to table-scoped targets (mysql/_table/orders), effective immediately.
  • Agent self-service — GET /api/v2/agent/catalog answers "what can I access?" with the agent's own role-filtered service catalog.
  • Agent-to-agent brokering — a domain-owning agent can review and resolve requests into services its own role covers (/api/v2/agent/inbox, /api/v2/agent/resolve), so routine grants don't escalate to a human. Brokered approvals are tagged and alerted.
  • Kill switch & sponsor rule — deactivating an agent immediately stops its key from authenticating; an agent may not outlive its owner (deactivating or deleting a user auto-suspends their agents, including via directory sync).
  • Activity ledger — one row per data-plane API operation: agent → owning user → role, verb, service, table, outcome, duration — joined on the platform trace id. Covers human/API traffic too.
  • Deterministic capability router — POST /api/v2/agents/route {"task": "..."} scores every active agent against a task description (+3 per matched skills keyword, +1 per name/description token) and returns the winner, its chat persona service, a match reason, and the scoreboard. No LLM call; never returns key material.

Alerts (df-alerts) — COMMERCIAL — NEW PACKAGE

  • Real-time alerting on system lifecycle events (role/service/user changes, agent activity) routed to Slack/webhook channels.
  • Database-backed alert rules with glob event patterns (system.role.*), severity, per-rule rate limiting, and {{variable}} message templates.
  • Default messages include acting-user attribution and field-level old → new diffs; secret-looking fields are masked as "changed".
  • Admin API backing the new Alerts UI: rules/channels CRUD, test delivery, delivery log (sent/failed/skipped/throttled with HTTP status), and an event catalog.
  • Works out of the box: seeds a Slack channel from DF_ALERTS_WEBHOOK_URL and default rules for role, service, and agent events.

Schema Contracts (df-schema-contracts) — COMMERCIAL — NEW PACKAGE

  • Versioned, hashed, immutable schema snapshots for SQL services, with drift detection and optional runtime enforcement. Opt-in per service; default behavior is unchanged.
  • Canonical, connector-neutral JSON schema shape validated against MySQL/MariaDB, PostgreSQL (multi-schema), Oracle, and SQLite.
  • Contract modes per service: none (default), auto (additive drift auto-promotes, breaking drift held), strict (all drift held for review).
  • Drift engine classifies changes into breaking / potentially breaking / additive / cosmetic, with per-table and service-wide diff reports.
  • Runtime enforcement — shape_response strips response fields not in the locked contract on every verb; strict additionally rejects writes referencing non-contract or read-only fields. Alias-aware, including embedded ?related= records.
  • Stable OpenAPI 3.0 schema generation from the locked contract; full system API under /api/v2/system/schema_contract; schema-contracts:describe and schema-contracts:prune artisan commands.

API Builder (df-api-builder) — OPEN SOURCE — NEW PACKAGE

Author custom REST endpoints atop existing DreamFactory services in a human-centric, script-free workspace; every custom API deploys as a standalone, first-class platform service.

  • Managed API lifecycle — moves through draft, published, and archived states; active APIs reside at /api/v2/{api_name}/ with native inheritance of RBAC roles, keys, logging, and platform-wide request tracing.
  • Deterministic execution plans — endpoints execute sequenced service_request steps against backing workspace resources; static admin-defined selectors ensure caller input only influences parameters and payloads, never the call target.
  • Declarative transform steps — no-code response reshaping (pick, rename, filter, sort, wrap/unwrap) replaces post-processing scripts for in-memory data manipulation between steps.
  • Workspace-scoped composition — APIs only orchestrate services explicitly granted to their workspace; inter-service links utilize platform-native db_virtual_relationships to prevent registry drift.
  • Granular test tracing — POST /api/v2/api-builder/test {"trace": true} delivers per-step telemetry (status, timing, output) to isolate failures, powering the new Admin UI Preview panel.
  • Automated OpenAPI documentation — dynamic spec generation via api-builder/docs ensures custom endpoints maintain parity with native service documentation.
  • Integrated designer service — a dedicated api_builder management surface is automatically provisioned to handle endpoint, relationship, and workspace configuration.

AI Gateway (df-ai) — COMMERCIAL

  • Google Gemini provider — full chat, tool-calling, streaming, embeddings, and model listing via Gemini's OpenAI-compatible surface.
  • Department chargeback — usage rollups now include by_department (requests, tokens, cost) using the directory department captured at LDAP/AD login; unmapped usage lands in "Unattributed" so totals always reconcile.
  • Langfuse audit sink — completions can be forwarded to a Langfuse instance as trace + generation observations (AI_INSIGHTS_* env vars); best-effort, never blocks responses.
  • Per-model rate table — cost tracking consults built-in per-model rates (e.g. per-model Claude pricing) before provider defaults; lookup order: per-model on the connection → per-service flat rates → built-in per-model → provider defaults.

AI Chat (df-ai-chat) — COMMERCIAL

  • Conversations run under the caller's own role by default; admins can still act-as an allowed role for down-scoping.
  • New mcp_servers scope on chat services and sessions — always intersected with the caller's role grants.
  • Data tools honestly track RBAC: allowed table names are baked into tool descriptions, and table-listing tools are withheld when a role grants only specific tables.
  • MCP tool schemas are sanitized so LLM providers accept tools from remote MCP servers.

Platform request tracing (df-core)

  • Every API request carries a trace id (accepted from a validated X-DreamFactory-Trace-Id header or minted server-side) and echoes it on the response. AI usage, prompt logs, MCP audit rows, and the agent activity ledger all join on the same id, so one agent action can be followed across every hop. A new AGENT requestor type lets role access rules and event scripts distinguish agent traffic from API and script traffic.

MCP Server (df-mcp-server) — OPEN SOURCE

  • Stateless daemon mode (MCP_STATELESS=true) — no session pinning, so multi-node deployments behind a load balancer work without sticky sessions; defaults unchanged when unset.
  • Session-authenticated /rpc bridge — first-party DreamFactory-session callers (e.g. AI Chat) reach the MCP daemon without an OAuth client, with role-scoped service lists resolved server-side and full audit parity with OAuth clients.
  • Agent negotiation tools — always-on discover_services and request_access MCP tools connect any MCP-capable agent to the access-negotiation flow.

Editions & packaging

  • Commercial editions: df-ai (AI Connections), df-ai-chat (AI Chat), df-agents (Agent Access Negotiation), df-alerts (Alerts), and df-schema-contracts (Schema Contracts) ship in the commercial composer builds only. They are not part of the open-source build.
  • df-mcp-server remains open source and ships in every build including OSS — MCP is protocol-level connectivity governed by the same roles and REST enforcement, not a tier differentiator.

Admin UI overhaul (df-admin-interface)

  • Design system — global theme-token layer with consistent light/dark theming, denser tables with hover-revealed actions, and shared primitives (stat tiles, badges, skeletons, toasts, empty states).
  • Command palette — Cmd/Ctrl-K jumps to real objects (services, roles, keys) and common actions; API Builder, Agents, and Alerts are nested under their pillar sections.
  • Home — dashboard-first redesign with quick actions, an activation checklist driven by live instance counts, and a deny-by-default governance metric (locked/scoped/open).
  • Services — Health column now probes the real backend connection; health issues appear on the service's own page; a Live API Card with working curl/JS/Python/MCP snippets sits on every service Overview; create an HTTP service by pasting a cURL command; service detail shows the effective request chain (scripts, limits, role filters) and a role-scope matrix.
  • API Docs — multi-column, spec-synced layout with a live request builder, a visual ?filter= builder, and cascading dropdown pickers for path tokens (tables, fields, procedures).
  • API Builder — restored and overhauled endpoint-authoring workspace: cross-service composition, relationship builder, contract mapper, step-by-step test trace, and duplicate-endpoint.
  • Agents & Alerts — new admin pages for the two new packages (agent CRUD, pending approvals, activity, kill switch, gateway log; alert rules/channels/log).
  • AI Chat — follow-up suggestion chips from a trailing ```suggestions fence, per-turn metering chips, grouped sessions, safe markdown rendering.
  • AI Usage — task-based what-if cost estimator: describe a task, get a calibrated low/high cost range and budget verdict, run it for real, and compare actuals. Plus spend-by-model breakdown and budget-vs-actual view.
  • Scheduler — run-as support: pick the API key and role a scheduled task executes under (pairs with df-scheduler below).
  • Logs — live tail mode in the log viewer.

Connectors & Services

  • BigQuery, rebuilt (df-bigquery) — full CRUD write support via parameterized DML (by filter or by id with id_field), fixed filtered/grouped/aggregated reads with real positional parameter binding, real BigQuery type normalization (Date/Time/Numeric/Geography → scalars), and exposure through MCP database discovery (service group moved to Database).
  • OIDC group-to-role mapping (df-oidc) — assign DreamFactory roles from the provider's groups claim with a per-group mapping table; priority: group mapping → role-per-app → default role. Azure AD group Object IDs supported (requires validated ID Token); the Azure groups-overage case falls back safely.
  • Scheduler run-as (df-scheduler) — scheduled tasks can execute under a real platform session (app + user), so RBAC roles and lookup keys apply; existing tasks are unchanged.
  • LDAP department capture (df-adldap) — AD/LDAP logins record the user's directory department to a side table, powering AI department chargeback. The directory stays the source of truth.
  • Git (df-git) — services no longer assume master: the repository's actual default branch is used when no branch is specified (GitHub/GitLab/Bitbucket).

Security Hardening

  • df-core: closed an authentication-bypass vector where any service named with an _oauth suffix skipped authentication via the OAuth-callback exemption; the exemption now also requires the service's type group to be OAuth/SSO.
  • df-ai (HIGH): per-service RBAC is now enforced on the OpenAI-compatible gateway (/api/v2/_ai/v1/*) — previously anonymous or under-privileged callers could enumerate model aliases and invoke any AI Connection. Chat completions now require POST on the resolved service; the model list is role-filtered.
  • df-ai-chat: privilege-escalation fix — non-admins can no longer select a foreign ai_role_id at session creation (sessions bind to the caller's login role); tool dispatch is gated by a hard allow-list so prompt-injected/hallucinated tool names are refused; session roles are re-validated on every message; restricted roles can no longer discover unauthorized table names.
  • df-agents: broker agents must hold every requested operation on every requested target before approving another agent's request, and applied grants are clamped to the broker's own permissions; every agent must reference an active human owner; the router never exposes key material.
  • df-oidc: ID Token signature validation now actually executes — the undeclared, abandoned namshi/jose dependency (which 500'd on instantiation) is replaced with firebase/php-jwt, preserving signature, algorithm-allowlist, issuer, audience, and expiry checks.
  • df-mcp-server: first-party /rpc bridge calls now write the same audit rows as OAuth clients — previously an invisible path in the audit ledger.
  • Installer: unattended installs no longer fall back to hardcoded default admin credentials — DF_ADMIN_EMAIL is required and a random password is generated (saved to /opt/dreamfactory/.admin_credentials, mode 600) when DF_ADMIN_PASSWORD is unset.
  • Admin UI: support-info clipboard export scrubs connection internals and prefers a least-privilege API key; AI chat renders model output through safe markdown to prevent script injection.

Fixes

Platform (df-core)

  • CORS preflight no longer fails or returns wrong headers: DreamFactory's database-backed CORS config is synced into Laravel's CORS layer, with case-insensitive and *-wildcard method matching.
  • df:setup fails fast with an actionable error in non-interactive runs instead of spinning at 100% CPU when the admin password is invalid.
  • Service create/update/delete/rename now reliably purge the service cache (post-commit), and config-schema caches invalidate automatically after migrations — no more stale service forms after upgrades.
  • Fixed a 500 on every request under a headless (path-less) app on file/database cache stores.

Databases

  • Oracle: fixed ORA-01861 on ISO datetime inserts into DATE columns; fixed ORA-01741 on paginated selects of columns with a SELECT db_function; yajra/laravel-oci8 widened for Laravel 11/12/13 hosts.
  • SQL Server: fixed stored procedure calls failing with SQLSTATE[07002] (regression from the May 2026 hardening).
  • MySQL-wire engines: RBAC component discovery no longer aborts wholesale on StarRocks/Doris (missing INFORMATION_SCHEMA.ROUTINES.DATA_TYPE); fixed 500 when filtering on a column with a FILTER db_function (e.g. case-insensitive search via upper()).

Scripting & SSO

  • Git-linked scripts fetch their content as a privileged internal request, so end users no longer need direct access to the source git service; failed fetches surface errors instead of silently running an empty (and cache-poisoning) script.
  • OIDC services instantiate correctly again (see security above).

AI

  • Anthropic provider omits temperature for models that reject it (fixes HTTP 400 on newer Claude models).
  • Test Connection / Get Models works on saved AI connections showing the masked API key.
  • PostgreSQL migration fix for chat sessions; long chat titles and long MCP tool names no longer 500/overflow.

MCP

  • Fixed migration failure on SQL Server (FK referential action guarded by driver).
  • Fixed argument-less tools/call requests failing on the session-authenticated /rpc bridge (empty {} was serialized as []).

Admin UI

  • Detail pages show record names (not ids) in breadcrumbs; side-nav active matching uses segment boundaries.
  • API Docs re-scopes on navigation (no more stale table pickers); swagger-ui pinned to 4.15.5.
  • Fixed a ~48px content overflow on every screen; screens fit 1080p at 100% zoom; app shell self-heals ChunkLoadError after deploys; index.html served no-cache.
  • Home KPI tiles no longer overlap the exposure strip; Schema Contracts dialogs themed in dark mode; reduced change-detection work across large tables and AI surfaces.

Platform & Compatibility

  • Laravel 13.25 / PHP 8.4 minimum (tested on PHP 8.5) via the Laravel 12.x and 13.x Shifts, including config/cache.php 'serializable_classes' => true (required — without it schema/script objects come back as __PHP_Incomplete_Class and every cached request 500s), and the PHP 8.5 \Pdo\Mysql SSL constant fix.
  • New cache stores (redis, dynamodb, octane, failover) and env-driven database/Redis tuning (DB_URL, DB_SOCKET, Redis retry/backoff).
  • Linux installer overhaul: stateful resume of interrupted installs, disk-space preflight, opt-in advanced connectors, full Oracle Linux support incl. SELinux, and a refreshed OS matrix (EL 8/9/10, Debian 12/13, Fedora 39+, Ubuntu 22/24).

Upgrade Notes

  • Platform baseline change: Laravel 13.25 / PHP 8.4+ — upgrade PHP before moving to 7.7.0. Use the shipped composer.lock; df-* packages are pinned by ref.
  • Run database migrations — 7.7.0 adds tables/columns across df-agents (4 objects), df-alerts (3 tables), df-schema-contracts (2 tables), df-ai-chat (mcp_servers, wider columns), df-adldap (user_department), df-compliance (nullable service_report.service_name), df-oidc (group mapping), and df-scheduler (run-as columns).
  • Cache changes: default cache store is now database when CACHE_STORE is unset (set CACHE_STORE=file to keep the old behavior); the cache key prefix changed, effectively invalidating existing entries; custom config/cache.php overrides must carry 'serializable_classes' => true.
  • AI gateway RBAC: previously-working unauthenticated or under-privileged calls to /api/v2/_ai/v1/* now return 403 / filtered model lists — intended result of the security fix. Grant the calling role access to the AI Connection service.
  • AI chat role binding: non-admin ai_role_id at session creation is now ignored; sessions use the caller's login role.
  • Agent keys expire (default 4h TTL): long-running integrations must handle 401 re-approval; backing apps named agent:{id} appear in the API Keys list and should not be edited directly. Offboarding a user suspends the agents they own.
  • Admin UI: navigation moved (API Builder/Agents/Alerts nested; Logs under System); the services Health column now reflects real connection probes only; cached clients may see one automatic reload after upgrade.
  • df:setup non-interactive: passwords under 16 characters now fail fast with exit 1 — update automation to pass a 16+ character --admin_password.
  • Git services: requests omitting branch now target the repo's default branch, not master.
  • All API responses include X-DreamFactory-Trace-Id — account for it in strict response-header allowlists.
  • BigQuery: id-based operations require id_field; DML writes require a billing-enabled GCP project; the service type now lists under Database.
  • Installer OS support dropped for CentOS/RHEL/OEL 7, Debian 10/11, Fedora 36/37, Ubuntu 20. Unattended installs require DF_ADMIN_EMAIL (see Security).
  • Editions: the AI/agents packages install only from the commercial composer bundles — open-source installs are unaffected by (and cannot enable) the AI features.
  • Security fixes are cumulative — upgrade promptly if you expose the AI gateway, MCP bridge, OIDC SSO, or agent surfaces.