Release 0.17.0

XMC Release 0.17.0

Release Version Determination: Minor bump (0.16.0 -> 0.17.0).
Rationale: Promotes Log Monitor (log_monitor_deployment_enabled: true / logMonitorDeployment.enabled: true) to enabled out-of-the-box by default across Cluster Toolkit and Helm deployments, adds database schema version health checks and db-config visibility in the Admin Panel, hardens XID ACL principal normalization and fail-closed production authentication guards, raises the minimum Python version for SDKs to Python 3.11+ (xmanager >= 0.8.5.post1), delivers the Pub/Sub Metrics Ingester and BigQuery DataEngine support for Dashboard Service, and completes the experimental PostgreSQL repository layer and database-agnostic Components dbstore.

---

🌟 1. Public Changes (Available Out-of-the-Box)

🚀 Major Highlights & Promoted Capabilities

• [Promoted to Default] Log Monitor GKE Deployment (log_monitor_deployment_enabled: true): The Log Monitor service is now enabled out-of-the-box by default in deploy/ctk_v2/xmanager-cloud.yaml (deploy/ctk_v2/xmc-deployment-template.yaml) and deploy/helm/values.yaml (logMonitorDeployment.enabled: true), automatically scanning Cloud Logging for workload failures and surfacing root-cause status messages on experiments and work units (#1494).
• [Action Required / Breaking] Python 3.11+ Minimum Requirement for SDKs & xmanager >= 0.8.5.post1 Default: Raised requires-python across the xmc and xdash Python SDKs to >=3.11 (#1452), bumped the default xmanager client version requirement in deploy/cli/install_xmanager.py and all example workloads to >=0.8.5.post1 (#1400), and updated install_xmanager.py to purge shadowed wheel xdash packages so local sdks/python/xdash installs cleanly (#1482).

✨ New Features & Improvements

• Database Configuration (db-config) & Schema Version Health Checks in Admin Panel: Added Spanner and PostgreSQL schema version verification to the health checker (#1394) and exposed the shared db-config ConfigMap in the Health Service and Admin Panel UI (/admin) (#1293, #1380).
• Declarative Kueue Queue & MultiKueue Dispatch Configuration in Helm: Extended deploy/helm/values.yaml and kueue-dev chart templates so deployments can declaratively configure CPU/GPU queues and their MultiKueue dispatch targets (#1359).

🐛 User-Facing Bug Fixes & Stability

• Case-Insensitive XID ACL Matching, Principal Normalization & Group Query Injection Guard: Consolidated and normalized email principals on ACL write and OAuth2 sign-in (#1447, #1453), matched XID ACL principals case-insensitively during authorization checks (#1446), and escaped group membership filter queries to prevent query injection (#1448).
• Build-Tag Isolation & Fail-Closed Production Guards for DISABLE_AUTHENTICATION and MOCK_AUTH: Gated DISABLE_AUTHENTICATION and MOCK_AUTH behind the dev build tag, built production container images exclusively from untagged binaries, and enforced fail-closed startup in production binaries (#1443, #1444, #1451).
• Spanner bit_reversed_xid Filter Rewrite & Thread-Safe QueryGenerator: Stopped leaking Spanner-specific bit_reversed_xid filter predicates from ESS ListExperiments and ListArtifacts by rewriting xid equality filters inside the Spanner XID repository (#1481), and removed racy mutable lastFromTable state from QueryGenerator (#1480).

---

🚀 2. Production-Ready Features Behind Feature Flags (Safe to Enable)

The following capabilities are fully developed, verified, and safe to be enabled. Their default enablement is temporarily postponed solely to align with managed solution milestones or external BYOD prerequisites.

🔹 [Flag: enable_metrics & dashboard_service_deployment_enabled] Metrics Pipeline & Dashboard Service

• How to enable:
  • Metrics Ingestion: Set enable_metrics: true in deploy/ctk_v2/xmc-deployment-template.yaml (or metrics.enabled: true in deploy/helm/values.yaml).
  • Dashboard Service: Set dashboard_service_deployment_enabled: true in deploy/ctk_v2/xmc-deployment-template.yaml (or dashboardServiceDeployment.enabled: true in deploy/helm/values.yaml). (Note: Dashboard Service requires BYOD storage devices—see the deployment README—and also requires enable_metrics / metrics.enabled to be enabled for workload metrics data to be generated).
• What's New:
  • Pub/Sub Metrics Ingester (cmd/metrics_ingester) with ClickHouse & BigQuery Writers: Added the Pub/Sub metrics ingester service with ClickHouse batch ingestion, subscription validation, health/readiness probes, and graceful shutdown batch flushing (#1002, #1003, #1004), alongside a BigQuery schema, row normalizer, and streaming writer (#1405, #1406).
  • Dashboard Service BigQuery DataEngine, Parallel Multi-Plot Queries & Live Component/bigquery Hot-Reload: Decoupled the DataEngine provider and parallelized multi-plot queries in Dashboard Service (#1407), added BigQuery client and static DataEngine query support (#1408), supported live Component/bigquery configuration hot-reloading (#1409), and added a BigQuery golden E2E parity harness and seeder (#1410).
  • xdash Auto-Discovery of xmanager login Credentials & Resilient Metrics Addons: Updated the xdash Python SDK and nightly verifiers to automatically discover cached xmanager login credentials (#1462), and hardened ClickHouse and Redpanda metrics addon manifests to survive pod rescheduling (#1459).

---

🧪 3. In-Development / Experimental Features Behind Flags

Warning

The following capabilities are under active ongoing development, experimental, and may not yet be functional. They are available for early preview and evaluation by enabling their deployment flags in deploy/ctk_v2/xmc-deployment-template.yaml (or deploy/ctk_v2/xmanager-cloud.yaml) and Helm values.yaml.

🔹 [Flag: enable_components] Components

• How to enable: Set enable_components: true in deploy/ctk_v2/xmc-deployment-template.yaml (or componentsDeployment.enabled: true in deploy/helm/values.yaml).
• What's New:
  • Database-Agnostic dbstore Component Store & Serializable Catalog Transactions: Promoted the component store to a database-agnostic pkg/components/store/dbstore package backed by storage.Storage and RunInSerializableTransaction (#1393, #1398, #1402, #1403, #1404, #1415, #1416, #1471, #1495).
  • Gateway Parameter Parity, Config Checksum Rollouts & Multi-Broker Preservation: Added gateway parameter parity, custom namespace support, and image resolution across stable-2026a component charts (#1428), rotated config checksum annotations on Helm deployments while preserving multi-broker Redpanda endpoints in ingestion-gateway (#1427), and added --profile-values support with Helm >= 3.14 enforcement in dev_env.py (#1429).

🔹 [Flag: database.experimentalPostgres] Experimental PostgreSQL Control-Plane Backend

• How to enable: Set database.backend: postgres and database.experimentalPostgres: true in deploy/helm/values.yaml.
• What's New:
  • Shared dbconfig Loader & Backend-Agnostic Storage Factory Across Services: Added the shared dbconfig loader, postgres-secret delivery, explicit database.experimentalPostgres Helm opt-in, and backend-agnostic storage factory across all control-plane and auxiliary services (ess, xid_service, actuator, reconciler, dashboard_service, log_monitor, rod_data_provider, health_service) (#1293, #1302, #1387, #1388, #1396, #1401, #1419, #1493).
  • Complete PostgreSQL Domain Repositories & Spanner Parity: Implemented PostgreSQL repositories and row converters for Experiments (Create, Get, Update, ListExperiments, FetchDriftedComponents), WorkUnits (reads, writes, ID allocation, severity queries, ListWorkUnits, FetchActiveWorkUnitsByQueues), Artifacts, Dashboards & Charts, StatusMessages, WorkUnitStatusHistory, Operations, PausedTasks, CLILoginCodes, XIDs, and Components, along with BIGINT symbolic enum query rewrites (#1355, #1397, #1417, #1424, #1434, #1435, #1436, #1437, #1439, #1442, #1454, #1463, #1464, #1465, #1488, #1489, #1492).

---

⚙️ 4. Operational, Infrastructure & Engineering Velocity

Changes relevant to cluster operators, CI/CD, and internal architecture that do not alter the end-user interface.

• CI, Pre-Commit & Nightly Verification Tooling: Dropped apt-bound --with-deps from Playwright browser installs on ARC runners (#1483), handled str and bytes HTTP responses in xmc_entities_validator.py (#1460), auto-restaged formatter fixes during .githooks/pre-commit (#1474), added check_agent_doc_paths.py to fail on broken repository file paths in agent documentation (#1476), and added wait_for_ci.py and PR review thread resolution helpers under tools/pr_review/ (#1473, #1475, #1477).
• Dependency Security Updates: Bumped github.com/go-jose/go-jose/v4, serialize-javascript, and npm packages in cmd/ui (#1450, #1457, #1458).