Release 0.12.0

XMC Release 0.12.0

Release Version Determination: Minor bump (0.11.0 -> 0.12.0).
Rationale: Promotes the Resource Observability Dashboard (ROD) and Health Service / Admin Panel to default-enabled (true) out-of-the-box across all Cluster Toolkit v2 blueprints and Helm deployments, introduces a comprehensive UI design system overhaul, hardens core task queue and workload scheduling reliability, and advances the preview Pause Scheduling and Custom Components frameworks.

---

🌟 1. Public Changes (Available Out-of-the-Box)

🚀 Major Highlights & Promoted Capabilities

• Resource Observability Dashboard (ROD) Enabled by Default: The Resource Observability Dashboard is now deployed and active out-of-the-box across all Cluster Toolkit v2 blueprints and Helm manifests (rod_deployment_enabled: true, rodDeployment.enabled: true), giving operators immediate, zero-configuration visibility into cluster queue velocity, resource utilization, and queue backlog predictability (#981).
• Health Service & Admin Panel Enabled by Default: The Health Service and administrative interface are now enabled out-of-the-box (health_service_deployment_enabled: true, healthServiceDeployment.enabled: true), enabling real-time platform component health inspection, readiness probing, and centralized administrative controls without custom configuration (#981).
• Comprehensive UI Design System & Visual Refresh: Overhauled the frontend visual styling, typography, and page structure across 8 synchronized updates to deliver a modern, cohesive developer experience:
  • Collapsible Sidenav Rail: Introduced a responsive navigation rail expandable from compact 72px icon view to 240px full-width label view, with restored documentation quick-links (#953, #973).
  • Global Theming & Typography: Aligned color tokens, typography hierarchies, and status chip styling across light and dark interfaces (#952).
  • Top Toolbar & Global Search: Refreshed the global application header with standardized height, borders, and an integrated search bar (#954).
  • Experiments & Work Units Layouts: Redesigned the Experiments List container, data table borders, metadata split cards, and Work Units table row headers and ID badges (#955, #956, #957, #958).
  • Artifact Filters & Cards: Polished artifact category toggle buttons and interactive artifact cards (#959).

✨ New Features & Runtime Improvements

• Automated Default Resource Requests for Kueue Workloads: Workloads submitted without explicit resource limits are now automatically assigned safe default resource requests, preventing submitted jobs from becoming indefinitely stuck in Kueue scheduling queues (#916).
• Strict Task Queue Error Propagation: The Experiment State Server (ESS) now rigorously checks and propagates Redis message broker errors during work unit task enqueueing, preventing silent task drops and ensuring immediate operational visibility into queue saturation (#937).

🐛 User-Facing Bug Fixes & Stability

• Protobuf Compatibility & Python SDK Stability: Pinned googleapis-common-protos below version 1.75.4 across example workloads and deployments, eliminating runtime descriptor conflicts and import failures in Python SDK clients (#985).
• Dev Environment Service Synchronization: Fixed the local development orchestrator (dev_env.py) to automatically rebuild, push, and restart the rod_data_provider container upon local code changes (#994).

---

🚀 2. Production-Ready Features Behind Feature Flags (Safe to Enable)

The following capabilities are fully verified and safe to enable in production environments. Default enablement is optional to accommodate bring-your-own-device or cloud infrastructure prerequisites.

🔹 [Flag: dashboard_service_deployment_enabled] Dashboard Service (Custom Dashboards & Visualization)

• How to enable: Set dashboard_service_deployment_enabled: true in deploy/ctk_v2/xmanager-cloud.yaml (or dashboardServiceDeployment.enabled: true in Helm values.yaml).
• Why it's opt-in: Fully production-ready and safe to enable, but kept opt-in by default as it requires bring-your-own-devices (BYOD / external storage backends such as ClickHouse for metric ingestion and analytical query processing).
• Prerequisite Flag: Note that for the dashboard service to fully function and ingest telemetry, the metrics pipeline must also be activated by enabling enable_metrics: true in deploy/ctk_v2/xmanager-cloud.yaml (or metrics.enabled: true in Helm values.yaml) for metric data to be generated and streamed.
• What's New:
  • Powers custom metric charts, interactive dashboard creation, and high-throughput metric aggregation for experiment workloads.

🔹 [Flag: enable_managed_mldiagnostics] Google Cloud Managed ML Diagnostics on GKE

• How to enable: Set enable_managed_mldiagnostics: true and managed_xprof_gcs_bucket: "gs://<bucket-name>" in deploy/ctk_v2/xmanager-cloud.yaml (or managedMLDiagnostics.enabled: true in Helm values.yaml).
• What's New:
  • Automates performance profiling and diagnostic trace capture for JAX, PyTorch, and TensorFlow workloads running on GKE TPU/GPU clusters.
  • Injects XProf profiling sidecars, enables Google Cloud Hypercompute Cluster APIs, and integrates seamlessly with Diagon profiling consoles.

---

🛠️ 3. Internal Development / Upcoming Features (Under Construction)

Caution

The following capabilities are under active internal development and are not ready for use, preview, or testing. They are guarded by strict feature flags (defaulting to false). Enabling them manually is unsupported and will lead to environment instability.

🔹 Pause Scheduling & Task Resumption Subsystem

• Status: 🚧 Under Construction (Do not enable)
• What's New:
  • Real-Time Redis Broadcast Engine: Implemented production Redis Pub/Sub publisher and consumer channels for broadcasting cluster-wide pause and resume events between microservices (#864).
  • Actuator Task Interception & Buffering: Actuator now intercepts and safely buffers pending work unit tasks in Spanner storage when scheduling is paused, preventing premature execution on drained compute nodes (#927).
  • Resumption Worker Loop & Replay Execution: Added dedicated ESS resumption workers that progressively replay buffered tasks in configurable batches upon cluster resumption (#922).
  • Replay Claim Verification & Safety Checks: Replay workers verify cluster pause state and worker ownership claims on every batch to prevent split-brain processing during rapid pause/unpause toggles (#926).
  • Active Task State Filtering: Enhanced replay filtering to skip cancelled, expired, or superseded tasks during cluster unpause (#972).

🔹 Pluggable Custom Components Framework

• Status: 🚧 Under Construction (Do not enable)
• What's New:
  • ClickHouse 3P Component Blueprint: Added official 3rd-party component blueprint for deploying and managing ClickHouse analytical engines (#963).
  • Simplified Wrapper Component Architecture: Made wrapperRef optional for 3P components and migrated Redpanda to the unified 3P Wrapper Component blueprint (#939, #978).
  • Ingestion Gateway CR Resolution: Ingestion Gateway now automatically resolves Redpanda Custom Resources and configures RBAC policies when enable_components=true (#948).
  • Shared Component Dependency DAG: Introduced a unified directed acyclic graph (DAG) engine for validating and resolving complex inter-component dependencies (#930).
  • Dynamic Chart Resolution & Cold-Start Hydration: Bridge reconciler supports dynamic per-component chart resolution and cold-start state hydration on controller restarts (#931).
  • Integration Testing & Linter Hardening: Added Cloud Spanner emulator drift integration suites and extended the component linter to support Helm sub-charts and CRD enums (#826, #945).

---

⚙️ 4. Operational, Infrastructure & Engineering Velocity

Changes relevant to cluster operators, CI/CD, and internal architecture that do not alter the end-user interface.

• Automated Release Tooling Suite:
  • Introduced the end-to-end AI release notes generator skill and taxonomy reference (xmc_generate_release_notes) (#836).
  • Added core release publisher CLI tool with headless GitHub REST API fallback (tools/release/publish_release.py) (#834, #835).
  • Built dynamic feature flag discovery, gated subsystem classification, and automated SemVer bump calculation engines (tools/release/extract_release_metadata.py) (#700, #701).
• Automated Visual UI Audits & E2E Validation:
  • Introduced weekly automated visual UI snapshot tooling and scheduled GitHub Actions visual audit workflow with step summary galleries (#960, #970).
  • Added Playwright end-to-end UI traversal verifiers covering Experiment headers, Work Units tabs, and row assertions (#913, #924).
  • Accelerated E2E testing by eliminating unnecessary 30s Dex auth timeouts, online Spanner DDL waits, and Tier-1 installations (#944).
  • Verified WorkUnit artifact parent linkage and timestamp ordering in example verification tests (#947).
• Multi-Scenario Benchmark Hardening & Engineering Guidelines:
  • Upgraded agent benchmark verification to grade candidate agents across 3 distinct test scenarios with pre-registered assertions and zero baseline-arm leakage (#965, #967, #968).
  • Codified codebase evidence scoping and reflection control ladder guidelines in agent rules (#949, #961, #964).
  • Cleaned up reviewer availability records in ABSENCE.md (#929).
• Dependency Updates:
  • Bumped @angular/common to 19.2.14 in /cmd/ui (#854).
  • Bumped oras.land/oras-go/v2 to 2.6.0 in Go modules (#914).