v0.17.7

Changes since v0.17.6:

Actions Required Before Upgrading

(No, really, you MUST read this before you upgrade)

• Minor releases: Review the .0 release notes for each new minor version you cross; see: v0.16.0, v0.17.0.

• Patch releases: Review the patch release notes leading up to this version, but only within this minor release line; see: v0.17.1, v0.17.2, v0.17.3, v0.17.4, v0.17.5, v0.17.6.

• RayJob: Fixed a bug where the Ray job submitter container's resources were not counted against quota when submissionMode: SidecarMode was used, causing the head PodSet to be under-counted. The head PodSet now includes the submitter sidecar (KubeRay's default 500m CPU / 200Mi memory).

  After upgrading, RayJobs using submissionMode: SidecarMode reserve the submitter sidecar's resources (default 500m CPU / 200Mi memory) on the head. ClusterQueues sized without this headroom may fail to admit such RayJobs; increase the affected ClusterQueue's CPU/memory quota accordingly. (#12725, @kevin85421)

Changes by Kind

Bug or Regression

• AFS: Fixed ConsumedResources CPU truncating to zero when the sampling interval guard was bypassed by informer cache lag during initialization. (#12694, @sohankunkerkar)
• AFS: Fixed a race where a sampling tick running concurrently with workload settlement could persist a skewed ConsumedResources value in LocalQueue fair-sharing status. (#12946, @mimowo)
• AFS: Fixed consumed-resources cache initialization and warm-start recovery so LocalQueue usage is not over-counted during cache seeding, and persisted historical usage is preserved after manager restarts when workload settlement runs before LocalQueue reconciliation. (#12891, @apullo777)
• CLI: Fix --dry-run flag being silently ignored in kueuectl resume/stop localqueue and clusterqueue subcommands. (#12627, @carterpewpew)
• ElasticJobsViaWorkloadSlices: Fix workload slice misordering that could finish a correctly-admitted elastic workload slice when 3+ slices were created within the same second. (#12965, @mimowo)
• ElasticJobsViaWorkloadSlices: Fixed a bug where scaling a Job below its accumulated succeeded count could permanently wedge the Workload reconciler and leak quota. (#12963, @mimowo)
• ElasticJobsViaWorkloadSlices: Fixed a bug where worker pods of an elastic job could be ungated after scale up,
  past the ClusterQueue quota; ungating is now capped to the replicas granted quota across the workload-slice chain. (#12045, @mcochner)
• Helm: Fix helm chart failing to install with a manager CrashLoopBackoff when cert-manager integration is enabled. (#12877, @meln5674)
• Kueue-populator: Fixed a bug where an error creating a LocalQueue was logged but not returned from Reconcile,
  preventing controller-runtime from retrying. LocalQueue creation failures are now aggregated and returned so the request is requeued. (#12930, @NasitSony)
• KueueViz: Fixed a Cross-Site WebSocket Hijacking (CSWSH) vulnerability in the KueueViz Backend by strictly validating WebSocket Origin headers to prevent unauthorized cross-origin data extraction. (#12876, @Vaishnav88sk)
• KueueViz: Fixed a Denial of Service vulnerability where an oversized WebSocket frame could exhaust backend memory (OOM). Connections now enforce an 8 KiB read limit. (#12703, @ABHIGYAN-MOHANTA)
• KueueViz: Fixed dashboard crash caused by missing optional chaining on flavor.resources (#12667, @ABHIGYAN-MOHANTA)
• KueueViz: Improved workloads dashboard performance by avoiding repeated Pod list operations per Workload (#12858, @cryo-zd)
• KueueViz: backend includes HTTP server timeouts (ReadHeaderTimeout, ReadTimeout, WriteTimeout, IdleTimeout) to prevent connection resource exhaustion. (#12867, @ABHIGYAN-MOHANTA)
• KueueViz: frontend container image now runs as a non-root user (node) to adhere to the principle of least privilege. (#12586, @ABHIGYAN-MOHANTA)
• LeaderWorkerSet: Fixed a bug where a LeaderWorkerSet with a negative or excessively large spec.replicas could crash the Kueue controller during reconciliation and MultiKueue workload processing. Kueue now rejects spec.replicas values that are negative or greater than 1000000 (#12756, @reruno)
• MultiKueue: Fixed a bug where obsolete remote Workloads could remain on temporarily unavailable worker clusters when the manager Workload lost its reservation or was deleted. Kueue now retries cleanup after worker clusters reconnect. (#11515, @vamsikrishna-siddu)
• MultiKueue: Fixed a data race where reconnecting a remote cluster could swap the remote client while other goroutines were reading it, which could crash-loop the controller manager. (#12612, @apullo777)
• MultiKueue: Fixed custom jobs using external-framework adapters being repeatedly created and deleted on worker clusters when source-cluster metadata was copied to the remote object. (#12643, @apullo777)
• Observability: Fixed LocalQueue gauge metrics not being reported after a LocalQueue starts matching the configured metrics selector. (#12912, @ikchifo)
• PodGroup integration: Fixed a bug that allowed Workloads corresponding to PodGroups with the WaitingForReplacementPods=True condition to be re-admitted immediately. (#12873, @mbobrovskyi)
• ProvisioningRequest: Fix a bug where ProvisioningRequest owned by finished or evicted Workloads are not cleaned up. The CleanupProvisioningRequestsOnEviction feature gate allows cleanup on eviction to be enabled by default. (#12654, @MatteoFari)
• RayJob: Fix the integration controller dropping Kueue admission placement constraints (nodeSelector, tolerations, nodeAffinity) for the submitter pod when submitterPodTemplate is not explicitly set and submissionMode is K8sJobMode. (#12695, @carterpewpew)
• RayService: Fixed a bug where deleting a Kueue-managed RayService with GCS fault tolerance enabled left KubeRay's Redis cleanup Job suspended forever, leaking the RayCluster's Redis metadata namespace. Kueue now defers finalizing the RayService's Workload until the cleanup Job completes. (#12778, @kevin85421)
• ResourceTransformations: Fixed a bug where milli-valued quantities were rounded before
  resource transformation multiplication. For example, multiplying 300m CPU by 1000
  now correctly produces 300 instead of 3000. (#12962, @mimowo)
• Scheduling: Fixed resource accounting and validation for Pods using Kubernetes pod-level
  resources (pod.spec.resources), including LimitRange defaulting and request/limit
  validation. (#12780, @anuragdalvi)
• Scheduling: Fixed stale scheduling queue entries for pending Workloads that transition
  to WorkloadOnHold. (#12948, @mimowo)
• SparkApplication: Fixed a bug where the global spec.nodeSelector could overwrite driver or executor node selectors when they were admitted to different ResourceFlavors. (#12688, @carterpewpew)
• StatefulSet: Fixed a bug where scaling a StatefulSet to zero caused its Workload to be incorrectly requeued for scheduling during the terminating-pod window, competing for quota it should no longer hold. (#12657, @gola)
• TAS: Fixed a bug that permanently leaked Topology-Aware Scheduling (TAS) resources if a workload was deleted while its ClusterQueue was temporarily missing a required Topology. (#12752, @Vaishnav88sk)
• VisibilityOnDemand: Fixed a data race between the Visibility API pending-workloads endpoint and preemption requeuing that could crash the queue manager for BestEffortFIFO ClusterQueues. (#12736, @somaz94)

Other (Cleanup or Flake)

• TAS: Reduced the CPU and memory overhead of building the topology snapshot on large clusters by no longer cloning per-node usage maps on every scheduling cycle. (#12706, @akshay-pm)