Skip to content

v0.13.0

Latest
Latest

Choose a tag to compare

View all tags
@github-actions github-actions released this 26 Jun 18:07
v0.13.0
This tag was signed with the committer’s verified signature.
klihub Krisztian Litkey
GPG key ID: 637F2939D50AF85D
Verified
Learn about vigilant mode.
f206c26

This is a new major release of NRI Reference Plugins. It comes with major new features, some bugfixes and dependency updates.

What's New

Balloons Policy

  • cpuClasses with turbo priority and PCT The balloons policy has a new cpuClasses configuration section. You can use it to set CPU frequency limits, energy-performance preference, C-state overrides, uncore frequency, and the CPUFreq governor on a per-class basis. Frequency fields accept plain numbers (kHz), units (3.2GHz, 2900MHz), or the symbolic names min, base, and turbo — resolved from sysfs at startup.

    turboPriority handles the common case where you want only one balloon type to get actual turbo headroom at any given time. Among all active balloon types, the one with the highest turboPriority gets turbo; the rest are capped at base clock. The scope is configurable per socket or system-wide via turboDomain.

    pctPriority and pctClosID wire up Intel Speed Select Technology Priority Core Turbo (SST-CP/SST-TF). In managed mode the plugin programs the CLOS assignments itself. In assoc-only mode it associates CPUs to CLOSes that are already configured by the operator or BIOS. Setting publishExtendedResource: true on a PCT class makes the plugin advertise a Kubernetes extended resource (cpuclass.balloons.nri.io/<class>) whose value tracks available HP CPUs on the node, so the scheduler can avoid over-subscribing a single node's HP capacity. See Quick start: PCT in Kubernetes with Balloons for more information.

New nri-resctrl-mon Plugin

  • nri-resctrl-mon is a new standalone plugin that creates per-pod RDT monitoring groups in /sys/fs/resctrl. Each pod gets its own mon_group for the lifetime of the pod sandbox, making per-pod L3 cache and memory bandwidth metrics available to Prometheus (for example via Kepler) without touching any policy configuration. Requires kernel RDT support. With CRI-O, version 1.36 or later is required. See Resctrl-Mon NRI Plugin documentation for more information.

Logging

  • slog-based logging The internal logger now uses Go's standard slog package. The pkg/log API is unchanged, but the underlying implementation is smaller and cleaner.

  • OpenTelemetry-based log exporting Produced logs can now be exported to an OpenTelemetry-compatible log collector. The following custom resource fragment enables gRPC based log exporting.

apiVersion: config.nri/v1alpha1
kind: TopologyAwarePolicy
metadata:
  name: default
spec:
...
  instrumentation:
    httpEndpoint: 8891
    logExportPeriod: 15s
    logExporter: otlp-grpc
  log:
    debug:
      - policy
...

Additionally you need to pass the collector endpoint to the resource policy plugin using the stock Opentelemetry environment variables. With the above configuration, you need to pass OTEL_EXPORTER_OTLP_LOGS_ENDPOINT=http://otel-collector:4317 if your collector is otel-collector and configured to use the standard port.

You can set both the configuration and the necessary extra environment variable with a Helm config fragment like this:

config:
  reservedResources:
    cpu: 750m
  pinCPU: true
  pinMemory: true
  instrumentation:
    httpEndpoint: :8891
    logExporter: otlp-grpc
    logExportPeriod: 15s
  log:
    debug:
      - policy
extraEnv:
  OTEL_EXPORTER_OTLP_LOGS_ENDPOINT: http://otel-collector:4317

new goresctrl v0.13.0 and updated SST API

goresctrl is updated to v0.13.0, which restructures the SST model around a Platform handle and per-package Punit slices. The sysfs and cpuallocator packages are updated accordingly. sysfs.System has a new Sst() method that returns the platform handle directly for code that needs it.

Fixes

  • Policies no longer fail to start on kernels built with CONFIG_NUMA=n.
  • resctrl-mon: mon_groups now persist until pod sandbox removal rather than disappearing when the first container in the pod stops.
  • resctrl-mon: pod UIDs are normalised to standard dashed-UUID format before directory lookups, fixing a lookup failure with certain runtimes.
  • resctrl-mon: CRI-O versions before 1.36 are rejected at startup with a descriptive error (they don't populate Container.Pid in NRI events).
  • memtierd: temporary file handling tightened to avoid stale files after a restart.
  • cgroups: fixed an integer truncation in cgroup stats parsing that could silently corrupt values.
  • instrumentation: fixed accidental Prometheus configuration activation on startup.
  • Logging: fixed stderrthreshold being ignored when logtostderr is set.
  • Security: the NRI socket directory is now mounted read-only in all Helm DaemonSets.
  • sysfs: replaced unsafe integer-cast helpers with typed generics, eliminating a class of integer-truncation issues flagged by CodeQL.

Dependency updates

  • goresctrl: v0.12.0 → v0.13.0
  • OpenTelemetry SDK: v1.42.0 → v1.44.0 (plus new log exporter packages)
  • prometheus/client_golang: v1.23.0 → v1.23.2
  • grpc: v1.79.3 → v1.81.1

CI

Added Trivy vulnerability scanning and CodeQL security analysis workflows.

What's Changed In Details

  • resource-annotator: Document command line options by @ozhuraki in #648
  • build: Remove extra clean in topology's Dockerfile by @ozhuraki in #647
  • Honor stderrthreshold when logtostderr is enabled by @pierluigilenoci in #646
  • topology-aware: Add metrics unit tests by @ozhuraki in #654
  • topology-aware: Skip size arbitration if both offers are nil by @ozhuraki in #649
  • topology-aware: Add libmem unit tests by @ozhuraki in #653
  • topology-aware: Add Node traversal stop by @ozhuraki in #652
  • sysfs: Add tests for parsing entries into units by @ozhuraki in #655
  • sysfs: run policies even if kernel CONFIG_NUMA=n by @askervin in #658
  • topology-aware: Add Node traversal unit tests by @ozhuraki in #650
  • build(deps): bump google.golang.org/grpc from 1.79.2 to 1.79.3 by @dependabot[bot] in #644
  • operator: update the maintainer details by @fmuyassarov in #661
  • build(deps): bump pygments from 2.17.0 to 2.20.0 in /docs by @dependabot[bot] in #651
  • operator: replace deprecated gcr.io/kubebuilder/kube-rbac-proxy by @fmuyassarov in #662
  • golang,golang-ci-lint: bump versions, fix linting errors. by @klihub in #664
  • resctrl-mon: add NRI plugin for per-pod resctrl monitoring groups by @cmcantalupo in #666
  • mount NRI socket directory as read-only by @fmuyassarov in #675
  • Parallelize makefile running by @uniemimu in #678
  • Move tar.go to testutils by @uniemimu in #677
  • Fix unescaped dot in regexp by @uniemimu in #680
  • Fix issues in instrumentation unit test by @askervin in #685
  • e2e: install cri-tools/crictl on Fedora 43 and later. by @klihub in #684
  • Fix "make test" issues caused by pkg/http by @askervin in #683
  • buildx: load the images to local docker daemon. by @fmuyassarov in #673
  • resctrl-mon: revert CRI-O pidfile fallback due to security concerns by @cmcantalupo in #671
  • resctrl-mon: accept pod UIDs without dashes too by @fmuyassarov in #674
  • Fix for incorrect conversion between integer types by @uniemimu in #679
  • pkg/sysfs: replace interface{}-based sysfs I/O with typed generics by @kad in #682
  • e2e: fix CIDRs added to vm's no_proxy to be effective by @askervin in #687
  • build: dispel ghosts from topology-aware images by @askervin in #689
  • instrumentation, log: add support for otel log collection. by @klihub in #663
  • fixes: fix hint allow-/deny-list filtering tests. by @klihub in #692
  • memtierd: harden temporary file handling by @askervin in #690
  • cache: avoid a test cache population data race by @klihub in #696
  • e2e: fix kubectl wait in otel log collection test. by @klihub in #695
  • e2e: strict topology hint test fixes by @klihub in #694
  • e2e: build custom guest OS kernels with AMD enabled by @askervin in #699
  • e2e: fix otel log test wait error, possible instability by @askervin in #698
  • Makefile: make sure test cleanup is not run in parallel with tests. by @klihub in #701
  • e2e: add e2e test for the nri-memtierd plugin by @askervin in #691
  • fix resctrl-mon: retain mon_group until pod sandbox removal by @cmcantalupo in #693
  • topology-aware: drop cpus attribute from shared pool zone metrics by @TonyxSun in #686
  • pkg/topology,Makefile: fix flaky tests, ensure subpackage tests run. by @klihub in #702
  • e2e: detect and try to fix potential post reboot image blob corruption. by @klihub in #700
  • e2e: show active test case in 'VM prompt'. by @klihub in #703
  • e2e: more robust otel log collection test. by @klihub in #704
  • e2e: fix topology-aware test04-nrt expected memory set output by @askervin in #705
  • e2e: give otel log dumping more slack. by @klihub in #706
  • Add trivy and codeql workflows by @uniemimu in #681
  • Dockerfiles: work around parallel make license extraction breakage. by @klihub in #707
  • Update to goresctrl v0.13.0 and use new SST API by @askervin in #697
  • e2e: use nop BTRFS/blob corruption handlers for cri-o. by @klihub in #708
  • balloons: add cpuClasses with cpufreq and PCT turbo allocation by @askervin in #667

New Contributors

Full Changelog: v0.12.2...v0.13.0

Contributors

uniemimu, kad, and 8 other contributors
Assets 11
Loading