Merge pull request #64 from mpatlasov/rebase-v2.12.0

STOR-1573: Rebase `livenessprobe` to v2.12.0 for OCP 4.16
openshift · Apr 10, 2024 · f5e3ff5 · f5e3ff5
2 parents 2d60f07 + e4e46f1
commit f5e3ff5
Show file tree

Hide file tree

Showing 1,035 changed files with 139,856 additions and 3,057 deletions.
diff --git a/CHANGELOG/CHANGELOG-2.12.md b/CHANGELOG/CHANGELOG-2.12.md
@@ -0,0 +1,105 @@
+# Release notes for v2.12.0
+
+[Documentation](https://kubernetes-csi.github.io/docs/)
+
+# Changelog since v2.11.0
+
+
+## Changes by Kind
+
+### Feature
+
+- Added support for structured logging (the log messages have been changed due to the activation of structured logging) ([#202](https://github.com/kubernetes-csi/livenessprobe/pull/202), [@bells17](https://github.com/bells17))
+
+### Bug or Regression
+
+- Liveness probe process does not crash when it cannot access the associated CSI driver. It only fails all kubelet probes, most probably with "connection refused". ([#240](https://github.com/kubernetes-csi/livenessprobe/pull/240), [@jsafrane](https://github.com/jsafrane))
+
+### Uncategorized
+
+- CVE fixes: CVE-2023-44487 ([#220](https://github.com/kubernetes-csi/livenessprobe/pull/220), [@dobsonj](https://github.com/dobsonj))
+- Update kubernetes dependencies to v1.29.0 ([#238](https://github.com/kubernetes-csi/livenessprobe/pull/238), [@sunnylovestiramisu](https://github.com/sunnylovestiramisu))
+
+## Dependencies
+
+### Added
+- github.com/benbjohnson/clock: [v1.1.0](https://github.com/benbjohnson/clock/tree/v1.1.0)
+- github.com/cpuguy83/go-md2man/v2: [v2.0.2](https://github.com/cpuguy83/go-md2man/v2/tree/v2.0.2)
+- github.com/creack/pty: [v1.1.9](https://github.com/creack/pty/tree/v1.1.9)
+- github.com/kisielk/errcheck: [v1.5.0](https://github.com/kisielk/errcheck/tree/v1.5.0)
+- github.com/kisielk/gotool: [v1.0.0](https://github.com/kisielk/gotool/tree/v1.0.0)
+- github.com/kr/pty: [v1.1.1](https://github.com/kr/pty/tree/v1.1.1)
+- github.com/kr/text: [v0.2.0](https://github.com/kr/text/tree/v0.2.0)
+- github.com/russross/blackfriday/v2: [v2.1.0](https://github.com/russross/blackfriday/v2/tree/v2.1.0)
+- go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc: v0.46.0
+
+### Changed
+- cloud.google.com/go/compute: v1.21.0 → v1.23.0
+- github.com/container-storage-interface/spec: [v1.8.0 → v1.9.0](https://github.com/container-storage-interface/spec/compare/v1.8.0...v1.9.0)
+- github.com/emicklei/go-restful/v3: [v3.9.0 → v3.11.0](https://github.com/emicklei/go-restful/v3/compare/v3.9.0...v3.11.0)
+- github.com/evanphx/json-patch: [v5.6.0+incompatible → v4.12.0+incompatible](https://github.com/evanphx/json-patch/compare/v5.6.0...v4.12.0)
+- github.com/go-logr/logr: [v1.2.4 → v1.3.0](https://github.com/go-logr/logr/compare/v1.2.4...v1.3.0)
+- github.com/golang/glog: [v1.1.0 → v1.1.2](https://github.com/golang/glog/compare/v1.1.0...v1.1.2)
+- github.com/google/go-cmp: [v0.5.9 → v0.6.0](https://github.com/google/go-cmp/compare/v0.5.9...v0.6.0)
+- github.com/google/uuid: [v1.3.0 → v1.4.0](https://github.com/google/uuid/compare/v1.3.0...v1.4.0)
+- github.com/grpc-ecosystem/grpc-gateway/v2: [v2.7.0 → v2.16.0](https://github.com/grpc-ecosystem/grpc-gateway/v2/compare/v2.7.0...v2.16.0)
+- github.com/kubernetes-csi/csi-lib-utils: [v0.14.0 → v0.17.0](https://github.com/kubernetes-csi/csi-lib-utils/compare/v0.14.0...v0.17.0)
+- github.com/kubernetes-csi/csi-test/v5: [v5.0.0 → v5.2.0](https://github.com/kubernetes-csi/csi-test/v5/compare/v5.0.0...v5.2.0)
+- github.com/onsi/ginkgo/v2: [v2.9.4 → v2.13.1](https://github.com/onsi/ginkgo/v2/compare/v2.9.4...v2.13.1)
+- github.com/onsi/gomega: [v1.20.0 → v1.30.0](https://github.com/onsi/gomega/compare/v1.20.0...v1.30.0)
+- github.com/stretchr/testify: [v1.8.2 → v1.8.4](https://github.com/stretchr/testify/compare/v1.8.2...v1.8.4)
+- github.com/yuin/goldmark: [v1.4.1 → v1.3.5](https://github.com/yuin/goldmark/compare/v1.4.1...v1.3.5)
+- go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp: v0.35.1 → v0.44.0
+- go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc: v1.10.0 → v1.19.0
+- go.opentelemetry.io/otel/exporters/otlp/otlptrace: v1.10.0 → v1.19.0
+- go.opentelemetry.io/otel/metric: v0.31.0 → v1.20.0
+- go.opentelemetry.io/otel/sdk: v1.10.0 → v1.19.0
+- go.opentelemetry.io/otel/trace: v1.10.0 → v1.20.0
+- go.opentelemetry.io/otel: v1.10.0 → v1.20.0
+- go.opentelemetry.io/proto/otlp: v0.19.0 → v1.0.0
+- go.uber.org/goleak: v1.2.1 → v1.1.10
+- golang.org/x/crypto: v0.11.0 → v0.15.0
+- golang.org/x/lint: d0100b6 → 1621716
+- golang.org/x/net: v0.13.0 → v0.18.0
+- golang.org/x/oauth2: v0.10.0 → v0.13.0
+- golang.org/x/sync: v0.3.0 → v0.4.0
+- golang.org/x/sys: v0.10.0 → v0.14.0
+- golang.org/x/term: v0.10.0 → v0.14.0
+- golang.org/x/text: v0.11.0 → v0.14.0
+- golang.org/x/tools: v0.8.0 → v0.14.0
+- google.golang.org/appengine: v1.6.7 → v1.6.8
+- google.golang.org/genproto/googleapis/api: 782d3b1 → d307bd8
+- google.golang.org/genproto/googleapis/rpc: 782d3b1 → bbf56f3
+- google.golang.org/genproto: 782d3b1 → d783a09
+- google.golang.org/grpc: v1.58.0 → v1.60.1
+- k8s.io/api: v0.28.0 → v0.29.0
+- k8s.io/apimachinery: v0.28.0 → v0.29.0
+- k8s.io/client-go: v0.28.0 → v0.29.0
+- k8s.io/component-base: v0.28.0 → v0.29.0
+- k8s.io/klog/v2: v2.100.1 → v2.110.1
+- k8s.io/kube-openapi: 2695361 → 2dd684a
+- k8s.io/utils: d93618c → 3b25d92
+- sigs.k8s.io/structured-merge-diff/v4: v4.2.3 → v4.4.1
+
+### Removed
+- cloud.google.com/go: v0.34.0
+- github.com/BurntSushi/toml: [v0.3.1](https://github.com/BurntSushi/toml/tree/v0.3.1)
+- github.com/antihax/optional: [v1.0.0](https://github.com/antihax/optional/tree/v1.0.0)
+- github.com/chzyer/logex: [v1.1.10](https://github.com/chzyer/logex/tree/v1.1.10)
+- github.com/chzyer/readline: [2972be2](https://github.com/chzyer/readline/tree/2972be2)
+- github.com/chzyer/test: [a1ea475](https://github.com/chzyer/test/tree/a1ea475)
+- github.com/client9/misspell: [v0.3.4](https://github.com/client9/misspell/tree/v0.3.4)
+- github.com/fsnotify/fsnotify: [v1.4.9](https://github.com/fsnotify/fsnotify/tree/v1.4.9)
+- github.com/ghodss/yaml: [v1.0.0](https://github.com/ghodss/yaml/tree/v1.0.0)
+- github.com/google/gnostic: [v0.5.7-v3refs](https://github.com/google/gnostic/tree/v0.5.7-v3refs)
+- github.com/grpc-ecosystem/grpc-gateway: [v1.16.0](https://github.com/grpc-ecosystem/grpc-gateway/tree/v1.16.0)
+- github.com/hpcloud/tail: [v1.0.0](https://github.com/hpcloud/tail/tree/v1.0.0)
+- github.com/ianlancetaylor/demangle: [28f6c0f](https://github.com/ianlancetaylor/demangle/tree/28f6c0f)
+- github.com/nxadm/tail: [v1.4.8](https://github.com/nxadm/tail/tree/v1.4.8)
+- github.com/onsi/ginkgo: [v1.16.4](https://github.com/onsi/ginkgo/tree/v1.16.4)
+- github.com/rogpeppe/fastuuid: [v1.2.0](https://github.com/rogpeppe/fastuuid/tree/v1.2.0)
+- go.opentelemetry.io/otel/exporters/otlp/internal/retry: v1.10.0
+- golang.org/x/exp: 509febe
+- gopkg.in/fsnotify.v1: v1.4.7
+- gopkg.in/tomb.v1: dd63297
+- honnef.co/go/tools: ea95bdf
diff --git a/README.md b/README.md
@@ -81,7 +81,7 @@ spec:
 
 * `--http-endpoint <endpoint>`: The TCP network address where the HTTP server for diagnostics, including CSI driver health check and metrics. The default is empty string, which means the server is disabled.
 
-* All glog / klog arguments are supported, such as `-v <log level>` or `-alsologtostderr`.
+* [Arguments set by the `k8s.io/component-base/logs` package for klog](https://github.com/kubernetes/component-base/blob/v0.28.0-rc.0/logs/api/v1/options.go#L337-L355) are supported, such as `--v <log level>` and `--logging-format <log format>`.
 
 ## Community, discussion, contribution, and support
 

diff --git a/cmd/livenessprobe/main.go b/cmd/livenessprobe/main.go
@@ -23,12 +23,15 @@ import (
 	"net"
 	"net/http"
 	"os"
-	"sync"
 	"time"
 
-	"google.golang.org/grpc"
 	"k8s.io/klog/v2"
 
+	"k8s.io/component-base/featuregate"
+	"k8s.io/component-base/logs"
+	logsapi "k8s.io/component-base/logs/api/v1"
+	_ "k8s.io/component-base/logs/json/register"
+
 	connlib "github.com/kubernetes-csi/csi-lib-utils/connection"
 	"github.com/kubernetes-csi/csi-lib-utils/metrics"
 	"github.com/kubernetes-csi/csi-lib-utils/rpc"
@@ -57,78 +60,54 @@ func (h *healthProbe) checkProbe(w http.ResponseWriter, req *http.Request) {
 	ctx, cancel := context.WithTimeout(req.Context(), *probeTimeout)
 	defer cancel()
 
-	conn, err := acquireConnection(ctx, h.metricsManager)
+	conn, err := connlib.Connect(*csiAddress, h.metricsManager, connlib.WithTimeout(*probeTimeout))
 	if err != nil {
 		w.WriteHeader(http.StatusInternalServerError)
 		w.Write([]byte(err.Error()))
-		klog.Errorf("failed to establish connection to CSI driver: %v", err)
+		klog.ErrorS(err, "Failed to establish connection to CSI driver")
 		return
 	}
 	defer conn.Close()
 
-	klog.V(5).Infof("Sending probe request to CSI driver %q", h.driverName)
+	klog.V(5).InfoS("Sending probe request to CSI driver", "driver", h.driverName)
 	ready, err := rpc.Probe(ctx, conn)
 	if err != nil {
 		w.WriteHeader(http.StatusInternalServerError)
 		w.Write([]byte(err.Error()))
-		klog.Errorf("health check failed: %v", err)
+		klog.ErrorS(err, "Health check failed")
 		return
 	}
 
 	if !ready {
 		w.WriteHeader(http.StatusInternalServerError)
 		w.Write([]byte("driver responded but is not ready"))
-		klog.Error("driver responded but is not ready")
+		klog.ErrorS(nil, "Driver responded but is not ready")
 		return
 	}
 
 	w.WriteHeader(http.StatusOK)
 	w.Write([]byte(`ok`))
-	klog.V(5).Infof("Health check succeeded")
-}
-
-// acquireConnection wraps the connlib.Connect but adding support to context
-// cancelation.
-func acquireConnection(ctx context.Context, metricsManager metrics.CSIMetricsManager) (conn *grpc.ClientConn, err error) {
-
-	var m sync.Mutex
-	var canceled bool
-	ready := make(chan bool)
-	go func() {
-		conn, err = connlib.Connect(*csiAddress, metricsManager)
-
-		m.Lock()
-		defer m.Unlock()
-		if err != nil && canceled && conn != nil {
-			conn.Close()
-		}
-
-		close(ready)
-	}()
-
-	select {
-	case <-ctx.Done():
-		m.Lock()
-		defer m.Unlock()
-		canceled = true
-		return nil, ctx.Err()
-
-	case <-ready:
-		return conn, err
-	}
+	klog.V(5).InfoS("Health check succeeded")
 }
 
 func main() {
-	klog.InitFlags(nil)
-	flag.Set("logtostderr", "true")
+	fg := featuregate.NewFeatureGate()
+	logsapi.AddFeatureGates(fg)
+	c := logsapi.NewLoggingConfiguration()
+	logsapi.AddGoFlags(c, flag.CommandLine)
+	logs.InitLogs()
 	flag.Parse()
+	if err := logsapi.ValidateAndApply(c, fg); err != nil {
+		klog.ErrorS(err, "LoggingConfiguration is invalid")
+		os.Exit(1)
+	}
 
 	if *healthzPort != defaultHealthzPort && *httpEndpoint != "" {
-		klog.Error("only one of `--health-port` and `--http-endpoint` can be explicitly set.")
+		klog.ErrorS(nil, "Only one of `--health-port` and `--http-endpoint` can be explicitly set")
 		os.Exit(1)
 	}
 	if *metricsAddress != "" && *httpEndpoint != "" {
-		klog.Error("only one of `--metrics-address` and `--http-endpoint` can be explicitly set.")
+		klog.ErrorS(nil, "Only one of `--metrics-address` and `--http-endpoint` can be explicitly set")
 		os.Exit(1)
 	}
 	var addr string
@@ -139,20 +118,27 @@ func main() {
 	}
 
 	metricsManager := metrics.NewCSIMetricsManager("" /* driverName */)
-	csiConn, err := acquireConnection(context.Background(), metricsManager)
+	// Connect to the CSI driver without any timeout to avoid crashing the probe when the driver is not ready yet.
+	// Goal: liveness probe never crashes, it only fails the probe when the driver is not available (yet).
+	// Since a http server for the probe is not running at this point, Kubernetes liveness probe will fail immediately
+	// with "connection refused", which is good enough to fail the probe.
+	csiConn, err := connlib.Connect(*csiAddress, metricsManager, connlib.WithTimeout(0))
 	if err != nil {
 		// connlib should retry forever so a returned error should mean
-		// the grpc client is misconfigured rather than an error on the network
-		klog.Fatalf("failed to establish connection to CSI driver: %v", err)
+		// the grpc client is misconfigured rather than an error on the network or CSI driver.
+		klog.ErrorS(err, "Failed to establish connection to CSI driver")
+		klog.FlushAndExit(klog.ExitFlushTimeout, 1)
 	}
 
-	klog.Infof("calling CSI driver to discover driver name")
+	klog.InfoS("Calling CSI driver to discover driver name")
 	csiDriverName, err := rpc.GetDriverName(context.Background(), csiConn)
 	csiConn.Close()
 	if err != nil {
-		klog.Fatalf("failed to get CSI driver name: %v", err)
+		// The CSI driver does not support GetDriverName, which is serious enough to crash the probe.
+		klog.ErrorS(err, "Failed to get CSI driver name")
+		klog.FlushAndExit(klog.ExitFlushTimeout, 1)
 	}
-	klog.Infof("CSI driver name: %q", csiDriverName)
+	klog.InfoS("CSI driver name", "driver", csiDriverName)
 
 	hp := &healthProbe{
 		driverName:     csiDriverName,
@@ -171,18 +157,19 @@ func main() {
 		metricsMux := http.NewServeMux()
 		metricsManager.RegisterToServer(metricsMux, *metricsPath)
 		go func() {
-			klog.Infof("Separate metrics ServeMux listening at %q", *metricsAddress)
+			klog.InfoS("Separate metrics ServeMux listening", "address", *metricsAddress)
 			err := http.ListenAndServe(*metricsAddress, metricsMux)
 			if err != nil {
-				klog.Fatalf("Failed to start prometheus metrics endpoint on specified address (%q) and path (%q): %s", *metricsAddress, *metricsPath, err)
+				klog.ErrorS(err, "Failed to start prometheus metrics endpoint on specified address and path", "addr", *metricsAddress, "path", *metricsPath)
+				klog.FlushAndExit(klog.ExitFlushTimeout, 1)
 			}
 		}()
 	}
 
 	mux.HandleFunc("/healthz", hp.checkProbe)
-	klog.Infof("ServeMux listening at %q", addr)
+	klog.InfoS("ServeMux listening", "address", addr)
 	err = http.ListenAndServe(addr, mux)
 	if err != nil {
-		klog.Fatalf("failed to start http server with error: %v", err)
+		klog.ErrorS(err, "Failed to start http server")
 	}
 }
diff --git a/go.mod b/go.mod
@@ -3,31 +3,54 @@ module github.com/kubernetes-csi/livenessprobe
 go 1.21
 
 require (
-	github.com/container-storage-interface/spec v1.8.0
+	github.com/container-storage-interface/spec v1.9.0
 	github.com/golang/mock v1.6.0
-	github.com/kubernetes-csi/csi-lib-utils v0.14.0
-	github.com/kubernetes-csi/csi-test/v5 v5.0.0
-	google.golang.org/grpc v1.58.0
-	k8s.io/klog/v2 v2.100.1
+	github.com/kubernetes-csi/csi-lib-utils v0.17.0
+	github.com/kubernetes-csi/csi-test/v5 v5.2.0
+	k8s.io/component-base v0.29.0
+	k8s.io/klog/v2 v2.110.1
 )
 
 require (
 	github.com/beorn7/perks v1.0.1 // indirect
 	github.com/blang/semver/v4 v4.0.0 // indirect
 	github.com/cespare/xxhash/v2 v2.2.0 // indirect
-	github.com/go-logr/logr v1.2.4 // indirect
+	github.com/go-logr/logr v1.3.0 // indirect
+	github.com/go-logr/stdr v1.2.2 // indirect
+	github.com/go-logr/zapr v1.2.3 // indirect
+	github.com/gogo/protobuf v1.3.2 // indirect
 	github.com/golang/protobuf v1.5.4 // indirect
+	github.com/google/go-cmp v0.6.0 // indirect
+	github.com/google/gofuzz v1.2.0 // indirect
+	github.com/inconshreveable/mousetrap v1.1.0 // indirect
+	github.com/json-iterator/go v1.1.12 // indirect
+	github.com/kr/text v0.2.0 // indirect
 	github.com/matttproud/golang_protobuf_extensions v1.0.4 // indirect
+	github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
+	github.com/modern-go/reflect2 v1.0.2 // indirect
 	github.com/prometheus/client_golang v1.16.0 // indirect
 	github.com/prometheus/client_model v0.4.0 // indirect
 	github.com/prometheus/common v0.44.0 // indirect
 	github.com/prometheus/procfs v0.10.1 // indirect
+	github.com/spf13/cobra v1.7.0 // indirect
 	github.com/spf13/pflag v1.0.5 // indirect
-	golang.org/x/net v0.17.0 // indirect
-	golang.org/x/sys v0.13.0 // indirect
-	golang.org/x/text v0.13.0 // indirect
-	google.golang.org/genproto/googleapis/rpc v0.0.0-20230711160842-782d3b101e98 // indirect
+	go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.46.0 // indirect
+	go.opentelemetry.io/otel v1.20.0 // indirect
+	go.opentelemetry.io/otel/metric v1.20.0 // indirect
+	go.opentelemetry.io/otel/trace v1.20.0 // indirect
+	go.uber.org/atomic v1.10.0 // indirect
+	go.uber.org/multierr v1.11.0 // indirect
+	go.uber.org/zap v1.19.0 // indirect
+	golang.org/x/net v0.18.0 // indirect
+	golang.org/x/sys v0.14.0 // indirect
+	golang.org/x/text v0.14.0 // indirect
+	google.golang.org/genproto/googleapis/rpc v0.0.0-20231106174013-bbf56f31fb17 // indirect
+	google.golang.org/grpc v1.60.1 // indirect
 	google.golang.org/protobuf v1.33.0 // indirect
-	k8s.io/apimachinery v0.28.0 // indirect
-	k8s.io/component-base v0.28.0 // indirect
+	gopkg.in/inf.v0 v0.9.1 // indirect
+	gopkg.in/yaml.v2 v2.4.0 // indirect
+	k8s.io/apimachinery v0.29.0 // indirect
+	k8s.io/utils v0.0.0-20230726121419-3b25d923346b // indirect
+	sigs.k8s.io/json v0.0.0-20221116044647-bc3834ca7abd // indirect
+	sigs.k8s.io/structured-merge-diff/v4 v4.4.1 // indirect
 )