Use of outdated OpenTelemetry Go package

[mx-psi]

What happened?

Importing Kubernetes' k8s.io/apiserver module together with the latest version of go.opentelemetry.io/otel results in a failed build due to breaking changes in the OpenTelemetry Go library. This also happens with k8s.io/component-base.

The solution is to bump the opentelemetry dependencies to a stable version.

What did you expect to happen?

k8s.io/apiserver should be usable in a module that uses a stable version of go.opentelemetry.io/otel

How can we reproduce it (as minimally and precisely as possible)?

A simple way to reproduce it is to create a new Go module via go mod init example.com/kube-issue that depends on the latest version of both k8s.io/apiserver and some new go.opentelemetry.io/otel module, e.g. go.opentelemetry.io/otel/exporters/otlptrace/otlptracegrpc.

main.go contents

package main

import (
	_ "go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc"
	_ "k8s.io/apiserver/pkg/endpoints/filters"
)

If you try to do go mod tidy you can see a bunch of errors (which come from breaking changes on the OpenTelemetry Go libraries).

go mod tidy output

❯ go mod tidy
go: finding module for package go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc
go: downloading go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.2.0
go: downloading go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.2.0
go: found go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc in go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.2.0
go: downloading go.opentelemetry.io/proto/otlp v0.10.0
go: finding module for package go.opentelemetry.io/otel/semconv
go: finding module for package go.opentelemetry.io/otel/unit
go: finding module for package go.opentelemetry.io/otel/internal/metric
go: found go.opentelemetry.io/otel/internal/metric in go.opentelemetry.io/otel/internal/metric v0.25.0
go: finding module for package go.opentelemetry.io/otel/metric/registry
go: finding module for package go.opentelemetry.io/otel/semconv
example.com/kube-issue imports
        k8s.io/apiserver/pkg/endpoints/filters imports
        go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp imports
        go.opentelemetry.io/otel/semconv: module go.opentelemetry.io/otel@latest found (v1.2.0), but does not contain package go.opentelemetry.io/otel/semconv
example.com/kube-issue imports
        k8s.io/apiserver/pkg/endpoints/filters imports
        go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp tested by
        go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp.test imports
        go.opentelemetry.io/otel/oteltest imports
        go.opentelemetry.io/otel/metric/registry: module go.opentelemetry.io/otel/metric@latest found (v0.25.0), but does not contain package go.opentelemetry.io/otel/metric/registry

Anything else we need to know?

Probably @dashpole is the right person to address this (but I am happy to help with it!)

Kubernetes version

Any version that includes #94942, that is, v0.22.0 or later.

Cloud provider

n/a

OS version

n/a

Install tools

Container runtime (CRI) and and version (if applicable)

Related plugins (CNI, CSI, ...) and versions (if applicable)

[mx-psi]

/sig instrumentation

[mx-psi]

/area dependency

[mx-psi]

This depends on etcd being bumped to a version that includes etcd-io/etcd#13361 (funnily enough, this issue was mentioned on etcd-io/etcd#13361 (comment)).

[ehashman]

/area code-organization
/sig architecture
/assign @dashpole

I believe this is by design due to API incompatibilities and limitations; we cannot use the latest version of the package.

[ehashman]

/triage accepted

[NikhilSharmaWe]

@ehashman I would like to collaborate to change the OpenTelemetry package to a stable version in k8s.io/apiserver and k8s.io/component-base Could you please give me some direction what steps can I take to find out what will be appropriate version to use at place of v0.20.0.

[dashpole]

This needs a release of etcd before we can update the opentelemetry dependency. See etcd-io/etcd#13361 (comment)

[dims]

@NikhilSharmaWe as @dashpole mentioned, please update etcd and possibly containerd as well before we can move up. we'll need released versions (tags) of those projects

[NikhilSharmaWe]

/assign

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle stale
• Mark this issue or PR as rotten with /lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[dashpole]

/remove-lifecycle stale

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle stale
• Mark this issue or PR as rotten with /lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[dashpole]

/remove-lifecycle stale

[ityuhui]

Hi developers

Is this work still on going?

[mx-psi]

@ityuhui This is still not fixed; it is blocked by upgrading the etcd version to the 3.6.x series

[ityuhui]

Thank you @mx-psi !

May I know the etcd upgrade progress ? ( e.g. an issue or PR link )

I'd like to contribute since I need this upgrade for OpenTelemetry in Kubernetes repo to resolve some security vulnerabilities.

[mx-psi]

I am afraid my knowledge ends here :) Maybe @dashpole can help answer that?

[dashpole]

etcd-io/etcd#13538 tracks the release, and references "data inconsistency" as blocking the release, which I believe is etcd-io/etcd#14039. That is part of etcd-io/etcd#14138, which is (I think) where work is being done right now

[Ishmeet]

any workaround ?

[kaiyou]

It seems etcd just backported and merged otel 1.x support in 3.5 : etcd-io/etcd#14312

I believe we can try and make this work without waiting for etcd 3.6!

[dims]

See here @kaiyou @dashpole - #112545

[kaiyou]

Oh thank you, I was already head down on the same track. I'll try and help with the PR if I can.