Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[bug] OpenTelemetry Tracing resource merge error #2503

Closed
Tsuribori opened this issue Jan 6, 2024 · 4 comments · Fixed by #2546
Closed

[bug] OpenTelemetry Tracing resource merge error #2503

Tsuribori opened this issue Jan 6, 2024 · 4 comments · Fixed by #2546
Labels
bug Something isn't working

Comments

@Tsuribori
Copy link
Contributor

Describe the bug with a clear and concise description of what the bug is.

There is an uncaught error that occurs with tracing in internal/tracing/tracing.go:

	r, _ := resource.Merge(
		resource.Default(),
		resource.NewWithAttributes(
			semconv.SchemaURL,
			semconv.ServiceName("GoToSocial"),
		),
	)

The error that occurs is "cannot merge resource due to conflicting Schema URL", and is caused by the mismatch of otel library versions being used is v1.21.0 but the semconv schema version in use is v1.20.0.

	semconv "go.opentelemetry.io/otel/semconv/v1.20.0"
	httpconv "go.opentelemetry.io/otel/semconv/v1.20.0/httpconv"

Traces are still produced, but Grafana Tempo won't show them at all I guess due to missing attributes and they are shown funky in Jaeger too (they are shown as being produced by OTLPResourceNoServiceName instead of GoToSocial)

Bumping the semconv schema version used to v1.21.0 makes it work again but the problem is that the httpconv package was moved to being an internal package of opentelemetry-go-contrib starting from v1.21.0, so the version for it can't be bumped.

What's your GoToSocial Version?

This occurs in all versions above v.11.0

GoToSocial Arch

No response

What happened?

OpenTelemetry traces don't show correctly in services that consume them

What you expected to happen?

That they would be shown correctly like they did in GoToSocial version v0.11.0

How to reproduce it?

Enable tracing grpc export to Grafana Tempo or Jaeger

Anything else we need to know?

No response

@Tsuribori Tsuribori added the bug Something isn't working label Jan 6, 2024
@tsmethurst
Copy link
Contributor

Ah, thanks!

@Tsuribori
Copy link
Contributor Author

Would using

	httpconv "go.opentelemetry.io/otel/semconv/v1.20.0/httpconv"
	semconv "go.opentelemetry.io/otel/semconv/v1.21.0"

mismatch be an acceptable quick fix? I guess it technically makes the whole thing schema non-compliant and is not a future-proof solution but it makes traces show up correctly. I haven't found anything to replace httpconv, though I'm not really an OTel expert

@tsmethurst
Copy link
Contributor

Mmm I'm not sure, possibly... I'll try to have a fiddle with this before we do our next minor release.

@daenney
Copy link
Member

daenney commented Jan 10, 2024

I'm a bit surprised this is causing an issue. There's many versions of the semantic conventions and it shouldn't be a stretch for v1.21 of the library to still request things be emitted with the v1.20 conventions.

I can spend a bit on fixing this, but that's going to have to wait a week or two. Instead of trying to randomly cobble together a version of the conventions that sort of just happens to work, I'd suggest just rolling back the dependency to 1.20. We just update them from now and then, but there's no real need to do so here in this case. That would solve the immediate issue, and then we can spend some time fixing it in a future-proof way after.

daenney added a commit that referenced this issue Jan 20, 2024
For some reason httpconv seems to have disappeared from 1.21, which
results in a 1.21 runtime with 1.20 semconv/httpconv which seems to
break some things.

For now, this rolls the OTEL dependencies back to 1.20 which should fix
the observability issues. We'll need to take a look at how to upgrade
safely and correctly in the future.

Relates to #2503.
@tsmethurst tsmethurst linked a pull request Jan 21, 2024 that will close this issue
9 tasks
tsmethurst pushed a commit that referenced this issue Jan 21, 2024
For some reason httpconv seems to have disappeared from 1.21, which
results in a 1.21 runtime with 1.20 semconv/httpconv which seems to
break some things.

For now, this rolls the OTEL dependencies back to 1.20 which should fix
the observability issues. We'll need to take a look at how to upgrade
safely and correctly in the future.

Relates to #2503.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants