Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bump OTEL version to latest #2355

Merged
merged 7 commits into from
Aug 21, 2020
Merged

Conversation

pavolloffay
Copy link
Member

@joe-elliott
Copy link
Member

This has been unblocked by: open-telemetry/opentelemetry-collector#1574

I have been working through the Jaeger Otel Collector integration and would like to give this PR a review when its ready.

Signed-off-by: Pavol Loffay <ploffay@redhat.com>
@pavolloffay pavolloffay marked this pull request as ready for review August 20, 2020 10:56
@pavolloffay pavolloffay requested a review from a team as a code owner August 20, 2020 10:56
Signed-off-by: Pavol Loffay <ploffay@redhat.com>
@pavolloffay
Copy link
Member Author

@joe-elliott would you like to review? I have updated the PR

@pavolloffay
Copy link
Member Author

We might have to update proto

cmd/collector/app/zipkin/http_handler_test.go:30:2: Package github.com/golang/protobuf/proto is deprecated: Use the "google.golang.org/protobuf/proto" package instead.  (SA1019)
model/ids_test.go:23:2: Package github.com/golang/protobuf/proto is deprecated: Use the "google.golang.org/protobuf/proto" package instead.  (SA1019)
model/prototest/model_test.pb.go:8:2: Package github.com/golang/protobuf/proto is deprecated: Use the "google.golang.org/protobuf/proto" package instead.  (SA1019)
model/prototest/model_test.pb.go:41:9: proto.EnumName is deprecated: Do not use.  (SA1019)
model/prototest/model_test.pb.go:65:9: xxx_messageInfo_SpanRef.Unmarshal is deprecated: Do not use; this method existed for intenal-use only.  (SA1019)
model/prototest/model_test.pb.go:68:9: xxx_messageInfo_SpanRef.Marshal is deprecated: Do not use; this method existed for intenal-use only.  (SA1019)
model/prototest/model_test.pb.go:71:2: xxx_messageInfo_SpanRef.Merge is deprecated: Do not use; this method existed for intenal-use only.  (SA1019)
model/prototest/model_test.pb.go:74:9: xxx_messageInfo_SpanRef.Size is deprecated: Do not use; this method existed for intenal-use only.  (SA1019)
model/prototest/model_test.pb.go:77:2: xxx_messageInfo_SpanRef.DiscardUnknown is deprecated: Do not use; this method existed for intenal-use only.  (SA1019)

Signed-off-by: Pavol Loffay <ploffay@redhat.com>
@pavolloffay
Copy link
Member Author

The issue is solved by -checks=-SA1019 on staticcheck (see golang/protobuf#1077).

Weird is that the badger test fails on

GOROOT=/home/ploffay/bin/go #gosetup
GOPATH=/home/ploffay/projects/golang #gosetup
/home/ploffay/bin/go/bin/go test -c -o /tmp/___TestDependencyReader_in_github_com_jaegertracing_jaeger_plugin_storage_badger_dependencystore github.com/jaegertracing/jaeger/plugin/storage/badger/dependencystore #gosetup
/home/ploffay/bin/go/bin/go tool test2json -t /tmp/___TestDependencyReader_in_github_com_jaegertracing_jaeger_plugin_storage_badger_dependencystore -test.v -test.run ^TestDependencyReader$
=== RUN   TestDependencyReader
--- FAIL: TestDependencyReader (0.74s)
panic: invalid Go type model.TraceID for field jaeger.api_v2.Span.trace_id [recovered]
	panic: invalid Go type model.TraceID for field jaeger.api_v2.Span.trace_id

goroutine 6 [running]:
testing.tRunner.func1.1(0xb21d00, 0xc005610480)
	/home/ploffay/bin/go/src/testing/testing.go:940 +0x2f5
testing.tRunner.func1(0xc0001fa000)
	/home/ploffay/bin/go/src/testing/testing.go:943 +0x3f9
panic(0xb21d00, 0xc005610480)
	/home/ploffay/bin/go/src/runtime/panic.go:969 +0x166
google.golang.org/protobuf/internal/impl.newSingularConverter(0xd4eb80, 0xbbaba0, 0xd4ea60, 0xc000160000, 0xc000186270, 0x30)
	/home/ploffay/projects/golang/pkg/mod/google.golang.org/protobuf@v1.24.0/internal/impl/convert.go:143 +0xff5
google.golang.org/protobuf/internal/impl.NewConverter(0xd4eb80, 0xbbaba0, 0xd4ea60, 0xc000160000, 0x2, 0x30)
	/home/ploffay/projects/golang/pkg/mod/google.golang.org/protobuf@v1.24.0/internal/impl/convert.go:60 +0xd9
google.golang.org/protobuf/internal/impl.fieldInfoForScalar(0xd4ea60, 0xc000160000, 0xb60450, 0x7, 0x0, 0x0, 0xd4eb80, 0xbbaba0, 0xb60459, 0x5b, ...)
	/home/ploffay/projects/golang/pkg/mod/google.golang.org/protobuf@v1.24.0/internal/impl/message_reflect_field.go:234 +0x110
google.golang.org/protobuf/internal/impl.(*MessageInfo).makeKnownFieldsFunc(0xc000142f00, 0xe0, 0xffffffffffffffff, 0xc8, 0xffffffffffffffff, 0xc00561c060, 0xc00561c090, 0xc00561c0c0, 0xc00561c0f0)
	/home/ploffay/projects/golang/pkg/mod/google.golang.org/protobuf@v1.24.0/internal/impl/message_reflect.go:67 +0x938
google.golang.org/protobuf/internal/impl.(*MessageInfo).makeReflectFuncs(0xc000142f00, 0xd4eb80, 0xc033e0, 0xe0, 0xffffffffffffffff, 0xc8, 0xffffffffffffffff, 0xc00561c060, 0xc00561c090, 0xc00561c0c0, ...)
	/home/ploffay/projects/golang/pkg/mod/google.golang.org/protobuf@v1.24.0/internal/impl/message_reflect.go:36 +0x63
google.golang.org/protobuf/internal/impl.(*MessageInfo).initOnce(0xc000142f00)
	/home/ploffay/projects/golang/pkg/mod/google.golang.org/protobuf@v1.24.0/internal/impl/message.go:90 +0x182
google.golang.org/protobuf/internal/impl.(*MessageInfo).init(0xc000142f00)
	/home/ploffay/projects/golang/pkg/mod/google.golang.org/protobuf@v1.24.0/internal/impl/message.go:72 +0x3c
google.golang.org/protobuf/internal/impl.(*messageReflectWrapper).ProtoMethods(0xc005610060, 0xc00565a001)
	/home/ploffay/projects/golang/pkg/mod/google.golang.org/protobuf@v1.24.0/internal/impl/message_reflect_gen.go:150 +0x2f
google.golang.org/protobuf/proto.protoMethods(...)
	/home/ploffay/projects/golang/pkg/mod/google.golang.org/protobuf@v1.24.0/proto/proto_methods.go:18
google.golang.org/protobuf/proto.UnmarshalOptions.unmarshal(0x101, 0xd365e0, 0xc00007ca50, 0xc005658000, 0x45, 0x50, 0xd49480, 0xc005610060, 0x126fa98, 0x203001, ...)
	/home/ploffay/projects/golang/pkg/mod/google.golang.org/protobuf@v1.24.0/proto/decode.go:79 +0x7f
google.golang.org/protobuf/proto.UnmarshalOptions.UnmarshalState(...)
	/home/ploffay/projects/golang/pkg/mod/google.golang.org/protobuf@v1.24.0/proto/decode.go:63
github.com/golang/protobuf/proto.UnmarshalMerge(0xc005658000, 0x45, 0x50, 0xd3cc80, 0xc00565a000, 0x1, 0xc00565a000)
	/home/ploffay/projects/golang/pkg/mod/github.com/golang/protobuf@v1.4.2/proto/wire.go:67 +0x19b
github.com/golang/protobuf/proto.Unmarshal(0xc005658000, 0x45, 0x50, 0xd3cc80, 0xc00565a000, 0xc000034d20, 0x19)
	/home/ploffay/projects/golang/pkg/mod/github.com/golang/protobuf@v1.4.2/proto/wire.go:58 +0x67
github.com/jaegertracing/jaeger/plugin/storage/badger/spanstore.decodeValue(0xc005658000, 0x45, 0x50, 0x2, 0xc005658000, 0x45, 0x50)
	/home/ploffay/projects/jaegertracing/jaeger/plugin/storage/badger/spanstore/reader.go:101 +0x85
github.com/jaegertracing/jaeger/plugin/storage/badger/spanstore.(*TraceReader).getTraces.func1(0xc00560e090, 0x0, 0x0)
	/home/ploffay/projects/jaegertracing/jaeger/plugin/storage/badger/spanstore/reader.go:138 +0x1e0
github.com/dgraph-io/badger.(*DB).View(0xc000216000, 0xc00022d9c8, 0x0, 0x0)
	/home/ploffay/projects/golang/pkg/mod/github.com/dgraph-io/badger@v1.5.3/transaction.go:567 +0x9d
github.com/jaegertracing/jaeger/plugin/storage/badger/spanstore.(*TraceReader).getTraces(0xc0055d0140, 0xc005622000, 0x28, 0x28, 0xc005622000, 0x28, 0x28, 0x0, 0x0)
	/home/ploffay/projects/jaegertracing/jaeger/plugin/storage/badger/spanstore/reader.go:120 +0x1e4
github.com/jaegertracing/jaeger/plugin/storage/badger/spanstore.(*TraceReader).FindTraces(0xc0055d0140, 0xd3fdc0, 0xc000032138, 0xc0001a2000, 0xbfc7ba304a530781, 0xfffffcb9cfc40099, 0x1246aa0, 0x2a, 0x11e4189)
	/home/ploffay/projects/jaegertracing/jaeger/plugin/storage/badger/spanstore/reader.go:456 +0xb6
github.com/jaegertracing/jaeger/plugin/storage/badger/dependencystore.(*DependencyStore).GetDependencies(0xc0055d0150, 0xbfc7bdb44a530781, 0x7ca099, 0x1246aa0, 0x34630b8a000, 0x0, 0x0, 0x1, 0x0, 0x0)
	/home/ploffay/projects/jaegertracing/jaeger/plugin/storage/badger/dependencystore/storage.go:50 +0x18e
github.com/jaegertracing/jaeger/plugin/storage/badger/dependencystore_test.TestDependencyReader.func1(0xd49660, 0xc0001fa000, 0xd2e400, 0xc0055c82a0, 0xd2e3e0, 0xc0055d0150)
	/home/ploffay/projects/jaegertracing/jaeger/plugin/storage/badger/dependencystore/storage_test.go:97 +0x5ba
github.com/jaegertracing/jaeger/plugin/storage/badger/dependencystore_test.runFactoryTest(0xd49660, 0xc0001fa000, 0xc00022df60)
	/home/ploffay/projects/jaegertracing/jaeger/plugin/storage/badger/dependencystore/storage_test.go:63 +0x5eb
github.com/jaegertracing/jaeger/plugin/storage/badger/dependencystore_test.TestDependencyReader(0xc0001fa000)
	/home/ploffay/projects/jaegertracing/jaeger/plugin/storage/badger/dependencystore/storage_test.go:67 +0x5a
testing.tRunner(0xc0001fa000, 0xc54098)
	/home/ploffay/bin/go/src/testing/testing.go:991 +0xdc
created by testing.(*T).Run
	/home/ploffay/bin/go/src/testing/testing.go:1042 +0x357

Process finished with exit code 1

@pavolloffay
Copy link
Member Author

Seems like a problem with customtype annotation on the trace ID https://github.com/jaegertracing/jaeger-idl/blob/52fb4c944067f7661e6a5fa23ba4c44c6f9c2923/proto/api_v2/model.proto#L101

Signed-off-by: Pavol Loffay <ploffay@redhat.com>
Signed-off-by: Pavol Loffay <ploffay@redhat.com>
@pavolloffay
Copy link
Member Author

pinning the gogo and protobuf verisons that are currently used in Jaeger worked.

We should upgrade proto the grpc deps in a separate PR.

Copy link
Member

@joe-elliott joe-elliott left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly just follows from upstream changes. Overall looks good, but had a few comments/questions.

Makefile Outdated
@@ -165,7 +165,8 @@ lint-gosec:
.PHONY: lint-staticcheck
lint-staticcheck:
@cat /dev/null > $(LINT_LOG)
time staticcheck ./... \
# TODO remove SA1019 once https://github.com/golang/protobuf/issues/1077 is resolved
time staticcheck -checks=-SA1019 ./... \
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of ignoring SA1019 universally does it make more sense to add the offending files to the list below?

		| grep -v \
			-e model/model.pb.go \
			-e model/prototest/model_test.pb.go \
			-e thrift-gen/ \

The other two files (cmd/collector/app/zipkin/http_handler_test.go, model/ids_test.go) are not autogenerated and could have nolint directives added. Although to keep things in one spot, might be better to add all three here with a note to remove once golang/protobuf#1077 is resolved.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This does not seem to be needed after I removed proto version bump. I will remove it for now.

We will run into this when upgrading to the newer proto.

factories.Receivers["jaeger"] = &jaegerreceiver.Factory{
Wrapped: jaegerRec,
Wrapped: otelJaegerReceiver.NewFactory(),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Previously the same factory was passed into both jaegerreceiver.Factory shims. I don't see any harm in doing it this way, and actually prefer this stylistically. Unsure if there are consequences I'm unaware of.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They removed factory types and instead they use just factory method from the helper package.

@@ -15,29 +15,31 @@
package resourceprocessor
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Clever reuse of the otel resource processor. As far as I can tell this is being configured to support agent tags?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To inject tags passed as flags or env property. The OTEL collector does not support flags or env vars at the moment.

Signed-off-by: Pavol Loffay <ploffay@redhat.com>
Signed-off-by: Pavol Loffay <ploffay@redhat.com>
@codecov
Copy link

codecov bot commented Aug 21, 2020

Codecov Report

Merging #2355 into master will not change coverage.
The diff coverage is n/a.

Impacted file tree graph

@@           Coverage Diff           @@
##           master    #2355   +/-   ##
=======================================
  Coverage   95.60%   95.60%           
=======================================
  Files         206      206           
  Lines       10549    10549           
=======================================
  Hits        10085    10085           
+ Misses        396      395    -1     
- Partials       68       69    +1     
Impacted Files Coverage Δ
pkg/discovery/grpcresolver/grpc_resolver.go 98.57% <0.00%> (-1.43%) ⬇️
plugin/storage/integration/integration.go 80.38% <0.00%> (-0.48%) ⬇️
plugin/storage/badger/spanstore/reader.go 96.79% <0.00%> (ø)
...lugin/sampling/strategystore/adaptive/processor.go 100.00% <0.00%> (+0.79%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 510042d...27d750d. Read the comment docs.

@pavolloffay pavolloffay merged commit 3eeedf0 into jaegertracing:master Aug 21, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants