-
Notifications
You must be signed in to change notification settings - Fork 2.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add Zipkin Thrift as kafka ingestion format #1256
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks to contributing! This will be the last missing piece for Zipkin compat.
I left a few stylistic comments. The build appears to be failing on go fmt
Go fmt, license check, or import ordering failures, run 'make fmt'
./plugin/storage/kafka/options.go
./plugin/storage/kafka/unmarshaller.go
Last piece is to add unit test for the marshaling.
Codecov Report
@@ Coverage Diff @@
## master #1256 +/- ##
======================================
Coverage 100% 100%
======================================
Files 159 160 +1
Lines 7163 7181 +18
======================================
+ Hits 7163 7181 +18
Continue to review full report at Codecov.
|
0ffd48e
to
83c8a2d
Compare
Signed-off-by: Geoffrey Beausire <g.beausire@criteo.com>
Signed-off-by: Geoffrey Beausire <g.beausire@criteo.com>
Signed-off-by: Geoffrey Beausire <g.beausire@criteo.com>
83c8a2d
to
50b11f1
Compare
plugin/storage/kafka/unmarshaller.go
Outdated
if err != nil { | ||
return nil, err | ||
} | ||
return mSpan[0], err |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@yurishkuro I am not sure it's some good code, any advices to avoid using [0]. I thought about changing the interface to return []model.Span and handle the array in the calling function.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think the ingester is currently built with the assumption that a single Kafka message contains just a single span. Even in this case a single Zipkin span can sometimes be transformed to two Jaeger spans, so it would be better to extend the ingester to support the notion of many spans per message, at least after unmarshaling. However, I would not recommend doing it in this PR, it should be a separate PR as it might get quite large. For now I would suggest logging a warning.
There are good reasons to expect a single span per Kafka message, it makes the streaming jobs easier to write.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks
Formating done, now going for tests |
Signed-off-by: Geoffrey Beausire <g.beausire@criteo.com>
Signed-off-by: Geoffrey Beausire <g.beausire@criteo.com>
Signed-off-by: Geoffrey Beausire <g.beausire@criteo.com>
a273a5c
to
9038399
Compare
Should be good now, @yurishkuro |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks great! Thanks for contributing 🎉
Have you been able to test this with real Zipkin clients writing to Kafka?
Yes, I tested it mostly that way. It has not run for a long continuous time thought. And I am not sure we are using a clean version of Zipkin. I should probably build a small PoC using docker images to test better |
I don't mind merging this as is, but it would be nice to add an integration test for Travis. We can probably extend https://github.com/jaegertracing/xdock-zipkin-brave to support configuration to use Kafka as the span reporting transport. |
Which problem is this PR solving?
Short description of the changes