Add BinaryFormat interface and implementation #74

mayurkale22 · 2019-07-01T22:03:36Z

Specs: https://github.com/open-telemetry/opentelemetry-specification/blob/master/specification/propagators-api.md#binary-format

bg451 · 2019-07-02T23:39:34Z

qq, where does the binary format for w3c trace context come from? is it compatible with, say, the java implementation?

rochdev · 2019-07-02T15:03:06Z

packages/opentelemetry-core/src/context/propagation/BinaryTraceContext.ts

+     *  | `---------------------------------- traceID field ID (0)
+     *  `------------------------------------ version (0)
+     */
+    const result = Buffer.alloc(FORMAT_LENGTH, 0);


allocUnsafe is a lot faster when the buffer is written to right away.

While there are performance advantages to using Buffer.allocUnsafe(), the new Buffer is uninitialized (the allocated segment of memory might contain old data - tests are failing because of that).

Not sure what you mean. Since data is written to the buffer right away on the next few lines, the original data of the buffer should be irrelevant.

Is it possible to use only ArrayBuffer in this code?

rochdev · 2019-07-02T15:04:53Z

packages/opentelemetry-core/src/context/propagation/BinaryTraceContext.ts

+  4 * ID_SIZE + TRACE_ID_SIZE + SPAN_ID_SIZE + TRACE_OPTION_SIZE;
+
+export class BinaryTraceContext implements BinaryFormat {
+  toBytes(spanContext: SpanContext): Buffer {


I'm not sure I understand this class. Isn't it a propagator that should have an inject and extract method? If not, why the difference and how is this supposed to be used?

Actually, there could have an interface that is compatible with other propagators with something like this:

inject(spanContext: SpanContext, format: string, carrier: Injector) {

And then the injector would be a function/class passed by the user that would take the buffer as input. Then they can control how to add the buffer created by OpenTelemetry in their own existing buffer, while keeping the API compatible.

This is based on specs here https://github.com/open-telemetry/opentelemetry-specification/blob/master/specification/propagators-api.md#propagators-api. Even I was confused about these two interfaces (BinaryFormat and HTTPTextFormat), but decided to have something ready for discussion.

@bogdandrutu Could you please address these questions?

This could be done similar to OpenTracing, but with an ArrayBuffer to support the browser too:

interface BinaryCarrier { buffer?: ArrayBuffer } class BinaryPropagator implements Propagator { inject(spanContext: SpanContext, format: string, carrier: BinaryCarrier): this { ... } extract(format: string, carrier: BinaryCarrier): SpanContext | null { ... } }

Sure makes sense to me, will wait for open-telemetry/opentelemetry-specification#174 to get accepted. Mean time, Could you please review again.

If I recall correctly, the reason in OpenCensus that this was a Buffer was to support gRPC, which isn't relevant in the browser case (there is gRPC web but its transport is actually different from native gRPC and it uses a proxy that speaks HTTP actually)

A Buffer can be initialized from an ArrayBuffer in Node and same for a TypedArray in the browser. The idea then would be to use an ArrayBuffer which is compatible with both platforms, and assign it to an object instead of returning it to make it compatible with inject and extract.

I think we can move forward with these APIs in the meantime while open-telemetry/opentelemetry-specification#174 and open-telemetry/opentelemetry-specification#165 are being discussed

@mayurkale22 could you make this an ArrayBuffer per our conversation in the SIG today?

It is browser compatible and is efficiently convertable to a Node Buffer via Buffer.from

mayurkale22 · 2019-07-02T23:49:35Z

qq, where does the binary format for w3c trace context come from? is it compatible with, say, the java implementation?

This is copied from OpenCensus specs and it is compatible with java's implementation. Having said that, users/vendor can write their own implementation (sdk-level) based on BinaryFormat interface.

rochdev · 2019-07-05T22:31:11Z

Having said that, users/vendor can write their own implementation (sdk-level) based on BinaryFormat interface.

These wouldn't be available to users without vendor lock in. The declarative approach of inject and extract with a string format allows extensions without requiring the user to use custom code from the vendor.

rochdev · 2019-07-09T15:38:55Z

packages/opentelemetry-core/src/context/propagation/BinaryTraceContext.ts

+     *  | `---------------------------------- traceID field ID (0)
+     *  `------------------------------------ version (0)
+     */
+    const result = Buffer.alloc(FORMAT_LENGTH, 0);


Not sure what you mean. Since data is written to the buffer right away on the next few lines, the original data of the buffer should be irrelevant.

rochdev · 2019-07-09T15:43:25Z

packages/opentelemetry-core/src/context/propagation/BinaryTraceContext.ts

+  4 * ID_SIZE + TRACE_ID_SIZE + SPAN_ID_SIZE + TRACE_OPTION_SIZE;
+
+export class BinaryTraceContext implements BinaryFormat {
+  toBytes(spanContext: SpanContext): Buffer {


A Buffer can be initialized from an ArrayBuffer in Node and same for a TypedArray in the browser. The idea then would be to use an ArrayBuffer which is compatible with both platforms, and assign it to an object instead of returning it to make it compatible with inject and extract.

I think we can move forward with these APIs in the meantime while open-telemetry/opentelemetry-specification#174 and open-telemetry/opentelemetry-specification#165 are being discussed

codecov-io · 2019-07-11T04:43:02Z

Codecov Report

Merging #74 into master will decrease coverage by 0.73%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master      #74      +/-   ##
==========================================
- Coverage     100%   99.26%   -0.74%     
==========================================
  Files          12       14       +2     
  Lines         200      271      +71     
  Branches        8       20      +12     
==========================================
+ Hits          200      269      +69     
- Misses          0        2       +2

Impacted Files	Coverage Δ
test/context/BinaryTraceContext.test.ts	`100% <0%> (ø)`
src/context/propagation/BinaryTraceContext.ts	`96.15% <0%> (ø)`

mayurkale22 · 2019-07-11T16:49:59Z

@rochdev charges have been made as per discussion in the SIG meeting (Use ArrayBuffer instead of Buffer to support both Node and browser), PTAL.

rochdev

LGTM. The API is problematic for vendors but this should be addressed in the spec.

mayurkale22 · 2019-07-12T16:56:49Z

LGTM. The API is problematic for vendors but this should be addressed in the spec.

SGTM, will update the interface once we have agreed on the specs.

mayurkale22 requested review from danielkhan, iredelmeier, justindsmith and rochdev as code owners July 1, 2019 22:03

mayurkale22 requested a review from bg451 July 2, 2019 00:18

rochdev reviewed Jul 2, 2019

View reviewed changes

mayurkale22 added the Discussion Issue or PR that needs/is extended discussion. label Jul 2, 2019

rochdev mentioned this pull request Jul 5, 2019

Add Tracer API #85

Merged

mayurkale22 force-pushed the BinaryFormat branch from a9788a5 to c60ffae Compare July 8, 2019 21:45

rochdev approved these changes Jul 9, 2019

View reviewed changes

mayurkale22 force-pushed the BinaryFormat branch from c60ffae to a45d6b0 Compare July 11, 2019 04:41

draffensperger approved these changes Jul 11, 2019

View reviewed changes

mayurkale22 force-pushed the BinaryFormat branch from a45d6b0 to 531c497 Compare July 11, 2019 16:44

mayurkale22 added this to the API complete milestone Jul 11, 2019

mayurkale22 self-assigned this Jul 11, 2019

rochdev approved these changes Jul 12, 2019

View reviewed changes

mayurkale22 added 3 commits July 12, 2019 09:42

Add BinaryFormat interface and implementation

a59fb21

Incorporate newly added TraceOptions & merge master

a306e05

Use ArrayBuffer instead of Buffer

4de8849

mayurkale22 force-pushed the BinaryFormat branch from 531c497 to 4de8849 Compare July 12, 2019 16:43

mayurkale22 merged commit 5870457 into open-telemetry:master Jul 12, 2019

mayurkale22 deleted the BinaryFormat branch July 12, 2019 16:57

markwolff mentioned this pull request Jul 23, 2019

add GlobalTracer API #118

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add BinaryFormat interface and implementation #74

Add BinaryFormat interface and implementation #74

mayurkale22 commented Jul 1, 2019

bg451 commented Jul 2, 2019

rochdev Jul 2, 2019

mayurkale22 Jul 8, 2019

rochdev Jul 9, 2019

draffensperger Jul 11, 2019

rochdev Jul 2, 2019

rochdev Jul 2, 2019

mayurkale22 Jul 3, 2019

rochdev Jul 4, 2019 •

edited

mayurkale22 Jul 8, 2019

draffensperger Jul 9, 2019

rochdev Jul 9, 2019

draffensperger Jul 11, 2019

mayurkale22 Jul 11, 2019

mayurkale22 commented Jul 2, 2019

rochdev commented Jul 5, 2019

rochdev Jul 9, 2019

rochdev Jul 9, 2019

codecov-io commented Jul 11, 2019

mayurkale22 commented Jul 11, 2019

rochdev left a comment

mayurkale22 commented Jul 12, 2019

Add BinaryFormat interface and implementation #74

Add BinaryFormat interface and implementation #74

Conversation

mayurkale22 commented Jul 1, 2019

bg451 commented Jul 2, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rochdev Jul 4, 2019 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mayurkale22 commented Jul 2, 2019

rochdev commented Jul 5, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codecov-io commented Jul 11, 2019

Codecov Report

mayurkale22 commented Jul 11, 2019

rochdev left a comment

Choose a reason for hiding this comment

mayurkale22 commented Jul 12, 2019

rochdev Jul 4, 2019 •

edited