Add Getter.Keys() with Jaeger Baggage support. #1549

carlosalberto · 2020-08-18T13:01:14Z

Fixes #967

Implements the open-telemetry/opentelemetry-specification#825 update, and implements Jaeger Baggage support to verify that the API is correct/useful (based on their own propagator)

Used Iterable for keys().
Had to remove the usage of anonymous classes in the integration test's trace-context section, as it broke some versions of Java 11 (see this).
We can play with removing the (most likely) slow regex in a follow-up.
Don't put an empty Baggage in Context in case there was not Baggage to be extracted at all.

For Jaeger reviewers:

Didn't encode the Baggage values. In Jaeger there's two TextMap propagators registered by default (encoded and non-encoded propagation). We need to define the encoding case in a follow-up (somehow).

@open-telemetry/java-approvers
cc @pavolloffay @jpkrohling

codecov · 2020-08-18T13:13:35Z

Codecov Report

❗ No coverage uploaded for pull request base (master@28905b7). Click here to learn what that means.
The diff coverage is 97.14%.

@@            Coverage Diff            @@
##             master    #1549   +/-   ##
=========================================
  Coverage          ?   84.30%           
  Complexity        ?     1819           
=========================================
  Files             ?      215           
  Lines             ?     7354           
  Branches          ?      800           
=========================================
  Hits              ?     6200           
  Misses            ?      852           
  Partials          ?      302

Impacted Files	Coverage Δ	Complexity Δ
.../io/opentelemetry/opentracingshim/Propagation.java	`95.83% <0.00%> (ø)`	`5.00 <0.00> (?)`
.../extension/trace/propagation/JaegerPropagator.java	`90.74% <100.00%> (ø)`	`41.00 <13.00> (?)`
...lemetry/sdk/extension/otproto/TraceProtoUtils.java	`76.19% <0.00%> (ø)`	`6.00% <0.00%> (?%)`
...n/java/io/opentelemetry/api/common/Attributes.java	`92.00% <0.00%> (ø)`	`18.00% <0.00%> (?%)`
.../metrics/aggregator/DoubleLastValueAggregator.java	`100.00% <0.00%> (ø)`	`7.00% <0.00%> (?%)`
.../io/opentelemetry/sdk/trace/NoopSpanProcessor.java	`100.00% <0.00%> (ø)`	`8.00% <0.00%> (?%)`
...io/opentelemetry/sdk/trace/data/ImmutableLink.java	`100.00% <0.00%> (ø)`	`5.00% <0.00%> (?%)`
...lemetry/context/PersistentHashArrayMappedTrie.java	`45.45% <0.00%> (ø)`	`4.00% <0.00%> (?%)`
...opagation/B3PropagatorInjectorMultipleHeaders.java	`100.00% <0.00%> (ø)`	`4.00% <0.00%> (?%)`
.../opentelemetry/sdk/metrics/InstrumentRegistry.java	`100.00% <0.00%> (ø)`	`6.00% <0.00%> (?%)`
... and 207 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 28905b7...1999c72. Read the comment docs.

richardstartin · 2020-08-18T13:45:38Z

The problem with making this a collection is it will often need to be materialised, even if you can get away with calling Map.keySet in a lot of cases. Making it an Iterable would be lighter-weight in most cases, e.g. Kafka client, old enumeration based context carriers (e.g. JMS, HttpServletRequest)

carlosalberto · 2020-08-18T16:42:51Z

I'd be fine to have keys() return an Iterable or Iterator.

jkwatson · 2020-08-18T18:56:41Z

Can you add the concrete example of how the Jaeger propagator will use this new feature to this PR? I think that will be very useful for the spec PR, as well, to demonstrate how you're doing to use it.

Ew, Java and your improper type inference...

context_prop/src/main/java/io/opentelemetry/context/propagation/TextMapPropagator.java

jkwatson · 2020-10-26T18:25:33Z

context_prop/src/main/java/io/opentelemetry/context/propagation/TextMapPropagator.java

+     * Returns all the keys in the given carrier.
+     *
+     * @param carrier carrier of propagation fields, such as an http request.
+     * @since 0.10.0


I think we've gotten rid of all the @since tags for now; we'll set them all to 1.0 when we're ready to release.

Oh, I totally forgot about it. Will do.

jkwatson · 2020-10-26T18:30:29Z

...ropagators/src/main/java/io/opentelemetry/extensions/trace/propagation/JaegerPropagator.java

+    return builder == null ? null : builder.build();
+  }
+
+  @SuppressWarnings("StringSplitter")


not relevant for this PR in particular, but I think we should turn off this warning globally, since it's telling us to use guava, which we don't to do for the API, and won't, ever.

jkwatson · 2020-10-26T18:33:23Z

...gators/src/test/java/io/opentelemetry/extensions/trace/propagation/JaegerPropagatorTest.java

  @Test
  void extract_Nothing() {
    // Context remains untouched.
    assertThat(
            jaegerPropagator.extract(
-                Context.current(), Collections.<String, String>emptyMap(), Map::get))
+                Context.current(), Collections.<String, String>emptyMap(), getter))


might be a little safer/cleaner to just use Context.root() here, so we know that we don't have a pre-polluted context.

Fair enough, will update.

jkwatson · 2020-10-26T18:38:10Z

This looks ok (although it makes me sad that our Getter is no longer a SMI). Based on the discussion above, I had thought you would go with an Iterable/Iterator, rather Collection as the return type for keys(). Did something change your mind?

carlosalberto · 2020-10-27T01:11:30Z

I had thought you would go with an Iterable/Iterator, rather Collection as the return type for keys(). Did something change your mind?

Wanted to give a last try at Collection and to verify it's still fine ;) But if you think we should try enumerable instead (as mentioned in the previous related conversation), then I will go with enumerable for sure.

jpkrohling

The Jaeger specific parts are looking good to me, but I'm not an expert in the propagation format. Perhaps @objectiser or @yurishkuro would want to double-check as well?

anuraaga

Mostly looks nice thanks

...ropagators/src/main/java/io/opentelemetry/extensions/trace/propagation/JaegerPropagator.java

anuraaga · 2020-10-27T10:19:58Z

...ropagators/src/main/java/io/opentelemetry/extensions/trace/propagation/JaegerPropagator.java

+
+    for (String key : getter.keys(carrier)) {
+      if (key.startsWith(BAGGAGE_PREFIX)) {
+        if (key.length() <= BAGGAGE_PREFIX.length()) {


Since we already checked startsWith I think this can be equals check

I'd rather stick to check the length, as it only depends on a numerical comparison.

Ah I meant an equal check on the length - if it starts with it, then the length is guarateed to be at least that length and the < here looks redundant

...ropagators/src/main/java/io/opentelemetry/extensions/trace/propagation/JaegerPropagator.java

...gators/src/test/java/io/opentelemetry/extensions/trace/propagation/JaegerPropagatorTest.java

yurishkuro · 2020-10-27T13:55:13Z

In Jaeger there's two TextMap propagators registered by default (encoded and non-encoded propagation). Which one is the default?

Neither is default. In OT, HTTP and TextMap are different, so Jaeger url-encodes baggage for HTTP, but not for TextMap (eg in Kafka headers there's no need to url-encode).

The jaeger-baggage header support only happens at extraction, not injection, which makes me believe it's deprecated? Should we mention this in the docs?

That is correct. The official baggage format (in and out) is uberctx-{key}. jaeger-baggage header was invented later for manual injection via curl.

context_prop/src/main/java/io/opentelemetry/context/propagation/TextMapPropagator.java

carlosalberto · 2020-10-27T17:16:07Z

Neither is default. In OT, HTTP and TextMap are different, so Jaeger url-encodes baggage for HTTP, but not for TextMap (eg in Kafka headers there's no need to url-encode).

Oh, interesting. @yurishkuro , what do you think we should do here? Maybe have two versions, one that does encoding, one that does not? Or maybe something else? e.g.

  private JaegerPropagator(bolean doBaggageEncoding) {
     this.doBaggageEncoding = doBaggageEncoding; // Depending on this, baggage will be encoded or not.
  }

...ropagators/src/main/java/io/opentelemetry/extensions/trace/propagation/JaegerPropagator.java

carlosalberto · 2020-11-04T00:55:41Z

@open-telemetry/java-approvers Please review the latest updates.

@yurishkuro Updated the code based on your feedback. In a follow up we can add the encoded-values case, if that works for you.

carlosalberto · 2020-11-04T00:58:47Z

@yurishkuro Also, there's the question on whether, upon no existing baggage at all in the carrier, we should put an empty Baggage in Context (for example, if SpanContext was extracted but no baggage was detected at alll).

yurishkuro · 2020-11-04T01:05:23Z

upon no existing baggage at all in the carrier, we should put an empty Baggage in Context

I would try to avoid that. Even if there is an immutable singleton Baggage.empty(), adding it into an immutable context still requires re-allocation of the context, which is a malloc for little benefit. If would just make Baggage.fromContext(ctx) return empty singleton when key is not present.

jkwatson · 2020-11-04T04:16:27Z

looks like this needs a rebase, @carlosalberto

carlosalberto · 2020-11-04T22:35:45Z

@jkwatson Rebased ;)

Initial addition of Getter.keys().

823c0ad

carlosalberto mentioned this pull request Aug 18, 2020

Add Getter's Keys operation. open-telemetry/opentelemetry-specification#825

Merged

carlosalberto added 9 commits October 19, 2020 00:45

Merge branch 'master' into getter_keys

38246f7

WIP.

1146897

Merge branch 'master' into getter_keys

28ef667

More updates to the Jaeger propagator.

f38e375

Merge branch 'master' into getter_keys

95d5f54

Fix the build.

14b4134

Ew, Java and your improper type inference...

Add jaeger propagation tests.

dd21377

Merge branch 'master' into getter_keys

91dda85

Getter.keys() exists since 0.10.0

62f506d

carlosalberto marked this pull request as ready for review October 26, 2020 13:40

carlosalberto requested review from anuraaga, arminru, bogdandrutu, jkwatson, pavolloffay, thisthat and tylerbenson as code owners October 26, 2020 13:40

carlosalberto changed the title ~~Initial addition of Getter.keys().~~ Add Getter.Keys() with Jaeger Baggage support. Oct 26, 2020

jkwatson reviewed Oct 26, 2020

View reviewed changes

context_prop/src/main/java/io/opentelemetry/context/propagation/TextMapPropagator.java Outdated Show resolved Hide resolved

jkwatson reviewed Oct 26, 2020

View reviewed changes

jpkrohling reviewed Oct 27, 2020

View reviewed changes

anuraaga reviewed Oct 27, 2020

View reviewed changes

yurishkuro reviewed Oct 27, 2020

View reviewed changes

context_prop/src/main/java/io/opentelemetry/context/propagation/TextMapPropagator.java Outdated Show resolved Hide resolved

yurishkuro reviewed Oct 27, 2020

View reviewed changes

...ropagators/src/main/java/io/opentelemetry/extensions/trace/propagation/JaegerPropagator.java Outdated Show resolved Hide resolved

carlosalberto added 2 commits November 3, 2020 23:11

Merge branch 'master' into getter_keys

a171e0e

Use Iterable instead of Collection.

01e2341

carlosalberto force-pushed the getter_keys branch from 6582c36 to 01e2341 Compare November 3, 2020 23:31

Add a test case for input with spaces.

b6ef0cb

yurishkuro approved these changes Nov 4, 2020

View reviewed changes

anuraaga approved these changes Nov 4, 2020

View reviewed changes

jkwatson approved these changes Nov 4, 2020

View reviewed changes

carlosalberto added 2 commits November 4, 2020 23:32

Merge branch 'master' into getter_keys

1fdd610

Tune comparison against BAGGAGE_HEADER.

1999c72

carlosalberto merged commit 5195622 into open-telemetry:master Nov 5, 2020

carlosalberto mentioned this pull request Nov 6, 2020

Encode values for the Jaeger propagator #2032

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add Getter.Keys() with Jaeger Baggage support. #1549

Add Getter.Keys() with Jaeger Baggage support. #1549

carlosalberto commented Aug 18, 2020 •

edited

Loading

codecov bot commented Aug 18, 2020 •

edited

Loading

richardstartin commented Aug 18, 2020

carlosalberto commented Aug 18, 2020

jkwatson commented Aug 18, 2020

jkwatson Oct 26, 2020

carlosalberto Oct 27, 2020

jkwatson Oct 26, 2020

carlosalberto Oct 27, 2020

jkwatson Oct 26, 2020

carlosalberto Oct 27, 2020

jkwatson commented Oct 26, 2020

carlosalberto commented Oct 27, 2020

jpkrohling left a comment

anuraaga left a comment

anuraaga Oct 27, 2020

carlosalberto Oct 27, 2020

anuraaga Nov 4, 2020

carlosalberto Nov 4, 2020

yurishkuro commented Oct 27, 2020

carlosalberto commented Oct 27, 2020

carlosalberto commented Nov 4, 2020

carlosalberto commented Nov 4, 2020

yurishkuro commented Nov 4, 2020

jkwatson commented Nov 4, 2020

carlosalberto commented Nov 4, 2020

Add Getter.Keys() with Jaeger Baggage support. #1549

Add Getter.Keys() with Jaeger Baggage support. #1549

Conversation

carlosalberto commented Aug 18, 2020 • edited Loading

codecov bot commented Aug 18, 2020 • edited Loading

Codecov Report

richardstartin commented Aug 18, 2020

carlosalberto commented Aug 18, 2020

jkwatson commented Aug 18, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jkwatson commented Oct 26, 2020

carlosalberto commented Oct 27, 2020

jpkrohling left a comment

Choose a reason for hiding this comment

anuraaga left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

yurishkuro commented Oct 27, 2020

carlosalberto commented Oct 27, 2020

carlosalberto commented Nov 4, 2020

carlosalberto commented Nov 4, 2020

yurishkuro commented Nov 4, 2020

jkwatson commented Nov 4, 2020

carlosalberto commented Nov 4, 2020

carlosalberto commented Aug 18, 2020 •

edited

Loading

codecov bot commented Aug 18, 2020 •

edited

Loading