Java11 httpclient instrumentation #251

XiXiaPdx · 2021-03-22T16:25:46Z

This adds instrumentation for the Java 11 HttpClient, for #128

ISSUE: The transactions take a long time to report to NR because there is an unexpired token in every. transaction. That token takes a default 180 seconds to timeout, finishing the transaction.

https://staging.one.nr/01qwL4EArw5

The token is created in the process of CompletableFutures. The issue is being tracked here #344

Non-instrumentation changes:

for getRuntimeMaxSupportedClassVersion in WeaveUtils, increased the max supported version to Java 11.
added addReadUnnamedModuleToHttpModule so java.net.http module can call into the unnamed module that holds the agent instrumentation utility classes. This required updating the module-utils dependency with this PR - add unnamed module to java.net.http for java 11 newrelic-module-util-java#13
Update Github actions to include JDK11

For the instrumentation:

A user of the HttpClient would build a HttpRequest. There is where we instrument and add headers.
The request would then be sent with the HttpClient. Whether by send or sendAsync, under the hood, both call into sendAsync on the HttpClientImpl. This is where the agent reports as external, using a segment
The HttpClient throws UnresolvedAddressExceptioninstead of UnknownHostException

XiXiaPdx · 2021-03-23T16:40:37Z

I thought the unexpired token issue was resolved so the integration test was merged. The test caught the unexpired tokens are still an issue. The merged test has been added to the ait stop list.

https://source.datanerd.us/java-agent/agent-integration-tests/pull/42

welintoncoradini · 2021-05-21T15:54:37Z

Early tests yielded positive results, NR trace.id is now being passed to the downstream entities. About a ~1/5 of the requests could be seen in the portal (10min delay).

XiXiaPdx · 2021-08-19T17:56:47Z

This is 🟢 for the AITs - https://javaagent-build.pdx.vm.datanerd.us/job/JavaAgent_PR_AIT/113/

But this 🟢 does not include the Java11 HttpClient specific AIT. That AIT is on the "stop list" because the instrumentation here has not been merged yet.

However, I've tested this branch against the specific Java11 HttpClient AIT locally and it passes.

twcrone · 2021-09-09T12:42:38Z

...n/httpclient-jdk11/src/main/java/com.nr.agent.instrumentation.httpclient/InboundWrapper.java

+
+public class InboundWrapper implements InboundHeaders {
+
+    HttpResponse delegate;


Why is this package private?

Good catch, will fix in update PR

twcrone · 2021-09-09T12:44:22Z

...rumentation/httpclient-jdk11/src/main/java/com.nr.agent.instrumentation.httpclient/Util.java

+import java.util.function.BiConsumer;
+import java.util.logging.Level;
+
+public class Util {


Might not be a big deal, but we might want to qualify our "Util" classes with a semi-unique prefix so when searching in IntelliJ its easier to resolve. Once again, maybe not a big deal for our work but in other projects I've been on it was.

Sounds good. What do you suggest here?

Once again, not suggesting you change now after the fact, but if you think its helpful in the future SomethingSpecificUtil might make it easier to find and distinguish.

twcrone · 2021-09-09T12:47:17Z

...ion/httpclient-jdk11/src/main/java/jdk/internal/net/http/HttpClientImpl_Instrumentation.java

+            throws IOException, InterruptedException {
+        URI uri = req.uri();
+        Segment segment = startSegment(uri);
+        HttpResponse<T> response;


This block would be one of those opportunities we discussed where a private (maybe static) method might help document what it does real quick. 'tryToProcessResponse' perhaps is quicker to understand than scanning the code. Just a thought, not meant to be a criticism by ANY means.

Yeah, I think it would not work or be helpful here now that I look at more closely. The Weaver.callOriginal() and segment stuff probably works better inline here rather than pulled out. I'll try to give a better example at some point.

twcrone · 2021-09-09T12:48:28Z

newrelic-agent/src/main/java/com/newrelic/bootstrap/BootstrapAgent.java

+        }
+    }
+
+    /**


Does Javadoc even get generated on private methods? If not, you can probably boil this one down to a regular comment.

Good to know on the javadoc for private methods. I’ve never thought about it so I did some research and it seems there are good opinions on this. I'm going to keep it as is with the current comment style. There are other private method javadocs in that class and probably throughout the agent. Whether right or wrong, there appears to be precedent in the agent.

Happy to change it if the team decides this format is not preferred!

I guess my only follow up point here is if our javadoc generator does not run against private methods currently, there is little point in using the verbose Javadoc formatting here.

/* Modify the java.net.http module so that it can read from the platform classloader's unnamed module. The agent http client instrumentation utility classes are in this specific unnamed module. ModuleUtil is compiled in a multi-release jar. In Java < 11, this results in a no-op implementation */

is probably sufficient without extra @param etc. that probably just clutters things up.

vijaykramesh · 2021-12-17T22:03:18Z

this seems to have broken scala on java 11 - see #438

XiXiaPdx mentioned this pull request Mar 22, 2021

Jdk11 httpclient #191

Closed

XiXiaPdx marked this pull request as draft April 16, 2021 16:23

XiXiaPdx marked this pull request as ready for review August 19, 2021 18:02

XiXiaPdx force-pushed the java11-httpclient-instrumentation branch from 9d0bf61 to 1f83b8f Compare September 2, 2021 18:16

XiXiaPdx added 4 commits September 2, 2021 11:37

Java 11 httpclient instrumentation

4e69a00

fix reporting external for unknown host

3319926

instrument HttpClientImpl

f776a45

reuse code in util

cde9435

XiXiaPdx force-pushed the java11-httpclient-instrumentation branch from 1f83b8f to cde9435 Compare September 2, 2021 18:40

add README.md

b65e1c7

jasonjkeller approved these changes Sep 3, 2021

View reviewed changes

XiXiaPdx merged commit 0d89111 into main Sep 8, 2021

XiXiaPdx deleted the java11-httpclient-instrumentation branch September 8, 2021 21:45

twcrone reviewed Sep 9, 2021

View reviewed changes

oryband mentioned this pull request Nov 26, 2021

apm support for java 11 http client DataDog/dd-trace-java#1478

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Java11 httpclient instrumentation #251

Java11 httpclient instrumentation #251

XiXiaPdx commented Mar 22, 2021 •

edited

Loading

XiXiaPdx commented Mar 23, 2021 •

edited

Loading

welintoncoradini commented May 21, 2021

XiXiaPdx commented Aug 19, 2021

twcrone Sep 9, 2021

XiXiaPdx Sep 9, 2021

twcrone Sep 9, 2021

XiXiaPdx Sep 9, 2021

twcrone Sep 9, 2021

twcrone Sep 9, 2021

twcrone Sep 9, 2021

twcrone Sep 9, 2021

XiXiaPdx Sep 9, 2021

twcrone Sep 9, 2021

vijaykramesh commented Dec 17, 2021


		public class InboundWrapper implements InboundHeaders {

		HttpResponse delegate;

+                      }
+                  }
+                  /**

Java11 httpclient instrumentation #251

Java11 httpclient instrumentation #251

Conversation

XiXiaPdx commented Mar 22, 2021 • edited Loading

XiXiaPdx commented Mar 23, 2021 • edited Loading

welintoncoradini commented May 21, 2021

XiXiaPdx commented Aug 19, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

vijaykramesh commented Dec 17, 2021

XiXiaPdx commented Mar 22, 2021 •

edited

Loading

XiXiaPdx commented Mar 23, 2021 •

edited

Loading