Unable to fetch SamplingRules configured in AWS console

[anaynayak]

Hi,

With X-ray 2.0, I wanted to try out the option to manage sampling rules within the console itself. However the rules are not being applied/fetched from the console. In the stacktrace, I can see that it fails in the RulePoller trying to fetch the sampling rules.

We use a config equivalent to the following to initialize the global recorder:

            AWSXRayRecorderBuilder builder = AWSXRayRecorderBuilder.standard()
                .withPlugin(new EC2Plugin()).withPlugin(new ElasticBeanstalkPlugin())
                .withSamplingStrategy(new CentralizedSamplingStrategy(MyConfig.class.getResource("/xray/rules.json")))
                .withContextMissingStrategy(new NoOpXrayContextMissingStratergy()); //skips logging
            AWSXRay.setGlobalRecorder(builder.build());

Stacktrace:

2018-09-04 11:27:41,312 ERROR [pool-5-thread-1] c.a.x.s.sampling.pollers.RulePoller  Encountered error polling GetSamplingRules:
com.amazonaws.xray.exceptions.SegmentNotFoundException: Failed to begin subsegment named 'AWSXRay': segment cannot be found.
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
        at org.springsource.loaded.ri.ReflectiveInterceptor.jlrConstructorNewInstance(ReflectiveInterceptor.java:1002)
        at com.amazonaws.xray.strategy.RuntimeErrorContextMissingStrategy.contextMissing(RuntimeErrorContextMissingStrategy.java:17)
        at com.amazonaws.xray.contexts.ThreadLocalSegmentContext.beginSubsegment(ThreadLocalSegmentContext.java:23)
        at com.amazonaws.xray.AWSXRayRecorder.beginSubsegment(AWSXRayRecorder.java:418)
        at com.amazonaws.xray.handlers.TracingHandler.beforeRequest(TracingHandler.java:152)
        at com.amazonaws.http.AmazonHttpClient$RequestExecutor.runBeforeRequestHandlers(AmazonHttpClient.java:782)
        at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:726)
        at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:719)
        at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:701)
        at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:669)
        at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:651)
        at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:515)
        at com.amazonaws.services.xray.AWSXRayClient.doInvoke(AWSXRayClient.java:1257)
        at com.amazonaws.services.xray.AWSXRayClient.invoke(AWSXRayClient.java:1226)
        at com.amazonaws.services.xray.AWSXRayClient.invoke(AWSXRayClient.java:1215)
        at com.amazonaws.services.xray.AWSXRayClient.executeGetSamplingRules(AWSXRayClient.java:568)
        at com.amazonaws.services.xray.AWSXRayClient.getSamplingRules(AWSXRayClient.java:539)
        at com.amazonaws.xray.strategy.sampling.pollers.RulePoller.pollRule(RulePoller.java:53)
        at com.amazonaws.xray.strategy.sampling.pollers.RulePoller.lambda$start$0(RulePoller.java:41)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

Few details:

Versions:
com.amazonaws:aws-xray-recorder-sdk-core:2.0.0
com.amazonaws:aws-xray-recorder-sdk-aws-sdk:2.0.0
com.amazonaws:aws-xray-recorder-sdk-aws-sdk-instrumentor:2.0.0
com.amazonaws:aws-xray-recorder-sdk-apache-http:2.0.0
com.amazonaws:aws-java-sdk-xray:1.11.400

Deployed on:
ElasticBeanstalk Java 8 running on 64bit Amazon Linux/2.7.4

[steven-aerts]

I think this ticket might be a duplicate of #27

[anaynayak]

Yes #27 is a duplicate of this issue.

[awssandra]

Hi anaynayak,

Sorry for the issue. As per comment on #27 the AWSXRay client requests to the new Centralized Sampling API should not be traced as they are not part of the application's request code path. As of now, it looks like an exception is thrown due to the segment context expected but missing, and is not able to fetch the sampling rules as a result. We'll be investigating this and pushing a fix as soon as we can.

[haotianw465]

This bug is fixed at 2.0.1 which was just released. Please note it takes some time for the new version to propagate on maven central. Thanks.