Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added rate limit sampler via a property #1175

Closed
wants to merge 3 commits into from
Closed
Changes from 1 commit
Commits
File filter...
Filter file types
Jump to…
Jump to file or symbol
Failed to load files and symbols.

Always

Just for now

Next

Added rate limit sampler via a property; fixes gh-1162

  • Loading branch information...
marcingrzejszczak committed Jan 6, 2019
commit 3d12b6f8f1ee102892ad47eac7e2cad4b9ee0c6d
@@ -331,6 +331,8 @@ include::../../../../spring-cloud-sleuth-core/src/test/java/org/springframework/
TIP: You can set the HTTP header `X-B3-Flags` to `1`, or, when doing messaging, you can set the `spanFlags` header to `1`.
Doing so forces the current span to be exportable regardless of the sampling decision.

In order to use the rate-limited sampler set the `spring.sleuth.sampler.ratelimit` property to choose an amount of traces to accept on a per-second interval. The minimum number is 0 and the max is 2,147,483,647 (max int).
This conversation was marked as resolved by marcingrzejszczak

This comment has been minimized.

Copy link
@adriancole

adriancole Jan 6, 2019

Contributor

maybe spring.sleuth.sampler.rate to prevent accidental spring.sleuth.sampler.rateLimit?


== Propagation

Propagation is needed to ensure activities originating from the same root are collected together in the same trace.
@@ -121,6 +121,11 @@
<groupId>org.springframework</groupId>
<artifactId>spring-context</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-context</artifactId>
<optional>true</optional>
</dependency>
<dependency>
<groupId>com.netflix.hystrix</groupId>
<artifactId>hystrix-core</artifactId>
@@ -0,0 +1,47 @@
/*
* Copyright 2013-2018 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package org.springframework.cloud.sleuth.sampler;

import brave.sampler.Sampler;

/**
* The rate-limited sampler allows you to choose an amount of traces to accept on a
* per-second interval. The minimum number is 0 and the max is 2,147,483,647 (max int).
*
* You can read more about it in {@link brave.sampler.RateLimitingSampler}
*
* @author Marcin Grzejszczak
* @since 2.1.0
*/
public class RateLimitingSampler extends Sampler {
This conversation was marked as resolved by marcingrzejszczak

This comment has been minimized.

Copy link
@adriancole

adriancole Jan 6, 2019

Contributor

does this need to be public? just curious if this is only a convenience for auto-config.


private final Sampler sampler;

public RateLimitingSampler(SamplerProperties configuration) {
this.sampler = brave.sampler.RateLimitingSampler.create(rateLimit(configuration));
}

private Integer rateLimit(SamplerProperties configuration) {
return configuration.getRatelimit() != null ? configuration.getRatelimit() : 0;
}

@Override
public boolean isSampled(long traceId) {
return this.sampler.isSampled(traceId);
}

}
@@ -0,0 +1,72 @@
/*
* Copyright 2013-2018 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.springframework.cloud.sleuth.sampler;

import brave.sampler.Sampler;

import org.springframework.boot.autoconfigure.condition.ConditionalOnBean;
import org.springframework.boot.autoconfigure.condition.ConditionalOnMissingBean;
import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty;
import org.springframework.boot.context.properties.EnableConfigurationProperties;
import org.springframework.cloud.context.config.annotation.RefreshScope;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

/**
* {@link org.springframework.boot.autoconfigure.EnableAutoConfiguration
* Auto-configuration} to setup sampling for Spring Cloud Sleuth.
*
* @author Marcin Grzejszczak
* @since 2.1.0
*/
@Configuration
@ConditionalOnProperty(value = "spring.sleuth.enabled", matchIfMissing = true)
@EnableConfigurationProperties(SamplerProperties.class)
public class SamplerAutoConfiguration {

@Configuration
@ConditionalOnBean(type = "org.springframework.cloud.context.scope.refresh.RefreshScope")
protected static class RefreshScopedSamplerConfiguration {

@Bean
@RefreshScope
@ConditionalOnMissingBean
public Sampler defaultTraceSampler(SamplerProperties config) {
return samplerFromProps(config);
}

}

@Configuration
@ConditionalOnMissingBean(type = "org.springframework.cloud.context.scope.refresh.RefreshScope")
protected static class NonRefreshScopeSamplerConfiguration {

@Bean
@ConditionalOnMissingBean
public Sampler defaultTraceSampler(SamplerProperties config) {
return samplerFromProps(config);
}

}

static Sampler samplerFromProps(SamplerProperties config) {
if (config.getRatelimit() != null) {
return new RateLimitingSampler(config);
}
return new ProbabilityBasedSampler(config);
}

}
@@ -35,6 +35,18 @@
*/
private float probability = 0.1f;

/**
* A rate per second can be a nice choice for low-traffic endpoints as it allows you
* surge protection. For example, you may never expect the endpoint to get more than
* 50 requests per second. If there was a sudden surge of traffic, to 5000 requests
* per second, you would still end up with 50 traces per second. Conversely, if you
* had a percentage, like 10%, the same surge would end up with 500 traces per second,
* possibly overloading your storage. Amazon X-Ray includes a rate-limited sampler
* (named Reservoir) for this purpose. Brave has taken the same approach via the
* {@link brave.sampler.RateLimitingSampler}.
*/
private Integer ratelimit;

public float getProbability() {
return this.probability;
}
@@ -43,4 +55,12 @@ public void setProbability(float probability) {
this.probability = probability;
}

public Integer getRatelimit() {
return this.ratelimit;
}

public void setRatelimit(Integer ratelimit) {
this.ratelimit = ratelimit;
}

}
@@ -1,6 +1,7 @@
# Auto Configuration
org.springframework.boot.autoconfigure.EnableAutoConfiguration=\
org.springframework.cloud.sleuth.annotation.SleuthAnnotationAutoConfiguration,\
org.springframework.cloud.sleuth.sampler.SamplerAutoConfiguration,\
org.springframework.cloud.sleuth.autoconfig.TraceAutoConfiguration,\
org.springframework.cloud.sleuth.log.SleuthLogAutoConfiguration,\
org.springframework.cloud.sleuth.propagation.SleuthTagPropagationAutoConfiguration,\
@@ -0,0 +1,34 @@
package org.springframework.cloud.sleuth.sampler;

import brave.sampler.Sampler;
import org.assertj.core.api.BDDAssertions;
import org.junit.Test;

import static org.junit.Assert.*;

/**
* @author Marcin Grzejszczak
* @since
*/
public class SamplerAutoConfigurationTests {

@Test
public void should_use_rate_limit_sampler_when_property_set() {
SamplerProperties properties = new SamplerProperties();
properties.setRatelimit(10);

Sampler sampler = SamplerAutoConfiguration.samplerFromProps(properties);

BDDAssertions.then(sampler).isInstanceOf(RateLimitingSampler.class);
}

@Test
public void should_use_probability_sampler_when_rate_limiting_not_set() {
SamplerProperties properties = new SamplerProperties();

Sampler sampler = SamplerAutoConfiguration.samplerFromProps(properties);

BDDAssertions.then(sampler).isInstanceOf(ProbabilityBasedSampler.class);
}

}
@@ -45,11 +45,6 @@
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-commons</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-context</artifactId>
<optional>true</optional>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-actuator</artifactId>
@@ -16,59 +16,56 @@

package org.springframework.cloud.sleuth.zipkin2;

import brave.sampler.Sampler;
import java.util.concurrent.TimeUnit;

import zipkin2.Span;
import zipkin2.codec.BytesEncoder;
import zipkin2.reporter.AsyncReporter;
import zipkin2.reporter.InMemoryReporterMetrics;
import zipkin2.reporter.Reporter;
import zipkin2.reporter.ReporterMetrics;
import zipkin2.reporter.Sender;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.autoconfigure.AutoConfigureAfter;
import org.springframework.boot.autoconfigure.AutoConfigureBefore;
import org.springframework.boot.autoconfigure.condition.ConditionalOnBean;
import org.springframework.boot.autoconfigure.condition.ConditionalOnClass;
import org.springframework.boot.autoconfigure.condition.ConditionalOnMissingBean;
import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty;
import org.springframework.boot.autoconfigure.web.ServerProperties;
import org.springframework.boot.context.properties.EnableConfigurationProperties;
import org.springframework.cloud.client.serviceregistry.Registration;
import org.springframework.cloud.commons.util.InetUtils;
import org.springframework.cloud.context.config.annotation.RefreshScope;
import org.springframework.cloud.sleuth.autoconfig.TraceAutoConfiguration;
import org.springframework.cloud.sleuth.sampler.ProbabilityBasedSampler;
import org.springframework.cloud.sleuth.sampler.SamplerProperties;
import org.springframework.cloud.sleuth.sampler.SamplerAutoConfiguration;
import org.springframework.cloud.sleuth.zipkin2.sender.ZipkinSenderConfigurationImportSelector;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.Import;
import org.springframework.core.env.Environment;
import org.springframework.web.client.RestTemplate;
import zipkin2.Span;
import zipkin2.codec.BytesEncoder;
import zipkin2.reporter.AsyncReporter;
import zipkin2.reporter.InMemoryReporterMetrics;
import zipkin2.reporter.Reporter;
import zipkin2.reporter.ReporterMetrics;
import zipkin2.reporter.Sender;

import java.util.concurrent.TimeUnit;

/**
* {@link org.springframework.boot.autoconfigure.EnableAutoConfiguration
* Auto-configuration} enables reporting to Zipkin via HTTP. Has a default {@link Sampler}
* set as {@link ProbabilityBasedSampler}.
* Auto-configuration} enables reporting to Zipkin via HTTP. Has a default sampler
* set from the {@link SamplerAutoConfiguration}
*
* The {@link ZipkinRestTemplateCustomizer} allows you to customize the
* {@link RestTemplate} that is used to send Spans to Zipkin. Its default implementation -
* {@link DefaultZipkinRestTemplateCustomizer} adds the GZip compression.
*
* @author Spencer Gibb
* @since 1.0.0
* @see ProbabilityBasedSampler
* @see SamplerAutoConfiguration
* @see ZipkinRestTemplateCustomizer
* @see DefaultZipkinRestTemplateCustomizer
*/
@Configuration
@EnableConfigurationProperties({ ZipkinProperties.class, SamplerProperties.class })
@EnableConfigurationProperties(ZipkinProperties.class)
@ConditionalOnProperty(value = "spring.zipkin.enabled", matchIfMissing = true)
@AutoConfigureBefore(TraceAutoConfiguration.class)
@AutoConfigureAfter(name = "org.springframework.cloud.autoconfigure.RefreshAutoConfiguration")
@Import(ZipkinSenderConfigurationImportSelector.class)
@Import({ZipkinSenderConfigurationImportSelector.class, SamplerAutoConfiguration.class})
public class ZipkinAutoConfiguration {

/**
@@ -107,31 +104,6 @@ ReporterMetrics sleuthReporterMetrics() {
return new InMemoryReporterMetrics();
}

@Configuration
@ConditionalOnBean(type = "org.springframework.cloud.context.scope.refresh.RefreshScope")
protected static class RefreshScopedProbabilityBasedSamplerConfiguration {

@Bean
@RefreshScope
@ConditionalOnMissingBean
public Sampler defaultTraceSampler(SamplerProperties config) {
return new ProbabilityBasedSampler(config);
}

}

@Configuration
@ConditionalOnMissingBean(type = "org.springframework.cloud.context.scope.refresh.RefreshScope")
protected static class NonRefreshScopeProbabilityBasedSamplerConfiguration {

@Bean
@ConditionalOnMissingBean
public Sampler defaultTraceSampler(SamplerProperties config) {
return new ProbabilityBasedSampler(config);
}

}

@Configuration
@ConditionalOnMissingBean(EndpointLocator.class)
@ConditionalOnProperty(value = "spring.zipkin.locator.discovery.enabled", havingValue = "false", matchIfMissing = true)
ProTip! Use n and p to navigate between commits in a pull request.
You can’t perform that action at this time.