Skip to content
This repository has been archived by the owner on Jul 1, 2022. It is now read-only.

Refactor public builder API #346

Merged
merged 11 commits into from
Feb 26, 2018

Conversation

pavolloffay
Copy link
Member

@pavolloffay pavolloffay commented Feb 21, 2018

Resolves #258 #44
Supersedes #336

The idea is to use Configuration only for creating Tracer.Builder from environmental variables (or in the future from whatever source).

Reporter, Sender, Sampler classes are arbitrary difficult to construct therefore they might implement a builder interface defined in main Reporter, Sender, Sampler interface class.

Other high-level concepts:

  • tracer builder is opinionated and without any configuration it creates a tracer with service name unknown and sends data to the local agent.
  • impl classes like senders, reporter holds the default configuration e.g. default flush interval, default hostname of the agent.

This is WIP without javadoc, tests. Please just look at the overall approach whether it is something we want.

@ghost ghost assigned pavolloffay Feb 21, 2018
@ghost ghost added the review label Feb 21, 2018
@pavolloffay
Copy link
Member Author

cc @jpkrohling @yurishkuro

@@ -449,6 +453,10 @@ public Scope startActive(boolean finishSpanOnClose) {
private ScopeManager scopeManager = new ThreadLocalScopeManager();
private BaggageRestrictionManager baggageRestrictionManager = new DefaultBaggageRestrictionManager();

public Builder() {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe Tracer.builder(serviceName) instead? What are the conventions in Java these days? I personally prefer static methods to constructors.

return new Tracer(this.serviceName, reporter, sampler, registry, clock, metrics, tags,
if (serviceName == null) {
serviceName = "unknown";
log.warn("Service name hasn't been configured, using " + serviceName + " instead");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the argument for allowing "unknown"? Service name is a fundamental concept in Jaeger, I would rather require it in the builder() func.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 there si a comment in a PR description about this

}
Reporter reporter = new RemoteReporter(sender, flushInterval, maxQueueSize, metrics);
if (withLogging) {
reporter = new CompositeReporter(new LoggingReporter(), reporter);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is one reason why the separation of configuration exists. I think it breaks encapsulation if RemoteReporter suddenly becomes aware of things like logging reporter.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It can be removed from here, I think it's better to add it directly in TracerBuilder

@@ -29,4 +29,8 @@
* Release any resources used by the sampler.
*/
void close();

interface Builder {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why is this needed?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The reason is mentioned in PR description. Some samplers are easy to create, however some (like RemoteControlledSampler) has several configuration options therefore there is a generic builder.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

when someone creates a concrete sampler/reporter, they are dealing with a concrete class. If it happened to have a builder, it would be specific to that class. So I am not seeing the value of a common builder interface, unless the intention is to pass builders around.

* This constructor expects Jaeger running running on {@value #DEFAULT_AGENT_UDP_HOST} and port {@value #DEFAULT_AGENT_UDP_COMPACT_PORT}
*/
public UdpSender() {
this(DEFAULT_AGENT_UDP_HOST, DEFAULT_AGENT_UDP_COMPACT_PORT, 0);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is maxPacketSize=0 valid?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is, look how it was called from the configuration class. I don't like that default values are defined in configuration classes.

@@ -175,10 +175,12 @@
*/
private Tracer tracer;

@Deprecated
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does this still allow populating Configuration object from a config file?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ignore this, I want to say that only Tracer.Builder builder = Configuration.fomEnv() should be used. See PR description

@pavolloffay
Copy link
Member Author

@yurishkuro thanks for the review. Do you think we want to proceed with this approach? Don't do nit pitching, just see the overall approach described in the first comment in this PR.

@yurishkuro
Copy link
Member

I do like the idea of coalescing defaults into one place. Whether that's configs or builders - no strong opinion.

The original issue was about config classes being difficult to construct programmatically. I think this PR's approach simply bypasses that problem by offering construction via builders directly in the components, i.e. if you're constructing programmatically you don't need to use the configs. +1 to that.

My only additional requirement is that changes to configs should be kept backwards compatible (deprecated - fine). The primary use case for config class at Uber is when it is populated from a config file, so that should still work (not sure if we have tests for that, maybe in dw module).

@pavolloffay
Copy link
Member Author

I won't remove config classes in this PR. internals in config classes will use these new builders. I will iterate over tomorrow to the final state.

The high level idea of this PR is to move builders to impl classes so impl classes are responsible of creating. Then there is a configuration layer which just maps configfiles/env/props to these builders.

The primary use case for config class at Uber is when it is populated from a config file, so that should still work (not sure if we have tests for that, maybe in dw module).

Are is it possible to share this? e.g. if there is a code which loads yaml/json and creates Jaeger tracer that could be moved here I guess.

@codecov
Copy link

codecov bot commented Feb 22, 2018

Codecov Report

Merging #346 into master will decrease coverage by 0.23%.
The diff coverage is 85.58%.

Impacted file tree graph

@@             Coverage Diff              @@
##             master     #346      +/-   ##
============================================
- Coverage     84.17%   83.93%   -0.24%     
- Complexity      583      590       +7     
============================================
  Files            92       92              
  Lines          2268     2390     +122     
  Branches        266      272       +6     
============================================
+ Hits           1909     2006      +97     
- Misses          257      281      +24     
- Partials        102      103       +1
Impacted Files Coverage Δ Complexity Δ
...com/uber/jaeger/samplers/ProbabilisticSampler.java 81.81% <ø> (ø) 12 <0> (ø) ⬇️
...ain/java/com/uber/jaeger/senders/ThriftSender.java 73.46% <ø> (ø) 10 <0> (ø) ⬇️
.../com/uber/jaeger/samplers/HttpSamplingManager.java 100% <100%> (ø) 6 <1> (+1) ⬆️
...c/main/java/com/uber/jaeger/senders/UdpSender.java 73.33% <100%> (+4.1%) 5 <1> (+1) ⬆️
.../uber/jaeger/samplers/RemoteControlledSampler.java 81.81% <63.15%> (-11.14%) 20 <1> (+1)
...ger-core/src/main/java/com/uber/jaeger/Tracer.java 84.61% <63.63%> (-2.46%) 21 <0> (ø)
...java/com/uber/jaeger/reporters/RemoteReporter.java 84.7% <87.5%> (+0.42%) 8 <0> (ø) ⬇️
...e/src/main/java/com/uber/jaeger/Configuration.java 88.96% <94.17%> (+1.36%) 35 <10> (+4) ⬆️
.../main/java/com/uber/jaeger/senders/HttpSender.java 90% <96.87%> (+1.76%) 8 <5> (ø) ⬇️
... and 1 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e152db4...91a3c7f. Read the comment docs.

@pavolloffay
Copy link
Member Author

I have moved all default values to impl classes.

To summarize this:

  • default values are moved to impl classes
  • Configuration classes are dedicated to hold and set/get only primitive values and in the end create tracer.
  • Configuration supports creating itself (then the tracer) from env vars or Map<string, string> (to be done separately)
  • Configuration supports creating from env and then overriding any property
  • Complicated classes with more properties have builders

Here is a code snippet:

    // Tracer Builder directly
    Tracer tracer = new Tracer.Builder("name")
        .withReporter(new Builder()
            .withSender(new HttpSender("http://localhost:8080"))
            .withFlushInterval(100)
            .build())
        .withStatsReporter(new NullStatsReporter())
        .withSampler(new RemoteControlledSampler.Builder("name")
            .withInitialSampler(new ConstSampler(true))
            .build())
        .build();

    // Configuration from env
    System.setProperty(Configuration.JAEGER_SERVICE_NAME, "name");
    Configuration configuration = Configuration.fromEnv();
    configuration.getReporterConfig()
        .getSenderConfiguration()
        .getBuilder()
        .endpoint("http://jaegereverywhere.com");
    configuration.getTracer();

    // Configuration directly
    new Configuration("name")
        .reporterConfiguration(new ReporterConfiguration()
            .senderConfiguration(new SenderConfiguration.Builder()
                .endpoint("http://jaegereverywhere.com")
                .build()))
      .getTracer();
  }

@pavolloffay
Copy link
Member Author

pavolloffay commented Feb 22, 2018

@yurishkuro @vprithvi @jpkrohling @objectiser could you please have a look?

PR is not ready to merge, but the overall ideas shouldn't change

Copy link
Member

@yurishkuro yurishkuro left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

general comment, just my preference but what if we make all new builder constructors private and instead have a static builder() method on the respective class, e.g. Tracer.builder(), RepoteReporter.builder()? I think it improves readability of initialization code.

@@ -263,56 +261,79 @@ public void setStatsFactory(StatsFactory statsFactory) {
this.statsFactory = statsFactory;
}

public Configuration reporterConfiguration(ReporterConfiguration reporterConfig) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: naming not consistent with setStatsFactory or with builders. Any reason not to converge on withXyz naming?

nit2: Configuration suffix can probably be dropped

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can, but there will be more deprecated methods.

}
};
}

public static class Builder {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we deprecate this class and just use SenderConfiguration directly?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am definitely +1 on that, I will try to do it

stringOrDefault(this.agentHost, UdpSender.DEFAULT_AGENT_UDP_HOST),
numberOrDefault(this.agentPort, UdpSender.DEFAULT_AGENT_UDP_COMPACT_PORT).intValue(),
stringOrDefault(this.builder.agentHost, UdpSender.DEFAULT_AGENT_UDP_HOST),
numberOrDefault(this.builder.agentPort, UdpSender.DEFAULT_AGENT_UDP_COMPACT_PORT).intValue(),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should the defaults be moved into respective builders, like UdpSender.Builder?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All defaults have been moved to impl classes

this.sampler = sampler;

public Builder(String serviceName) {
this.withServiceName(serviceName);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Q: what is the scenario where withServiceName() method is needed instead of providing the name to the ctor?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not really, It can be removed.

.withMetrics(metrics)
.build();
}
if (withLoggingSpans) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

copying from other PR: I prefer not to have this option on the Tracer.Builder, it creates unnecessary coupling with CompositeReporter, LoggingReporter, etc. This feature can be left to Config classes only.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would recommend to people start using Tracer.Builder When they use programmatic API. I think it's useful to have it here directly. I am not sure if the composite reporter is easy enough to find/use.

@@ -24,4 +24,8 @@
void report(Span span);

void close();

interface Builder<T extends Reporter> {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do not believe this interface is needed. No class actually uses it.

@@ -29,4 +29,8 @@
* Release any resources used by the sampler.
*/
void close();

interface Builder<T extends Sampler> {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

similarly, this is never used on the left hand side.

@@ -23,4 +23,8 @@
int flush() throws SenderException;

int close() throws SenderException;

interface Builder<T extends Sender> {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

never used

@@ -490,15 +553,15 @@ public String getAgentHost() {
return null;
}

return this.senderConfiguration.agentHost;
return this.senderConfiguration.builder.agentHost;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Possibly we should deprecate these accessors (getAgentHost/getAgentPort) as looks like they are only used in tests?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was thinking about this, We definitely need accessors like Configuration.getReporterConfiguration then getSenderConfiguration to be able to override sender without re-creating other configs.

I am wondering whether there is an use case when app gets the configuration from env and then changes behaviour - e.g. if used endpoint it will get it and override to https for example.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't think we should be supporting such use cases - but if someone wanted to make a case for it later, it could be discussed then.

@Override
public RemoteReporter build() {
if (sender == null) {
sender = new UdpSender(UdpSender.DEFAULT_AGENT_UDP_HOST, UdpSender.DEFAULT_AGENT_UDP_COMPACT_PORT, 0);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should just use the default constructor.

OkHttpClient.Builder clientBuilder = new OkHttpClient.Builder();
if (authInterceptor != null) {
clientBuilder.addInterceptor(authInterceptor);

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: remove line

@pavolloffay
Copy link
Member Author

general comment, just my preference but what if we make all new builder constructors private and instead have a static builder() method on the respective class, e.g. Tracer.builder(), RepoteReporter.builder()? I think it improves readability of initialization code.

Can do, we can talk about this cosmetics at the end.

@pavolloffay pavolloffay force-pushed the refactor-builders branch 2 times, most recently from 581d360 to fc17680 Compare February 23, 2018 15:45
@pavolloffay
Copy link
Member Author

I was able to rename conf method to with variants and deprecate SenderConfiguration.Builder. Overall I think this is getting to be ready soon

Copy link
Member

@yurishkuro yurishkuro left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

I wish lombok supported generation of with methods

public void setStatsFactory(StatsFactory statsFactory) {
this.statsFactory = statsFactory;
}

public Configuration withStatsFactory(StatsFactory statsFactory) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

an overlap with #349, I suggest we don't touch setStatsFactory in this PR and let it be deprecated in #349

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

reerted

}

/**
* Holds the configuration related to the sender. A sender can be a {@link HttpSender} or {@link UdpSender}
*
*/
@Getter
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks like a breaking change

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added back, although I think this was added only for tests by @jpkrohling. I don't think we had a requirement of exposing it.

@pavolloffay pavolloffay changed the title WIP: Refactor public builder API Refactor public builder API Feb 26, 2018
@pavolloffay pavolloffay force-pushed the refactor-builders branch 2 times, most recently from a950d09 to 395e90a Compare February 26, 2018 13:44
Signed-off-by: Pavol Loffay <ploffay@redhat.com>
Signed-off-by: Pavol Loffay <ploffay@redhat.com>
Signed-off-by: Pavol Loffay <ploffay@redhat.com>
Signed-off-by: Pavol Loffay <ploffay@redhat.com>
Copy link
Member

@yurishkuro yurishkuro left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@pavolloffay do you want to ramp up the coverage back into green? E.g. by simply invoking all the deprecated constructors.

@pavolloffay
Copy link
Member Author

@yurishkuro yes, that is what I am trying. I will merge on green

Signed-off-by: Pavol Loffay <ploffay@redhat.com>
Signed-off-by: Pavol Loffay <ploffay@redhat.com>
Signed-off-by: Pavol Loffay <ploffay@redhat.com>
Signed-off-by: Pavol Loffay <ploffay@redhat.com>
Signed-off-by: Pavol Loffay <ploffay@redhat.com>
@codecov-io
Copy link

codecov-io commented Feb 26, 2018

Codecov Report

Merging #346 into master will increase coverage by 0.37%.
The diff coverage is 91.62%.

Impacted file tree graph

@@             Coverage Diff              @@
##             master     #346      +/-   ##
============================================
+ Coverage     84.17%   84.54%   +0.37%     
- Complexity      583      591       +8     
============================================
  Files            92       92              
  Lines          2268     2388     +120     
  Branches        266      272       +6     
============================================
+ Hits           1909     2019     +110     
- Misses          257      270      +13     
+ Partials        102       99       -3
Impacted Files Coverage Δ Complexity Δ
...ain/java/com/uber/jaeger/senders/ThriftSender.java 73.46% <ø> (ø) 10 <0> (ø) ⬇️
...com/uber/jaeger/samplers/ProbabilisticSampler.java 81.81% <ø> (ø) 12 <0> (ø) ⬇️
...c/main/java/com/uber/jaeger/senders/UdpSender.java 73.33% <100%> (+4.1%) 5 <1> (+1) ⬆️
.../com/uber/jaeger/samplers/HttpSamplingManager.java 100% <100%> (ø) 6 <1> (+1) ⬆️
...ger-core/src/main/java/com/uber/jaeger/Tracer.java 87.85% <100%> (+0.78%) 21 <0> (ø) ⬇️
.../uber/jaeger/samplers/RemoteControlledSampler.java 85.85% <73.68%> (-7.1%) 20 <1> (+1)
...java/com/uber/jaeger/reporters/RemoteReporter.java 84.7% <87.5%> (+0.42%) 8 <0> (ø) ⬇️
...e/src/main/java/com/uber/jaeger/Configuration.java 89.86% <95.14%> (+2.27%) 36 <11> (+5) ⬆️
.../main/java/com/uber/jaeger/senders/HttpSender.java 90% <96.87%> (+1.76%) 8 <5> (ø) ⬇️
... and 1 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e152db4...9958d13. Read the comment docs.

Signed-off-by: Pavol Loffay <ploffay@redhat.com>
}

public Map<String, String> getTracerTags() {
return tracerTags;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be wrapped in a Collections#unmodifiableMap.

}

public Configuration withTracerTags(Map<String, String> tracerTags) {
this.tracerTags = tracerTags;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You should use a new map, like new HashMap(tracerTags), so that the caller won't change your copy of the map.

Signed-off-by: Pavol Loffay <ploffay@redhat.com>
@pavolloffay pavolloffay merged commit 581e20f into jaegertracing:master Feb 26, 2018
@ghost ghost removed the review label Feb 26, 2018
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants