Implementing Collector for Azure EventHub

[aliostad]

Hi,

I am working on a Zipkin collector for Azure EventHub. The work itself is not demanding since most of the heavy lifting is done by the Azure Java SDK itself.

However I am not sure if I should contribute as part of Zipkin itself or it is able to load plugins and configurations from outside.

Can you please guide what is the best route?

Thanks

[codefromthecrypt]

Thanks for the ping, Ali! We sometimes get people asking about various aspects of azure. Ex https://groups.google.com/d/msg/zipkin-dev/bTBKUEjLtKg/yLbvVkFYAQAJ for storage There are two alternate collector projects: https://github.com/openzipkin/zipkin-aws < for amazon components, notable SQS https://github.com/GoogleCloudPlatform/stackdriver-zipkin <- actually more a storage bridge to Google Stackdriver on SDK vs small internal library Zipkin-aws uses the Amazon SDK to access SQS, though that may not always be the case. There are pros and cons to using an SDK for something like pulling from a queue. For example, usually the SDKs have quite a lot of dependencies, and the act of pulling from a queue is a simple rest call. Moreover, the rest calls involved in collections are usually extremely stable apis. Additional dependencies can often complicate interactions with existing code, and they typically not easy to instrument, for example internal debugging we call "self-tracing". On the other hand, sometimes SDKs are directly integrated in Spring Boot, which could cut down work. Also, SDKs often help with platform-specific concerns. For example, Amazon's includes the ability to refresh credentials which is not technically difficult, but work better deferred until it has to be changed. In the zipkin-aws project, code currently seems better off using Amazon's SDK than not, so that's why it does! back to the point.. Anyway, when making a zipkin-azure, I'd make the SDK an internal detail, in case it needs to change later, and follow the same pattern as zipkin-aws as that's almost exactly the same interaction you've mentioned. You can probably ping @llinder for advice, too. In both projects mentioned, there's a separate docker image at the moment, though we've regularly discussed a "layer" approach which could make it possible to add something like Azure EventHub to an existing container. next steps (imho).. The key next steps would be to look at the existing projects, and introduce yourself in gitter, as that's where we chat https://gitter.im/openzipkin/zipkin Then, decide whether you want to make this third-party or not. For example, zipkin-aws is released in the openzipkin org, which means a few of us can release it. Outside the zipkin org, you would be responsible for determining your own path, for example, Google have their own continuous integration and release setup (which seems to work well). Regardless, you'd want to make sure that there's some user who can vet that the result works. For example, one common problem with new code is that it is only used by the author. If you find a user or another stakeholder, the work will be more sustainable. Hope this helps! -A

[codefromthecrypt]

ps totally missed that there was a link in your description! https://github.com/aliostad/zipkin-collector-eventhub cool that the collector work is in progress. If I were you, I'd try to add offline tests to it (this allows others to help even if they don't have a cloud credential). We often use MockWebServer to do things like this, though google created a fake server of their own (since it was more gRPC than plain http). The key would be getting the SDK to allow you to override the URL to azure so that you can fake responses.

[codefromthecrypt]

I pinged twitter to hopefully interest others in contributing or helping https://twitter.com/zipkinproject/status/820801744833691651

[aliostad]

@adriancole Awesome mate! That is excellent.

Thanks I believe I have all I need. As you can see, it is only the skeleton with no embellishment, etc. As you might have guessed, Java is not my primary language but I will make the effort to ensure this is of the quality rest of zipkin is akin to.

I will close the issue but I might get back to you soon.

[codefromthecrypt]

looking forward to it!

[aliostad]

OK, I have made progress and trying to get it run. I know this could be a silly question but I am new to all this: how is zipkin-aws supposed to run for example?

My expectation for zipkin-collector-eventhub is that I drop zipkin-collector-eventhub.jar along with the zipkin-server.jar and run the zipkin-server.jar and specify configurations in application.yaml or application.properties and the Spring Boot autoconfig will load it and start running my host in addition to zipkin-server. But I have not been able to get this to work. Do I need to make my jar a Spring Boot application, like Stackdriver Trace Zipkin Collector?

Can you please provide some pointers?

Thanks a lot.

[codefromthecrypt]

hi, @aliostad Thanks for the update.

Right now, people are working with custom images, though there is a means to change how things work by using "zip" layout on the server, which allows use of the more customizable property launcher

There's another approach, too, a thin launcher.. I mention things about it here: #1219 (comment)

It is high time we had a suggested and testable way to compose server binaries.. let's follow-through on this.

[llinder]

@aliostad to answer the question about how I do this with zipkin-aws. Right now we make a fat jar by depending on zipkin and our extra auto configuration libs. From there we create a new layer on the zipkin docker container by exploding the fat jar in the same way the upstream build does. This is fairly easy though it does make the container larger than needed since the old layer still remains.

Ideally we need to figure out an elegant way to only layer in new jars. Both the property launcher and thin launcher sound promising though I don't have experience with either so I'm anxious to see what the developer experience would be like.

[aliostad]

@llinder Hi, thanks for the comments. So this fat jar, what script gets used to build this? I would like to see what the process is.

This weekend I will spend some time getting my head around thin launcher.

[codefromthecrypt]

I don't think this exists in zipkin-aws at the moment.. there is a version of what was originally to be merged here: https://github.com/openzipkin/zipkin-aws/pull/26/files#diff-67f9e3155157d7189b9915f13d8a3669 the spring-boot-maven-plugin makes the fat jar,

[aliostad]

OK I have been looking into samples of thin launcher. I cannot see any ZIP layout so properties can be changed, am I missing something or we can configure thin launcher in other means?

[UPDATE]

I finally got zipkin-server to read application.properties using --spring.config.location. So I guess I don't need any magic for supplying config as I can use the same trick.

[codefromthecrypt]

actually, the thin plugin is available, you just need to use snapshot repo

ex.
http://repo.spring.io/libs-snapshot/org/springframework/boot/experimental/spring-boot-thin-maven-plugin/

[codefromthecrypt]

Update: figured out how to get modularity working. It is a little constrained, but it works now.

Here's how.

Step 1: Add spring boot plugin to your autoconfig module, adding a "module" jar

The classifier name "module" is arbitrary, but I think it works. Take extreme care to not duplicate jars already in zipkin-server's exec jar. Here's an example configuration for the SQS collector:

  <dependencyManagement>
    <dependencies>
      <dependency>
        <!-- Import dependency management from Spring Boot -->
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-dependencies</artifactId>
        <version>${spring-boot.version}</version>
        <type>pom</type>
        <scope>import</scope>
      </dependency>
    </dependencies>
  </dependencyManagement>

  <build>
    <plugins>
      <plugin>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-maven-plugin</artifactId>
        <executions>
          <execution>
            <goals>
              <goal>repackage</goal>
            </goals>
          </execution>
        </executions>
        <configuration>
          <layout>MODULE</layout>
          <classifier>module</classifier>
          <!-- https://github.com/spring-projects/spring-boot/issues/3426 transitive exclude doesn't work -->
          <excludeGroupIds>io.zipkin.java,org.springframework.boot,org.springframework,commons-codec,com.fasterxml.jackson.core,com.fasterxml.jackson.dataformat,org.apache.httpcomponents,commons-logging,joda-time,software.amazon.ion</excludeGroupIds>
          <!-- already packaged in zipkin-server -->
          <excludeArtifactIds>aws-java-sdk-core,aws-java-sdk-sts,jmespath-java</excludeArtifactIds>
        </configuration>
      </plugin>
    </plugins>
  </build>

This will make a file like.. autoconfigure/collector-sqs/target/zipkin-autoconfigure-collector-sqs-0.0.4-SNAPSHOT-module.jar

Step 2: Extract this module jar when composing a server layer

I cannot get the PropertiesLauncher configuration to accept the module-jar directly. However, the following does work.

1. extract your module jar somewhere (Ex to an extensions folder)
2. explicitly start zipkin using PropertiesLauncher with loader.path set to include that directory

Example:

$ java  -Dloader.path=/tmp/extensions -cp zipkin.jar org.springframework.boot.loader.PropertiesLauncher --zipkin.collector.sqs.queue-url=http://foobar

Note: the module jar is self-contained.. when doing your devopsian things, feel free to just curl it from jcenter, like we do for the server jar.

[aliostad]

Wow! @adriancole that is so nice! Thanks a lot. I will hopefully get to play with this today or tomorrow.

[aliostad]

So unless I am missing something, this is not launcher, no?

So do you think it is worth to try that out (I was trying to get my head around it and made some progress) or this is the canonical way to go (using MODULE to create a fat jar)?

[codefromthecrypt]

So unless I am missing something, this is not launcher, no?

No, this isn't https://github.com/dsyer/spring-boot-thin-launcher it is using a modularity feature that's been in spring boot for a while

So do you think it is worth to try out that (I was trying to get my head around it and made some progress) or this is the canonical way to go?

Since the thin launcher is still experimental, we'd have a race condition on it finishing or us :) There's also matters like docs support etc which are trickier on experimental features. I'd say that the best way forward for today is to use the module approach, and then have an experimental (or even alternate docker image) for the thin launcher. In other words, module for tactical and short term, and thin launcher as an experiment (possible future replacement) my 2p

[aliostad]

Thank you. I will carry on with this. Do I need to define Start-Class (was not in your configuration above)? Here is what I get:

Exception in thread "main" java.lang.IllegalStateException: No 'loader.main' or 'Start-Class' specified
	at org.springframework.boot.loader.PropertiesLauncher.getMainClass(PropertiesLauncher.java:317)
	at org.springframework.boot.loader.Launcher.launch(Launcher.java:50)
	at org.springframework.boot.loader.PropertiesLauncher.main(PropertiesLauncher.java:521)

And if so, I guess it should be from zipkin-server?

Using java -jar works but does not load my stuff.

PS. Sorry for asking many questions... I am actually pleased I have made it this far. With little background soon will be publishing my contribution 👍

[codefromthecrypt]

@aliostad so keep in mind this is using the default zipkin exec jar (which has the start class). You would be making the module that adds to it.

Ex. you should be able to download and use an unmodified zipkin-server jar like below.. (which was zipkin.jar in my example)

https://github.com/openzipkin/zipkin#quick-start

also feel free to hop on https://gitter.im/openzipkin/zipkin if you get stuck! Thanks for the help on this

[aliostad]

Sure, I will go to gitter. Just to clarify I WAS using the original zipkin.jar but as I said, it was not working with -cp.

[codefromthecrypt]

Sure, I will go to gitter. Just to clarify I WAS using the original zipkin.jar but as I said, it was not working with -cp.

sorry about that. It might be best to push what you have to your repo (or somewhere), and add instructions for how you get the error. Then, myself or someone else could try. It might be better than guessing what's up

[aliostad]

OK, this is working now 👍

Impossible without your help. I am closing this now, I have everything I need.

[codefromthecrypt]

OK, this is working now 👍

yay

Impossible without your help. I am closing this now, I have everything I need.

hah.. sorry about that. you caught us at a good time, though as we were vetting ideas on how to do modularity!

[aliostad]

I am pleased this helped pushing for better modularity! It looks awesomely clean yet pluggable.