Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Consider FQ transport #1233

Open
codefromthecrypt opened this issue Aug 10, 2016 · 20 comments
Open

Consider FQ transport #1233

codefromthecrypt opened this issue Aug 10, 2016 · 20 comments

Comments

@codefromthecrypt
Copy link
Member

From @adriancole on May 26, 2016 21:27

Circonus are zipkin users who have had to work off forks due to the lack of a high-performance span transport.

Their library libmtev uses fq.

By adding an FQ transport, we can bring circonus and users of mtev back into the zipkin ecosystem, and no longer reliant on forks.

Copied from original issue: openzipkin/zipkin-java#252

@codefromthecrypt
Copy link
Member Author

This came out of a discussion with @postwait. Depending on how many folks are interested in this, it could end up in a separate repo or in the monorepo (this repo)

@codefromthecrypt
Copy link
Member Author

From @eirslett on May 26, 2016 21:58

What isn't high-performant about Kafka?

@codefromthecrypt
Copy link
Member Author

@eirslett well when they last used zipkin, it only supported scribe. I don't recall a performance comparison with kafka, although less complexity was mentioned (ex no zookeeper dep)

@codefromthecrypt
Copy link
Member Author

From @postwait on May 26, 2016 22:10

Not sure how a performance benchmark is relevant to the discussion? Fq can
easily saturate multiple 10Gbe links, but that isn't the point. The two
systems have different use cases and semantics. Fq is open, tiny, wildly
efficient, has very few dependencies and fits the use case for trace
dissemination quite well.

Additionally, I don't think anyone was suggesting the removal of Kafka -
that would be nuts.

@codefromthecrypt
Copy link
Member Author

Summary so far:

With some relative ease one could make a proof of concept which employs FQ as a custom collector.

It doesn't look viable to do more than that until fq has binaries and a jar published to artifact repositories. Our build and dev story is far easier to manage when we don't need to compile c as a prerequisite to running tests.

details..


Steps to create fq are like below main things is that it creates a fqd and friends, as well a jar file for the client connector. It wasn't hard to get it started though I haven't tested it, yet

I think the challenge we'd have with this is we usually get dependencies from packages or otherwise.

Testing would be easier to integrate if there were a current fq homebrew and apt (for our CI server) package. Otherwise we'd end up with what could be brittle pre-install routines that build the code.

There's also the java side, where we typically consume published jars that include some version of whatever the connector is. The fq code has no dependencies, so it could be feasible to build locally (ie source import), but my concern would be about support issues around that. Ex we don't have experience or a means to run unit tests against it.

# build and install ck
$ git clone https://github.com/concurrencykit/ck.git; cd ck/
$ ./configure && make && make install
$ cd ..

# build and install jlog
$ git clone https://github.com/omniti-labs/jlog.git; cd jlog/
$ autoconf; ./configure; make; make install
$ cd ..

# build and install fq
$ git clone https://github.com/circonus-labs/fq.git; cd fq/
$ make; make install

@codefromthecrypt
Copy link
Member Author

From @postwait on May 27, 2016 1:5

We'll look at publishing the fq java client jars to the maven central repo.

On Thu, May 26, 2016 at 6:58 PM, Adrian Cole notifications@github.com
wrote:

Summary so far:

With some relative ease one could make a proof of concept which employs FQ
as a custom collector.

It doesn't look viable to do more than that until fq has binaries and a
jar published to artifact repositories. Our build and dev story is far
easier to manage when we don't need to compile c as a prerequisite to
running tests.

details..

Steps to create fq are like below main things is that it creates a fqd and
friends, as well a jar file for the client connector. It wasn't hard to get
it started though I haven't tested it, yet

I think the challenge we'd have with this is we usually get dependencies
from packages or otherwise.

Testing would be easier to integrate if there were a current fq homebrew
and apt (for our CI server) package. Otherwise we'd end up with what could
be brittle pre-install routines that build the code.

There's also the java side, where we typically consume published jars that
include some version of whatever the connector is. The fq code has no
dependencies, so it could be feasible to build locally (ie source import),
but my concern would be about support issues around that. Ex we don't have
experience or a means to run unit tests against it.

build and install ck

$ git clone https://github.com/concurrencykit/ck.git; cd ck/
$ ./configure && make && make install
$ cd ..

build and install jlog

$ git clone https://github.com/omniti-labs/jlog.git; cd jlog/
$ autoconf; ./configure; make; make install
$ cd ..

build and install fq

$ git clone https://github.com/circonus-labs/fq.git; cd fq/
$ make; make install


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/openzipkin/zipkin-java/issues/252#issuecomment-222034932,
or mute the thread
https://github.com/notifications/unsubscribe/AAUfhCGmAJrgqqET8kgdTLictJOGAbZAks5qFkGqgaJpZM4In-Hj
.

@codefromthecrypt
Copy link
Member Author

Sounds good. So, I took a few stabs while in the air and probably didn't
concentrate hard enough.

A sample using the java api in a producer consumer context would be nice,
even better if it models how libmtev sends spans. Super kudos for a
health-check snippet. This sort of sample is something I could help put a
collector and test around.

Without a sample, it is harder (at least for me) to understand which
methods in the client interface I should be handling or paying attention to.

@codefromthecrypt
Copy link
Member Author

From @eirslett on May 27, 2016 8:39

I wasn't saying it's either Kafka or fq; I was just pointing out we already have ScribeFluentd and Kafka transports, both are widely known to be high-performant and fit the bill well, so I don't agree with @adriancole's claim. I think the primary reason one would use fq instead of Fluentd or Kafka, would be that you already have fq running on the servers, and don't have any of the other transports present?

I'm all for adding a transport for fq if people are willing to contribute one (adriancole sounds really motivated) but we should handle it the same way we handle zipkin-hbase, which is similar, community/ecosystem-wise.

@codefromthecrypt
Copy link
Member Author

Sheesh sorry about mis-introducing this.

There are many who have had problems with scribe due to its synchronous
protocol and often mention the double encoding in base64 as a performance
concern. At craftconf last year, these topics were brought up comparing
scribe to fq. I didn't expect that to be contentious, but seems to have
hijacked this conversation. Apologies for the distraction.

@codefromthecrypt
Copy link
Member Author

From @postwait on May 27, 2016 23:24

Backs away slowly and exits the room.
On May 27, 2016 7:03 PM, "Adrian Cole" notifications@github.com wrote:

Sheesh sorry about mis-introducing this.

There are many who have had problems with scribe due to its synchronous
protocol and often mention the double encoding in base64 as a performance
concern. At craftconf last year, these topics were brought up comparing
scribe to fq. I didn't expect that to be contentious, but seems to have
hijacked this conversation. Apologies for the distraction.


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/openzipkin/zipkin-java/issues/252#issuecomment-222271344,
or mute the thread
https://github.com/notifications/unsubscribe/AAUfhGBnyzRF_8nCecBo7CzRzkEFacq1ks5qF3hAgaJpZM4In-Hj
.

@codefromthecrypt
Copy link
Member Author

Ps to be transparent about this, I looked into this because I met circonus and we chatted :) It was interesting enough for me to want to take a look to see if I could help them move off a fork.

Spikes are important. I get swamped with other things and I don't usually have the energy to try lots of things simultaneously. I opened this issue to log thoughts so the energy I spent might become helpful to circonus.

Frankly, I am out of energy and already focusing on other things. That said, if we get a code snippet, I am interested enough to help sketch a proof of concept that Circonus might be able to use.

So motivation on my end is that I am motivated to finish the exploration, but I wouldn't conflate that with a priority of a somewhat niche transport being higher than our more requested backlog :)

@codefromthecrypt
Copy link
Member Author

Sorry about this, Theo. I'll close this issue as it has become a mess.

I'm still keen to help, but I think it would be less distracted to explore this out of the mono repo, as it would be less distracting.

@codefromthecrypt
Copy link
Member Author

From @postwait on June 7, 2016 14:28

@adriancole fqclient artifact is now published via maven http://search.maven.org/#artifactdetails%7Ccom.circonus%7Cfqclient%7C0.0.1%7Cjar

A default program for zipkin thrift-encoded messages would be: "prefix:"zipkin.thrift.""

The program we use to bridge fq onto scribe is here:

https://github.com/circonus-labs/fq2scribe

So a fully functional fq subscription to zipkin thrift data lies within:

https://github.com/circonus-labs/fq2scribe/blob/master/src/main/java/com/circonus/fq2scribe/Fq2Scribe.java

@codefromthecrypt
Copy link
Member Author

From @eirslett on June 7, 2016 16:35

Great work! I guess fq2scribe could be ported to c or go, for an additional performance boost?

@codefromthecrypt
Copy link
Member Author

From @postwait on June 7, 2016 16:40

It performs okay today, but it quite Rube Goldberg. If performance is
important,the next logical step would be to have openzipkin act directly as
an fq client...

On Tue, Jun 7, 2016 at 12:35 PM, Eirik Sletteberg notifications@github.com
wrote:

Great work! I guess fq2scribe could be ported to c or go, for an
additional performance boost?


You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/openzipkin/zipkin-java/issues/252#issuecomment-224338296,
or mute the thread
https://github.com/notifications/unsubscribe/AAUfhEXF4jlcfVtMEhShBITUYB7-Wqcgks5qJZ27gaJpZM4In-Hj
.

@codefromthecrypt
Copy link
Member Author

From @eirslett on June 7, 2016 17:8

Latency is not a big deal (1 second is fine end-to-end), but high throughput is critical.

@codefromthecrypt
Copy link
Member Author

From @postwait on June 7, 2016 17:10

The avoiding scribe itself is critical as it is RPC. Integrating an
fqclient directly makes even more sense.
On Jun 7, 2016 1:08 PM, "Eirik Sletteberg" notifications@github.com wrote:

Latency is not a big deal (1 second is fine end-to-end), but high
throughput is critical.


You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/openzipkin/zipkin-java/issues/252#issuecomment-224347882,
or mute the thread
https://github.com/notifications/unsubscribe/AAUfhMSgUb4Y13A7wi3XAdTDjYMFA9QMks5qJaV2gaJpZM4In-Hj
.

@codefromthecrypt
Copy link
Member Author

re-opening as the topic has been gratefully resurrected

@postwait
Copy link
Contributor

The code for an fqclient is here: https://github.com/circonus-labs/fq/tree/master/java

The published jars are here: https://mvnrepository.com/artifact/com.circonus/fqclient/0.0.1

I'm sure work can be done to improve performance over what is there already, we primarily use C to communicate with Fq.

@codefromthecrypt
Copy link
Member Author

codefromthecrypt commented Aug 10, 2016 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants