Integrate OpenTracing support

[alexmiller-apple]

https://opentracing.io gives a way for the lifetime of an RPC to be traced across multiple components in a distributed system. Think of this as the full solution to what debug transactions try to provide. It's also the one debugging tool that I miss having.

This will require RPCs to forward a UUID identifying the overall trace, and to forward that UUID along with any subsequent RPCs they make as a result.

Once we do this, it'd also be good to allow a UUID to be passed in from the client API for a transaction, so layers using opentracing could get the full view into FDB of what happens with their requests.

[ryanworl]

Where would trace data be emitted to? New entries in the existing trace logs?

[alexmiller-apple]

The variant of this I used before (I think) buffered RPC traces in memory, which were periodically pushed to the trace collection service. I imagine Zipkin does something similar, but I haven't looked.

I'd be somewhat concerned if the answer was they get dumped to a log file, as that would put a significant limiter on how many traces we could propagate information for.

[mfrw]

Just a humble opinion, instead of integrating with OpenTracing we should aim to integrate with OpenTelemetry.

tl;dr;

From the OpenTelemetry home page:

The leadership of OpenCensus and OpenTracing have come together to create OpenTelemetry, and it will supersede both projects. More details here.