RFC: continuation instrumentation

IMPORTANT: discontinued effort in favor of https://github.com/vert-x3/wiki/wiki/RFC:-context-tracing

Continuation instrumentation

A set of hooks in Vert.x for instrumenting continuations, part of Vert.x value add.

• Goals
  • allow transparent propagation of thread local semantics (continuation/flow local) when using Vert.x APIs
  • no overhead when disabled
  • minimal work for integrating third party libraries
• Non goals
  • perform actual propagation (at least in vertx-core)
  • demarcate continuation / flows, i.e when to start/stop instrumenting, this is left to the actual metrics SPI

Tech preview in 4.0.

Current status:

• WIP for 3.6
• https://github.com/eclipse/vert.x/tree/continuation-instrumentation
• Prototype the instrumentation API
• API covered and tested
  • io.vertx.core.net
  • io.vertx.core.http
  • io.vertx.core.Context
  • io.vertx.core.shareddata
  • io.vertx.core.eventbus
  • Timers
• Todo
  • worker verticle support (with HttpServer, NetServer, etc...)
  • other vertx-core API (verticle deployment, etc...)
  • check how it works with API build on top of vertx-ore (vertx-web, vertx-web-client ,etc...)
  • Kotlin coroutines support
  • RxJava support
  • ???

SPI

The main interface in io.vertx.core.spi.instrumentation package:

Interface Instrumentation {

  // provides the opportunity to capture the actual context
  // and return an handler that will do some kind of propagation

  <T> Handler<T> captureContinuation(Handler<T> handler);

  ...
}

Currently obtained when creating a Vert.x instance from a static factory that is statically populated with a service loader SPI, there is also a static getter/setter for testing purposes.

Usage

The basic and simple idea is to capture an handler and provide the opportunity to return a wrapper that will be used instead

• when the handler is captured, the instrumentation has the opportunity to capture context (thread local, etc...)
• when the wrapped handler is called, the instrumentation has the opportunity to perform work around its invocation

Future like method

Most asynchronous methods.

public void performRequest(HttpClientRequest req, Handler<AsyncResult<HttpClientResponse>> callback) {
   // Capture flow
   Handler<AsyncResult<HttpClientResponse>> continuation = vertx.captureContinuation(callback);
   doRequest(req, ar -> {
     // Call the continuation with the result
     continuation.handle(ar.map(resp -> toHttpClientResponse(req, resp));
   });
}

Multiple callbacks

Various use cases

• the HttpClientRequest drain handler is called when the underlying channel is writable again (HTTP/1.x and HTTP/2 are difference cases)
• the HttpClientResponse handler will be called for every buffer received from the server
• setPeriodic
• etc...

private Handler<Buffer> handler;

public void handler(Handler<Buffer> handler) {
   // Capture continuation
   this.handler = vertx.captureContinuation(handler);
}

void handleBuffer(Buffer buff) {
  // Call the continuation with the result
  this.handler.handle(buff);
}

Instrumentation example

class myTracingInterceptor {
  public <T> Handler<T> wrap(Handler<T> handler) {
    MyContext ctx = getContext();
    if (ctx == null) {
      return handler;
    } else {
      return event -> {
        setContext(ctx);
        try { handler.handle(event) }
        finally { unsetContext(); }
      };
    }
  }

For a more complete example, see https://github.com/eclipse/vert.x/blob/continuation-instrumentation/src/test/java/io/vertx/test/core/instrumentation/TestInstrumentation.java

Vert.x API

Each Vert.x component will need to be checked with integration tests.

Some components will out of the box benefit from vertx-core instrumentation, e.g vertx-jdbc-client relies on executeBlocking, vertx-mqtt relies on NetClient/NetServer, so there should not be much to do.

Some components will require instrumentation, e.g vertx-mongo-client, etc...

User API

When user needs to integrate with third party asynchronous libraries, they often use runOnContext to go back on the Vert.x event loop.

Unfortunately code relying on runOnContext will not propagate context:

someMethod(result -> vertx.runOnContext(v -> {
  // Too late...
  callback.handle(result);
}));

So we need to provide an API for properly resuming a Vert.x continuation, e.g:

Handler<T> handler = vertx.captureContinuation(callback); 
someMethod(result -> vertx.runOnContext(v -> {
  handler.handle(result);
}));

This is a bit low level so we will likely provide an API for this, something like:

// An event dispatcher that when called dispatch the event on the current Vert.x context
// wrapping the actual handler
Handler<T> dispatcher = vertx.contextDispatcher(callback);
someMethod(result -> dispatcher.handle(result));

For Vert.x 4, we could use the CompletableFuture API:

// A completable future that when completed dispatches the event on the current Vert.x context
// wrapping the actual handler
CompletableFuture<T> future = vertx.completableFuture(callback);
someMethod(result -> {
  completion.complete(result);
});