Vert.x Roadmap

Julien Viet edited this page Aug 8, 2018 · 97 revisions

This document present the Vert.x projects global roadmap for the next releases.

Vert.x 4.0

  • Vert.x 4 modernizes a few important aspects of Vert.x 3 and breaking changes can be expected
  • Breaking changes should be kept minimal and used when it is necessary
  • 3.6 users will have an easy and documented migration path

4.0 main themes

  • dual asynchronous programming model CompletionStage / callback
  • split package removal
  • focus on backend clients usability and offering

CompletionStage support

The asynchronous programming model will evolve to support CompletionStage in addition of the callback approach. It is a fundamental change for users and thus the callback model will remain (in addition a portion of our users are fine with callbacks according to a survey).

Asynchronous methods are overloaded with a CompletionStage variant:

Future<T> asyncMethod(String s);
void asyncMethod(String s, Handler<AsyncResult<T>>);

Most Vert.x asynchronous methods can be migrated to this. There are a few edge cases to care about:

  • EventBus#send(String, Object) and EventBus#send(String, Object, Handler<AsyncResult<Message>>) don't have the same semantic
  • HttpClient methods are Handler based and not Handler<AsyncResult> based

Composing a resolved future should trampoline on the event loop instead of doing a direct execution, in order to provide a non racy model that people can reason about.

Polyglot is affected since the new programming model needs to be translated into other languages.

  • Java API languages shall be fine out of the box
  • Ruby, JS should have their model upgraded to a promise or keep the original wrapping ?
  • Code translation needs to be adapted as well

new JS stack

  • NPM first
    • Modules hosted on npm instead of maven central
    • TypeScript definition files bundled (hinting + intellisense)
    • Just Works(™) with modern IDEs
  • Benefits
    • Normal workflow for JS developers
    • Less maintenance (just 1 maven artifact)

Detyped redis Client

The 3.x client provides a type API modelled after Redis commands, this makes a strong coupling between the client API and the Redis version.

An detyped version of the client will provide a more generic framework for interacting with Redis and the client version will be independant of the Redis server version.

Redis redis = Redis.create(vertx);
redis.open(SocketAddress.inetSocketAddress(6379, "localhost"), open -> {
  if (open.succeeded()) {
    redis.send("set", Args.args().add("key").add("value"), ar -> {
      ...
    });
  }
});

The detyped API will also provide better Buffer support to minimize conversions.

Vert.x Web

Split package resolution

The package io.vertx.ext.web.templ is split between vertx-web and every engine implementation.

  • In 3.6 each engine is deprecated and the interface is copied to its own package.
  • in 4.0 the deprecated interface is removed

Modularity (not JPMS)

todo

Redis backed session

todo

Networking

Removals

List of project dropped after 3.x

  • vertx-rx-js
  • vertx-rx-groovy
  • vertx-mysql-postgres-client
  • vertx-jca
  • vertx-embedded-mongo-db: provides a way to start / stop mongo embedded as a vertx service, it is not really needed and worth to support
  • vertx-client-services: in practice client services (Mongo, Consul and Mail) are not much used and instead people use clients
  • vertx-lang-ceylon
  • vertx-amqp-bridge (TBD)
  • vertx-camel-bridge (TBD)
  • vertx-maven-service-factory (TBD)
  • vertx-service-factory (TBD)
  • vertx-http-service-factory (TBD)

Vert.x 3.6

Vert.x 3.6.x will be the last minor version of the 3.x release, it will remain LTS (until 12/2020) for bug fix releases and library version updates.

Asynchronous flow instrumentation

It is sometimes necessary for third-party libraries to follow the flow of an asynchronous program. Traditionally such libraries rely on the current thread to achieve it. Libraries falling this this category are most notably:

  • distributed tracing integration
  • thread local storage
  • MDC logging

Vert.x shall provide instrumentation of its asynchronous flows to enable these use cases.

See this RFC for the details.

Stream improvements

A Vert.x stream isn't trivial to implement and actual adapters using request flow control (RxJava,Reactive-Streams) have to maintain an extra queue for buffering unrequested items until they are requested.

The ReadStream interface should be augmented with a fetch operation that allows to provide request based control. The stream interface continues to operate with pause/resume control that is easier to use (and also more realistic with actual transports such as TCP, as request based flow control has to prefetch for efficiency in practice).

The state changes from two states (paused / flowing) to (fetching[n] / flowing), fetching[0] is equivalent to pause.

A fetch method is added to ReadStream:

public interface ReadStream<T> {

  // Set the stream into flowing mode
  @Fluent
  ReadStream<T> resume();

  // Set the stream into fetch mode with no fetched items
  @Fluent
  ReadStream<T> pause();

  // Set the stream into fetch mode with the specified amount of fetched items
  // or increase the current amount of fetched items if the stream was previously fetching
  @Fluent
  ReadStream<T> fetch(long amount);

  ...
}

ReadStream is difficult to implement and it is often implemented multiples times. We want to provide a queue interface that relieves the implementor to handle buffering, threading, reentrancy and data races. Currently this queue is maintained in several places implicitly (like HttpServerRequest or AsyncFile) and sometimes not:

public interface Queue<T> {

  // Add an item to the queue
  // returns whether more items can be added
  boolean add(T item);

  // Signal when the queue becomes writable again
  Queue<T> writableHandler(Handler<Void> handler);

  // Set the handler consuming the elements
  Queue<T> handler(Handler<T> elementHandler);

  // Queue flow control
  Queue<T> pause();
  Queue<T> resume();
  Queue<T> take(long amount);
}

Stack language generation

Until 3.6, each language provided a generation annotation processing plugin consumed by the components of the Vert.x stack to produce the polyglot bits in the component itself (so the component jar contained them). This worked well until 3.6 but created some friction between languages and also with JPMS.

Starting 3.6 we reverse this scheme and languages instead integrates with polyglot components instead. It will give the opportunity also for a language to improve the integration, e.g add custom extension methods in Kotlin. It also makes easier to support new languages (e.g es4x).

This feature has already been implemented in 3.6.

MVEL pruning

MVEL has compatibilities issues with Java 10 (blocker)

  • codegen without dependencies
  • replaced by pure Java code

At least MVEL removal in common generators (data object helper), since the stack structure has been modified the languages don't impact the components.

Other generators should be rewritten in a best effort fahsion, we aim at least RxJava and Kotlin (contributed), or they will be rewritten in 4.0 (mandatory).

There is a global task for this effort.

Codegen any Java type support

Currently codegen rejects methods declaring non any java types (e.g Thread getThread()) and such method must be annotated with @GenIgnore.

We want to weave this restriction, such methods can be used already with languages using the Vert.x API directly (Kotlin and Groovy) or capable of exposing Java types with no translation (RxJava1, RxJava2, ES4X, Scala?).

Such method gets extra flag in MethodInfo#isContainingJavaType() indicating wether it declares such types or not, allowing the code generator to filter the methods and only retain the one used.

Such method are explicitly tagged to be accepted by the processor, so we can easily distinguish in the source which method are available for which generator:

@SuppressWarnings("codegen-allow-any-java-type")
void deployVerticle(Verticle verticle);

See this RFC for the details.

Multi-release jar for vertx-codegen

Currently we need to use com.sun for Java 8 in this jar and Jdeps complains about it. MR jar shall allow to hide for Java 9. Implemented by https://github.com/vert-x3/vertx-codegen/issues/161

Kotlin coroutine generation

Provide code generation of Kotlin extension methods that augment the Vert.x APIs with suspending methods:

suspend fun <T : Any> HttpRequest<T>.send(): HttpResponse<T> {
  return awaitResult { this.send(it) }
}

// write this
val response = request.send();

// instead of
val response = awaitResult { request.send(it) }

This is a community contribution.

RxJava

toObserver() method generation

Generate WriteStream#toObserver() with back-pressure

Codegen any Java type support

Support codegen any java type methods, i.e expose methods having non code-generated types, with simple delegation. This will improve the usability of the API and require less helpers (such as verticle deploy helpers).

Integrate with ReadStream enhancement

Use the new stream fetch mode to avoid intermediary queue between in the stream to Observable/Flowable adapters.

Cassandra Client

Work In Progress as a Google Summer of Code contribution, based on top of the DataStax Java Driver.

Infinispan Client

A new module in the vertx-infinispan project that provides a lightweight / simple client for Infinispan based on the HotRod Java client

  • Polyglot client (including RxJava)
  • Manages the HotRod thread / Vert.x context switch
  • Compatibility with SharedData API
  • Client listeners
  • Continuous queries

Best effort delivery for 3.6.0

Mail client

Support for Oauth2 authentication.

Other features

These features could either be in 3.6 or 4.x depending on community contributions, they are currently not scheduled for a specific version.

Web client

  • authentication (basic, JWT, Oauth)
  • session / cookies
  • response expectations (status code, content-type)
  • URL templates
  • caching
  • websockets

SQL API evolution

  • Generic reactive SQL API
    • JDBC backend
    • ADBA sql2 support (depending on ADBA status and JDK)
  • MySQL protocol support for Reactive PG Client
  • vertx-jdbc-client and vertx-mysql-postgres-client dropped after 4.x

Extra redis features

  • Extendable to allow cluster and HA
  • Rewrite 3.x API on top of the new API for less maintenance (users can continue to use the typed API when needed)

Scala next

Wrap the Vert.x API using Scala Value Classes

  • generate asynchronous methods returning Scala's Future
  • @Nullable parameters wrapped with Scala's Option
  • Collection and primitive types left as is

The code generator will get simpler and this approach will expose all Vert.x types and have a smaller allocation rate.

Streaming over EventBus

The EventBus is not adapted for flow control or sending large payloads which are typical properties provided by streams. Instead of implementing these features in the event bus, we want instead to provide stream over the event bus, where the bus is used to lookup and negotiate streams between two parties. In addition it can be beneficial to Vert.x service proxies to provide streaming support in a similar fashion gRPC does.

Streaming the combination of a protocol for negotiating streams over the event bus and a transport. The base transport can be the event bus.

Opening a stream for read:

  • The client binds a unique consumer at $addr
  • The client sends a request/response message to the service addres
    • with header stream=open
    • With header addr=$addr
  • The service receives and
    • acks the message
    • emits messages to $addr until done

Opening a stream for write:

  • The client sends a request/response message to the service
    • with header stream=open
  • The service upon request
    • binds a consumer at $addr
    • acks the request with $addr
  • The client
    • receives the response
    • emits message to $addr until done

Service proxy generation:

Support streams in Proxygen in a similar fashion @ProxyClose does, extending the service proxy protocol:

JsonObject json = msg.body();
   switch (msg.headers().get("action")) {
      case “stream”:
        String addr = msg.headers().get(“addr”);
        … 
   }

Codegen extended function support

Codegen supports java.util.Function as argument or return type. It can be extended to support any functional interface declaring one parameter and returning non void type: Function, IntToLongFunction, etc...

gRPC

  • reimplement gRPC transport with Vert.x HTTP/2 support
    • gRPC is coupled to a specific version of Netty
    • removes the tight coupling to a Netty version
    • flow control is complicated to implement with gRPC
  • support reactive-grpc compiler extension
    • provides a code generator for RxJava2 API
    • possibly replacing the actual vertx specific code generator (less support)
You can’t perform that action at this time.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.
Press h to open a hovercard with more details.