9. Distributed services

Rui Gu edited this page Oct 26, 2016 · 42 revisions

9.1. Remote service

Redis based distributed remote service for Java allows to execute object methods by remote interface on different Redisson instances. In other words it allows Java remote calls with Redis. Uses POJO objects, method parameters and result object type can literally be anything.

The RemoteService provides two types of RRemoteService instances:

  • Server side instance - executes remote method (worker instance). Example:
RRemoteService remoteService = redisson.getRemoteService();
SomeServiceImpl someServiceImpl = new SomeServiceImpl();

// register remote service before any remote invocation
// can handle only 1 invocation concurrently
remoteService.register(SomeServiceInterface.class, someServiceImpl);

// register remote service able to handle up to 12 invocations concurrently
remoteService.register(SomeServiceInterface.class, someServiceImpl, 12);
  • Client side instance - invokes remote method. Example:
RRemoteService remoteService = redisson.getRemoteService();
SomeServiceInterface service = remoteService.get(SomeServiceInterface.class);

String result = service.doSomeStuff(1L, "secondParam", new AnyParam());

Client and server side instances shall be using the same remote interface and backed by redisson instances created using the same server connection configuration. Client and server side instances could be run in same JVM. There are no limits to the amount of client and/or server instances. (Note: While redisson does not enforce any limits, limitations from Redis still apply.)

Remote invocations executes in parallel mode if 1+ workers are available.

The total number of parallel executors is calculated as such: T = R * N

T - total available parallel executors R - Redisson server side instance amount N - executors amount defined during service registration

Commands exceeding this number will be queued for the next available executor.

Remote invocations executes in sequential mode if only 1 workers are available. Only one command can be handled concurrently in this case and the rest of commands will be queued.

9.1.1. Remote service. Message flow

RemoteService creates two queues per invocation. One queue for request (being listened by server side instance) and another one is for ack-response and result-response (being listened by client side instance). Ack-response used to determine if method executor has got a request. If it doesn't during ack timeout then RemoteServiceAckTimeoutException will be thrown.

Below is depicted a message flow for each remote invocation.

9.1.2. Remote service. Fire-and-forget and ack-response modes

RemoteService offers some options for each remote invocation via org.redisson.core.RemoteInvocationOptions object. Such options allow to change timeouts and skip ack-response and/or result-response. Examples:

// 1 second ack timeout and 30 seconds execution timeout
RemoteInvocationOptions options = RemoteInvocationOptions.defaults();

// no ack but 30 seconds execution timeout
RemoteInvocationOptions options = RemoteInvocationOptions.defaults().noAck();

// 1 second ack timeout then forget the result
RemoteInvocationOptions options = RemoteInvocationOptions.defaults().noResult();

// 1 minute ack timeout then forget about the result
RemoteInvocationOptions options = RemoteInvocationOptions.defaults().expectAckWithin(1, TimeUnit.MINUTES).noResult();

// no ack and forget about the result (fire and forget)
RemoteInvocationOptions options = RemoteInvocationOptions.defaults().noAck().noResult();

RRemoteService remoteService = redisson.getRemoteService();
YourService service = remoteService.get(YourService.class, options);

9.1.3. Remote service. Asynchronous calls

Remote invocations could be made in asynchronous manner. Separate interface marked with @RRemoteAsync annotation should be used. Method signatures should be match with same methods in remote interface. Each method should return org.redisson.api.RFuture object. Asynchronous interface validation will be performed during RRemoteService.get method invocation. It's not necessary to list all of them, only those which need to be called in asynchronous manner.

public interface RemoteInterface {

    Long someMethod1(Long param1, String param2);

    void someMethod2(MyObject param);

    MyObject someMethod3();

}

// async interface for RemoteInterface
@RRemoteAsync(RemoteInterface.class)
public interface RemoteInterfaceAsync {

    RFuture<Long> someMethod1(Long param1, String param2);

    RFuture<Void> someMethod2(MyObject param);

}

RRemoteService remoteService = redisson.getRemoteService();
RemoteInterfaceAsync asyncService = remoteService.get(RemoteInterfaceAsync.class);

9.1.4. Remote service. Asynchronous call cancellation

It's easy to cancel asynchronous invocation using Future.cancel() method. Remote service offers ability to cancel invocation in any stages of its execution. There are three stages:

  1. Remote invocation request in queue
  2. Remote invocation request received by remote service but not lunched and Ack-response hasn't send yet
  3. Remote invocation execution in progress

To handle third stage you need to check for Thread.currentThread().isInterrupted() status in your Remote service code. Here is an example:

public interface MyRemoteInterface {

    Long myBusyMethod(Long param1, String param2);

}

// async interface for RemoteInterface
@RRemoteAsync(MyRemoteInterface.class)
public interface MyRemoteInterfaceAsync {

    RFuture<Long> myBusyMethod(Long param1, String param2);

}

// remote service implementation
public class MyRemoteServiceImpl implements MyRemoteInterface {

   public Long myBusyMethod(Long param1, String param2) {
       for (long i = 0; i < Long.MAX_VALUE; i++) {
           iterations.incrementAndGet();
           if (Thread.currentThread().isInterrupted()) {
                System.out.println("interrupted! " + i);
                return;
           }
       }
   }

}

RRemoteService remoteService = redisson.getRemoteService();
ExecutorService executor = Executors.newFixedThreadPool(5);
// register remote service using separate
// ExecutorService used to execute remote invocation
MyRemoteInterface serviceImpl = new MyRemoteServiceImpl();
remoteService.register(MyRemoteInterface.class, serviceImpl, 5, executor);

// call method
MyRemoteInterfaceAsync asyncService = remoteService.get(MyRemoteInterfaceAsync.class);
RFuture<Long> future = asyncService.myBusyMethod(1L, "someparam");
// cancel invocation
future.cancel(true);

9.2. Live Object service

9.2.1. Introduction

A Live Object can be understood as an enhanced version of standard Java object, of which an instance reference can be shared not only between threads in a single JVM, but can also be shared between different JVMs across different machines.Wikipedia discribes it as:

Live distributed object (also abbreviated as live object) refers to a running instance of a distributed multi-party (or peer-to-peer) protocol, viewed from the object-oriented perspective, as an entity that has a distinct identity, may encapsulate internal state and threads of execution, and that exhibits a well-defined externally visible behavior.

Redisson Live Object (RLO) realised this idea by mapping all the fields inside a Java class to a redis hash through a runtime-constructed proxy class. All the get/set methods of each field are translated to hget/hset commands operated on the redis hash, making it accessable to/from any clients connected to the same redis server. As we all know, the field values of an object represent its state; having them stored in a remote repository, redis, makes it a distributed object. This object is a Redisson Live Object.

By using RLO, sharing an object between applications and/or servers is the same as sharing one in a standalone application. This removes the need for serialization and deserialization, and at the same time reduces the complexity of the programming model: Changes made to one field is (almost^) immediately accessable to other processes, applications and servers. (^Redis' eventual consistant replication rule still applies when connected to slave nodes)

Since the redis server is a single-threaded application, all field access to the live object is automatically executed in atomic fashion: a value will not be changed when you are reading it.

With RLO, you can treat the redis server as a shared Heap space for all connected JVMs.

9.2.2. Usage

In order to enjoy all the benefits brought by RLO, only thing you need to do is annotate the Class you desire to use with @REntity, then annotate a field with @RId.

@REntity
public class MyLiveObject {
    @RId
    private String name;
    //other fields
    ...
    ...

    //getters and setters
    ...
    ...
}

Now you have made an otherwise standard Java object class into a Redisson Live Object class. You are able to get an instance of it with the RedissonLiveObjectService, which you can get from a RedissonClient.

...
RLiveObjectService service = redisson.getLiveObjectService();
MyLiveObject myObject1 = service.<MyLiveObject, String>getOrCreate(MyLiveObject.class, "myObjectId");
...

Using the Redisson Live Object is the same as using a standard Java object. Let's assume you have this object:

@REntity
public class MyObject {
    @RId
    private String name;
    private String value;

    public MyObject(String name) {
        this.name = name;
    }

    public MyObject() {
    }

    public String getName() {
        return name;
    }

    public String getValue() {
        return value;
    }

    public void setName(String name) {
        this.name = name;
    }

    public void setValue(String value) {
        this.value = value;
    }
}

Somewhere else in the code you may want to create it as a standard Java instance.

//Standard Java object instance
MyObject standardObject1 = new MyObject();
standardObject1.setName("standard1");

//Of course you can use non-default constructor
MyObject standardObject2 = new MyObject("standard2");

Elsewhere, you may also want to create it as a RLO instance

//first create the service
RLiveObjectService service = redisson.getLiveObjectService();

//instantiate the object with the service
MyObject liveObject1 = service.<MyObject, String>getOrCreate(MyObject.class, "liveObjectId");
//Behind scense, it tries to locate the constructor with one argument and invoke with the id value,
//"liveObjectId" in this case. If the constructor is not found, falls back on default constructor
//and then call setName("liveObjectId") before returns back to you.

There is literally no difference when it comes to using these instances.

//Setting the "value" field is the same
standardObject1.setValue("abc");//the value "abc" is stored in heapspace in VM

standardObject2.setValue("abc");//same as above

liveObject1.setValue("abc");
//the value "abc" is stored inside redis, no value is stored in heap. (OK, there
//is a string pool, but the value is not referenced here in the object, so it can
//be garbage collected.)

//Getting the "value" out is just the same
System.out.println(standardObject1.getValue());
//It should give you "abc" in the console, the value is retrieved from heapspace in the VM;

System.out.println(standardObject2.getValue());//same as above.

System.out.println(liveObject1.getValue());
//output is the same as above, but the value is retrieved from redis.

While these two snippets of code look exactly the same, there is a slight difference between them. Let me explain it with another example.

@REntity
public class MyLiveObject {
    @RId
    private String name;
    private MyOtherObject value;

    public MyLiveObject(String name) {
        this.name = name;
    }

    public MyObject() {
    }

    public String getName() {
        return name;
    }

    public MyOtherObject getValue() {
        return value;
    }

    public void setName(String name) {
        this.name = name;
    }

    public void setValue(MyOtherObject value) {
        this.value = value;
    }
}

In this case, the type of the "value" field is a mutable type. In a standard Java object, when you invoke the getValue() method, the reference to this MyOtherObject instance is returned to you. When you invoke the same method on a RLO, a reference of a new instance is returned. This can have the following two effects:

//Redisson Live Object behaviour:
MyLiveObject myLiveObject = service.getOrCreate(MyLiveObject.class, "1");
myLiveObject.setValue(new MyOtherObject());
System.out.println(myLiveObject.getValue() == myLiveObject.getValue());
//False (unless you use a custom Codec with object pooling)

//Standard Java Object behaviour:
MyLiveObject notLiveObject = new MyLiveObject();
notLiveObject.setValue(new MyOtherObject());
System.out.println(notLiveObject.getValue() == notLiveObject.getValue());
//True
//Redisson Live Object behaviour:
MyLiveObject myLiveObject = service.getOrCreate(MyLiveObject.class, "1");
MyOtherObject other = new MyOtherObject();
other.setOtherName("ABC");
myLiveObject.setValue(other);
System.out.println(myLiveObject.getValue().getOtherName());
//ABC

other.setOtherName("BCD");
System.out.println(myLiveObject.getValue().getOtherName());
//still ABC

myLiveObject.setValue(other);
System.out.println(myLiveObject.getValue().getOtherName());
//now it's BCD


//Standard Java Object behaviour:
MyLiveObject myLiveObject = service.getOrCreate(MyLiveObject.class, "1");
MyOtherObject other = new MyOtherObject();
other.setOtherName("ABC");
myLiveObject.setValue(other);
System.out.println(myLiveObject.getValue().getOtherName());
//ABC

other.setOtherName("BCD");
System.out.println(myLiveObject.getValue().getOtherName());
//already is BCD

myLiveObject.setValue(other);
System.out.println(myLiveObject.getValue().getOtherName());
//still is BCD

The reason for this difference in behaviour is because we are not keeping any of the object states, and each setter and getter call will serialize and deserialize the value to and from redis back to local VM. This effectively detaches the field value from the object state. This behaviour usually is not a problem when the value type is an immutable type, such as String, Double, Long, etc. When you are dealing with a mutable type, you may want to benefit from this behaviour, since the value instance is detached off from the object state, you can consider all the read/write actions to this value instance is effectively in a transaction with ACID property. It can be extremely useful when the application is designed to incorporate this behaviour properly. If you prefer to stick to standard Java behaviour, you can always convert the MyOtherObject into a Redisson Live Object.

//A Redisson Live Object with nested Redisson Live Object behaviour:
MyLiveObject myLiveObject = service.getOrCreate(MyLiveObject.class, "1");
MyOtherObject other = service.getOrCreate(MyOtherObject.class, "2");
other.setOtherName("ABC");
myLiveObject.setValue(other);
System.out.println(myLiveObject.getValue().getOtherName());
//ABC

other.setOtherName("BCD");
System.out.println(myLiveObject.getValue().getOtherName());
//you see, already is BCD

myLiveObject.setValue(other);
System.out.println(myLiveObject.getValue().getOtherName());
//and again still is BCD

Field types in the RLO can be almost anything, from Java util classes to collection/map types and of course your own custom objects, as long as it can be encoded and decoded by a supplied codec. More details about the codec can be found in the Advanced Usage section.

As much as I like to say it's free with no limits, there are still some restrictions on the choices of field types you can have. The field annotated with RId can not be an array type, i.e. int[], long[], double[], byte[], etc. More details and explainations can be found in Restrictions section

In order to keep RLOs behaving as closely to standard Java objects as possible, Redisson automatically converts the following standard Java field types to its counter types supported by Redisson RObject.

Standard Java Class Converted Redisson Class
SortedSet.class RedissonSortedSet.class
Set.class RedissonSet.class
ConcurrentMap.class RedissonMap.class
Map.class RedissonMap.class
BlockingDeque.class RedissonBlockingDeque.class
Deque.class RedissonDeque.class
BlockingQueue.class RedissonBlockingQueue.class
Queue.class RedissonQueue.class
List.class RedissonList.class

The conversion prefers the one nearer to the top of the table if a field type matches more than one entries. i.e. LinkedList implements Deque, List, Queue, it will be converted to a RedissonDeque because of this.

Instances of these Redisson classes retains their states/values/entries in Redis too, changes to them are directly reflected into Redis without keeping values in local VM.

9.2.3. Advanced Usage

As described before, RLO classes are proxy classes which can be fabricated when needed and then get cached in a RedissonClient instance against its original class. This process can be a bit slow and it is recommended to pre-register all the Redisson Live Object classes via RedissonLiveObjectService for any kind of delay-sensitive applications. The service can also be used to unregister a class if it is no longer needed. And of course it can be used to check if the class has already been registered.

RLiveObjectService service = redisson.getLiveObjectService();
service.registerClass(MyClass.class);
service.unregisterClass(MyClass.class);
Boolean registered = service.isClassRegistered(MyClass.class);

@REntity

The behaviour of each type of RLO can be customised through properties of the @REntity annotation. You can specify each of those properties to gain fine control over its behaviour.

  • namingScheme - You can specify a naming scheme which tells Redisson how to assign key names for each instance of this class. It is used to create a reference to an existing Redisson Live Object and materialising a new one in redis. It defaults to use Redisson provided DefaultNamingScheme.
  • codec - You can tell Redisson which Codec class you want to use for the RLO. Redisson will use an instance pool to locate the instance based on the class type. It defaults to JsonJacksonCodec provided by Redisson.
  • fieldTransformation - You can also specify a field transformation mode for the RLO. As mentioned before, in order to keep everything as close to standard Java as possible, Redisson will automatically transform fields with commonly-used Java util classes to Redisson compatible classes. This uses ANNOTATION_BASED as the default value. You can set it to IMPLEMENTATION_BASED which will skip the transformation.

@RId

The @RId annotation is used on a field that can be used to distinguish between one instance and another. Think of this field as the primary key field of this class. The value of this field is used to create a reference to existing RLO. The field with this annotation is the only field that has its value also kept in the local VM. You can only have one RId annotation per class.

You can supply a generator strategy to the @RId annotation if you want the value of this field to be programatically generated. The default generator is null.

@RObjectField

When the transformationMode in @REntity is set to ANNOTATION_BASED, which is the default value, you can optionally use it to annotate a field that does not have @RId annotation at the same time. This is often used to give a different namingScheme and/or a different codec class to the ones specified in @REntity.

As you can see the codec and namingScheme are quite often used in providing a Redisson Live Object and its service, in order to reduce the amount of redundant instances. Redisson, by default, caches these instances internally to be reused. You can supply your own providers for each of them via Redisson's Config instance.

9.2.4. Restrictions

As mentioned above, the type of the RId field cannot be an Array type. This is due to the DefaultNamingScheme which cannot serialize and deserialize the Array type as of yet. This restriction can be lifted once the DefaultNamingScheme is improved. Since the RId field is encoded as part of the key name used by the underlying RMap, it makes no sense to create a RLO with just have one field. It is better to use a RBucket for this type of usage.

9.3. Distributed executor service

9.3.1. Distributed executor service. Overview

Redisson distributed Executor service for Java implements java.util.concurrent.ExecutorService and allows to run java.util.concurrent.Callable and java.lang.Runnable tasks on different Redisson nodes. Tasks have an access to Redisson instance, can do any manipulations with Redis data and execute distributed computations in fast and efficient way.

9.3.2. Distributed executor service. Tasks

Redisson node don't need to have task classes in classpath. They are loaded automatically by Redisson node ClassLoader. Thus Redisson node reloading is not needed for each new task class.

Example with Callable task:

public class CallableTask implements Callable<Long> {

    @RInject
    private RedissonClient redissonClient;

    @Override
    public Long call() throws Exception {
        RMap<String, Integer> map = redissonClient.getMap("myMap");
        Long result = 0;
        for (Integer value : map.values()) {
            result += value;
        }
        return result;
    }

}

RExecutorService executorService = redisson.getExecutorService("myExecutor");
Future<Long> future = executorService.submit(new CallableTask());
Long result = future.get();

Example with Runnable task:

public class RunnableTask implements Runnable {

    @RInject
    private RedissonClient redissonClient;

    private long param;

    public RunnableTask() {
    }

    public RunnableTask(long param) {
        this.param = param;
    }

    @Override
    public void run() {
        RAtomicLong atomic = redissonClient.getAtomicLong("myAtomic");
        atomic.addAndGet(param);
    }

}

RExecutorService executorService = redisson.getExecutorService("myExecutor");
executorService.submit(new RunnableTask(123));

Each Redisson node has ready to use RedissonClient which injected to task each time via @RInject annotation.

9.3.3. Distributed executor service. Extended asynchronous mode

All standard ExecutorService.submit methods implemented by RedissonExecutorService send tasks synchronously but result receiving in asynchronous way via standard Future object. Redisson offers RExecutorServiceAsync.submitAsync methods which send tasks fully asynchronous and allows to bind FutureListener objects.

RExecutorService executorService = redisson.getExecutorService("myExecutor");
RFuture<MyResultObject> future = executorService.submitAsync(new CallableTask());
future.addListener(new FutureListener<MyResultObject>() {
     public void operationComplete(Future<MyResultObject> f) {
         // ...
     }
});

9.3.4. Distributed executor service. Task execution cancellation

It's easy to cancel any submitted task via Future.cancel() method. To handle case then task execution is already in progress check for interrupted Thread status should be added via Thread.currentThread().isInterrupted() invocation:

public class CallableTask implements Callable<Long> {

    @RInject
    private RedissonClient redissonClient;

    @Override
    public Long call() throws Exception {
        RMap<String, Integer> map = redissonClient.getMap("myMap");
        Long result = 0;
        // map contains many entries
        for (Integer value : map.values()) {
           if (Thread.currentThread().isInterrupted()) {
                // task has been canceled
                return null;
           }
           result += value;
        }
        return result;
    }

}

RExecutorService executorService = redisson.getExecutorService("myExecutor");
Future<Long> future = executorService.submit(new CallableTask());
// or
RFuture<Long> future = executorService.submitAsync(new CallableTask());
// ...
future.cancel(true);

9.4. Distributed scheduled executor service

9.4.1. Distributed scheduled executor service. Overview

Redisson distributed scheduler service for Java implements java.util.concurrent.ScheduledExecutorService and allows to schedule java.util.concurrent.Callable and java.lang.Runnable tasks on different Redisson nodes. Redisson node run jobs from Redis queue. Scheduled task is a job which needs to be execute in the future at a particular time one or more times.

9.4.2. Distributed scheduled executor service. Scheduling a task

Redisson node don't need to have task classes in classpath. They are loaded automatically by Redisson node ClassLoader. Thus Redisson node reloading is not needed for each new task class.

Example with Callable task:

public class CallableTask implements Callable<Long> {

    @RInject
    private RedissonClient redissonClient;

    @Override
    public Long call() throws Exception {
        RMap<String, Integer> map = redissonClient.getMap("myMap");
        Long result = 0;
        for (Integer value : map.values()) {
            result += value;
        }
        return result;
    }

}

RScheduledExecutorService executorService = redisson.getExecutorService("myExecutor");
ScheduledFuture<Long> future = executorService.schedule(new CallableTask(), 10, TimeUnit.MINUTES);
Long result = future.get();

Example with Runnable task:

public class RunnableTask implements Runnable {

    @RInject
    private RedissonClient redissonClient;

    private long param;

    public RunnableTask() {
    }

    public RunnableTask(long param) {
        this.param= param;
    }

    @Override
    public void run() {
        RAtomicLong atomic = redissonClient.getAtomicLong("myAtomic");
        atomic.addAndGet(param);
    }

}

RScheduledExecutorService executorService = redisson.getExecutorService("myExecutor");
ScheduledFuture<?> future1 = executorService.schedule(new RunnableTask(123), 10, TimeUnit.HOURS);
// ...
ScheduledFuture<?> future2 = executorService.scheduleAtFixedRate(new RunnableTask(123), 10, 25, TimeUnit.HOURS);
// ...
ScheduledFuture<?> future3 = executorService.scheduleWithFixedDelay(new RunnableTask(123), 5, 10, TimeUnit.HOURS);

9.4.3. Distributed scheduled executor service. Scheduling a task with cron expression

Tasks scheduler service allows to define more complex schedule with cron expressions. Which fully compatible with Quartz cron format.

Example:

RScheduledExecutorService executorService = redisson.getExecutorService("myExecutor");
executorService.schedule(new RunnableTask(), CronSchedule.of("10 0/5 * * * ?"));
// ...
executorService.schedule(new RunnableTask(), CronSchedule.dailyAtHourAndMinute(10, 5));
// ...
executorService.schedule(new RunnableTask(), CronSchedule.weeklyOnDayAndHourAndMinute(12, 4, Calendar.MONDAY, Calendar.FRIDAY));

9.4.4. Distributed scheduled executor service. Task scheduling cancellation

Scheduled executor service provides two ways to cancel any scheduled task via ScheduledFuture.cancel() method and RScheduledExecutorService.cancelScheduledTask method. To handle case then task execution is already in progress check for interrupted Thread status should be added via Thread.currentThread().isInterrupted() invocation:

public class RunnableTask implements Callable<Long> {

    @RInject
    private RedissonClient redissonClient;

    @Override
    public Long call() throws Exception {
        RMap<String, Integer> map = redissonClient.getMap("myMap");
        Long result = 0;
        // map contains many entries
        for (Integer value : map.values()) {
           if (Thread.currentThread().isInterrupted()) {
                // task has been canceled
                return null;
           }
           result += value;
        }
        return result;
    }

}

RScheduledExecutorService executorService = redisson.getExecutorService("myExecutor");
RScheduledFuture<Long> future = executorService.scheduleAsync(new RunnableTask(), CronSchedule.dailyAtHourAndMinute(10, 5));
// ...
future.cancel(true);
// or
String taskId = future.getTaskId();
// ...
executorService.cancelScheduledTask(taskId);