New version 2.1.4 or greater performance is low.

[lefay1982]

When I use redisson 2.1.3 the ubuntu's load average is 1.8 ~ 2.3; but I use 2.1.4 or greater, the load average is often greater than 3.00, my java application often overload.

[mrniko]

Which Redisson objects do you use in your application? Redis configuration also important to know

[mrniko]

Also try currenty latest Redisson version - 2.2.3. Because Redisson has a lot of changes since 2.1.4 version.

[lefay1982]

I have test 2.1.4 2.1.6 2.2.0 2.2.3 cpu and memory also increase more than 2.1.3.

[mrniko]

Ok, will check it too

[mrniko]

@lefay1982 also try to change netty version. Redisson uses netty 4.0.32 since 2.1.4 version. 2.1.3 uses 4.0.30. Maybe this affects performance

[lefay1982]

Run 17 hours, the new version after 2.1.3 res increase 1.019g and 800M.
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
1020 root 20 0 3563004 1.019g 6436 S 7.9 26.4 381:40.31 java
1019 root 20 0 3609728 274328 4552 S 5.7 6.8 81:15.25 java
1022 root 20 0 3553692 866980 6040 S 4.5 21.4 406:50.26 java
1029 root 20 0 3563380 898648 6188 S 3.4 22.2 452:04.97 java

the other machine the res is 300M and 500M
26459 root 20 0 3737492 369828 7716 S 63.0 9.1 536:16.80 java
26418 root 20 0 3705568 380572 7864 S 37.5 9.4 488:18.65 java
26434 root 20 0 3693408 382884 7896 S 36.2 9.5 540:06.79 java
26416 root 20 0 3674520 505140 7608 S 6.0 12.5 108:08.33 java

[mrniko]

Let's move to gitter chat to discuss :)

[mrniko]

I ran load-tests with one Redisson client, 25 threads each of them executed task with two Redisson operations: RDeque.addLast(randomString) and RDeque.getFirst(). Total tasks amount is 1.000.000
JDK 1.8.0_60

Here is my results:
For Redisson 2.1.3

For Redisson 2.2.3

I don't see much difference in CPU consumption. As for memory, with 2.2.3 version memory footprint looks better.

[mrniko]

@lefay1982 Does it work well now?

[lefay1982]

no, after deploy 1 machine the 2.2.4-SNAPSHOT, all my apps is run abnormal.

[mrniko]

ok, I'll continue investigation

[mrniko]

@lefay1982 i have reduced memory allocation by listeners. Please check last build

[lefay1982]

my gitter is not use, I check the latest code, and run, all my server is not load, redis no output bytes.
I set the master pool size and slave pool size is 50, if the connection is not the cause, maybe have a method consumed many connection. so the connection pool is empty.

[lefay1982]

I only use RMap RAtomicLong RSet RLock RQueue, RBucket.

[mrniko]

I tried this config and it works.

    config.useSentinelServers().setMasterName("mymaster").addSentinelAddress("127.0.0.1:26379", "127.0.0.1:26389")
    .setSlaveConnectionPoolSize(50)
    .setMasterConnectionPoolSize(50);

[lefay1982]

Now I used is config.userMasterSlaveServers();

[lefay1982]

Run test is OK
But deploy to work server. is down , but v2.1.3 is OK. I returned to 2.1.3.

[mrniko]

But deploy to work server. is down

Sorry, what does it mean?

[mrniko]

Could you describe steps to reproduce it?

[mrniko]

I tried follow scenarios with useMasterSlaveServers (1 master, 2 slaves):

1. stop 1 slave, then start it again. Client continues to work
2. stop two slaves, then start them again. Client continues to work
3. stop master, then start it again. Client continues to work

All tests are passed for me.

[lefay1982]

"epollEventLoopGroup-4-2" prio=10 tid=0x00007f03e0004000 nid=0x6153 in Object.wait() [0x00007f0450e32000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:461)
at io.netty.util.concurrent.DefaultPromise.await0(DefaultPromise.java:355)
- locked <0x00000007843f5d58> (a io.netty.util.concurrent.DefaultPromise)
at io.netty.util.concurrent.DefaultPromise.awaitUninterruptibly(DefaultPromise.java:306)
at org.redisson.RedissonLock.tryLock(RedissonLock.java:293)

"epollEventLoopGroup-4-1" prio=10 tid=0x00007f03e0007800 nid=0x6152 in Object.wait() [0x00007f0450f33000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:503)
at io.netty.util.concurrent.DefaultPromise.awaitUninterruptibly(DefaultPromise.java:286)
- locked <0x00000007dc3f8dc0> (a io.netty.util.concurrent.DefaultPromise)
at io.netty.util.concurrent.DefaultPromise.awaitUninterruptibly(DefaultPromise.java:32)
at org.redisson.CommandExecutorService.get(CommandExecutorService.java:181)
at org.redisson.CommandExecutorService.read(CommandExecutorService.java:196)
at org.redisson.RedissonMap.containsKey(RedissonMap.java:88)

"epollEventLoopGroup-4-5" prio=10 tid=0x00007f03e000a000 nid=0x616c in Object.wait() [0x00007f0450832000]
java.lang.Thread.State: BLOCKED (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:503)
at io.netty.util.concurrent.DefaultPromise.awaitUninterruptibly(DefaultPromise.java:286)
- locked <0x00000007db8cde78> (a io.netty.util.concurrent.DefaultPromise)
at io.netty.util.concurrent.DefaultPromise.awaitUninterruptibly(DefaultPromise.java:32)
at org.redisson.CommandExecutorService.get(CommandExecutorService.java:181)
at org.redisson.RedissonObject.get(RedissonObject.java:48)
at org.redisson.RedissonMap.put(RedissonMap.java:142)

"nioEventLoopGroup-6-8" prio=10 tid=0x00007f040c074800 nid=0x61e6 runnable [0x00007f03ce1eb000]
java.lang.Thread.State: RUNNABLE
at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
- locked <0x00000007834ff838> (a io.netty.channel.nio.SelectedSelectionKeySet)
- locked <0x00000007834ff898> (a java.util.Collections$UnmodifiableSet)
- locked <0x00000007834ff7a0> (a sun.nio.ch.EPollSelectorImpl)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:622)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:310)
at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)
at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
at java.lang.Thread.run(Thread.java:745)

[lefay1982]

when run some time in my server. and stop nginx not let client call server's, my app's cpu is 40%.
Then I jstack the app , show me above infomations.

[mrniko]

Here is my cpu graph. During graph pit servers are down, almost 0% during this time. Then servers are up CPU consumption backs to usual level.

[lefay1982]

Netty's EventLoopGroup bossGroup and workderGroup What Number you set?

[lefay1982]

I think is my netty's setting have set error.

[mrniko]

@lefay1982 I didn't touch any of those settings. My settings are:

      config.useMasterSlaveServers().setMasterAddress("127.0.0.1:6399").addSlaveAddress("127.0.0.1:6379").addSlaveAddress("127.0.0.1:6389")
    .setSlaveConnectionPoolSize(50)
    .setMasterConnectionPoolSize(50);

[mrniko]

Also, there is no bossGroup for io.netty.bootstrap.Bootstrap.Bootstrap client instance.

[lefay1982]

OK, I use Netty for Server instance.

[lefay1982]

hi @mrniko here has a exception of v2.2.4:

ERROR error
org.redisson.client.RedisException: ERR Error running script (call to f_9401052d872adfd0179ef8c8e8c028512707629a): @user_script:1: WRONGTYPE Operation against a key holding the wrong kind of value . channel: [id: 0xe61fdefb, /10.117.238.103:43493 => /10.204.16.23:7001] command: CommandData [promise=DefaultPromise@aff3759(incomplete), command=(EVAL), params=[if (redis.call('exists', KEYS[1]) == 0) then redis.call('hset', KEYS[1], ARGV[2], 1); redis.call('pexpire', KEYS[1], ARGV[1]); return nil; end; if (redis.call('hexists', KEYS[1], ARGV[2]) == 1) then redis.call('hincrby', KEYS[1], ARGV[2], 1); redis.call('pexpire', KEYS[1], ARGV[1]); return nil; end; return redis.call('pttl', KEYS[1]);, 1, ad: lock:adOfferMap:423, 29000, d1dae71c-d3fa-43c6-ad50-49c14a4d2836:56], codec=org.redisson.client.codec.LongCodec@798ab9b2]
at org.redisson.client.handler.CommandDecoder.decode(CommandDecoder.java:184) ~[redisson-2.2.4.jar:na]
at org.redisson.client.handler.CommandDecoder.decode(CommandDecoder.java:109) ~[redisson-2.2.4.jar:na]
at io.netty.handler.codec.ReplayingDecoder.callDecode(ReplayingDecoder.java:376) ~[netty-all-4.0.33.Final.jar:4.0.33.Final]
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:244) ~[netty-all-4.0.33.Final.jar:4.0.33.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:318) ~[netty-all-4.0.33.Final.jar:4.0.33.Final]

[lefay1982]

when using RLock.

[lefay1982]

2015-12-29 01:17:55.212 epollEventLoopGroup-4-1 ERROR error
org.redisson.client.RedisException: ERR Error running script (call to f_9401052d872adfd0179ef8c8e8c028512707629a): @user_script:1: WRONGTYPE Operation against a key holding the wrong kind of value . channel: [id: 0xe61fdefb, /10.117.238.103:43493 => /10.204.16.23:7001] command: CommandData [promise=DefaultPromise@6cd9f6cb(incomplete), command=(EVAL), params=[if (redis.call('exists', KEYS[1]) == 0) then redis.call('hset', KEYS[1], ARGV[2], 1); redis.call('pexpire', KEYS[1], ARGV[1]); return nil; end; if (redis.call('hexists', KEYS[1], ARGV[2]) == 1) then redis.call('hincrby', KEYS[1], ARGV[2], 1); redis.call('pexpire', KEYS[1], ARGV[1]); return nil; end; return redis.call('pttl', KEYS[1]);, 1, ad: lock:adOfferMap:1219, 29000, d1dae71c-d3fa-43c6-ad50-49c14a4d2836:38], codec=org.redisson.client.codec.LongCodec@798ab9b2]

[mrniko]

@lefay1982 you need to recreate lock it's not compatible with 2.1.3 version. RedissonLock passes the tests.

[lefay1982]

I start 7 instance, but 1 instace update 2.2.4. this is the cause.

[mrniko]

So now you are using 2.2.4 in all instances. Am I right?

[lefay1982]

when I deploy 2.2.4 in all instances, RMap.fastPut can not put value to Redis, but fastRemove can remove, cause my Redis's data removed. I can only return to 2.1.3.

In 2.1.3 the RBatch.executeAsync not put my data to Redis.
but the RBatch.execute can put the data to Redis,
2.2.4 no time to test the RBatch.
please check.

[mrniko]

I launched RMap tests and they all are passed. RBatch.execute based on RBatch.executeAsync. Therefore if RBatch.executeAsync doesn't work then RBatch.execute shouldn't too. Also RBatch passes the unit tests.

[mrniko]

@lefay1982 any news?

[lefay1982]

Have no time to test.

[mrniko]

@lefay1982 major bug with RBatch has been fixed in #361