Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

java.util.NoSuchElementException: Could not create a validated object, cause: ValidateObject failed #307

Open
swarupe04 opened this Issue · 6 comments

5 participants

@swarupe04

jedis v2.1.0

Intermittently our webservice is failing against jedis call with following error:

redis.clients.jedis.exceptions.JedisConnectionException: Could not get a resource from the pool
at redis.clients.util.Pool.getResource(Pool.java:22)
...
Caused by: java.util.NoSuchElementException: Could not create a validated object, cause: ValidateObject failed
at org.apache.commons.pool.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:1226)
at redis.clients.util.Pool.getResource(Pool.java:20)

We're noticing this on read and r+w(master) pool

Config pool settings:

-- readPool
rJedisPoolConfig.maxActive=200
rJedisPoolConfig.testOnBorrow=true
rJedisPoolConfig.minIdle=5
rJedisPoolConfig.maxIdle=10
rJedisPoolConfig.testWhileIdle=true
rJedisPoolConfig.numTestsPerEvictionRun=10
rJedisPoolConfig.timeBetweenEvictionRunsMillis=300000
#public static final byte WHEN_EXHAUSTED_GROW = 2;
rJedisPoolConfig.whenExhaustedAction=2

-- writePool
rwJedisPoolConfig.maxActive=200
rwJedisPoolConfig.testOnBorrow=true
rwJedisPoolConfig.minIdle=5
rwJedisPoolConfig.maxIdle=10
rwJedisPoolConfig.testWhileIdle=true
rwJedisPoolConfig.numTestsPerEvictionRun=10
rwJedisPoolConfig.timeBetweenEvictionRunsMillis=300000
#public static final byte WHEN_EXHAUSTED_BLOCK = 1;
rwJedisPoolConfig.whenExhaustedAction=1
rwJedisPoolConfig.maxWait=-1

@swarupe04

Hey just wanted to add some more thoughts on this since we're still consistently seeing this error..

I've increase connection pool WhenExhaustedAction=2 (WHEN_EXHAUSTED_GROW), and also increased to 5000 maxActive, even though it shouldn't affect by that parm, but still continue to see this error.
As per GenericPoolConfig, NoElementException is caused when WhenExhaustedAction=BLOCK..http://commons.apache.org/pool/api-1.6/org/apache/commons/pool/impl/GenericObjectPool.html#borrowObject%28%29

I'm wondering if whenExhaustedAction is really getting set. Any thoughts, comments very much appreciated.

@a-b
a-b commented

Any updates?

@xetorthio
Owner

I think there is a problem when the pool want to create new resources. Because the message is: "Could not create a validated object". It tries to grow, but can't.
Is there any connection problem that you are aware of when creating the resources?

@a-b
a-b commented

You're right. The real problem was thread unsafe code jedis = jedisPool.getResource(); stored as instance variable of Dropwizard resource class. To adress this issue we moved getResource() deeper into the core: now we use jedis as local variable at the method level. Everything work as expected.
Thanks!

@sheldoan

I'm getting this exact same issue. a-b can you expand on your comment?

I believe I'm using Jedis in the same way -- I have a public class that keeps the JedisPool as a static variable, and only when I need to issue a redis command do I request a resource, and then return it immediately. Is this what you mean by using "jedis as a local variable at the method level" ?

@lukess

I got the similar issue. In my scenario that is because of new JedisPool/3 timeout (default is 2000) so change to JedisPool/4 (with timeout) to solve the problem. I need a longer timeout because of several complex queries. BTW, my getResource() inside method by default.

the psudo processes are:
1. getResource()
2. GenericObjectPool.borrowObject()
3. Jedis.ping() --> timeout
4. JedisPool$JedisFactory.validateObject() --> failed because obj NOT instanceof Jedis inside JedisPool.validateObject(final Object obj)
5. java.util.NoSuchElementException: Could not create a validated object, cause: ValidateObject failed
at org.apache.commons.pool.impl.GenericObjectPool.borrowObject
at redis.clients.util.Pool.getResource

hope this helps

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.