ResourcePool errors #11

Closed
neonknight opened this Issue Mar 1, 2012 · 2 comments

Comments

Projects
None yet
2 participants
@neonknight

I have discovered hardly reproducable ResourcePool errors under load/parallel access to cassandra. Unfortunately, I have too little Perl knowledge to find the culprit myself. This is the error I get:

Use of uninitialized value $_[0] in sprintf at /usr/local/share/perl5/ResourcePool.pm line 247, <GEN70> line 3.
ResourcePool(): got failed resource from client
Use of uninitialized value $_[1] in sprintf at /usr/local/share/perl5/ResourcePool/LoadBalancer.pm line 307, <GEN70> line 3.
Use of uninitialized value $_[0] in sprintf at /usr/local/share/perl5/ResourcePool.pm line 247, <GEN70> line 3.
ResourcePool(): Downsizing
ResourcePool: Downsized... still 0 open (0)

I'm getting this error while connecting, reading, writing... Maybe you can give me a few pointers where to investigate further.

The number of errors could be reduced by increasing the Max option in ResourcePool->new() (I simply hardcoded a value of 256 instead of 5 in RessourcePool.pm).

This is how I connect to my cassandra cluster:

my $conn = Cassandra::Simple->new(keyspace => 'mykeyspace', server_name => '192.168.0.31', pool_from_ring => 0);
$conn->pool->add_pool('mykeyspace',  '192.168.0.32');
$conn->pool->add_pool('mykeyspace',  '192.168.0.33');

However, after the queries my application simply quits. Maybe my application gets stuck because of lingering connections. Is there a way to properly disconnect from cassandra?
Should I randomize to which server I connect first?

@fmgoncalves

This comment has been minimized.

Show comment Hide comment
@fmgoncalves

fmgoncalves Mar 1, 2012

Owner

Those ResourcePool errors happen whenever Cassandra throws an exception, they stem from that server being tagged as failed and bumped down the pool. They are messages hardcoded in ResourcePool without an option to be turned off, which makes them a very annoying side effect of using ResourcePool.
If they are happening a lot, it is a symptom of many errors in Cassandra, take notice of the messages before these.

There is no proper way to disconnect, and no query returns properly finishing its operation so there should be no lingering connection. Anyway, I don't understand what you mean by "after the queries my application simply quits", does it quit halfway through?

Owner

fmgoncalves commented Mar 1, 2012

Those ResourcePool errors happen whenever Cassandra throws an exception, they stem from that server being tagged as failed and bumped down the pool. They are messages hardcoded in ResourcePool without an option to be turned off, which makes them a very annoying side effect of using ResourcePool.
If they are happening a lot, it is a symptom of many errors in Cassandra, take notice of the messages before these.

There is no proper way to disconnect, and no query returns properly finishing its operation so there should be no lingering connection. Anyway, I don't understand what you mean by "after the queries my application simply quits", does it quit halfway through?

@neonknight

This comment has been minimized.

Show comment Hide comment
@neonknight

neonknight Mar 1, 2012

Sorry, I wrote this very imprecisely. The application terminates as expected. By "simply quits" I wanted to express that I don't do any cleanup.
I will try to get more debugging information from my application and from cassandra itself. Thank you for the explanation.

Sorry, I wrote this very imprecisely. The application terminates as expected. By "simply quits" I wanted to express that I don't do any cleanup.
I will try to get more debugging information from my application and from cassandra itself. Thank you for the explanation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment