cassandra complains about big batches of prepared statements #73

Closed
mabrek opened this Issue Nov 3, 2014 · 3 comments

Comments

Projects
None yet
3 participants
@mabrek

mabrek commented Nov 3, 2014

Here's an example from cassandra logs:

WARN  13:54:22 Batch of prepared statements for [metric.metric] is of size 15496, exceeding specified threshold of 5120 by 10376.
WARN  13:54:23 Batch of prepared statements for [metric.metric] is of size 15652, exceeding specified threshold of 5120 by 10532.

batch function from store.clj creates BatchStatements using default constructor which results in LOGGED batches. Logged batches have significant ovehead in the way they handled by cassandra (http://www.datastax.com/documentation/cql/3.1/cql/cql_reference/batch_r.html). I'm not familiar with closure enough to find out if updates are done using the same key (as explained in http://www.datastax.com/documentation/cql/3.1/cql/cql_using/useBatch.html) or not.
There is a good explanation on why it's better to use simple inserts instead of batches with different keys:
http://lostechies.com/ryansvihla/2014/08/28/cassandra-batch-loading-without-the-batch-keyword/

@pyr

This comment has been minimized.

Show comment
Hide comment
@pyr

pyr Nov 3, 2014

Owner

agreed @mabrek, I plan to switch to execute async soon.

Owner

pyr commented Nov 3, 2014

agreed @mabrek, I plan to switch to execute async soon.

@jippi

This comment has been minimized.

Show comment
Hide comment
@jippi

jippi Jun 30, 2015

bump bump

jippi commented Jun 30, 2015

bump bump

@pyr

This comment has been minimized.

Show comment
Hide comment
@pyr

pyr Jul 23, 2015

Owner

Hi,

I will be closing this, since batches have been deprecated by the cassandra team and I've opted for binding on prepared statements. (approach described in #80)

Sorry for the long wait

Owner

pyr commented Jul 23, 2015

Hi,

I will be closing this, since batches have been deprecated by the cassandra team and I've opted for binding on prepared statements. (approach described in #80)

Sorry for the long wait

@pyr pyr closed this Jul 23, 2015

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment