Cassandra Source: internal queue isn't drained fast enough under load resulting in loop

[CodeSmell]

When inserting data into Cassandra under even modest loads (approx >100 messages per minute), it can cause an "infinite loop" in the Cassandra Source that causes the connector to stop processing.

After firing off a query, the CassandraTableReader will take each row in the result set and place them onto an internal LinkedBlockingQueue. After the result set is processed, the CassandraSourceTask will drain some of the messages (default 100) off of the internal queue at the start of the next polling cycle and return them as a List of SourceRecord(s) to be published to Kafka.

Under load the internal queue continues to grow faster than the messages are being drained. Eventually it reaches the limit (default 10,000) and no more messages can be placed on the queue. However, the CassandraTableReader and the CassandraSourceTask get stuck in a loop where

• nothing can be added to the internal queue until some messages are drained
• no messages are drained from the queue until querying ends
• the querying can't end until the result set is fully processed

Willing to submit PR for the fix

[andrewstevenson]

@CodeSmell Can this one be closed?