Token aware routing

[conorbros]

• main pr: Token aware routing #824

TODO:

• should use the keyspace info to properly select the replica count (Use keyspace info for replication factor #895)
• should select rack replica first, then data center (Replica choice bug fix #894)
• Some easy performance wins can be made, mostly around avoiding calling the frame method on every message and instead - -- checking the opcode bytes directly first.
• Round Robin load balancing for CassandraSinkCluster #846
• Create new connections concurrently for prepared statements #842
• share prepared metadata between connections #841
• see if we only need to store parts of the prepared metadata (share prepared metadata between connections #841 (comment))
• investigate bug (issue 11)
  Found the causes for issue 11 to be the routing key not being encoded correctly and Shotover not supporting flags. The following PRs resolve the issue and add tests against the Scylla driver. A bug was found in cdrs-tokio routing which meant the bug was not discovered until ran against Joey's test suite (which uses the Java driver).
  Dependency pre-req PRs:
  • fix flags for tracing & warnings krojew/cdrs-tokio#118
  • don't include value length in serialised routing key krojew/cdrs-tokio#119
  • Add set_tracing method Metaswitch/cassandra-rs#147
    Shotover PRs:
  • Cassandra flags #853
  • Fix and test token aware routing #854
• add routing tests against cdrs-tokio once above pre-reqs are merged. (More routing tests and bug fix #932)
• more tests for partition keys:
  • composite, compound keys (More routing tests and bug fix #932)
  • different types
• make CassandraFrame flag invalid states impossible through modification
• Fix unbound query parameters in string statement bug (issue 19)
• test with multiple shotover nodes, make sure driver routes query to correct shotover node and then to correct cassandra node

[yuzhouchen-instaclustr]

I'd also like to add one more item to the TODO list:
Token aware routing picks the replica with lowest latency value

[conorbros]

Power of two choices algorithm is tracked by #962