From 477eee511d6a40512c24422a7f54b3e4b47711a9 Mon Sep 17 00:00:00 2001 From: Andrew Tolbert Date: Thu, 3 Jan 2019 16:16:13 -0600 Subject: [PATCH 1/4] JAVA-1964: Complete remaining "Coming Soon" sections in docs Completed docs for: * Authentication * Control Connection * Reconnection * Statements * UDTs --- changelog/README.md | 2 +- manual/auth/README.md | 33 +++++++++- manual/control_connection/README.md | 27 +++++--- manual/reconnection/README.md | 61 +++++++++++++++--- manual/statements/batch/README.md | 31 ++++++++- manual/udts/README.md | 98 ++++++++++++++++++++++++++++- 6 files changed, 230 insertions(+), 22 deletions(-) diff --git a/changelog/README.md b/changelog/README.md index 1eca207e3c4..a8b666bb110 100644 --- a/changelog/README.md +++ b/changelog/README.md @@ -9,7 +9,7 @@ - [bug] JAVA-2070: Call onRemove instead of onDown when rack and/or DC information changes for a host. - [improvement] JAVA-1256: Log parameters of BuiltStatement in QueryLogger. - [documentation] JAVA-2074: Document preference for LZ4 over Snappy. - +- [documentation] JAVA-1964: Complete remaining "Coming Soon" sections in docs. ### 3.6.0 diff --git a/manual/auth/README.md b/manual/auth/README.md index 502973702cc..a77834eff0a 100644 --- a/manual/auth/README.md +++ b/manual/auth/README.md @@ -1,5 +1,34 @@ ## Authentication -*Coming soon... In the meantime, see the javadoc for [AuthProvider].* +Cassandra’s binary protocol supports [SASL]-based authentication. To enable it, use +[Cluster.Builder.withCredentials] when building your `Cluster` instance to provide the credentials +you wish to authenticate with: -[AuthProvider]: http://docs.datastax.com/en/drivers/java/3.6/com/datastax/driver/core/AuthProvider.html \ No newline at end of file +```java +Cluster.builder() + .withCredentials("bob", "mypassword") + .build(); +``` + +This is a shortcut for using [PlainTextAuthProvider] for simple username/password authentication +(intended to work with the server-side `PasswordAuthenticator`). This may alternatively be +provided using the [Cluster.Builder.withAuthProvider] method: + + +```java +Cluster.builder() + .withAuthProvider(new PlainTextAuthProvider("bob", "mypassword")) + .build(); +``` + +Authentication must be configured before opening a session, it cannot be changed at runtime. + +You can also write your own provider; it must implement [AuthProvider]. + + +[SASL]: https://en.wikipedia.org/wiki/Simple_Authentication_and_Security_Layer + +[Cluster.Builder.withCredentials]: https://docs.datastax.com/en/drivers/java/3.6/com/datastax/driver/core/Cluster.Builder.html#withCredentials-java.lang.String-java.lang.String- +[AuthProvider]: https://docs.datastax.com/en/drivers/java/3.6/com/datastax/driver/core/AuthProvider.html +[Cluster.Builder.withAuthProvider]: https://docs.datastax.com/en/drivers/java/3.6/com/datastax/driver/core/Cluster.Builder.html#withAuthProvider-com.datastax.driver.core.AuthProvider- +[PlainTextAuthProvider]: https://docs.datastax.com/en/drivers/java/3.6/com/datastax/driver/core/PlainTextAuthProvider.html diff --git a/manual/control_connection/README.md b/manual/control_connection/README.md index eccef57e029..6b6385fd30a 100644 --- a/manual/control_connection/README.md +++ b/manual/control_connection/README.md @@ -1,10 +1,21 @@ ## Control connection -*Coming soon...* - - \ No newline at end of file +The control connection is a dedicated connection used for administrative tasks: + +* querying system tables to learn about the cluster's topology and + [schema](../metadata/#schema-metadata); +* checking [schema agreement](../metadata/#schema-agreement); +* reacting to server events, which are used to notify the driver of external topology or schema + changes. + +When the driver starts, the control connection is established to the first contacted node. If that +node goes down, a [reconnection](../reconnection/) is started to find another node; it is governed +by the same policy as regular connections and tries the nodes according to a query plan from the +[load balancing policy](../load_balancing/). + +The control connection is managed independently from [regular pooled connections](../pooling/), and +used exclusively for administrative requests. It is included in [Session.State.getOpenConnections], +as well as the `open-connections` [metric](../metrics); for example, if you've configured a pool +size of 2, the control node will have 3 connections. + +[Session.State.getOpenConnections]: https://docs.datastax.com/en/drivers/java/3.6/com/datastax/driver/core/Session.State.html#getOpenConnections-com.datastax.driver.core.Host- \ No newline at end of file diff --git a/manual/reconnection/README.md b/manual/reconnection/README.md index d1273197077..94a0461a67a 100644 --- a/manual/reconnection/README.md +++ b/manual/reconnection/README.md @@ -1,11 +1,58 @@ ## Reconnection -*Coming soon... In the meantime, see the javadoc for [ReconnectionPolicy].* +If the driver loses a connection to a node, it tries to re-establish it according to a configurable +policy. This is used in two places: - +* [connection pools](../pooling/): for each node, a session has a fixed-size pool of connections to + execute user requests. If one or more connections drop, a reconnection gets started for the pool; + each attempt tries to reopen the missing number of connections. This goes on until the pool is + back to its expected size; +* [control connection](../control_connection/): a session uses a single connection to an arbitrary + node for administrative requests. If that connection goes down, a reconnection gets started; each + attempt iterates through all active nodes until one of them accepts a connection. This goes on + until we have a control node again. -[ReconnectionPolicy]: http://docs.datastax.com/en/drivers/java/3.6/com/datastax/driver/core/policies/ReconnectionPolicy.html \ No newline at end of file +[ReconnectionPolicy] controls the interval between each attempt. The policy to use may be +provided using [Cluster.Builder.withReconnectionPolicy]. For example, the following configures +an [ExponentialReconnectionPolicy] with a base delay of 1 second, and a max delay of 10 minutes +(this is the default behavior). + +```java +Cluster.builder() + .withReconnectionPolicy(new ExponentialReconnectionPolicy(1000, 10 * 60 * 1000)) + .build(); +``` + +[ConstantReconnectionPolicy] uses the same delay every time, regardless of the +previous number of attempts. + +You can also write your own policy; it must implement [ReconnectionPolicy]. + +For best results, use reasonable values: very low values (for example a constant delay of 10 +milliseconds) will quickly saturate your system. + +The policy works by creating a *schedule* each time a reconnection starts. These schedules are +independent across reconnection attempts, meaning that each pool will start with a fresh delay even +if other pools are already reconnecting. For example, assuming that the pool size is 3, the policy +is the exponential one with the default values, and the control connection is initially on node1: + +* [t = 0] 2 connections to node2 go down. A reconnection starts for node2's pool, with the next + attempt in 1 second; +* [t = 1] node2's pool tries to open the 2 missing connections. One succeeds but the other fails. + Another attempt is scheduled in 2 seconds; +* [t = 1.2] 1 connection to node3 goes down. A reconnection starts for node3's pool, with the next + attempt in 1 second; +* [t = 1.5] the control connection to node1 goes down. A reconnection starts for the control + connection, with the next attempt in 1 second; +* [t = 2.2], node3's pool tries to open its missing connection, which succeeds. The pool is back to + its expected size, node3's reconnection stops; +* [t = 2.5] the control connection tries to find a new node. It invokes the + [load balancing policy](../load_balancing/) to get a query plan, which happens to start with + node4. The connection succeeds, node4 is now the control node and the reconnection stops; +* [t = 3] node2's pool tries to open the last missing connection, which succeeds. The pool is back + to its expected size, node2's reconnection stops. + +[ReconnectionPolicy]: https://docs.datastax.com/en/drivers/java/3.6/com/datastax/driver/core/policies/ReconnectionPolicy.html +[Cluster.Builder.withReconnectionPolicy]: https://docs.datastax.com/en/drivers/java/3.6/com/datastax/driver/core/Cluster.Builder.html#withReconnectionPolicy-com.datastax.driver.core.policies.ReconnectionPolicy- +[ExponentialReconnectionPolicy]: https://docs.datastax.com/en/drivers/java/3.6/com/datastax/driver/core/policies/ExponentialReconnectionPolicy.html +[ConstantReconnectionPolicy]: https://docs.datastax.com/en/drivers/java/3.6/com/datastax/driver/core/policies/ConstantReconnectionPolicy.html diff --git a/manual/statements/batch/README.md b/manual/statements/batch/README.md index ebc156d731b..2d53362cf3f 100644 --- a/manual/statements/batch/README.md +++ b/manual/statements/batch/README.md @@ -1,5 +1,32 @@ ## Batch statements -*Coming soon... In the meantime, see the javadoc for [BatchStatement].* +Use [BatchStatement] to execute a set of queries as an atomic operation (refer to +[Batching inserts, updates and deletes][batch_dse] to understand how to use batching effectively): -[BatchStatement]: http://docs.datastax.com/en/drivers/java/3.6/com/datastax/driver/core/BatchStatement.html +```java +PreparedStatement preparedInsertExpense = + session.prepare( + "INSERT INTO cyclist_expenses (cyclist_name, expense_id, amount, description, paid) " + + "VALUES (:name, :id, :amount, :description, :paid)"); +SimpleStatement simpleInsertBalance = + new SimpleStatement("INSERT INTO cyclist_expenses (cyclist_name, balance) VALUES (?, 0) IF NOT EXISTS", + "Vera ADRIAN"); + +BatchStatement batch = new BatchStatement() + .add(simpleInsertBalance) + .add(preparedInsertExpense.bind("Vera ADRIAN", 1, 7.95f, "Breakfast", false)); + +session.execute(batch); +``` + +As shown in the examples above, batches can contain any combination of simple statements and bound +statements. A given batch can contain at most 65536 statements. Past this limit, addition methods +throw an `IllegalStateException`. + +In addition, simple statements with named parameters are currently not supported in batches (this is +due to a [protocol limitation][CASSANDRA-10246] that will be fixed in a future version). If you try +to execute such a batch, an `IllegalArgumentException` is thrown. + +[BatchStatement]: https://docs.datastax.com/en/drivers/java/3.6/com/datastax/driver/core/BatchStatement.html +[batch_dse]: http://docs.datastax.com/en/dse/5.1/cql/cql/cql_using/useBatch.html +[CASSANDRA-10246]: https://issues.apache.org/jira/browse/CASSANDRA-10246 diff --git a/manual/udts/README.md b/manual/udts/README.md index 8fd946908d7..11727c404bd 100644 --- a/manual/udts/README.md +++ b/manual/udts/README.md @@ -1,5 +1,99 @@ ## User-defined types -*Coming soon... In the meantime, see the javadoc for [UserType].* +[CQL user-defined types][cql_doc] are ordered sets of named, typed fields. They must be defined in a +keyspace: -[UserType]: http://docs.datastax.com/en/drivers/java/3.6/com/datastax/driver/core/UserType.html \ No newline at end of file +``` +CREATE TYPE ks.type1 ( + a int, + b text, + c float); +``` + +And can then be used as a column type in tables, or a field type in other user-defined types in that +keyspace: + +``` +CREATE TABLE ks.collect_things ( + pk int, + ck1 text, + ck2 text, + v frozen, + PRIMARY KEY (pk, ck1, ck2) +); + +CREATE TYPE ks.type2 (v frozen); +``` + +### Fetching UDTs from results + +The driver maps UDT columns to the [UDTValue] class, which exposes getters and setters to access +individual fields by index or name: + +```java +Row row = session.execute("SELECT v FROM ks.collect_things WHERE pk = 1").one(); + +UDTValue udtValue = row.getUDTValue("v"); +int a = udtValue.getInt(0); +String b = udtValue.getString("b"); +Float c = udtValue.getFloat(2); +``` + +### Using UDTs as parameters + +Statements may contain UDTs as bound values: + +```java +PreparedStatement ps = + session.prepare( + "INSERT INTO ks.collect_things (pk, ck1, ck2, v) VALUES (:pk, :ck1, :ck2, :v)"); +``` + +To create a new UDT value, you must first have a reference to its [UserType]. There are +various ways to get it: + +* from the statement's metadata + + ```java + UserType udt = (UserType) ps.getVariables().getType("v"); + ``` + +* from the driver's [schema metadata](../metadata/#schema-metadata): + + ```java + UserType udt = session.getMetadata().getKeyspace("ks").getUserType("type1"); + ``` + +* from another UDT value: + + ```java + UserType udt = udtValue.getType(); + ``` + +Note that the driver's official API does not expose a way to build [UserType] instances manually. +This is because the type's internal definition must precisely match the database schema; +if it doesn't (for example if the fields are not in the same order), you run the risk of inserting +corrupt data, that you won't be able to read back. + +Once you have the type, call `newValue()` and set the fields: + +```java +UdtValue udtValue = udt.newValue().setInt(0, 1).setString(1, "hello").setFloat(2, 2.3f); +``` + +And bind your UDT value like any other type: + +```java +BoundStatement bs = + ps.bind() + .setInt("pk", 1) + .setString("ck1", "1") + .setString("ck2", "1") + .setUDTValue("v", udtValue); +session.execute(bs); +``` + +[cql_doc]: https://docs.datastax.com/en/cql/3.3/cql/cql_reference/cqlRefUDType.html + +[UDTValue]: https://docs.datastax.com/en/drivers/java/3.6/com/datastax/driver/core/UDTValue.html +[UserType]: https://docs.datastax.com/en/drivers/java/3.6/com/datastax/driver/core/UserType.html From c9d52688c407229ca7e7e1eb43643d1bb838afb9 Mon Sep 17 00:00:00 2001 From: Andrew Tolbert Date: Mon, 7 Jan 2019 12:42:53 -0600 Subject: [PATCH 2/4] Update reconnection docs based on feedback --- manual/reconnection/README.md | 25 +------------------------ 1 file changed, 1 insertion(+), 24 deletions(-) diff --git a/manual/reconnection/README.md b/manual/reconnection/README.md index 94a0461a67a..d54390813dc 100644 --- a/manual/reconnection/README.md +++ b/manual/reconnection/README.md @@ -4,9 +4,7 @@ If the driver loses a connection to a node, it tries to re-establish it accordin policy. This is used in two places: * [connection pools](../pooling/): for each node, a session has a fixed-size pool of connections to - execute user requests. If one or more connections drop, a reconnection gets started for the pool; - each attempt tries to reopen the missing number of connections. This goes on until the pool is - back to its expected size; + execute user requests. If a node is detected as down, a reconnection is started. * [control connection](../control_connection/): a session uses a single connection to an arbitrary node for administrative requests. If that connection goes down, a reconnection gets started; each attempt iterates through all active nodes until one of them accepts a connection. This goes on @@ -31,27 +29,6 @@ You can also write your own policy; it must implement [ReconnectionPolicy]. For best results, use reasonable values: very low values (for example a constant delay of 10 milliseconds) will quickly saturate your system. -The policy works by creating a *schedule* each time a reconnection starts. These schedules are -independent across reconnection attempts, meaning that each pool will start with a fresh delay even -if other pools are already reconnecting. For example, assuming that the pool size is 3, the policy -is the exponential one with the default values, and the control connection is initially on node1: - -* [t = 0] 2 connections to node2 go down. A reconnection starts for node2's pool, with the next - attempt in 1 second; -* [t = 1] node2's pool tries to open the 2 missing connections. One succeeds but the other fails. - Another attempt is scheduled in 2 seconds; -* [t = 1.2] 1 connection to node3 goes down. A reconnection starts for node3's pool, with the next - attempt in 1 second; -* [t = 1.5] the control connection to node1 goes down. A reconnection starts for the control - connection, with the next attempt in 1 second; -* [t = 2.2], node3's pool tries to open its missing connection, which succeeds. The pool is back to - its expected size, node3's reconnection stops; -* [t = 2.5] the control connection tries to find a new node. It invokes the - [load balancing policy](../load_balancing/) to get a query plan, which happens to start with - node4. The connection succeeds, node4 is now the control node and the reconnection stops; -* [t = 3] node2's pool tries to open the last missing connection, which succeeds. The pool is back - to its expected size, node2's reconnection stops. - [ReconnectionPolicy]: https://docs.datastax.com/en/drivers/java/3.6/com/datastax/driver/core/policies/ReconnectionPolicy.html [Cluster.Builder.withReconnectionPolicy]: https://docs.datastax.com/en/drivers/java/3.6/com/datastax/driver/core/Cluster.Builder.html#withReconnectionPolicy-com.datastax.driver.core.policies.ReconnectionPolicy- [ExponentialReconnectionPolicy]: https://docs.datastax.com/en/drivers/java/3.6/com/datastax/driver/core/policies/ExponentialReconnectionPolicy.html From bd2bc78eb2337b9d5367a08bb3176331387f2323 Mon Sep 17 00:00:00 2001 From: Andrew Tolbert Date: Mon, 7 Jan 2019 13:07:00 -0600 Subject: [PATCH 3/4] Process batch documentation feedback --- manual/statements/batch/README.md | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-) diff --git a/manual/statements/batch/README.md b/manual/statements/batch/README.md index 2d53362cf3f..fd8e50b9f41 100644 --- a/manual/statements/batch/README.md +++ b/manual/statements/batch/README.md @@ -1,6 +1,6 @@ ## Batch statements -Use [BatchStatement] to execute a set of queries as an atomic operation (refer to +Use [BatchStatement] to execute a set of queries as a single operation (refer to [Batching inserts, updates and deletes][batch_dse] to understand how to use batching effectively): ```java @@ -12,7 +12,7 @@ SimpleStatement simpleInsertBalance = new SimpleStatement("INSERT INTO cyclist_expenses (cyclist_name, balance) VALUES (?, 0) IF NOT EXISTS", "Vera ADRIAN"); -BatchStatement batch = new BatchStatement() +BatchStatement batch = new BatchStatement(BatchStatement.Type.UNLOGGED) .add(simpleInsertBalance) .add(preparedInsertExpense.bind("Vera ADRIAN", 1, 7.95f, "Breakfast", false)); @@ -23,10 +23,20 @@ As shown in the examples above, batches can contain any combination of simple st statements. A given batch can contain at most 65536 statements. Past this limit, addition methods throw an `IllegalStateException`. +By default, batches are configured as [LOGGED]. This ensures that if any statement in the batch +succeeds, all will eventually succeed. Ensuring all queries in a batch succeed has a +performance cost. Consider using [UNLOGGED] as shown above if you do not need this capability. + +Please note that the size of a batch is subject to the [batch_size_fail_threshold] configuration +option on the server. + In addition, simple statements with named parameters are currently not supported in batches (this is due to a [protocol limitation][CASSANDRA-10246] that will be fixed in a future version). If you try to execute such a batch, an `IllegalArgumentException` is thrown. [BatchStatement]: https://docs.datastax.com/en/drivers/java/3.6/com/datastax/driver/core/BatchStatement.html [batch_dse]: http://docs.datastax.com/en/dse/5.1/cql/cql/cql_using/useBatch.html +[LOGGED]: https://docs.datastax.com/en/drivers/java/3.6/com/datastax/driver/core/BatchStatement.Type.html#LOGGED +[UNLOGGED]: https://docs.datastax.com/en/drivers/java/3.6/com/datastax/driver/core/BatchStatement.Type.html#UNLOGGED +[batch_size_fail_threshold]: https://docs.datastax.com/en/cassandra/3.x/cassandra/configuration/configCassandra_yaml.html#configCassandra_yaml__batch_size_fail_threshold_in_kb [CASSANDRA-10246]: https://issues.apache.org/jira/browse/CASSANDRA-10246 From d88a8581373711b9d5af8f100c075a51454bd350 Mon Sep 17 00:00:00 2001 From: Andrew Tolbert Date: Mon, 7 Jan 2019 13:48:47 -0600 Subject: [PATCH 4/4] UdtValue -> UDTValue --- manual/udts/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/manual/udts/README.md b/manual/udts/README.md index 11727c404bd..08f37ac48c7 100644 --- a/manual/udts/README.md +++ b/manual/udts/README.md @@ -78,7 +78,7 @@ corrupt data, that you won't be able to read back. Once you have the type, call `newValue()` and set the fields: ```java -UdtValue udtValue = udt.newValue().setInt(0, 1).setString(1, "hello").setFloat(2, 2.3f); +UDTValue udtValue = udt.newValue().setInt(0, 1).setString(1, "hello").setFloat(2, 2.3f); ``` And bind your UDT value like any other type: