Unable to connect spark to cassandra DB

[renegademonkey]

Hi there,

I've spent the last week trying to figure out how to use sparlyr to get spark to connect to cassandra on our local cluster, and I've hit a wall - any help would be greatly appreciated. I'm the only one trying to use R to make this connection (everyone else uses Java on NetBeans and Maven), and have am not sure what I need to do to make this work.

The stack i'm using is:
Ubuntu 16.04 (in a VM)
sparklyr: 0.5.1
Spark: 2.0.0
Scala: 2.11
Cassandra: 3.7

relevant config.yml file settings:
`# cassandra settings
spark.cassandra.connection.host: <cluster_address>
spark.cassandra.auth.username: <user_name>
spark.cassandra.auth.password:

sparklyr.defaultPackages:
- com.databricks:spark-csv_2.11:1.3.0
- com.datastax.spark:spark-cassandra-connector_2.11:2.0.0-M1
- com.datastax.cassandra:cassandra-driver-core:3.0.2`

Sys.setnev setting set for local install of Java and spark, config set to use yml file. Spark connection initiated with:
sc <- spark_connect(master = "spark://<cluster_address>", config = spark_config(file = "config.yml"))

Spark session initiated with:
Spark.session <- sparklyr::invoke_static(sc, "org.apache.spark.sql.SparkSession", "builder") %>% sparklyr::invoke("config", "spark.cassandra.connection.host", "<cluster_address>") %>% sparklyr::invoke("getOrCreate")

It all seems fine up to here, (sc connection and Spark.session), but now attempting to access a cassandra table (table_1 in in keyspace_1), which I know exists:
cass_df <- invoke(Spark.session, "read") %>% invoke("format", "org.apache.spark.sql.cassandra") %>% invoke("option", "keyspace", "keyspace_1") %>% invoke("option", "table", "table_1") %>% invoke("load")

throws up the following errors:
`Error: java.lang.IllegalArgumentException: Cannot build a cluster without contact points
at com.datastax.driver.core.Cluster.checkNotEmpty(Cluster.java:123)
at com.datastax.driver.core.Cluster.(Cluster.java:116)
at com.datastax.driver.core.Cluster.buildFrom(Cluster.java:182)
at com.datastax.driver.core.Cluster$Builder.build(Cluster.java:1274)
at com.datastax.spark.connector.cql.DefaultConnectionFactory$.createCluster(CassandraConnectionFactory.scala:92) . . .'

[renegademonkey]

finally solved the issue! when initiating my sparkSession I was using the SPARK <cluster_address> with port number) rather than just the cluster address (where cassandra was located).