Support for HBase Data Source

[mramos1004]

I've been looking for a way to connect to HBase through SparklyR but I can't find any examples. Is it right so far, the only thing to do is to export the hbase tables to hdfs files?

[kevinykuo]

https://github.com/hortonworks-spark/shc

[javierluraschi]

Should be able to use sparklyr with something like this: https://therinspark.com/data.html#cassandra

But instead of connecting to Cassandra, you would use the Spark-HBase connector from Hortonworks: https://github.com/hortonworks-spark/shc

Something like the following might just work...

sc <- spark_connect(master = "local", version = "2.3", config = list(
  sparklyr.connect.packages = "com.hortonworks:shc-core:1.1.1-2.1-s_2.11",
  sparklyr.shell.repositories = "http://repo.hortonworks.com/content/groups/public/",
  sparklyr.shell.files = "/etc/hbase/conf/hbase-site.xml"))

spark_read_source(
  sc, 
  name = "<table>",
  source = "org.apache.spark.sql.execution.datasources.hbase",
  options = list("HBaseTableCatalog.tableCatalog" = "<catalog>"),
  memory = FALSE)