Load Avro data into Spark with sparklyr
R
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
R
inst/extdata
man
tests
.Rbuildignore
.gitignore
.travis.yml
DESCRIPTION
LICENSE
NAMESPACE
README.md
sparkavro.Rproj

README.md

Travis-CI Build Status

sparkavro

Load Avro data into Spark with sparklyr. It is a wrapper of spark-avro

Installation

Install using {devtools} as follows:

devtools::install_github("chezou/sparkavro")

Usage

library(sparklyr)
library(sparkavro)
sc <- spark_connect(master = "spark://HOST:PORT")
df <- spark_read_avro(sc, "test_table", "/user/foo/test.avro")

spark_write_avro(df, "/tmp/output")

Example data are from https://github.com/miguno/avro-cli-examples