Skip to content
master
Switch branches/tags
Code

Latest commit

 

Git stats

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
 
 
 
 
 
 
 
 
 
 

Docker image sparkfromr

Based on images by the Rocker Project. Used for building the sparkfromr.com bookdown website. Has RStudio exposed at http://localhost:8787.

Main features

  • R 3.6.1
  • RStudio Server
  • openjdk 8
  • Apache Spark 2.4.3
  • Apache Arrow C++ libraries

R Packages (including dependencies):

  • sparklyr (1.0.2)
  • arrow (0.14.1.1)
  • remotes (2.1.0)
  • microbenchmark (1.4-6)
  • knitr (1.24)
  • rmarkdown (1.15)
  • bookdown (0.13)
  • data.table (1.12.2)

Usage

Interactive with RStudio and sparklyr

# replace <password> with a password of your choice
docker run -d -p 8787:8787 -e PASSWORD=<password> --name rstudio jozefhajnala/sparkfromr:latest

Navigate to http://localhost:8787. Username for login is: rstudio, password is the one you chose above.

Building bookdown books

docker run --rm -it jozefhajnala/sparkfromr:latest /bin/bash
git clone https://github.com/rstudio/bookdown-demo
cd bookdown-demo
Rscript -e "bookdown::render_book('.')"

Rendering spark-related code with R Markdown

docker run --rm -it jozefhajnala/sparkfromr:latest R
rmarkdown::render("/root/.local/spark_script.R")

Interactive with R and sparklyr

docker run --rm -it jozefhajnala/sparkfromr:latest R

# R session should start
library(sparklyr)
sc <- spark_connect("local")

Interactive with spark shell

docker run --rm -it jozefhajnala/sparkfromr:latest /root/spark/spark-2.4.3-bin-hadoop2.7/bin/spark-shell

# Scala REPL should open with
# - Spark context available as `sc`
# - Spark session avaiable as `spark`

Running an example R script

docker run --rm jozefhajnala/sparkfromr:latest Rscript /root/.local/spark_script.R

About

Docker sources for the sparkfromr.com bookdown site. The image is auto build in DockerHub.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published