Skip to content
Docker sources for the sparkfromr.com bookdown site. The image is auto build in DockerHub.
Dockerfile Shell R
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
.github/workflows
resources
.gitignore
Dockerfile
LICENSE
README.md

README.md

Docker image sparkfromr

Based on images by the Rocker Project. Used for building the sparkfromr.com bookdown website. Has RStudio exposed at http://localhost:8787.

Main features

  • R 3.6.1
  • RStudio Server
  • openjdk 8
  • Apache Spark 2.4.3
  • Apache Arrow C++ libraries

R Packages (including dependencies):

  • sparklyr (1.0.2)
  • arrow (0.14.1.1)
  • remotes (2.1.0)
  • microbenchmark (1.4-6)
  • knitr (1.24)
  • rmarkdown (1.15)
  • bookdown (0.13)
  • data.table (1.12.2)

Usage

Interactive with RStudio and sparklyr

# replace <password> with a password of your choice
docker run -d -p 8787:8787 -e PASSWORD=<password> --name rstudio jozefhajnala/sparkfromr:latest

Navigate to http://localhost:8787. Username for login is: rstudio, password is the one you chose above.

Building bookdown books

docker run --rm -it jozefhajnala/sparkfromr:latest /bin/bash
git clone https://github.com/rstudio/bookdown-demo
cd bookdown-demo
Rscript -e "bookdown::render_book('.')"

Rendering spark-related code with R Markdown

docker run --rm -it jozefhajnala/sparkfromr:latest R
rmarkdown::render("/root/.local/spark_script.R")

Interactive with R and sparklyr

docker run --rm -it jozefhajnala/sparkfromr:latest R

# R session should start
library(sparklyr)
sc <- spark_connect("local")

Interactive with spark shell

docker run --rm -it jozefhajnala/sparkfromr:latest /root/spark/spark-2.4.3-bin-hadoop2.7/bin/spark-shell

# Scala REPL should open with
# - Spark context available as `sc`
# - Spark session avaiable as `spark`

Running an example R script

docker run --rm jozefhajnala/sparkfromr:latest Rscript /root/.local/spark_script.R
You can’t perform that action at this time.