Skip to content
Docker debian base image that contains a spark build which has been compiled with postgresql driver
Branch: master
Clone or download

Latest commit

Fetching latest commit…
Cannot retrieve the latest commit at this time.

Files

Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
Dockerfile
README.md

README.md

docker-spark-postgresql-base

Docker image that contains a spark build which has been compiled with postgresql driver.

The idea is that if we can include the larger libraries as part of the docker image, we won't have to spend time downloading them every time we start a Spark executor because Docker will cache them.

For simplicity, I've compiled the postgresql driver into Spark. This saves us from classpath issues, adding jars for spark to distribute, security contexts etc..

See:

You can’t perform that action at this time.