Skip to content

Docker debian base image that contains a spark build which has been compiled with postgresql driver

Notifications You must be signed in to change notification settings

nickpoorman/docker-spark-postgresql-base

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

18 Commits
 
 
 
 

Repository files navigation

docker-spark-postgresql-base

Docker image that contains a spark build which has been compiled with postgresql driver.

The idea is that if we can include the larger libraries as part of the docker image, we won't have to spend time downloading them every time we start a Spark executor because Docker will cache them.

For simplicity, I've compiled the postgresql driver into Spark. This saves us from classpath issues, adding jars for spark to distribute, security contexts etc..

See:

About

Docker debian base image that contains a spark build which has been compiled with postgresql driver

Resources

Stars

Watchers

Forks

Releases

No releases published

Sponsor this project

 

Packages

No packages published