Repository navigation
2. Architecture
This Spark Docker Implementation works on Desktop machines running Docker Desktop Software which supports Kubernetes. All relevant services run in the Kubernetes cluster and are exposed to the desktop via a NGINX Ingress providing web url's which can be browsed or approached using REST.
The Development Containers runs natively on Docker and Docker Compose as this is the most easy way to work with Development Containers in VSCode where a local workspace can be used as a working folder. The Development Containers needs to run on the virtual machine host (the desktop) in order to have connection with the Kubernetes services using the NGINX ingresses and the Kube API to start Spark Workloads.