BDE stack
The BDE stack consists of 3 layers: hardware, resource manager, and applications. Each layer is shortly described below. The installation guide of the base platform can be found here.
Hardware nodes which may be servers available in a data-center or hosted in the cloud.
Docker Swarm is a clustering tool turning a pool of Docker hosts into a single virtual host. It serves the standard Docker API, so any tool that already communicates with a Docker daemon can use Swarm to transparently scale to multiple hosts.
An application running on the BDE stack can be seen as a pipeline consisting of multiple components, which are wired together in order to solve a specific Big Data problem. The components will be packaged in Docker containers. In order to encourage technology reuse and to facilitate the development of a component, the BDE stack will provide base Docker images.
A base Docker image offers a template implementation for a specific technology. This template can be easily extended by a programmer with his own custom implementation to solve a particular problem. Should some base technology not be supported yet, a new base image can be built. The selection of technologies that will be provided as base images mainly depend on the pilot cases that need to be implemented. An overview of the supported components can be found here.