Red Cell: Cluster Configuration & Deployment
This repository contains scripts and configurations used at Ognitio to automate the deployment process of our clusters. As we run Apache Mesos and a few of his frameworks for scheduling, we plan to continuously share publicly related templates and scripts through this directory.
Status
This project is undergoing heavy refactoring, hardening and testing and isn't even considered alpha.
Technology stack
We mainly rely on Ansible to configure our VMs.
The following list contains some of the projects that composes our stack:
-
Consensus
- Apache ZooKeeper: Distributed systems coordination
-
Resource management
-
Scheduling (i.e. Mesos frameworks)
- Marathon: Long running services
- Aurora: Long-running services and cron jobs
- Chronos: Cron jobs
- Kafka: Apache Kafka on Apache Mesos
- Elasticsearch: Elasticsearch on Apache Mesos
-
Logging and metrics
-
Container registry
- Distribution: Docker toolset to pack, ship, store, and deliver content
-
Service discovery and load balancing
-
Application isolation
- Weave: Network of Docker containers
Further work
- Enable SSL within Mesos and frameworks
- Authorization and authentication
- Secret files distribution
- Finer cloud providers deployment support (GCE, AWS, ...)
- Generate Ansible inventory file