Luigi is a Python module that helps you build complex pipelines of batch jobs. It handles dependency resolution, workflow management, visualization etc. It also comes with Hadoop support built in.

A flexible, easy to use, automation framework allowing users to integrate their capabilities and devices to cut through the repetitive, tedious tasks slowing them down. #nsacyber

ESB, SOA, REST, APIs and Cloud Integrations in Python

AWS account provisioning and management service

SpiffWorkflow is a software development platform for building, running, and monitoring executable diagrams

Saisoku is a Python module that helps you build complex pipelines of batch file/directory transfer/sync jobs.

The worker node manager container which manages nebula nodes

The API endpoint that manages nebula orchestrator clusters

TwinGraph is a Python framework for distributed container orchestration using Kubernetes clusters, Docker Compose/Swarm or cloud resources on AWS (AWS Lambda, AWS Batch, Amazon EKS). Applications include high-throughput simulations, simulation-driven optimization, Digital Twins and machine learning.

An end-to-end Python-based Infrastructure as Code framework for network automation and orchestration.

BotCity Maestro SDK - Python

This project demonstrates how to build and automate an ETL pipeline using DAGs in Airflow and load the transformed data to Bigquery. There are different tools that have been used in this project such as Astro, DBT, GCP, Airflow, Metabase.

a massive scale docker container orchestrator REPO MOVED - DETAILS AT README

Johann, the lightweight and flexible scenario orchestrator

A next-generation open source orchestration platform for the development, production, and observation of data assets.

command line tool for managing nebula clusters

Tractor API extension for authoring reusable task hierarchies.

A lightweight yet powerful, event driven workflow orchestration manager for microservices.

docker-compose framework and installation scripts for creating bitcoin boxes

The optional reporter container which reads nebula reports from Kafka into the backend DB