This repository contains source code based on a guide on how to use Cloud Build and Cloud Composer to create a CI/CD pipeline for building, testing, and deploying a data processing workflow
-
Updated
Jul 12, 2021 - Shell
DataOps is an automated, process-oriented methodology, used by analytic and data teams, to improve the quality and reduce the cycle time of data analytics. While DataOps began as a set of best practices, it has now matured to become a new and independent approach to data analytics. DataOps applies to the entire data lifecycle from data preparation to reporting, and recognizes the interconnected nature of the data analytics team and information technology operations.
This repository contains source code based on a guide on how to use Cloud Build and Cloud Composer to create a CI/CD pipeline for building, testing, and deploying a data processing workflow
DataOps for the Modern Data Warehouse on Microsoft Azure. https://aka.ms/mdw-dataops.
HokStack - Run Hadoop Stack on Kubernetes
❤for real-time DataOps - where the application and data fabric blends - Lenses
DataOps for the Modern Data Warehouse on Microsoft Azure. https://aka.ms/mdw-dataops.
Kafka Docker for development. Kafka, Zookeeper, Schema Registry, Kafka-Connect, Landoop Tools, 20+ connectors