Data Engineering

Useful Tools

• Debezium for change data capture, architecture example
• Meltano for DataOps
• DataHub: The Metadata Platform for the Modern Data Stack
• Prefect - Orchestration

Reading

• "Data Warehouse Guide" by Panoply
• "What is Data Engineering: Explaining the Data Pipeline, Data Warehouse, and Data Engineer Role" by Altexsoft
• "Machine Learning Project Structure: Stages, Roles, and Tools" by Altexsoft
• Data Warehousing Tutorial
• Data Engineer Cookbook
• Engineers Shouldn’t Write ETL