Here's how to get DataQuest's Data Engineering Track missions' content to work on your localhost. Using data from my Valenbisi ARIMA modeling project, I document my steps using PostgreSQL, Postico, and the Command Line to get our DataQuest exercises running out of a Jupyter Notebook.
An extension enabling the monitoring of Apache Airflow DAGs directly from Jupyter notebooks. Tailored for developers and data scientists, it simplifies tracking specific DAGs, reduces unnecessary friction, and allows severity levels setup for failed DAGs.
This project was made in Jupyter Notebook, it included data ingestion from a CSV file, PDF file, and web scraping. Then transforming and cleaning the data, and lastly storing the data in a storage solution, in which I used MongoDB to store the data.
Spotify Data Engineering Project
A compute framework for turning multimodal data structures into vector embeddings, to improve quality and control when working with LLMs. Generate custom multimodal embeddings with ease and weigh the vector parts separately at query time, removing the need for custom re-ranking models. Deploy straight from notebook to production.
Add a description, image, and links to the data-pipeline topic page so that developers can more easily learn about it.
To associate your repository with the data-pipeline topic, visit your repo's landing page and select "manage topics."