The Goal of this project is to provide documentation for the Lakehouse Engine framework.

Using U-Net Model to Detect Wildfire from Satellite Imagery

From image to text - a handwriting recognition tool prototype using Image Classification - Deep Learning in DataBricks.

Stocks Data Analysis In DataBricks - Using SQL and Pyspark

🏂 A machine learning model that performs topic classification of news articles for media bias analysis. Final project for UC Berkeley MIDS 266 (Natural Language Processing)

practice of Spark on two datasets using PySpark

Scalable Data Science, course sets in big data Using Apache Spark over databricks and their mathematical, statistical and computational foundations using SageMath.

End-to-end data engineer project

POC projects working on Cloud Platforms

Alumni Profile Matching is a project aimed at facilitating networking between graduate students and alumni with similar backgrounds and career goals. By leveraging machine learning techniques and data processing pipelines, the project aims to provide graduate students with personalized recommendations of alumni profiles to connect with.

Spotify API Data Engineering + Machine Learning Project

A place to learn data engineering

Distributed processing challenge

Built a data pipeline with Azure Databricks and Azure Data Factory using Formula 1 data

Implementation of the "CCF: Fast and Scalable Connected Component Computation in MapReduce" paper with Spark. Study of its scalability on several datasets using various clusters' sizes on Databricks and Google Cloud Platform (GCP)

Wind energy prediction employing PySpark in Databricks.

DEPRECATED: Integrating Jupyter with Databricks via SSH

This is a code sample repository for demonstrating how to perform Databricks Delta Table operations.

A project on classification of GitHub readme sections using Machine Learning