Formative 2 - Data Preprocessing Assignment for Machine Learning Pipeline

Project Overview

This project focuses on preprocessing two real-world datasets to enhance data quality and prepare them for machine learning. The process involved data augmentation, merging datasets using transitive properties, feature engineering, and ensuring data consistency.

Group Members & Contributions

• Geofrey Tumwesigye: Part 1 - Data Augmentation on CSV Files
• Serge Kamanzi: Part 2 - Merging Datasets with Transitive Properties
• Willy Kalisa: Part 3 - Data Consistency and Quality Checks

Video link = https://www.youtube.com/watch?v=D6oaPMmh098

## Project Structure

├── Bonus Challenge
│   └── Bonus_Challenge-colab.ipynb         # bonus challenge colab
├── csv_files
│   ├── customer_transactions_augmented.csv  # csv generated on task 1
│   ├── final_customer_data_5.csv            # csv generated on task 2
│   └── final_dataset_ready_5.csv            # csv generated on task 3
├── Data Preprocessing Project Report.pdf    # pdf report file
├── Formative_2_Data_Preprocessing.ipynb     # main colab
└── README.md

How to Use This Project

1. Clone the repository:

   git clone https://github.com/sergekamanzi/Formative2-Data-Preprocessing

2. Navigate to the project folder and run the notebooks in Google Colab:
   • Open Formative_2_Data_Preprocessing.ipynb and Bonus_Challenge-colab.ipynb in Google Colab.
3. Download and use the preprocessed datasets for training machine learning models:
   • Preprocessed datasets are stored in the csv_files folder.

Challenges Faced

• Handling Missing Values: Predictive modeling was required for filling gaps in certain features.
• Complex Merging: Transitive mapping added an extra layer of complexity in dataset integration.
• Feature Engineering: Required experimentation with different transformations to extract useful insights.