This project analyzes a semiconductor manufacturing dataset (uci-secom.csv
) from Kaggle to extract insights and identify potential applications. It employs various data science methodologies, including data preprocessing, machine learning, and data visualization.
The dataset includes:
- A
Time
column with the timestamp of data collection. - 590 feature columns (
Sensor_0
toSensor_589
) representing sensor data or process measurements. - A
Pass/Fail
label column indicating the manufacturing outcome (1 for fail, -1 for pass).
- To understand the data structure, quality, and distribution.
- To address missing values and imbalanced data.
- To uncover patterns and insights through data visualization.
- To predict the Pass/Fail outcome using machine learning models.
- To evaluate the performance of preprocessing techniques and models.
- Handling Missing Values: Explored deletion, mean/median imputation, and hybrid approaches.
- Addressing Imbalanced Data: Applied undersampling and oversampling (SMOTE) techniques.
- Normalization: Utilized Min-Max scaling to prepare the dataset for machine learning algorithms.
- Conducted exploratory data analysis (EDA) using box plots and bar charts.
- Analyzed the correlation between features and the Pass/Fail outcome.
- Baseline Model: Implemented XGBoost to handle imbalanced data, evaluating its performance with precision, recall, F1 score, and confusion matrices.
- Model Comparison: Compared different preprocessing approaches to optimize handling missing values and imbalanced data.
- Advanced Techniques: Explored dimensionality reduction (PCA) and custom oversampling techniques.
- Preprocessing significantly impacts handling missing values and imbalanced data, with certain strategies proving more effective.
- Machine learning models, particularly those adjusted for imbalanced data, can predict the Pass/Fail outcome effectively.
- Novel oversampling techniques, inspired by CTGAN and SMOTE, improved model recall without significantly affecting precision.
- Experiment with other machine learning algorithms and parameter tuning.
- Explore additional oversampling and undersampling techniques.
- Validate the model on different datasets to ensure generalizability.
This project demonstrates the potential of data science in improving decision-making in semiconductor manufacturing. It addresses missing values and imbalanced data challenges, paving the way for accurate predictive modeling.