This project demonstrates the application of Principal Component Analysis (PCA) and Logistic Regression on the load_digits() dataset from scikit-learn. It includes:
- Training a Logistic Regression classifier on the original 64-dimensional data
- Applying PCA for dimensionality reduction
- Retraining Logistic Regression on the PCA-reduced data
- Comparing model accuracy, performance, and execution time before and after PCA
The load_digits() dataset contains:
- 1797 grayscale images of handwritten digits (0 to 9)
- Each image is 8×8 pixels, flattened into a 64-dimensional vector
- Pixel values range from 0 (white) to 16 (black)
- Each entry has a corresponding label (0–9)
- Train a Logistic Regression classifier on original high-dimensional data
- Use PCA to reduce dimensionality while preserving variance
- Compare classification accuracy and efficiency before and after PCA
- Python 3.x
scikit-learn– for loading data, PCA, and Logistic Regressionmatplotlib&seaborn– for data visualizationnumpy,pandas– for data processing
| Metric | Without PCA (64 features) | With PCA (29 components) |
|---|---|---|
| Accuracy | 97.22% | 96.44% |
| Number of Features | 64 | 29 |
| Dimensionality Reduction | ❌ | ✅ ~55% fewer features |
| Training Time | Slightly higher | Slightly faster |
- Logistic Regression achieves 97.22% accuracy on the original 64-feature space.
- After reducing the features to 29 principal components using PCA, the model still maintains a strong 96.44% accuracy.
- PCA helped in reducing dimensionality by ~55%, while retaining over 96% classification performance.
- This trade-off is often valuable when working with larger, high-dimensional datasets where training speed and generalization matter.
Contributions, issues, and feature requests are welcome! Feel free to open an issue or submit a pull request.