Want to add [ML] Implement PCA, Logistic Regression (Vectorized), Naive Bayes with Laplace Smoothing, and Decision Tree Pruning from Scratch #13320
Description
Feature description
Hello maintainers 👋,
I would like to contribute several machine learning algorithms implemented from scratch to enhance the machine_learning/ module.
These implementations will follow the repository’s coding guidelines — including type hints, docstrings, and unit tests.
📘 Proposed Algorithms
Principal Component Analysis (PCA) from Scratch
Implementation using NumPy for matrix operations.
Steps: standardization, covariance matrix computation, eigen decomposition, dimensionality reduction.
Output: projected dataset and explained variance ratio.
File: machine_learning/pca_from_scratch.py
Logistic Regression (Vectorized Version)
Implementation using full vectorization with NumPy.
Includes sigmoid activation, gradient descent optimization, and cost function.
File: machine_learning/logistic_regression_vectorized.py
Naive Bayes with Laplace Smoothing
Multinomial Naive Bayes classifier implementation with Laplace correction to handle zero probabilities.
File: machine_learning/naive_bayes_laplace.py
Decision Tree Pruning
Enhancement of the existing decision tree algorithm with post-pruning (reduced error pruning) or cost-complexity pruning.
Reduces overfitting on training data.
File: machine_learning/decision_tree_pruning.py