-
Notifications
You must be signed in to change notification settings - Fork 2
Machine Learning Basics: scikit‐learn Unveiled
UA DATALAB WORKSHOP SERIES
📓 These notes: https://github.com/ua-datalab/Workshops/wiki/Machine-Learning-Basics-Scikit-learn-Unveiled/
💻 Codebase with examples: ML Classification with scikit-learn
Machine learning (ML) focuses on the study of computer algorithms that are feed on collections of data, and can be improved automatically or by the intervention of the users.
Machine learning can be seen as a part of Artificial Intelligence. And within Machine learning there is a whole other field of knowledge called Deep learning (DL).
With Machine learning algorithms, models are built based on sample data, known as training data. The goal of these models is to make predictions or decisions without being explicitly programmed to perform this.
Machine learning algorithms are used in a wide variety of current applications, for example they can be used for:
- Object Recognition, a computer vision task to automatically identify objects form a digital image or video.
- Automatic Summarization, for extracting the most relevant information form a text document.
- Prediction, which form analyzing current or historical data a future event can be predicted.
- Classification, which is used to identify an object based on its main features and assign it to a set of similar objets.
- Clustering, being the action of grouping objects with similar characteristics.
- Recommender systems, generalized application in digital platforms that seeks to predict the "rating" or "preference" a user will sign to a specific item.
- and more.
(Image Credit: mygreatlearning.com)
In Python, there is Scikit-learn Library for solving machine learning problems built on top of Numpy and SciPy. It features a lot of machine learning algorithms such as support vector machines, and random forests, as well as a lot of utilities for general pre-and post-processing of data. There is a large set of worked examples we can use to adapt them to whatever is our case.
Scikit-learn's implementation of neural networks is limited. Unlike scikit-learn, TensorFlow and PyTorch allow you to use a custom architecture, and they support GPUs for a massive training scale.
There is also a well-known specialized Python library dedicated to digital image processing called scikit-image. It includes algorithms for image segmentation, geometric transformations, color space manipulation, image analysis, filtering, morphology, feature detection, and more.
Scikit-Learn also provides a collection of datasets for learning machine learning, that can be accessed thru the function sklearn.datasets
.
Machine Learning Process - Infograph
Classical Machine Learning Methods can be classified into 3 major classes:
- Supervised Learning. In supervised learning, the algorithms construct a function from input-output examples, and then it will predict the outcome given an unlabeled input.
- Unsupervised Learning. In unsupervised learning, the algorithms learn to detect regularity or patterns from untagged data.
- Weak Supervised Learning. This process lands between supervised and unsupervised learning methods. It occurs when there is only a noisy, limited or imprecise set of data used in supervised training for labeling large amounts of training data.
- Classification. Classification is an algorithm for predicting to which set of items, class,es or categories an object belongs to.
Some examples of Classification algorithms:
Decision Trees | Examples
K-Nearest Neighbors | Examples (Infograph)
Naive Bayes | Examples
Perceptron | Examples
Random Forest | Example
Support Vector Machine | Examples (Infograph)
- Regression (Prediction). Regression algorithms are used for predicting continuous values.
Some examples of Regression algorithms:
Linear Regression | Examples (Infograph)
Least Squares | Examples
Polynomial Regression | Examples (Infograph)
Nonlinear Regression
Logistic Regression | Examples (Infograph)
Classifier evaluation metrics:
- Precision and Recall.
- Precision = TP/(TP + FP), is the ratio of the correctly identified positive cases to all the predicted positive cases.
- Recall = TP/(TP + FN), also known as sensitivity, is the ratio of the correctly identified positive cases to all the actual positive cases.
- Accuracy. Accuracy is a statistical measure that is defined as the quotient of correct predictions (both True positives (TP) and True negatives (TN)) made by a classifier divided by the sum of all predictions made by the classifier, including False positives (FP) and False negatives (FN). Accuracy = (TP + TN)/(TP + TN + FP + FN).
- F1-score. F1 = 2(Precision * Recall)/(Precision + Recall) = 2TP/(2TP + FP + FN).
- Area under the curve - AUC
- The Confusion Matrix, is a specific table layout that allows visualization of the performance of an algorithm
Regression Learning Evaluation:
More on Evaluation Metrics in Scikit-Learn.
- Dimensionality reduction, is a transformation of data set from a high-dimensional space into a low-dimensional space so that this representation preserves the meaningful properties of the original data. Dimensionality reduction is common in fields that deal with large numbers of observations and/or large numbers of variables, such as signal processing, speech recognition, neuroinformatics, and bioinformatics.
Some examples of Dimensionality Reduction Algorithms:
Linear Discriminant Analysis - LDA | Examples
Latent Semantic Analysis - LSA | Examples
Principal Component Analysis - PCA | Examples
Singular Value Decomposition - SVD
t-distribuited Stochastic Neighbor Embedding - t-SNE
- Association Rule Learning, is a rule-based machine learning method meant to discover the rules that determine how or why certain items are connected.
Some examples of Association Rule Learning Algorithms:
Apriori
Equivalent Class Transformation - Eclat
Frequent Pattern - FP-Growth
- Clustering is the action of defining groups of objects (clusters) that are very similar to each other, compared with objects from other groups.
Some examples of Clustering algorithms Comparison:
Connectivity models: for example, hierarchical clustering builds models based on distance connectivity. | Examples (Infograph)
Centroid models: for example, the k-means algorithm represents each cluster by a single mean vector. | Examples (Infograph)
Distribution models: clusters are modeled using statistical distributions, such as the multivariate normal distributions.
Density models: for example, the Density-based spatial clustering of applications with noise - DBSCAN and the Ordering Points to Identify the Clustering Structure - OPTICS, which defines clusters as connected dense regions in the data space. | DBSCAN Examples | OPTICS Examples
Soft Clustering: Each object belongs to each cluster to a certain degree, example Fuzzy Clustering.
There is a large collection of Cluster Analysis Algorithms.
Dimensionality Reduction and Clustering Examples
Reinforcement learning is the third Machine Learning paradigm, alongside supervised learning and unsupervised learning.
Reinforcement learning focus on finding a balance between exploration (of uncharted territory) and exploitation (of current knowledge).
Some examples of Reinforcement Learning Algorithms:
Monte Carlo method
Q-learning
State-action-reward-state-action - SARSA
Decision Tree Learning
Decision Tree learning (Infograph)
Ensemble learning uses a combination of algorithms to improve its prediction performance. Examples of Ensemble learning algorithms:
- Random Forest (Examples) is an ensemble learning algorithm used for classification and regression (Infograph).
Adaptive Boosting - AdaBoost (Examples)
Gradient Boosting (Examples)
XGBoost - eXtreme Gradient Boosting
LightGBM - Light Gradient Boosting Machine
- Artificial Intelligence and Machine Learning Explained. Steve Blank.
- Machine Learning. Wikipedia.org.
- Scikit-Learn Library | User guide | API
- Andreas C. Müller, Sarah Guido. Introduction to Machine Learning with Python - Code Repository
- Bernd Klein. Intro to Machine Learning with Python. Python-course.eu.
- Machine Learning Tutorial for Complete Beginners. Great Learning Team. mygreatlearning.com.
- Pratik, Shukla and Iriondo, Roberto and Chen, Sherwin (2020). Machine Learning Algorithms For Beginners with Code Examples in Python. Towards AI.
- Raschka, Sebastian, Liu Yuxi (Hayden), and Mirjalili, Vahid (2022). Machine Learning with PyTorch and Scikit-Learn Book. Code & notes. Packt Publishing.
- 11 Dimensionality Reduction Techniques You Should Know in 2021. Rukshan Pramoditha. Towards Data Science.
- Avik Jain. 100 Days of ML Code.
- Shervine Amidi. Supervised Machine Learning Cheat Sheet
- Shervine Amidi. Unsupervised Machine Learning Cheat Sheet
Created: 04/24/2022 (C. Lizárraga); Last update: 10/24/2023 (C. Lizárraga)
UArizona DataLab, Data Science Institute, University of Arizona, 2024.