• Python Library i need to learn first:
  • NumPy
  • Pillow (PIL)
  • OpenCV
  • scikit-learn
  • TensorFlow or PyTorch

Machine Learning Algorithms

• Regression
• Decision Trees
• Support Vector Machines (SVM)
• Naive Bayes Classifiers
• K-Nearest neighbor
• Artificial Neural Networks (belongs to subfield of deep learning)

Each algorithm uses different techniques to learn patterns from data and make predictions or discover structure in data.

1. Supervised Learning Algorithms (SVM, Linear Regression, Logistic Regression, Decision Trees) - where the model is trained on labeled data.
2. Unsupervised Learning Algorithms (K-Means, PCA) - where the model learns from data without explicit labels.
3. Ensemble Methods (Random Forest, Gradient Boosting) - which reduce the number of features in data while preserving its important str.

1. Support Vector Machine (SVM)

• Type: Supervised learning algorithm (classification and regression).
• Use case: Primarily used for classification tasks (e.g., image recognition, text classification).
• How it works: SVM finds the optimal hyperplane that best separates data points of different classes. In higher dimensions, this hyperplane can be a line, plane, or hyperplane.
• Key feature: Can handle both linear and non-linear classification by using kernel tricks to project data into higher dimensions.

2. K-Means Clustering

• Type: Unsupervised learning algorithm (clustering).
• Use case: Used to group similar data points into clusters (e.g., market segmentation, image compression).
• How it works: Divides data into k clusters based on distance, where each cluster is represented by a centroid. The algorithm iterates by adjusting cluster centroids to minimize the variance within each cluster.
• Key feature: Fast and simple, but the number of clusters (k) needs to be predefined.

3. Linear Regression

• Type: Supervised learning algorithm (regression).
• Use case: Used for predicting continuous outcomes (e.g., predicting house prices, sales forecasts).
• How it works: Models the relationship between independent variables (features) and a dependent variable (target) by fitting a straight line that minimizes the error between the predicted and actual values.
• Key feature: Simple to understand and interpret, but assumes a linear relationship between variables.

4. Logistic Regression

• Type: Supervised learning algorithm (classification).
• Use case: Used for binary classification tasks (e.g., spam detection, disease prediction).
• How it works: Similar to linear regression but applies the sigmoid function to the output to constrain the predictions between 0 and 1. This makes it suitable for probability-based classification.
• Key feature: Handles binary and multi-class classification problems effectively.

5. Decision Trees

• Type: Supervised learning algorithm (classification and regression).
• Use case: Used for tasks like classification (e.g., fraud detection, customer segmentation).
• How it works: Splits the data into branches based on feature values to create a tree structure. Each branch represents a decision, and the leaf nodes represent the output (class label or value).
• Key feature: Easy to interpret and visualize, but can overfit without pruning or regularization.

6. Random Forest

• Type: Supervised learning algorithm (ensemble method).
• Use case: Used for classification and regression tasks (e.g., credit scoring, stock market prediction).
• How it works: Builds multiple decision trees (a "forest") and aggregates their results for better accuracy and generalization.
• Key feature: Reduces overfitting and improves accuracy by averaging the results from multiple trees.

7. k-Nearest Neighbors (k-NN)

• Type: Supervised learning algorithm (classification and regression).
• Use case: Used in recommendation systems, image recognition, and classification tasks.
• How it works: Classifies a data point based on the majority vote of its k nearest neighbors in the feature space.
• Key feature: Simple and intuitive, but can be computationally expensive for large datasets.

8. Naive Bayes

• Type: Supervised learning algorithm (classification).
• Use case: Used for text classification, spam filtering, sentiment analysis.
• How it works: Based on Bayes' theorem, it assumes that the features are independent given the target class, which simplifies the computations.
• Key feature: Fast and works well for high-dimensional data like text.

9. Principal Component Analysis (PCA)

• Type: Unsupervised learning algorithm (dimensionality reduction).
• Use case: Used to reduce the dimensionality of datasets (e.g., in image compression, data visualization).
• How it works: PCA identifies the principal components (i.e., directions of maximum variance) in the data and transforms the data to a lower-dimensional space while preserving as much variance as possible.
• Key feature: Helps in reducing the complexity of data without losing much information.

10. Gradient Boosting Machines (GBM)

• Type: Supervised learning algorithm (ensemble method).
• Use case: Used in regression and classification tasks (e.g., ranking, customer churn prediction).
• How it works: Builds models sequentially, where each new model corrects the errors of the previous ones by minimizing the loss function.
• Key feature: Powerful and accurate, but can be slow to train.

Deep Learning

• It is a type of machine learning where computers learn to do tasks by processing large amounts of data.
• Its called “Deep” because it uses layers of artificial “neurons” to make sense of complex info like recognizing faces, understanding speech
• More layer, better it can learn

With supervised learning, the data that the network is trained on is labeled, whereas with unsupervised learning, the data is unlabeled.

The Perceptron

• Structural building block of deep learning

Activation Function

• help model decide whether a neuron should be activated (fired) or not.
• Introduces Non-Linearity : allows the network to learn complex patterns and relationships in data.
• all activation functions are non-linear
• linear activation functions produce linear decisions no matter the network size
• non-linear allow us to approximate arbitrarily complex functions

Common activation functions used in neural networks include:

1. ReLU (Rectified Linear Unit)

• Formula: ( f(x) = \max(0, x) )
• Range: [0, ∞)
• Use case: Most widely used in deep learning for hidden layers. It's simple and effective for learning.
• Pros: Efficient, helps with vanishing gradient problems.
• Cons: Can suffer from the "dying ReLU" problem, where neurons get stuck at 0 and stop learning.

2. Sigmoid

• Formula: ( f(x) = \frac{1}{1 + e^{-x}} )
• Range: (0, 1)
• Use case: Typically used for binary classification tasks.
• Pros: Outputs values in the range of probabilities.
• Cons: Can cause vanishing gradient problems and slow convergence.

3. Tanh (Hyperbolic Tangent)

• Formula:

  

• Range: (-1, 1)

• Use case: Useful for hidden layers where negative values are needed.

• Pros: Centers output around zero, better for learning compared to Sigmoid.

• Cons: Can also cause vanishing gradient issues.

4. Leaky ReLU

• Formula: ( f(x) = \max(0.01x, x) )
• Range: (-∞, ∞)
• Use case: A variant of ReLU, used to fix the "dying ReLU" problem by allowing small negative values.
• Pros: Helps prevent neurons from becoming inactive.
• Cons: May still be less effective than other activations in some tasks.

5. Softmax

• Formula:

  

• Range: (0, 1), where all outputs sum to 1.

• Use case: Used for multi-class classification to output probabilities for different classes.

• Pros: Provides a probability distribution across multiple classes.

• Cons: More computationally expensive due to exponentials.

6. Swish

• Formula:

  

• Range: (-∞, ∞)

• Use case: Developed by Google, Swish is gaining popularity as it can outperform ReLU in some tasks.

• Pros: Smooth, non-linear, and differentiable.

• Cons: More computationally intensive than ReLU.

7. ELU (Exponential Linear Unit)

• Formula:

  

• Range: (-α, ∞), where α is a hyperparameter.

• Use case: Similar to ReLU but allows for smoother and faster convergence.

• Pros: Reduces the vanishing gradient problem.

• Cons: More complex and slower than ReLU.

Loss Function:

• measures how well or poorly the models predictions match the actual data (ground truth).
• Goal of training model is minimize the loss function,
• adjust models parameters (weights and biases) through a process called optimization

Optimization

Optimization refers to the process of adjusting the weights and biases in such a way that the model's performance improves. The aim is to minimize the loss function (the difference between the model’s predictions and the actual values).

Gradient Descent is one of the most common optimization algorithms used to adjust weights and biases. Here's how it works:

• Gradient: This refers to the direction and rate of change of the loss function. Imagine the loss function as a hill, and gradient descent is the process of finding the lowest point on that hill (the minimum loss).
• Descent: The algorithm calculates the gradient (slope) of the loss function with respect to the model's parameters (weights and biases). It then updates the parameters in the direction that decreases the loss. This process is repeated over many iterations until the model converges to the point where the loss is minimized.

King here

CNN

what is CNN/ Convolutional Neural Network (CNN)?

• Also know as ConvNet
• The process starts by sliding a filter designed to detect certain features over the input image, a process known as the convolution operation (hence the name "convolutional neural network"). The result of this process is a feature map that highlights the presence of the detected features in the image.
• Specialized type of deep learning algorithm mainly designed for tasks that necessitate object recognization, including image classification, detection, and segmentation.
• Practical Scenarios: auto car, security camera systems etc..

Layer	Number of Nodes
Input layer	Must have one node for each component/feature of the input data
Hidden layers	Number of nodes is chosen (arbitrarily or empirically) by the network architect
Output layer	Must have one node for each of the possible output classes

Key Components of a CNN

The convolutional neural network is made of four main parts.

But how do CNNs Learn with those parts?

They help the CNNs mimic how the human brain operates to recognize patterns and features in images:

• Convolutional layers
• Rectified Linear Unit (ReLU for short)
• Pooling layers
• Fully connected layers

Do we have to manually find these weights?

In real life, the weights of the kernels are determined during the training process of the neural network.

Another name associated with the kernel in the literature is feature detector because the weights can be fine-tuned to detect specific features in the input image.

For instance:

• Averaging neighboring pixels kernel can be used to blur the input image.
• Subtracting neighboring kernel is used to perform edge detection.

The more convolution layers the network has, the better the layer is at detecting more abstract features.

Convolution layers

• the main mathematical task performed is called convolution
• application of a sliding window function to a matrix of pixels representing an image
• each filter is used to recognize a specific pattern from the image, such as the curving of the digits, the edges, the whole shape of the digits, and more.
• As it moves across the photo, it creates a new grid that highlights where it found these patterns.

Rectified Linear Unit (ReLU for short)

• ReLU activation function is applied after each convolution operation.
• helps network learn non-linear relationship, between features in the image
• helps to mitigate the vanishing gradient problems.

Pooling Layer

• pull the most significant features from the convoluted matrix.
• reduces dimensions and computation
• model is tolerant towards variations, distortions
• applying some aggregation operations, which reduce the dimension of the feature map (convoluted matrix), hence reducing the memory used while training the network
• mitigate overfitting.
• The most common aggregation functions that can be applied are:
  • Max pooling, which is the maximum value of the feature map
  • Sum pooling corresponds to the sum of all the values of the feature map
  • Average pooling is the average of all the values.

Flattening

• Flattening is like taking a 3D box of numbers (or a 2D grid, or any shape with multiple dimensions) and squishing it down into a 1D line of numbers.
• This makes it easier for the computer to understand the features when it's time to make a final decision, like deciding if an image shows a cat or a dog.

how weight is determined

The weights for the 1D line of numbers are determined during the training process of the neural network. The network starts with random weights, then it makes predictions based on these weights. The differences between the predictions and the actual values are calculated, and the weights are adjusted slightly to make the predictions more accurate. This process is repeated many times, and the weights are adjusted each time, until the predictions are as accurate as possible. This is called learning or training the model.

What computers “SEE”?

===========================================================

features : (important attributes or patterns in the data)

3x3 Kernel: A filter that scans 3x3 sections of the image.

Stride of 1: The filter moves 1 pixel at a time across the image.

Covers Every Pixel: No pixel is skipped during the scan.

Detailed Feature Map: Provides a more precise output but requires more computations.

U-NET

Introduction

• U-Net is a convolutional neural network designed for biomedical image segmentation.
• It relies on a contracting path to capture context and a symmetric expanding path for precise localization.
• It is built upon FCN(Fully Convolutional Networks) and modified in a way that it yields better segmentation in medical imaging.
• U-Net is trained end-to-end from few images and outperforms previous methods in segmenting neural structures.
• The network is fast, taking less than a second to segment a 512x512 image on a recent GPU.

Architecture

• U-Net consists of a contracting path (left side) and an expansive path (right side).
• The contracting path follows the typical architecture of a convolutional network, with repeated applications of 3x3 convolutions (unpadded).
• Each contraction stage is followed by a 2x2 max pooling operation with stride 2 for downsampling.
• The encoder block reduces dimensionality while increasing feature complexity, essential for capturing spatial hierarchies in images.
• Skip connections play a crucial role in preserving spatial information, allowing the decoder to utilize features from the encoder effectively, improving segmentation quality.
• Transposed convolutions in the decoder not only upscale features but also introduce learnable parameters, enhancing the model’s ability to reconstruct images.
• The expansive path consists of up-convolutions (also known as transposed convolutions) that upsample the feature maps, followed by a 2x2 convolution that reduces the number of feature maps.
• Concatenation with high-resolution features from the contracting path is performed at each up-convolution step.
• The final layer is a 1x1 convolution that maps the feature maps to the desired number of classes.

Fig. 1. U-net architecture (example for 32x32 pixels in the lowest resolution). Each blue box corresponds to a multi-channel feature map. The number of channels is denoted on top of the box. The x-y-size is provided at the lower left edge of the box. White boxes represent copied feature maps. The arrows denote the different operations

Data Augmentation

• Data augmentation is crucial for U-Net to perform well with few images.
• Techniques like rotation, scaling, flipping, and elastic deformations are used to increase the size of the training set.
• Data augmentation helps the network learn invariance to these deformations, allowing it to generalize better.

Training and Inference

• U-Net is trained using a weighted loss function, where the separating background labels between touching cells obtain a large weight.
• During inference, the network can predict the segmentation of arbitrarily large images using an overlap-tile strategy.
• Missing context in the border region is extrapolated by mirroring the input image.

Applications

• U-Net has shown strong results in segmenting neuronal structures in EM stacks and won the ISBI cell tracking challenge 2015 in transmitted light microscopy categories.
• It can be applied to various biomedical segmentation problems.

Tips for Implementing U-Net

• Use a deep learning framework like TensorFlow or PyTorch to implement U-Net.
• Prepare your dataset by annotating images and splitting them into training and validation sets.
• Apply data augmentation techniques to increase the size of your training set.
• Evaluate

Terminology

1. Backpropagation: A supervised learning algorithm used for training artificial neural networks, where the error is calculated and propagated backward through the network to update weights.
2. Convolutional Networks: A class of deep neural networks primarily used for processing structured grid data, such as images, by applying convolutional layers to extract features.
3. Semantic Segmentation: The process of classifying each pixel in an image into a category, allowing for detailed understanding of the image content.
4. Weight Map: A mapping used in training neural networks to assign different importance to various pixels, helping the model focus on specific areas, such as borders between objects.
5. Morphological Operations: Image processing techniques that process images based on their shapes, often used to extract or enhance features in binary images.
6. Cascaded Hierarchical Models: A framework in image segmentation that uses multiple layers of models to progressively refine the segmentation results.
7. Feature Map: The output of a convolutional layer in a neural network, representing the presence of specific features in the input data.
8. Intersection over Union (IoU): A metric used to evaluate the accuracy of an object detection model, calculated as the area of overlap between the predicted segmentation and the ground truth divided by the area of their union.
9. Segmentation Mask: A binary or multi-class image that indicates the regions of interest in an image, where different pixel values represent different classes or objects.
10. Pixel-wise Loss: A loss function that evaluates the prediction error for each pixel individually, often used in segmentation tasks to improve model accuracy.
11. Deep Learning: A subset of machine learning that uses neural networks with many layers (deep networks) to model complex patterns in large datasets.
12. Training Data Set: A collection of data used to train a machine learning model, consisting of input-output pairs that the model learns from.
13. Benchmark: A standard or point of reference against which things may be compared or assessed, often used in evaluating the performance of algorithms.
14. Ground Truth: The accurate and true data used as a reference for evaluating the performance of a model, often obtained through expert annotation.
15. Convolutional Layer: A layer in a convolutional neural network that applies convolution operations to the input data, extracting features through learned filters.
16. Hyperparameters: Configurable parameters in machine learning models that are set before the training process begins, influencing the model's performance.
17. Data Augmentation: Techniques used to artificially expand the size of a training dataset by applying transformations such as rotation, scaling, and flipping to the original data.
18. Overfitting: A modeling error that occurs when a machine learning model learns the training data too well, capturing noise and outliers, which negatively impacts its performance on unseen data.
19. Underfitting: A situation where a model is too simple to capture the underlying patterns in the data, resulting in poor performance on both training and test datasets.
20. Transfer Learning: A machine learning technique where a model developed for a specific task is reused as the starting point for a model on a second task, often improving training efficiency and performance.
21. Batch Normalization: A technique used to improve the training of deep neural networks by normalizing the inputs of each layer, which helps stabilize learning and accelerate convergence.
22. Activation Function: A mathematical function applied to the output of a neuron in a neural network, introducing non-linearity and enabling the network to learn complex patterns.
23. Pooling Layer: A layer in a convolutional neural network that reduces the spatial dimensions of the input, helping to decrease computation and control overfitting.
24. Learning Rate: A hyperparameter that determines the step size at each iteration while moving toward a minimum of the loss function during training.
25. Epoch: One complete pass through the entire training dataset during the training process of a machine learning model.
26. Precision and Recall: Metrics used to evaluate the performance of a classification model; precision measures the accuracy of positive predictions, while recall measures the ability to find all relevant instances.
27. F1 Score: A metric that combines precision and recall into a single score, providing a balance between the two, particularly useful for imbalanced datasets.
28. Confusion Matrix: A table used to evaluate the performance of a classification model by summarizing the true positives, true negatives, false positives, and false negatives.
29. Regularization: Techniques used to prevent overfitting by adding a penalty to the loss function, such as L1 (Lasso) or L2 (Ridge) regularization.
30. Gradient Descent: An optimization algorithm used to minimize the loss function by iteratively adjusting the model parameters in the direction of the steepest descent of the loss.
31. Dropout: A regularization technique in neural networks where randomly selected neurons are ignored during training, helping to prevent overfitting.
32. Learning Curve: A graphical representation of the model's performance on the training and validation datasets over time, indicating how well the model is learning.
33. Ensemble Learning: A technique that combines multiple models to improve overall performance, often leading to better accuracy than individual models.
34. Autoencoder: A type of neural network used for unsupervised learning that aims to learn a compressed representation of the input data by encoding and then decoding it.
35. Latent Space: A representation of compressed data in a lower-dimensional space, often used in generative models to capture the underlying structure of the data.
36. Semantic Segmentation: The process of classifying each pixel in an image into a category, allowing for detailed understanding of the image content.

Pytorch

To train the image classifier with PyTorch, you need to complete the following steps:

1. Load the data. If you've done the previous step of this tutorial, you've handled this already.
2. Define a Convolution Neural Network.
3. Define a loss function.
4. Train the model on the training data.
5. Test the network on the test data.

===========================================================

• semantic segmentation
• instance segmentation

Instance segmentation can essentially be solved in 2 steps:

1. Perform a version of object detection to draw bounding boxes around each instance of a class
2. Perform semantic segmentation on each of the bounding boxes

For background removal in images, you'd typically use semantic segmentation or instance segmentation techniques rather than R-CNN. Here are some approaches you could consider:

1. U-Net: A popular architecture for image segmentation tasks.
2. DeepLab: A state-of-the-art semantic segmentation model.
3. Mask R-CNN: An extension of Faster R-CNN that can perform instance segmentation.

What are Atrous (Dilated) Convolutions?

• Atrous (dilated) convolutions are similar to regular convolutional filters, but with one key difference: they introduce gaps (or dilation) between the filter elements.
• Filter (Kernel): In a normal convolution, the filter (e.g., 3x3) moves across the image pixel by pixel, performing a dot product.
• Dilated/Atrous Convolutions: Instead of the filter looking at consecutive pixels, it "dilates" the filter by skipping pixels. This allows the filter to cover a larger area of the image without increasing the number of parameters.

Why Use Atrous Convolutions?

• They help the model to capture a larger context or view of the image without increasing the size of the filter or reducing the spatial resolution of the image.
• This makes them useful in tasks like semantic segmentation and image matting, where you need to capture fine details (small features) and broader context (large features) at the same time.

In the context of MODNet, the e-ASPP module uses atrous convolutions to extract features at multiple scales, which is important for capturing both fine and large-scale details efficiently.

Region Based CNN

Object detection + semantic segmentation = instance Segmentation

Seeds allow you to create a starting point for randomly generated numbers, so that each time your code is run the same answer is generated. The advantage of doing this in your sampling is that you or anyone else can recreate the exact same training and test sets by using the same seed.