Genetic Algorithms
Heuristic search algorithm and optimization algorithm. Inspired by Charles Darwin's theory, natural evolution.
Commonly used in :
- Feature selection: To find the best subset of features for a given machine learning model.
- Hyperparameter optimization: To find the best hyperparameters for a machine learning model.
- Neural network training: To train neural networks with complex architectures.
- Clustering: To divide a data set into different groups based on similarities.
- Pattern recognition: To identify patterns in data.
- Optimization problems: To find the global optimum for problems such as the traveling salesman problem or the knapsack problem.
- Scheduling problems: To find the best schedule for tasks such as resource allocation or job scheduling.
How does it work:
A genetic algorithm always start with a population, and then goes on a process where the fittest will be selected, produced offspring which become the new generation until convergence. It relies on 5 main phase :
- Initial population
- Fitness function
- Selection process
- Crossover
- Mutation
Workflow:
Initial population:
Generate initial population that consist of all the probable solutions. Binary strings are often used here.
Fitness: Score computing, used to determine the fittest individual at each iteration.
Selection: Select best individual based on fitness score.
Crossover: Two individual are combined to produce an offspring, allowing for exchange of genetic information.
Mutation: Small random changes are sometimes introduced to individuals during crossover to explore new solutions.
Algorithms stop either when number of max generation as been reached or if convergence has been reached (convergence is when after multiple crossover/mutation individual are identical).
Source:
https://en.wikipedia.org/wiki/Genetic_algorithm
https://www.section.io/engineering-education/the-basics-of-genetic-algorithms-in-ml/