Stochastic Algorithms
Stochastic algorithms are optimization methods that incorporate randomness or probabilistic elements to explore the solution space.
These algorithms are particularly useful in scenarios where the search space is large or poorly understood.
How can you locate the point with the highest temp?
Easy? Let look at this "Terminator" example ~~~
Classic Methods
Brute Force
Greedy Algorithm
Dynamic Programming
Stochastic Algorithms
Explores the solution space randomly
- Monte Carlo Methods: Uses randomness to solve problems that might be deterministic in nature.
- Stochastic Gradient Descent: Iteratively optimizes by updating parameters using random subsets of data.
Probability-Based Methods:
- Bayesian Optimization: Utilizes probabilistic models to guide the search for optimal solutions, particularly in expensive function evaluations.
- Cross-Entropy Method: Combines principles from statistics and optimization to refine a distribution of candidate solutions.
Inspired by Physical Processes
- Simulated Annealing (SA): Used to find approximate global optima of given functions
Inspired Evolutionary
- Genetic Algorithms (GA): Mimics natural selection.
- Evolutionary Strategies (ES): Focuses on self-adaptation of strategy parameters.
- Differential Evolution (DE): Focuses on generating mutant and trial vectors.
Inspired Biological Processes
- Particle Swarm Optimization (PSO): Inspired by social behavior of birds or fish.
- Ant Colony Optimization (ACO): Based on the foraging behavior of ants.