This repo consists of multiple implementations of neural agents that work with some of the OpenAI Gym classical control environments. The agents have primarily been developed for use with the NES training algorithm (see below).
Each agent has the following methods:
set_weights()
get_action()- The CartPole problem is a classical underactuated control example. The task is to balance a pole connected to a cart such that it stays upright. The system is underactuated since we cannot manipulate the pole angle directly, but instead we must move the cart in order to effect the pendulum.
- The MountainCar problem is a task where we have an underactuated car on a hill and would like the car to get over the hill. For this we need to build up momentum.
- Pendulum
- Acrobot
Resources
Install: pip install -r requirements.txt
Tested with Python 3.8.2
Natural Evolution Strategies (NES) is an evolutionary inspired optimization algorithm that can be used for adjusting the parameters of an ANN. It works by creating a fixed standard deviation Gaussian distribution around some current parameter set, and testing the fitness of a set of parameters within this distribution. The current parameters are updated each iteration. The optimization process is similar to hill climbing, but uses instead a population of agents where each agent represents a single parameter vector in the set.
- A Visual Guide to Evolution Strategies
- Evolving Stable Strategies
- OpenAI: Evolution Strategies as a Scalable Alternative to Reinforcement Learning
- numpy (no framework)
- alirezamika/evostra
- pyTorch
- uber-research/EvoGrad (uses pyTorch)
The EvoGrad package uses PyTorch to create networks and performs NES to optimize the networks parameters/weights.
See
Standalone install:
pip install evograd