Context • Documentation • Usage • Getting started • Tools • Related • Acknowledgments • Gallery
In this project we develop a reinforcement learning algorithm based on parallel Monte-Carlo tree search to tackle the problem of long-term path planning under uncertainty for offshore sailing. This domain of application is challenging as it combines unreliable and time varying wind conditions with complex and uncertain boat performances. Contrarily to state of the art approaches applied to the sailing problem, we build a generator that models state transitions considering these two types of uncertainty. The first one is on the boat dynamics : given a environment state the boat performances are not deterministic. And the second one is uncertain weather forecasts. In practice, the wind is estimated from multiple weather forecasts (each one of them being a weather scenario with a given probability of happening). The boat’s dynamics are evaluated with a noisy Velocity Prediction Program (VPP). Then, a Monte Carlo Tree Search (MCTS) algorithm is applied in parallel to all the weather scenarios to find the sequence of headings that minimizes the travel time between two points.
The documentation as well as the prerequisites can be found on the following webpage :
This repository is intended to be a tool for future work on the problem of long-term path planning under uncertainty for offshore sailing. The tools that are provided are the following :
- Processing of real weather forecast data
- A Parallel Monte-Carlo Tree Search algorithm
- A Isochrones algorithm as reference and performance validation
- Results visualisation and decision support
Before doing anything you should have a look at the Jupyter notebook tutorial Tutorial.ipynb and its .py version Tutorial.py.
This project falls under the IBOAT project and is related to the repo:
It tackles the problem of end-to-end control for stall avoidance using a Deep Q-Network.
This project has been carried out with the help of:
- Yves Brière - Professor of automatic at ISAE-Supaero.
- Emmanuel Rachelson - Professor in reinforcement learning at ISAE-Supaero.
- Valentin Guillet - ISAE-Supaero student, for advices on multiprocessing.
- Tristan Karch - ISAE-Supaero student, for advices on sweet README layouts.
- Paul Barde - Initiator of the project - Developement of the weather tools and simulators, proof of concept on non-parallel MCTS. Implementation of the parallel version of MCTS, results visualisation and documentation.
- Fabien Brulport - Implementation of the parallel version of MCTS, results visualisation and documentation.
- Jean-Michel Bellay - Implementation of the isochrones method.
