RL-OptS

Reinforcement Learning of Optimal Search strategies

This library builds the necessary tools needed to study, replicate and develop the results of the paper: “Optimal foraging strategies can be learned and outperform Lévy walks” by G. Muñoz-Gil, A. López-Incera, L. J. Fiderer and H. J. Briegel.

Installation

You can access all these tools installing the python package rl_opts via Pypi:

pip install rl-opts

You can also opt for cloning the source repository and executing the following on the parent folder you just cloned the repo:

pip install -e rl_opts

This will install both the library and the necessary packages.

Tutorials

We have prepared a series of tutorials to guide you through the most important functionalities of the package. You can find them in the Tutorials folder of the Github repository or in the Tutorials tab of our webpage, with notebooks that will help you navigate the package as well as reproducing the results of our paper via minimal examples. In particular, we have three tutorials:

• Reinforcement learning : shows how to train a RL agent based on Projective Simulation agents to search targets in randomly distributed environments as the ones considered in our paper.
• Imitation learning : shows how to train a RL agent to imitate the policy of an expert equipped with a pre-trained policy. The latter is based on the benchmark strategies common in the literature.
• Benchmarks : shows how launch various benchmark strategies with which to compare the trained RL agents.

Package structure

The package contains a set of modules for:

• Reinforcement learning framework (rl_opts.rl_framework) : building foraging environments as well as the RL agents moving on them.
• Learning and benchmarking (rl_opts.learn_and_bench) : training RL agents as well as benchmarking them w.r.t. to known foraging strategies.
• Imitation learning (rl_opts.imitation): training RL agents in imitation schemes via foraging experts.
• Analytical functions (rl_opts.analytics): builiding analytical functions for step length distributions as well as tranforming these to foraging policies.
• Utils (rl_opts.utils): helpers used throughout the package.

Cite

We kindly ask you to cite our paper if any of the previous material was useful for your work, here is the bibtex info:

@article{munoz2023optimal,
  doi = {10.48550/ARXIV.2303.06050},  
  url = {https://arxiv.org/abs/2303.06050},  
  author = {Muñoz-Gil, Gorka and López-Incera, Andrea and Fiderer, Lukas J. and Briegel, Hans J.},  
  title = {Optimal foraging strategies can be learned and outperform Lévy walks},  
  publisher = {arXiv},  
  archivePrefix = {arXiv},
  eprint = {2303.06050},
  primaryClass = {cond-mat.stat-mech},  
  year = {2023},
}