Merge branch 'develop' into brax-v1

adaptive-intelligent-robotics · Oct 9, 2023 · 2e8825b · 2e8825b
2 parents 545d351 + f791159
commit 2e8825b
Show file tree

Hide file tree

Showing 36 changed files with 1,559 additions and 1,236 deletions.
diff --git a/README.md b/README.md
@@ -29,7 +29,7 @@ pip install git+https://github.com/adaptive-intelligent-robotics/QDax.git@main
 ```
 Installing QDax via ```pip``` installs a CPU-only version of JAX by default. To use QDax with NVidia GPUs, you must first install [CUDA, CuDNN, and JAX with GPU support](https://github.com/google/jax#installation).
 
-However, we also provide and recommend using either Docker, Singularity or conda environments to use the repository which by default provides GPU support. Detailed steps to do so are available in the [documentation](https://qdax.readthedocs.io/en/latest/installation/).
+However, we also provide and recommend using either Docker or conda environments to use the repository which by default provides GPU support. Detailed steps to do so are available in the [documentation](https://qdax.readthedocs.io/en/latest/installation/).
 
 ## Basic API Usage
 For a full and interactive example to see how QDax works, we recommend starting with the tutorial-style [Colab notebook](./examples/mapelites.ipynb). It is an example of the MAP-Elites algorithm used to evolve a population of controllers on a chosen Brax environment (Walker by default).
@@ -134,6 +134,7 @@ QDax currently supports the following algorithms:
 | [Multi-Objective MAP-Elites (MOME)](https://arxiv.org/abs/2202.03057)                                                         | [![Open All Collab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/adaptive-intelligent-robotics/QDax/blob/main/examples/mome.ipynb)       |
 | [MAP-Elites Evolution Strategies (MEES)](https://dl.acm.org/doi/pdf/10.1145/3377930.3390217)                                  | [![Open All Collab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/adaptive-intelligent-robotics/QDax/blob/main/examples/mees.ipynb)       |
 | [MAP-Elites PBT (ME-PBT)](https://openreview.net/forum?id=CBfYffLqWqb)                                                        | [![Open All Collab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/adaptive-intelligent-robotics/QDax/blob/main/examples/me_sac_pbt.ipynb) |
+| [MAP-Elites Low-Spread (ME-LS)](https://dl.acm.org/doi/abs/10.1145/3583131.3590433)                                           | [![Open All Collab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/adaptive-intelligent-robotics/QDax/blob/main/examples/me_ls.ipynb)      |
 
 
 

diff --git a/docs/api_documentation/core/mels.md b/docs/api_documentation/core/mels.md
@@ -0,0 +1,7 @@
+# MAP-Elites Low-Spread (ME-LS)
+
+[ME-LS](https://dl.acm.org/doi/abs/10.1145/3583131.3590433) is a variant of
+MAP-Elites that thrives the search process towards solutions that are consistent
+in the behavior space for uncertain domains.
+
+::: qdax.core.mels.MELS
diff --git a/docs/installation.md b/docs/installation.md
@@ -56,59 +56,6 @@ sudo docker run --rm -it -v $QDAX_PATH:/app instadeep/qdax:$USER /bin/bash
     sudo docker run --rm -it --gpus '"device=0,1"' -v $QDAX_PATH:/app instadeep/qdax:$USER /bin/bash
     ```
 
-
-
-### Using singularity
-
-First, follow these initial steps:
-
-1. If it is not already done, install Singularity, following [these instructions](https://docs.sylabs.io/guides/3.0/user-guide/installation.html).
-
-2. Clone `qdax`
-```zsh
-git clone git@github.com:adaptive-intelligent-robotics/QDax.git
-```
-
-3. Enter the singularity folder
-```zsh
-cd qdax/singularity/
-```
-
-You can build two distinct types of images with singularity: "final images" or "sandbox images".
-A final image is a single file with the `.sif` extension, it is immutable.
-On the contrary, a sandbox image is not a file but a folder, it allows you to develop inside the singularity container to test your code while writing it.
-
-To build a final image, execute the `build_final_image` script:
-```zsh
-./build_final_image
-```
-It will generate a `.sif` file: `[image_name].sif`. If you execute this file using singularity, as follows, it will run the default application of the image, defined in the `singularity.def` file that you can find in the `singularity` folder as well. At the moment, this is just running the MAP-Elites algorithm on a simple task.
-```zsh
-singularity run --nv [image_name].sif
-```
-
-!!! warning "Using GPU"
-    The `--nv` flag of the `singularity run` command allows the container to use the GPU, it is thus important to use it for QDax.
-
-
-To build a sandbox image, execute the `start_container` script:
-```zsh
-./start_container -n
-```
-
-!!! warning "Using GPU"
-    The `-n` flag of the `start_container` command allow the container to use the GPU, it is thus important to use it for QDax.
-
-This command will generate a sandbox container `qdax.sif/` and enter it. If you execute this command again later, it will not generate a new container but enter directly the existing one.
-Once inside the sandbox container, enter the qdax development folder:
-```zsh
-cd /git/exp/qdax
-```
-This folder is linked with the `qdax` folder on your machine, meaning that any modification inside the container will directly modify the files on your machine. You can now use this development environment to develop your own QDax-based code.
-
-
-
-
 ### Using conda
 
 1. If it is not already done, install conda from [here](https://docs.conda.io/projects/conda/en/latest/user-guide/install/linux.html)

diff --git a/examples/dads.ipynb b/examples/dads.ipynb
@@ -122,7 +122,8 @@
     "alpha_init = 1.0 #@param {type:\"number\"}\n",
     "discount = 0.97 #@param {type:\"number\"}\n",
     "reward_scaling = 1.0 #@param {type:\"number\"}\n",
-    "hidden_layer_sizes = (256, 256) #@param {type:\"raw\"}\n",
+    "critic_hidden_layer_size = (256, 256) #@param {type:\"raw\"}\n",
+    "policy_hidden_layer_size = (256, 256) #@param {type:\"raw\"}\n",
     "fix_alpha = False #@param {type:\"boolean\"}\n",
     "normalize_observations = False #@param {type:\"boolean\"}\n",
     "# DADS config\n",
@@ -208,7 +209,8 @@
     "    alpha_init=alpha_init,\n",
     "    discount=discount,\n",
     "    reward_scaling=reward_scaling,\n",
-    "    hidden_layer_sizes=hidden_layer_sizes,\n",
+    "    critic_hidden_layer_size=critic_hidden_layer_size,\n",
+    "    policy_hidden_layer_size=policy_hidden_layer_size,\n",
     "    fix_alpha=fix_alpha,\n",
     "    # DADS config\n",
     "    num_skills=num_skills,\n",
@@ -526,11 +528,8 @@
   }
  ],
  "metadata": {
-  "interpreter": {
-   "hash": "9ae46cf6a59eb5e192bc4f27fbb5c33d8a30eb9acb43edbb510eeaf7c819ab64"
-  },
   "kernelspec": {
-   "display_name": "Python 3 (ipykernel)",
+   "display_name": "Python 3.9.2 64-bit ('3.9.2')",
    "language": "python",
    "name": "python3"
   },
@@ -544,7 +543,12 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.8.13"
+   "version": "3.9.2"
+  },
+  "vscode": {
+   "interpreter": {
+    "hash": "1508da3da994e8b4133db52fbfc99c200ce19b8717cb4612fe84174533968534"
+   }
   }
  },
  "nbformat": 4,

diff --git a/examples/diayn.ipynb b/examples/diayn.ipynb
@@ -122,7 +122,8 @@
     "alpha_init = 1.0 #@param {type:\"number\"}\n",
     "discount = 0.97 #@param {type:\"number\"}\n",
     "reward_scaling = 1.0 #@param {type:\"number\"}\n",
-    "hidden_layer_sizes = (256, 256) #@param {type:\"raw\"}\n",
+    "critic_hidden_layer_size = (256, 256) #@param {type:\"raw\"}\n",
+    "policy_hidden_layer_size = (256, 256) #@param {type:\"raw\"}\n",
     "fix_alpha = False #@param {type:\"boolean\"}\n",
     "normalize_observations = False #@param {type:\"boolean\"}\n",
     "# DIAYN config\n",
@@ -206,7 +207,8 @@
     "    alpha_init=alpha_init,\n",
     "    discount=discount,\n",
     "    reward_scaling=reward_scaling,\n",
-    "    hidden_layer_sizes=hidden_layer_sizes,\n",
+    "    critic_hidden_layer_size=critic_hidden_layer_size,\n",
+    "    policy_hidden_layer_size=policy_hidden_layer_size,\n",
     "    fix_alpha=fix_alpha,\n",
     "    # DIAYN config\n",
     "    num_skills=num_skills,\n",
@@ -516,11 +518,8 @@
   }
  ],
  "metadata": {
-  "interpreter": {
-   "hash": "9ae46cf6a59eb5e192bc4f27fbb5c33d8a30eb9acb43edbb510eeaf7c819ab64"
-  },
   "kernelspec": {
-   "display_name": "Python 3 (ipykernel)",
+   "display_name": "Python 3.9.2 64-bit ('3.9.2')",
    "language": "python",
    "name": "python3"
   },
@@ -534,7 +533,12 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.8.13"
+   "version": "3.9.2"
+  },
+  "vscode": {
+   "interpreter": {
+    "hash": "1508da3da994e8b4133db52fbfc99c200ce19b8717cb4612fe84174533968534"
+   }
   }
  },
  "nbformat": 4,

diff --git a/examples/me_sac_pbt.ipynb b/examples/me_sac_pbt.ipynb
@@ -63,7 +63,8 @@
     "episode_length = 1000\n",
     "tau = 0.005\n",
     "alpha_init = 1.0\n",
-    "hidden_layer_sizes = (256, 256)\n",
+    "critic_hidden_layer_size = (256, 256) \n",
+    "policy_hidden_layer_size = (256, 256) \n",
     "fix_alpha = False\n",
     "normalize_observations = False\n",
     "\n",
@@ -148,7 +149,8 @@
     "    tau=tau,\n",
     "    normalize_observations=normalize_observations,\n",
     "    alpha_init=alpha_init,\n",
-    "    hidden_layer_sizes=hidden_layer_sizes,\n",
+    "    critic_hidden_layer_size=critic_hidden_layer_size,\n",
+    "    policy_hidden_layer_size=policy_hidden_layer_size,\n",
     "    fix_alpha=fix_alpha,\n",
     ")\n",
     "\n",
@@ -527,11 +529,8 @@
   }
  ],
  "metadata": {
-  "interpreter": {
-   "hash": "9ae46cf6a59eb5e192bc4f27fbb5c33d8a30eb9acb43edbb510eeaf7c819ab64"
-  },
   "kernelspec": {
-   "display_name": "Python 3 (ipykernel)",
+   "display_name": "Python 3.9.2 64-bit ('3.9.2')",
    "language": "python",
    "name": "python3"
   },
@@ -545,7 +544,12 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.8.16"
+   "version": "3.9.2"
+  },
+  "vscode": {
+   "interpreter": {
+    "hash": "1508da3da994e8b4133db52fbfc99c200ce19b8717cb4612fe84174533968534"
+   }
   }
  },
  "nbformat": 4,