Add scaling section to A2C documentation (#1250)

* add scaling section to A2C documentation * add cross-reference to vectorized envs article * turn it as note * update changelog * add Bonifatius94 to the list of contributors * fix issue number --------- Co-authored-by: Quentin Gallouédec <45557362+qgallouedec@users.noreply.github.com> Co-authored-by: Quentin GALLOUÉDEC <gallouedec.quentin@gmail.com> Co-authored-by: Antonin RAFFIN <antonin.raffin@ensta.org>
DLR-RM · Feb 2, 2023 · d0c1a87 · d0c1a87
1 parent 82bc63f
commit d0c1a87
Show file tree

Hide file tree

Showing 2 changed files with 20 additions and 1 deletion.
diff --git a/docs/misc/changelog.rst b/docs/misc/changelog.rst
@@ -39,6 +39,7 @@ Others:
 Documentation:
 ^^^^^^^^^^^^^^
 - Renamed ``load_parameters`` to ``set_parameters`` (@DavyMorgan)
+- Clarified documentation about subproc multiprocessing for A2C (@Bonifatius94)
 - Fixed typo in ``A2C`` docstring (@AlexPasqua)
 
 
@@ -1222,4 +1223,4 @@ And all the contributors:
 @Gregwar @ycheng517 @quantitative-technologies @bcollazo @git-thor @TibiGG @cool-RR @MWeltevrede
 @Melanol @qgallouedec @francescoluciano @jlp-ue @burakdmb @timothe-chaumont @honglu2875 @yuanmingqi
 @anand-bala @hughperkins @sidney-tio @AlexPasqua @dominicgkerr @Akhilez @Rocamonde @tobirohrer @ZikangXiong
-@DavyMorgan @luizapozzobon
+@DavyMorgan @luizapozzobon @Bonifatius94
diff --git a/docs/modules/a2c.rst b/docs/modules/a2c.rst
@@ -76,6 +76,24 @@ Train a A2C agent on ``CartPole-v1`` using 4 environments.
       env.render()
 
 
+.. note::
+
+  A2C is meant to be run primarily on the CPU, especially when you are not using a CNN. To improve CPU utilization, try turning off the GPU and using ``SubprocVecEnv`` instead of the default ``DummyVecEnv``:
+
+  .. code-block::
+
+    from stable_baselines3 import A2C
+    from stable_baselines3.common.env_util import make_vec_env
+    from stable_baselines3.common.vec_env import SubprocVecEnv
+
+    if __name__=="__main__":
+        env = make_vec_env("CartPole-v1", n_envs=8, vec_env_cls=SubprocVecEnv)
+        model = A2C("MlpPolicy", env, device="cpu")
+        model.learn(total_timesteps=25_000)
+  
+  For more information, see :ref:`Vectorized Environments <vec_env>`, `Issue #1245 <https://github.com/DLR-RM/stable-baselines3/issues/1245>`_ or the `Multiprocessing notebook <https://colab.research.google.com/github/Stable-Baselines-Team/rl-colab-notebooks/blob/sb3/multiprocessing_rl.ipynb>`_.
+
+
 Results
 -------