Update doc: Top-down renderer (#587)

* pull asset

* pull at post create envrionment

* pull at post create envrionment

* c

* new fix

* add vehicle to section

* top_down renderer no kwargs

* add top-down renderer example

* format

* defaula show crosswalk

* format

.readthedocs.yaml

@@ -9,6 +9,9 @@ build:
   os: ubuntu-22.04
   tools:
     python: "3.11"
+  jobs:
+    post_install: 
+      - python -m metadrive.pull_asset
     # You can also specify other tool versions:
     # nodejs: "20"
     # rust: "1.70"

documentation/source/action.ipynb

@@ -581,9 +581,10 @@
     "from metadrive.envs.metadrive_env import MetaDriveEnv\n",
     "from metadrive.policy.env_input_policy import ExtraEnvInputPolicy\n",
     "import gymnasium as gym\n",
-    "ExtraEnvInputPolicy.set_extra_input_space(gym.spaces.Discrete(10))\n",
     "import random\n",
     "\n",
+    "ExtraEnvInputPolicy.set_extra_input_space(gym.spaces.Discrete(10))\n",
+    "\n",
     "env=MetaDriveEnv(dict(map=\"S\",\n",
     "                      agent_policy=ExtraEnvInputPolicy,\n",
     "                      action_check=True,\n",
@@ -1005,7 +1006,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.9.17"
+   "version": "3.7.13"
   },
   "mystnb": {
    "execution_mode": "auto"

documentation/source/index.rst

@@ -54,14 +54,14 @@ Please feel free to contact us if you have any suggestions or ideas!
    :caption: Concepts and Customization
 
    system_design.ipynb
-   development.rst
    top_down_render.ipynb
-   policy.rst
+   panda_render.ipynb
+   vehicle.ipynb
    navigation.ipynb
    sensors.ipynb
-   gui.ipynb
    description.ipynb
    record_replay.ipynb
+   development.rst
 
 .. toctree::
    :hidden:

documentation/source/gui.ipynb → documentation/source/panda_render.ipynb

@@ -5,7 +5,7 @@
    "id": "3a2965f7",
    "metadata": {},
    "source": [
-    "# GUI"
+    "# 3D Renderer"
    ]
   }
  ],

documentation/source/system_design.ipynb

@@ -645,7 +645,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.9.17"
+   "version": "3.7.13"
   },
   "mystnb": {
    "execution_mode": "auto"

documentation/source/top_down_render.ipynb

@@ -6,7 +6,246 @@
    "metadata": {},
    "source": [
     "# Top-down Renderer\n",
-    "<a id='top-down-renderer'></a>"
+    "\n",
+    "2D Top-down renderer is widely used in this documentation for rendering the results, as it is lightweight and can run on any platforms without GPU requirement. If your system has poor support for OpenGL like Apple M1/M2 chips, a good choice is to use top-down renderer. And the simulation results are exactly the same using either 3D renderer or top-down renderer.\n",
+    "\n",
+    "## Lifetime\n",
+    "You are free to launch this renderer at any timestep by calling `env.render(mode=\"topdown\")`. The renderer will be created and work until the `env.reset()` is called. It will shutdown the `top_down_renderer` and destroy it. Thus the lifetime of a renderer is the period between calling `env.render` for the first time and executing next `env.reset`. \n",
+    "\n",
+    "The following example running an environment for 100 steps. It launches the renderer when episode_step=50, and thus the generated gif only records the last 50 frames. Also, it demonstrate how to record screen and generate gif."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "b0c054bf-510e-4689-8c7c-b102a374f6bd",
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [],
+   "source": [
+    "from metadrive.envs import MetaDriveEnv\n",
+    "from IPython.display import Image\n",
+    "from metadrive.utils import print_source, get_source\n",
+    "import cv2"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "294b0ed1-a7d8-44f9-b20e-2b5671adeec6",
+   "metadata": {
+    "tags": []
+   },
+   "outputs": [],
+   "source": [
+    "env = MetaDriveEnv(dict(log_level=50))\n",
+    "env.reset()\n",
+    "for i in range(100):\n",
+    "    env.step([0,0])\n",
+    "    if i>=50:\n",
+    "        env.render(mode=\"topdown\",\n",
+    "                   window=False,\n",
+    "                   screen_size=(400, 200),\n",
+    "                   screen_record=True,\n",
+    "                   text={\"Step\": i})\n",
+    "env.top_down_renderer.generate_gif()\n",
+    "print(\"Before reset the renderer is\", env.top_down_renderer)\n",
+    "env.reset()\n",
+    "print(\"After reset the renderer is\", env.top_down_renderer)\n",
+    "\n",
+    "env.close()\n",
+    "\n",
+    "Image(open(\"demo.gif\", 'rb').read())"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "573d0363-38b2-4538-850f-0e90e708de47",
+   "metadata": {},
+   "source": [
+    "## Configs\n",
+    "The `env.render()` accepts parameters like `screen_size`, `window` and so on as input which defines the behavior of the top-down renderer. **Note that these parameters only take effect when you call `env.render` for the first time in one episode.** \n",
+    "\n",
+    "All accepted arguments for creating the top-down renderer are as follows."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "a9a890f4",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from metadrive.obs.top_down_renderer import TopDownRenderer\n",
+    "from metadrive.utils import CONFIG, FUNC_2\n",
+    "print_source(TopDownRenderer.__init__, [\"def\", \"# doc-end\"], colorscheme=FUNC_2)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "48d017d4",
+   "metadata": {},
+   "source": [
+    "## Region Size in Screen\n",
+    "\n",
+    "If you wanna adjust the region size shown on the screen/window, change `scaling` to a reasonable value. The region size in meter is determined by `screen_size[0]/scaling` and `screen_size[1]/scaling`. For example, if your screen size is (1200, 800) and scaling is 5, then it draws a 240m x 160m region. \n",
+    "\n",
+    "To demonstrate this, The following example draws exactly the same region with different `screen_size` and `scaling`."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "9937fa20",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "env = MetaDriveEnv(dict(log_level=50, num_scenarios=1, map=\"X\"))\n",
+    "\n",
+    "env.reset()\n",
+    "frame_1 = env.render(mode=\"topdown\", window=False, camera_position=(50, 7.5),\n",
+    "       screen_size=(400, 200), scaling=4)\n",
+    "\n",
+    "env.reset()\n",
+    "frame_2 = env.render(mode=\"topdown\", window=False, camera_position=(50, 7.5),\n",
+    "       screen_size=(200, 100), scaling=2)\n",
+    "\n",
+    "env.reset()\n",
+    "frame_3 = env.render(mode=\"topdown\", window=False, camera_position=(50, 7.5),\n",
+    "       screen_size=(100, 50), scaling=1)\n",
+    "\n",
+    "env.reset()\n",
+    "frame_4 = env.render(mode=\"topdown\", window=False, camera_position=(50, 7.5),\n",
+    "   screen_size=(200, 100), scaling=1)\n",
+    "\n",
+    "env.close()\n",
+    "\n",
+    "import matplotlib.pyplot as plt\n",
+    "fig, axes = plt.subplots(2, 2, figsize=(10, 5)) # You can adjust the figsize as needed\n",
+    "axes[0][0].imshow(frame_1)\n",
+    "axes[0][0].axis('off')  # Turn off axis\n",
+    "axes[0][0].set_title(\"screen_size=(400, 200), scaling=4\")\n",
+    "axes[0][1].imshow(frame_2)\n",
+    "axes[0][1].axis('off')  # Turn off axis\n",
+    "axes[0][1].set_title(\"screen_size=(200, 100), scaling=2\")\n",
+    "axes[1][0].imshow(frame_3)\n",
+    "axes[1][0].axis('off')  # Turn off axis\n",
+    "axes[1][0].set_title(\"screen_size=(100, 50), scaling=1\")\n",
+    "axes[1][1].imshow(frame_4)\n",
+    "axes[1][1].axis('off')  # Turn off axis\n",
+    "axes[1][1].set_title(\"screen_size=(200, 100), scaling=1\")\n",
+    "plt.subplots_adjust(wspace=0.05)\n",
+    "plt.show()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "9e91883c",
+   "metadata": {},
+   "source": [
+    "## Map Region Size\n",
+    "The Map region size is determined by `film_size` and `scaling` like how to determine the region shown in window. Users have to make sure the map region size exceeds the actual map size to make sure the map is shown complete. Usually, maps in MetaDrive are smaller than 400m x 400m. Thus the default `film_size=(2000, 2000)` and `scaling=5` are able to handle most cases. \n",
+    "\n",
+    "If you find the map in top-down rendering is incomplete, consider increase the `film_size` or decrease the `scaling`. The following example shows what will happen if the film_size is too small."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "bc696d47",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "env = MetaDriveEnv(dict(log_level=50, num_scenarios=1, map=\"X\"))\n",
+    "\n",
+    "env.reset()\n",
+    "frame_1 = env.render(mode=\"topdown\", window=False, camera_position=(50, 7.5),\n",
+    "       screen_size=(800, 400), scaling=4, film_size=(200, 200))\n",
+    "map_1 = env.top_down_renderer.get_map()\n",
+    "\n",
+    "env.reset()\n",
+    "frame_2 = env.render(mode=\"topdown\", window=False, camera_position=(50, 7.5),\n",
+    "       screen_size=(800, 400), scaling=4, film_size=(400, 400))\n",
+    "map_2 = env.top_down_renderer.get_map()\n",
+    "\n",
+    "env.close()\n",
+    "\n",
+    "import matplotlib.pyplot as plt\n",
+    "fig, axes = plt.subplots(2, 2, figsize=(10, 5))\n",
+    "axes[0][0].imshow(map_1)\n",
+    "axes[0][0].axis('off')  # Turn off axis\n",
+    "axes[0][0].set_title(\"Map region 50m x 50m\")\n",
+    "axes[0][1].imshow(map_2)\n",
+    "axes[0][1].axis('off')  # Turn off axis\n",
+    "axes[0][1].set_title(\"Map region 100m x 100m\")\n",
+    "\n",
+    "axes[1][0].imshow(frame_1)\n",
+    "axes[1][0].axis('off')  # Turn off axis\n",
+    "axes[1][0].set_title(\"Rendering result\")\n",
+    "axes[1][1].imshow(frame_2)\n",
+    "axes[1][1].axis('off')  # Turn off axis\n",
+    "axes[1][1].set_title(\"Rendering result\")\n",
+    "\n",
+    "plt.subplots_adjust(wspace=0.05)\n",
+    "plt.show()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "1bcf09a0",
+   "metadata": {},
+   "source": [
+    "## Semantic Top-down View\n",
+    "The top-down view can be changed to semantic view by adding `semantic_map=True` when creating the renderer."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "064a27dd",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from metadrive.envs import ScenarioEnv\n",
+    "\n",
+    "env = ScenarioEnv(dict(log_level=50, \n",
+    "                        num_scenarios=2))\n",
+    "\n",
+    "env.reset(seed=0)\n",
+    "frame_1 = env.render(mode=\"topdown\", window=False,\n",
+    "                     screen_size=(800, 800), scaling=5)\n",
+    "\n",
+    "env.reset(seed=0)\n",
+    "frame_2 = env.render(mode=\"topdown\", window=False,\n",
+    "                     screen_size=(800, 800), scaling=5, semantic_map=True)\n",
+    "\n",
+    "env.reset(seed=1)\n",
+    "frame_3 = env.render(mode=\"topdown\", window=False,\n",
+    "                     screen_size=(800, 800), scaling=5)\n",
+    "\n",
+    "env.reset(seed=1)\n",
+    "frame_4 = env.render(mode=\"topdown\", window=False,\n",
+    "                     screen_size=(800, 800), scaling=5, semantic_map=True)\n",
+    "\n",
+    "env.close()\n",
+    "\n",
+    "import matplotlib.pyplot as plt\n",
+    "fig, axes = plt.subplots(2, 2, figsize=(10, 10)) # You can adjust the figsize as needed\n",
+    "axes[0][0].imshow(frame_1)\n",
+    "axes[0][0].axis('off')  # Turn off axis\n",
+    "axes[0][0].set_title(\"Seed: 0, Normal\")\n",
+    "axes[0][1].imshow(frame_2)\n",
+    "axes[0][1].axis('off')  # Turn off axis\n",
+    "axes[0][1].set_title(\"Seed: 0, Semantic View\")\n",
+    "axes[1][0].imshow(frame_3)\n",
+    "axes[1][0].axis('off')  # Turn off axis\n",
+    "axes[1][0].set_title(\"Seed: 1, Normal\")\n",
+    "axes[1][1].imshow(frame_4)\n",
+    "axes[1][1].axis('off')  # Turn off axis\n",
+    "axes[1][1].set_title(\"Seed: 1, Semantic View\")\n",
+    "plt.subplots_adjust(wspace=0.)\n",
+    "plt.show()"
    ]
   }
  ],
@@ -27,6 +266,9 @@
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
    "version": "3.7.13"
+  },
+  "mystnb": {
+   "execution_mode": "force"
   }
  },
  "nbformat": 4,

documentation/source/vehicle.ipynb

@@ -0,0 +1,33 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "a229471a-921a-43dc-ad2e-b7c39ee607d8",
+   "metadata": {},
+   "source": [
+    "# Vehicle"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.7.13"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

metadrive/engine/core/light.py

@@ -50,4 +50,4 @@ def __init__(self):
         self.ambient_np.node().setColor(LVector4(0.25, 0.25, 0.25, 1))
         self.ambient_np.reparentTo(self.origin)
 
-        self._node_path_list.append(self.ambient_np)
+        self._node_path_list.append(self.ambient_np)

metadrive/envs/base_env.py

@@ -225,7 +225,7 @@
     # If set to False, only the center region of the terrain has the physics body
     full_size_mesh=True,
     # Whether to show crosswalk
-    show_crosswalk=False,
+    show_crosswalk=True,
     # Whether to show sidewalk
     show_sidewalk=True,