Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ensure performance meets internal benchmark #47

Open
Gamenot opened this issue Oct 21, 2020 · 1 comment
Open

Ensure performance meets internal benchmark #47

Gamenot opened this issue Oct 21, 2020 · 1 comment
Labels

Comments

@Gamenot
Copy link
Collaborator

Gamenot commented Oct 21, 2020

Benchmark,

1 agent, 50 SVs, loop scenario: 100 FPS
10 agents, 0 SVs, loop scenario, 25 FPS

Feel free to suggest a different benchmark than ^

Context:

davidrusu: Trying to run both agents and social vehicles together hides the performance hit from each individually.
Here's the curves of how increasing either value, slows our performance:
image

Tests were run on the loop scenario and Agents were instances of the following AgentSpec:

class Policy(AgentPolicy):
    def act(self, obs):
        return "keep_lane"

agent_spec = AgentSpec(
    interface=AgentInterface.from_type(AgentType.Laner, max_episode_steps=1000),
    policy_builder=Policy,
)

davidrusu: @jun Well, we can't directly control how many social vehicles are in the scenario, but here's a few episodes with a traffic flow of 30SV/min + 10 agents.

╭────────────────────┬────────────────────┬────────────────────┬────────────────────┬────────────────────┬╮
│            Episode │     Sim T / Wall T │        Total Steps │        Steps / Sec │       Scenario Map ││
├────────────────────┼────────────────────┼────────────────────┼────────────────────┼────────────────────┼┤
│              0/100 │               2.18 │                999 │              21.78 │               loop ││
│              1/100 │               1.32 │                999 │              13.23 │               loop ││
│              2/100 │               1.24 │                999 │              12.35 │               loop ││
│              3/100 │               1.59 │                999 │              15.87 │               loop ││
@Gamenot Gamenot added this to the 0.4.3 milestone Oct 21, 2020
@Gamenot Gamenot added the meta label Oct 31, 2020
@Gamenot Gamenot modified the milestones: 0.5, 0.6 Nov 26, 2020
@Gamenot Gamenot modified the milestones: 0.6, Backlog Jan 27, 2021
@wuyukun-tong
Copy link

wuyukun-tong commented Nov 19, 2021

Uploading image.png…
运行vis.py的时候出现了这个错误是怎么回事,麻烦大神解答一下,谢谢
Successfully registered the following environments: ['MetaDrive-test-v0', 'MetaDrive-validation-v0', 'MetaDrive-v0', 'MetaDrive-10envs-v0', 'MetaDrive-1000envs-v0', 'MetaDrive-training0-v0', 'MetaDrive-training1-v0', 'MetaDrive-training2-v0'].
Traceback (most recent call last):
File "vis.py", line 83, in
env.render(mode="top_down", num_stack=25)
File "/home/gzz/miniconda3/envs/copo/lib/python3.7/site-packages/metadrive/envs/base_env.py", line 265, in render
return self._render_topdown(*args, **kwargs)
File "/home/gzz/miniconda3/envs/copo/lib/python3.7/site-packages/metadrive/envs/base_env.py", line 480, in _render_topdown
self._top_down_renderer = TopDownRenderer(*args, **kwargs)
File "/home/gzz/miniconda3/envs/copo/lib/python3.7/site-packages/metadrive/obs/top_down_renderer.py", line 143, in init
assert self.current_track_vehicle is not None, "Specify which vehicle to track"
File "/home/gzz/miniconda3/envs/copo/lib/python3.7/site-packages/metadrive/obs/top_down_renderer.py", line 235, in current_track_vehicle
return self.engine.agent_manager.active_agents["default_agent"]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants