Agent frozen #145
Comments
|
what you're describing suggests that the network weights have not been
loaded properly.
are you using the colab as provided? i tested it recently and it seemed to
be working fine.
…On Tue, Jul 28, 2020 at 5:58 AM satst27 ***@***.***> wrote:
I'm trying to visualize Breakout and Pong using example_viz_lib and dqn
agent. However, in both cases for all 5 checkpoints provided, the agent
seems to be stuck in the corner and does not move. In another words agent
does not seem to play. The same experiment with SpaceInvaders seems to work
fine. Is it because for Breakout and Pong, agent is waiting for FIRE action
that never arrives or is it some other reason?
Thanks
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#145>, or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AE3CCMJWQNUZLREDLRK3NS3R52OLZANCNFSM4PKK7R7A>
.
|
|
The agent_visualizer.py as it is provided does not seem to run in colab. With suggested python 2.7, it gives the following error There seems to be no install error if i set pip install dopamine-rl==1.0.0 but then I see the error I therefore tried to run the file locally with Python 3.6, tensorflow=1.15.0 and dopamine installed from source. It is here I see the issue with the agent getting stuck. thanks, |
|
apologies, something seems to have broken since our latest release
yesterday. looking into it, should hopefully have a fix today or tomorrow.
…On Tue, Jul 28, 2020 at 11:56 AM satst27 ***@***.***> wrote:
The agent_visualizer.py as it is provided does not seem to run in colab.
With suggested python 2.7, it gives the following error
while installing dopamine-rl
ERROR: Command errored out with exit status 1: python setup.py egg_info
Check the logs for full command output.
Is it still supposed to be used with python 2.7? Is there a way around
this error? To download Breakout I used the following:
!gsutil -q -m cp -R
gs://download-dopamine-rl/lucid/dqn/Breakout/1/tf_ckpt-199.data-00000-of-00001
/tmp
!gsutil -q -m cp -R
gs://download-dopamine-rl/lucid/dqn/Breakout/1/tf_ckpt-199.index /tmp
!gsutil -q -m cp -R
gs://download-dopamine-rl/lucid/dqn/Breakout/1/tf_ckpt-199.meta /tmp
There seems to be no install error if i set pip install dopamine-rl==1.0.0
but then I see the error
Import: Error: cannot import name example_viz_lib
I therefore tried to run the file locally with Python 3.6,
tensorflow=1.15.0 and dopamine installed from source. It is here I see the
issue with the agent getting stuck.
thanks,
satst27
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#145 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AE3CCMKWFUKZWDOJWIMV4ITR53YLBANCNFSM4PKK7R7A>
.
|
|
we just pushed some fixed and it seems to work from my tests. |
|
Thank you very much for upgrading the script! It works very well for SpaceInvaders. However for Breakout, the agent is still frozen. I used the following commands to download the trained breakout agent: !gsutil -q -m cp -R gs://download-dopamine-rl/lucid/dqn/Breakout/5/tf_ckpt-199.data-00000-of-00001 /tmp Can you please help me understand why agent does not seem to play? How do I get trained agent for games like Breakout and Pong? In particulars, have I used the correct download paths above? Many thanks, |
|
ah, you're using the lucid checkpoints which were trained a bit differently
(for this paper <https://arxiv.org/abs/1812.07069>), so reloading them may
not work properly.
you can use this for Pong:
!gsutil -q -m cp -R
gs://download-dopamine-rl/colab/samples/rainbow/Pong_v4/checkpoints/tf_ckpt-
199.data-00000-of-00001 /tmp
!gsutil -q -m cp -R
gs://download-dopamine-rl/colab/samples/rainbow/Pong_v4/checkpoints/tf_ckpt-
199.index /tmp
!gsutil -q -m cp -R
gs://download-dopamine-rl/colab/samples/rainbow/Pong_v4/checkpoints/tf_ckpt-
199.meta /tmp
you can replace the game name with either SpaceInvaders or Asterix, but
unfortunately we didn't push checkpoints for Breakout.
however, we do have checkpoints for all agents and all games available for
download as separate .tar.gz files here:
https://github.com/google/dopamine/tree/master/docs#downloads
you will probably need to add the `use_legacy_checkpoint=True` parameter to
the call to `example_viz_lib.run()` in the colab, but i'll be pushing an
updated version of that today.
hope this helps!
…-psc
On Wed, Jul 29, 2020 at 4:33 AM satst27 ***@***.***> wrote:
Thank you very much for upgrading the script! It works very well for
SpaceInvaders. However for Breakout, the agent is still frozen. I used the
following commands to download the trained breakout agent:
!gsutil -q -m cp -R
gs://download-dopamine-rl/lucid/dqn/Breakout/5/tf_ckpt-199.data-00000-of-00001
/tmp
!gsutil -q -m cp -R
gs://download-dopamine-rl/lucid/dqn/Breakout/5/tf_ckpt-199.index /tmp
!gsutil -q -m cp -R
gs://download-dopamine-rl/lucid/dqn/Breakout/5/tf_ckpt-199.meta /tmp
Can you please help me understand why agent does not seem to play? How do
I get trained agent for games like Breakout and Pong? In particulars, have
I used the correct download paths above?
Many thanks,
satst27
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#145 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AE3CCMOFRQPBPMLIKROGWITR57NFDANCNFSM4PKK7R7A>
.
|
|
Very helpful. It resolved all my issues. thanks, |
|
The issue is resolved. |
and others
I'm trying to visualize Breakout and Pong using example_viz_lib and dqn agent. However, in both cases for all 5 checkpoints provided, the agent seems to be stuck in the corner and does not move. In another words agent does not seem to play. The same experiment with SpaceInvaders seems to work fine. Is it because for Breakout and Pong, agent is waiting for FIRE action that never arrives or is it some other reason?
Thanks
The text was updated successfully, but these errors were encountered: