Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Tried to reset environment which is not done. While the monitor is active ... #56

Closed
weiliu620 opened this issue Nov 11, 2016 · 4 comments
Closed

Tried to reset environment which is not done. While the monitor is active ... #56

weiliu620 opened this issue Nov 11, 2016 · 4 comments

Comments

@weiliu620
Copy link

I'm using the github latest versio of gym, and also the latest rllab, and run the following code:

stub(globals())

    # env = normalize(GymEnv("Point-v0"))
    env = normalize(GymEnv("CartPole-v0"))

    policy = CategoricalMLPPolicy(
        env_spec=env.spec,
    )

    baseline = LinearFeatureBaseline(env_spec=env.spec)

    algo = TRPO(
        env=env,
        policy=policy,
        baseline=baseline,
        batch_size=4000,
        whole_paths=True,
        max_path_length=100,
        n_itr=1,
        discount=0.99,
        step_size=0.01,
    )

    run_experiment_lite(
        algo.train(),
        # Number of parallel workers for sampling
        n_parallel=1,
        # Only keep the snapshot parameters for the last iteration
        snapshot_mode="last",
        seed=1,
        # plot=True,
    )

Then sometimes I saw these errors

In [35]: algo = rl_prac.train_point()
python /home/weiliu/packages/rllab/scripts/run_experiment_lite.py  --log_dir '/home/weiliu/packages/rllab/data/local/experiment/experiment_2016_11_11_12_21_32_0019'  --use_cloudpickle 'False'  --snapshot_mode 'last'  --args_data 'gANjcmxsYWIubWlzYy5pbnN0cnVtZW50ClN0dWJNZXRob2RDYWxsCnEAKYFxAX1xAihYBgAAAF9fYXJnc3EDKGNybGxhYi5taXNjLmluc3RydW1lbnQKU3R1Yk9iamVjdApxBCmBcQV9cQYoWAYAAABrd2FyZ3NxB31xCChYBQAAAG5faXRycQlLAVgJAAAAc3RlcF9zaXplcQpHP4R64UeuFHtYCAAAAGJhc2VsaW5lcQtoBCmBcQx9cQ0oaAd9cQ5YCAAAAGVudl9zcGVjcQ9jcmxsYWIubWlzYy5pbnN0cnVtZW50ClN0dWJBdHRyCnEQKYFxEX1xEihYBAAAAF9vYmpxE2gEKYFxFH1xFShoB31xFlgDAAAAZW52cRdoBCmBcRh9cRkoaAd9cRpYCAAAAGVudl9uYW1lcRtYCwAAAENhcnRQb2xlLXYwcRxzWAsAAABwcm94eV9jbGFzc3EdY3JsbGFiLmVudnMuZ3ltX2VudgpHeW1FbnYKcR5YBAAAAGFyZ3NxHyl1YnNoHWNybGxhYi5lbnZzLm5vcm1hbGl6ZWRfZW52Ck5vcm1hbGl6ZWRFbnYKcSBoHyl1YlgKAAAAX2F0dHJfbmFtZXEhWAQAAABzcGVjcSJ1YnNoHWNybGxhYi5iYXNlbGluZXMubGluZWFyX2ZlYXR1cmVfYmFzZWxpbmUKTGluZWFyRmVhdHVyZUJhc2VsaW5lCnEjaB8pdWJYCgAAAGJhdGNoX3NpemVxJE2gD1gIAAAAZGlzY291bnRxJUc/764UeuFHrlgGAAAAcG9saWN5cSZoBCmBcSd9cSgoaAd9cSloD2gQKYFxKn1xKyhoE2gUaCFoInVic2gdY3JsbGFiLnBvbGljaWVzLmNhdGVnb3JpY2FsX21scF9wb2xpY3kKQ2F0ZWdvcmljYWxNTFBQb2xpY3kKcSxoHyl1YlgPAAAAbWF4X3BhdGhfbGVuZ3RocS1LZFgLAAAAd2hvbGVfcGF0aHNxLohoF2gUdWgdY3JsbGFiLmFsZ29zLnRycG8KVFJQTwpxL2gfKXViWAUAAAB0cmFpbnEwKX1xMXRxMlgIAAAAX19rd2FyZ3NxM31xNHViLg=='  --seed '1'  --n_parallel '1'  --exp_name 'experiment_2016_11_11_12_21_32_0019'
/home/weiliu/anaconda3/envs/rllab3/lib/python3.5/site-packages/theano/tensor/signal/downsample.py:6: UserWarning: downsample module has been moved to the theano.tensor.signal.pool module.
  "downsample module has been moved to the theano.tensor.signal.pool module.")
using seed 1
2016-11-11 12:47:49.070577 EST | Setting seed to 1
using seed 1
[2016-11-11 12:47:49,122] Making new env: CartPole-v0
2016-11-11 12:47:49.800481 EST | [experiment_2016_11_11_12_21_32_0019] Populating workers...
2016-11-11 12:47:49.800802 EST | [experiment_2016_11_11_12_21_32_0019] Populated
0%                          100%
[                              ][2016-11-11 12:47:49,853] Starting new video recorder writing to /home/weiliu/packages/rllab/data/local/experiment/experiment_2016_11_11_12_21_32_0019/gym_log/openaigym.video.0.29157.video000000.mp4
[2016-11-11 12:47:50,883] Starting new video recorder writing to /home/weiliu/packages/rllab/data/local/experiment/experiment_2016_11_11_12_21_32_0019/gym_log/openaigym.video.0.29157.video000001.mp4
[#                             ] | ETA: 00:01:05[2016-11-11 12:47:52,234] Starting new video recorder writing to /home/weiliu/packages/rllab/data/local/experiment/experiment_2016_11_11_12_21_32_0019/gym_log/openaigym.video.0.29157.video000008.mp4
[#####                         ] | ETA: 00:00:15[2016-11-11 12:47:52,944] Starting new video recorder writing to /home/weiliu/packages/rllab/data/local/experiment/experiment_2016_11_11_12_21_32_0019/gym_log/openaigym.video.0.29157.video000027.mp4
[######                        ] | ETA: 00:00:18Traceback (most recent call last):
  File "/home/weiliu/packages/rllab/scripts/run_experiment_lite.py", line 137, in <module>
    run_experiment(sys.argv)
  File "/home/weiliu/packages/rllab/scripts/run_experiment_lite.py", line 124, in run_experiment
    maybe_iter = concretize(data)
  File "/home/weiliu/packages/rllab/rllab/misc/instrument.py", line 1213, in concretize
    return method(*args, **kwargs)
  File "/home/weiliu/packages/rllab/rllab/algos/batch_polopt.py", line 120, in train
    paths = self.sampler.obtain_samples(itr)
  File "/home/weiliu/packages/rllab/rllab/algos/batch_polopt.py", line 28, in obtain_samples
    scope=self.algo.scope,
  File "/home/weiliu/packages/rllab/rllab/sampler/parallel_sampler.py", line 125, in sample_paths
    show_prog_bar=True
  File "/home/weiliu/packages/rllab/rllab/sampler/stateful_pool.py", line 150, in run_collect
    result, inc = collect_once(self.G, *args)
  File "/home/weiliu/packages/rllab/rllab/sampler/parallel_sampler.py", line 94, in _worker_collect_one_path
    path = rollout(G.env, G.policy, max_path_length)
  File "/home/weiliu/packages/rllab/rllab/sampler/utils.py", line 12, in rollout
    o = env.reset()
  File "/home/weiliu/packages/rllab/rllab/envs/normalized_env.py", line 52, in reset
    ret = self._wrapped_env.reset()
  File "/home/weiliu/packages/rllab/rllab/envs/gym_env.py", line 92, in reset
    return self.env.reset()
  File "/home/weiliu/packages/gym/gym/core.py", line 139, in reset
    self.monitor._before_reset()
  File "/home/weiliu/packages/gym/gym/monitoring/monitor.py", line 271, in _before_reset
    self.stats_recorder.before_reset()
  File "/home/weiliu/packages/gym/gym/monitoring/stats_recorder.py", line 65, in before_reset
    raise error.Error("Tried to reset environment which is not done. While the monitor is active for {}, you cannot call reset() unless the episode is over.".format(self.env_id))
gym.error.Error: Tried to reset environment which is not done. While the monitor is active for CartPole-v0, you cannot call reset() unless the episode is over.

If you suspect this is an IPython bug, please report it at:
    https://github.com/ipython/ipython/issues
or send an email to the mailing list at ipython-dev@scipy.org

You can print a more detailed traceback right now with "%tb", or use "%debug"
to interactively debug it.

Extra-detailed tracebacks for bug-reporting purposes can be enabled via:
    %config Application.verbose_crash=True

I don't seem to have seen such error before, when I used 'pip install gym'...
@dementrock
Copy link
Member

dementrock commented Nov 12, 2016 •
edited
Loading

Quick solution is to change max_path_length to 200.

Basically the error is because the horizon does not match the one used in the gym environment spec.

@dementrock
Copy link
Member

Fixed in 4fd25fe.

@dqii
Copy link

dqii commented Mar 10, 2017

Hi, I'm getting a similar error when running ddpg on CarRacing-v0:

File "/Users/diqi/iw_s17/rllab/scripts/run_experiment_lite.py", line 136, in
run_experiment(sys.argv)
File "/Users/diqi/iw_s17/rllab/scripts/run_experiment_lite.py", line 120, in run_experiment
method_call(variant_data)
File "examples/ddpg_car.py", line 42, in run_task
algo.train()
File "/Users/diqi/iw_s17/rllab/rllab/algos/ddpg.py", line 229, in train
next_observation, reward, terminal, _ = self.env.step(action)
File "/Users/diqi/iw_s17/rllab/rllab/envs/normalized_env.py", line 86, in step
wrapped_step = self._wrapped_env.step(scaled_action)
File "/Users/diqi/iw_s17/rllab/rllab/envs/gym_env.py", line 116, in step
next_obs, reward, done, info = self.env.step(action)
File "/Users/diqi/anaconda/envs/rllab3/lib/python3.5/site-packages/gym/core.py", line 100, in step
return self._step(action)
File "/Users/diqi/anaconda/envs/rllab3/lib/python3.5/site-packages/gym/wrappers/monitoring.py", line 32, in _step
self._before_step(action)
File "/Users/diqi/anaconda/envs/rllab3/lib/python3.5/site-packages/gym/wrappers/monitoring.py", line 167, in _before_step
self.stats_recorder.before_step(action)
File "/Users/diqi/anaconda/envs/rllab3/lib/python3.5/site-packages/gym/monitoring/stats_recorder.py", line 46, in before_step
raise error.ResetNeeded("Trying to step environment which is currently done. While the monitor is active for {}, you cannot step beyond the end of an episode. Call 'env.reset()' to start the next episode.".format(self.env_id))
gym.error.ResetNeeded: Trying to step environment which is currently done. While the monitor is active for CarRacing-v0, you cannot step beyond the end of an episode. Call 'env.reset()' to start the next episode.

@dementrock
Copy link
Member

@dqii see #87 (comment)

jonashen pushed a commit to jonashen/rllab that referenced this issue May 29, 2018
Cleanup list of dependencies in conda environment (rll#56) (rll#60)
92c41f3 
It was verified that each of the dependencies removed were not
imported or had any package dependants in rllab.
The new environment file was tested with a fresh copy of the
rllab project by running the setup_linux script, and with the
training of the swimmer in Theano and cartpole in TensorFlow
with plot=True.
The channels of the removed packages were discarded as well.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@dementrock @weiliu620 @dqii