Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Possible ways to replicate a env. #939

Closed
YuhangSong opened this issue Mar 8, 2018 · 10 comments
Closed

Possible ways to replicate a env. #939

YuhangSong opened this issue Mar 8, 2018 · 10 comments

Comments

@YuhangSong
Copy link

@YuhangSong YuhangSong commented Mar 8, 2018

In my research, I need to replicate a env object at current step. And then run the copies of the env object separately.

I have tried copy.deepcopy() and pickle.dumps(), but it does not seems to work as expected. I am wondering if above methods cannot really copy the env object, maybe something at the bottom level of the env object is not copied as expected?

Many thanks, the OpenAI gym is really pushing forward the exciting AI research.

@mpSchrader

This comment has been minimized.

Copy link
Contributor

@mpSchrader mpSchrader commented Mar 15, 2018

I don't know which environment you are using if it is an environment, which wraps C/C++ code, such as Atari. You might wanna have a look at ezpickle. With ezpickle you should be able to replicate the states of these environments.

@YuhangSong

This comment has been minimized.

Copy link
Author

@YuhangSong YuhangSong commented Mar 15, 2018

Thank you so much for the reply.

I am using Atari, which is created by, say gym.make('AssaulNoFrameskip-v4'). Should I look into ezpickle?

Many thanks!

@mpSchrader

This comment has been minimized.

Copy link
Contributor

@mpSchrader mpSchrader commented Mar 15, 2018

@YuhangSong

This comment has been minimized.

Copy link
Author

@YuhangSong YuhangSong commented Mar 15, 2018

Hi, can you be more specific on how should I do it? It seems that I should make my Atari environment inherit the ezpickle, then apply pickle & unpickle to it? But how can I do it?

Doesn't Atari environment already inherit ezpickle, base on the code here:

class AtariEnv(gym.Env, utils.EzPickle):

?

@mpSchrader

This comment has been minimized.

Copy link
Contributor

@mpSchrader mpSchrader commented Mar 15, 2018

@YuhangSong

This comment has been minimized.

Copy link
Author

@YuhangSong YuhangSong commented Mar 15, 2018

Thanks a lot, that helps a lot.

But another small question came up, I use a lot of Wrapper, how can I call these methods after using a Wrapper like:

class NoopResetEnv(gym.Wrapper):
    def __init__(self, env, noop_max=30):
        """Sample initial states by taking random number of no-ops on reset.
        No-op is assumed to be action 0.
        """
        gym.Wrapper.__init__(self, env)
        self.noop_max = noop_max
        self.override_num_noops = None
        self.noop_action = 0
        assert env.unwrapped.get_action_meanings()[0] == 'NOOP'

    def _reset(self, **kwargs):
        """ Do no-op action for a number of steps in [1, noop_max]."""
        self.env.reset(**kwargs)
        if self.override_num_noops is not None:
            noops = self.override_num_noops
        else:
            noops = self.unwrapped.np_random.randint(1, self.noop_max + 1) #pylint: disable=E1101
        assert noops > 0
        obs = None
        for _ in range(noops):
            obs, _, done, _ = self.env.step(self.noop_action)
            if done:
                obs = self.env.reset(**kwargs)
        return obs

And I wrap my_env with:

env = NoopResetEnv(env, noop_max=30)

I tried

my_env.env.clone_state()

It shows:

AttributeError: 'TimeLimit' object has no attribute 'clone_state'

But if I remove env = NoopResetEnv(env, noop_max=30), my_env.env.clone_state() works.

@mpSchrader

This comment has been minimized.

Copy link
Contributor

@mpSchrader mpSchrader commented Mar 15, 2018

You might wanna have a look at self.unwrapped from the wrapper class, this allows you to access the inner environment.

@fuxianh

This comment has been minimized.

Copy link

@fuxianh fuxianh commented Oct 23, 2018

Hi, bro. Have you succeed in replicating an env object and then run the copies of the env object separately?
@YuhangSong

@YuhangSong

This comment has been minimized.

Copy link
Author

@YuhangSong YuhangSong commented Oct 23, 2018

@christopherhesse

This comment has been minimized.

Copy link
Contributor

@christopherhesse christopherhesse commented Feb 9, 2019

Thanks for all the help @mpSchrader! As has been pointed out, you can look at env.unwrapped. clone_full_state() and env.unwrapped.restore_full_state() to get/set state for ALE environments.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants
You can’t perform that action at this time.