registering a new environment #142

nealmcb · 2019-04-17T02:22:15Z

Is there any way to specify a custom environment on the command line, e.g. as a fully-qualified name or lambda? Using the name of a custom environment as is leads to this error:

python -m spinup.run ppo --hid "[32,32]" --env Banana-v0 --exp_name banana --gamma 0.999

Traceback (most recent call last):
  File "/usr/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/srv/s/aima/spinningup/spinningup/spinup/run.py", line 231, in <module>
    parse_and_execute_grid_search(cmd, args)
  File "/srv/s/aima/spinningup/spinningup/spinup/run.py", line 156, in parse_and_execute_grid_search
    assert env_name in valid_envs, err_msg
AssertionError: 
Banana-v0 is not registered with Gym.
Recommendations:
    * Check for a typo (did you include the version tag?)
    * View the complete list of valid Gym environments at
        https://gym.openai.com/envs/

I can get past that point in the code by throwing an import statement in run.py for my custom environment import gym_banana, but then I run in to this later on:

....

gym.error.UnregisteredEnv: No registered env with id: Banana-v0

Traceback (most recent call last):
  File "/home/neal/Envs/rl2/lib/python3.6/site-packages/gym/envs/registration.py", line 159, in spec
    return self.env_specs[id]
KeyError: 'Banana-v0'

...

Traceback (most recent call last):
  File "/usr/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/srv/s/aima/spinningup/spinningup/spinup/run.py", line 231, in <module>
    parse_and_execute_grid_search(cmd, args)
  File "/srv/s/aima/spinningup/spinningup/spinup/run.py", line 163, in parse_and_execute_grid_search
    eg.run(algo, **run_kwargs)
  File "/srv/s/aima/spinningup/spinningup/spinup/utils/run_utils.py", line 546, in run
    data_dir=data_dir, datestamp=datestamp, **var)
  File "/srv/s/aima/spinningup/spinningup/spinup/utils/run_utils.py", line 171, in call_experiment
    subprocess.check_call(cmd, env=os.environ)
  File "/usr/lib/python3.6/subprocess.py", line 291, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/home/neal/Envs/rl2/bin/python', '/srv/s/aima/spinningup/spinningup/spinup/utils/run_entrypoint.py', 'eJyNUs9v0zAUdpquywYDxNAQO03i0gnR7oI4MIZEJThE2mFwRZZbu4mJ44TYLgxpEtJ+tJMsLntw4X/izl/BkSsvzdiAC8RK/N7nfH7fZ78P7U9HIZk//vZIFY6XcpQp0fstBn+NjqVSdOz0yMpCwxl0U+Kv05xlgppMNEsII7pCh04qKzW1+6VAzEeDgouXdXIMe9CNyXyE8ergGQ9mraxTbfEWJ4qcYsaDO+SUTMk0GLd4yNtHEWIL26ReeUxsMAsPg4DM2mNEPmLEyQsC3V0fCT2hmuUCYpIu/SrSwiLLs+A1mZHDAH/cPQbfrlWjlE3/qG+qSd/0mcxZ35RSa6kTV/4d4uSsVKZfOU3nUa/cR1vbiuVDznYg/jIgkF7xYbKfwwlsWrTpH4wYHpl4V4pK5kLb3rYqEDI7PZs6ndFSOXOJXezlO7WRsQYL6aqP8lLScVFl9bbpDb98yYX48+AaaUVBPW4GnTAKkZy9ZVViwC9ql9NR6cAvzCkwRYHp6gnU2uLwAPM9OICuX0xUMUQJmKTrfq3x22tc1sVtUeBiun4GxkdcjJlT1sCub3M5skjynbzgrm6SP7kXZ4Xdg71kXCXohCknDLyCbl15IWF5zuD5k+8/vn5bm97DA1x6yjSO+5Mt8EtsRM/dYJWrqeRcaGrk+2aHeCPeEMa3jRAcr9yvqCJJRHVJ6frlwtnSWcplBf7hv++aM8v6w7mC84kaFBLhHTad5TsNDM7Fgb917peppMCOKIv6BR/W3zMQPnrjmGp4d/+jF8DZvd5P3BROIA==']' returned non-zero exit status 1.

so evidently I need a more robust way to get it registered.

The text was updated successfully, but these errors were encountered:

christopherhesse · 2019-06-07T21:23:33Z

Can you use the env_id "gym_banana:Banana-v0"? This requires that gym_banana does the required registration as described here: https://github.com/openai/gym/blob/master/docs/creating-environments.md

bionicles · 2019-06-10T17:54:28Z

@christopherhesse @nealmcb I'm also having this issue. We register the env before we invoke the ExperimentGrid and we confirm the env is registered by making it and printing custom env attributes in our script. However the experiment threads do not have the env registered (perhaps because the registration is limited in scope to only the master thread which runs our script)

it may be necessary to edit gym itself to register the custom env in all threads instead of just the master thread as a local registration might not be globally available

christopherhesse · 2019-06-10T17:56:08Z

@bionicles do you have a short example script that demonstrates the issue?

bionicles · 2019-06-10T20:09:12Z

not really, our custom env is a file, and it's proprietary for now, but I did fix the registration issue by:

adding the new env to /home/USERNAME/anaconda3/envs/CONDAENVNAME/lib/python3.6/site-packages/gym/envs/.
adding __init__.py in same folder which imports the env,
an additional import in /gym/envs/__init__.py to register the new env. (with custom config attrdict in kwargs)

Now our custom env can be seen by all the threads because we spliced it into gym. Our environment also includes a custom numpy array space which i added into gym/spaces/array.py and added it to the __init__.py in the same folder. Result is all threads can find our custom env and the new env can find our custom space

now hitting a new issue; the replay buffers used in the td3 algorithm only support 1D observations, but our molecular environment has arbitrary dimensionality spaces (the aforementioned "Array" space I built) which do not fit into the current algos/td3/td3.py ReplayBuffer class (which just assumes observations are 17D, when in our case they are num_atoms x 17D

i believe it would save considerable work for us to transpose our custom env observation and action spaces to be instead shape (num_features, num_atoms) and (3, num_atoms) for the xyz dimensions, as this would allow us to avoid changing a bunch of code inside spinup to index different dimensions of these spaces

but we'll still need to augment the replay buffer class to support the additional dimension, and probably the training algorithm will need to change to read this buffer properly also

christopherhesse · 2019-06-10T20:22:42Z

If someone has a small example script that demonstrates this issue (it can use the roboschool package, which definitely works with gym). I can look into it, but please file a new issue on the gym repo with that example script.

bionicles · 2019-06-10T20:53:50Z

this issue only happens because the master script registers the env in one thread, and then other mpi threads run the actors, and those threads re-import gym separately, such that the original version of gym has the custom env, but the threads do not. If we could pass a path to the custom env to the threads, and they could register the env automatically, then we wouldn't need to splice custom envs into gym, we could just pass a path to the env into the script:

from spinup.utils.run_utils import ExperimentGrid
import tensorflow as tf
from spinup import td3


if __name__ == '__main__':
    import argparse
    parser = argparse.ArgumentParser()
    parser.add_argument('--cpu', type=int, default=1)
    parser.add_argument('--num_runs', type=int, default=3)
    args = parser.parse_args()

    kwargs = {"config": {"STOP_LOSS_MULTIPLE": 10}}

    eg = ExperimentGrid(name='neuromax')
    eg.add('env_name', 'CustomEnv-v0', '', True)
    **eg.add('custom_env_path', "/hax/envs/custom.py")**
    **eg.add('custom_env_kwargs', kwargs)**
    eg.add('seed', [10*i for i in range(args.num_runs)])
    eg.add('epochs', 10)
    eg.add('steps_per_epoch', 4000)
    eg.add(
        'ac_kwargs:hidden_sizes',
        [(18, 18)],
        'hid')
    eg.add('ac_kwargs:activation', [tf.tanh], '')
    eg.run(td3, num_cpu=args.cpu)

then the algo can just register the env at the given path under the provided name

krishpop · 2019-06-16T09:04:38Z

My solution to this problem was doing the following:
Passing env_fn to the experiment as

def env_fn():
    import <custom env module>
    return gym.make(env_name)

eg.add('env_fn', env_fn)
...

custom env module is responsible for registering the environment when loaded. Importing this module in the env_fn takes care of making sure the imports are done by the subprocesses.

alitk · 2019-06-26T20:08:45Z

@krishpop Did this solve the problem? How did you pass it as an argument to the experiment? Did you change the spinup.run? Or you defined this function and passed it to the experiment as:

!python -m spinup.run ddpg --hid "[32,32]" --env 'env_fn' --exp_name installtest --gamma 0.999

Because this does not work for me. I am not sure if you meant to modify the spinup.run file or your main code. Inside the spinup.run file they have the process() function which process the argument and run them using eval() if they need to be run. If there was a exec() instead of eval I could pass by import gym_my_custom_env to that. I am not sure if it solves the problem though.

krishpop · 2019-06-28T00:34:58Z

@alitk I am using my own run script and using the experiments API

here's a minimal example: https://gist.github.com/krishpop/f4b2aa8d60d7b22bce8c258fd68ab11c

alitk · 2019-06-28T03:54:42Z

Thank you @krishpop . I managed to fix it using these commands:

!echo  "from gym.envs.box2d.environment import Env_name_function" >> /usr/local/lib/python3.6/dist-packages/gym/envs/box2d/__init__.py
!echo "register(id=my_env-v0', entry_point='gym.envs.box2d:Env_name_function')" >>  /usr/local/lib/python3.6/dist-packages/gym/envs/__init__.py
!mv Source-folder/environment.py /usr/local/lib/python3.6/dist-packages/gym/envs/box2d/

I basically added it as a box2d environment to gym (which was the idea mentioned by @bionicles ). I append a line in the two init.py in the original gym environment folder. It might be a different folder for Anaconda. These folders are in Google Colab since I am using it. Finally, the problem was solved and I am able to use it in the form I mentioned

!python -m spinup.run ddpg --hid "[32,32]" --env 'env_fn' --exp_name installtest --gamma 0.999

billtubbs · 2020-02-10T01:40:35Z

What's the best solution here? I see:

@krishpop recommends adding a function to import the custom env. module
@alitk has a different solution which I don't quite understand

I basically got this error when switching the environment name in the bench_ppo_cartpole.py example from the default env ('CartPole-v0') to my custom environment ('CartPole-BT-dL-v0') — after making sure my custom environment was installed in the current conda environment and working.

Should I be expecting this example code to work with my custom environment this easily?

  File "/anaconda3/envs/spinningup/lib/python3.6/site-packages/gym/envs/registration.py", line 128, in spec
    raise error.UnregisteredEnv('No registered env with id: {}'.format(id))
gym.error.UnregisteredEnv: No registered env with id: CartPole-BT-dL-v0

thanks.

ahmeda14960 · 2020-03-07T03:38:29Z

I'm also running into the same issue and would love an answer to @billtubbs question!

tyoung-JHUAPL · 2020-03-11T17:38:46Z

Same here, it's not clear the right way to inform spinup of your custom environments. I want to be able to indicate to others on my team how to install spinup and our custom environment and run them together

hmate9 · 2020-08-19T21:33:57Z

Any updates on this? Looking to try on a custom environment.

krishpop · 2020-09-02T02:31:37Z

@hmate9 one modification worth trying is adding a file to utils that contains env_fn factory to create custom env generators that can be passed directly to the run script from the command line. For example, see my forked branch here. In spinup/utils/rrc_utils.py, I have added an env_fn factory (with the ability to add wrappers) that can make an arbitrary wrapped env. This way, env_fn can be passed in as a command line argument, for example,
python -m spinup.run ppo --env_fn rrc_utils.rrc_ppo_env_fn --exp_name ppo_rrc_t1 --cpu auto

varunjammula · 2020-11-16T05:58:29Z

Any updates here?

This commit contains a solution to the problems explained in openai#142. It adds two ways for people to use a custom environment with the Spinningup package: 1. People can add the code that initializes the custom gym environmeent in the 'env_config.py' file. 2. People can use the 'env_pkg' cmd line argument to specify which package should be imported for the custom environment to work.

This commit contains a solution to the problems explained in openai#142. It adds two ways for people to use a custom environment with the Spinningup package: 1. People can add the code that initializes the custom gym environment in the 'env_config.py' file. 2. People can use the 'env_pkg' cmd line argument to specify which the package should be imported for the custom environment to work.

rickstaa · 2021-02-23T16:38:53Z

@varunjammula I just created a pull request with our department's solution to this issue in a package based on the SpinningUp Package. You merge it into your fork to see if it suits your needs.

rickstaa · 2021-02-23T16:47:58Z

@varunjammula I also just found the solution of @pkol in #201 which I think is an even more elegant solution.

krishpop mentioned this issue Sep 2, 2020

Why passing a lambda function of making the environment to the algorithm? #270

Closed

rickstaa mentioned this issue Feb 23, 2021

Adds custom gym environment support #327

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

registering a new environment #142

registering a new environment #142

nealmcb commented Apr 17, 2019

christopherhesse commented Jun 7, 2019

bionicles commented Jun 10, 2019

christopherhesse commented Jun 10, 2019

bionicles commented Jun 10, 2019 •

edited

christopherhesse commented Jun 10, 2019

bionicles commented Jun 10, 2019

krishpop commented Jun 16, 2019 •

edited

alitk commented Jun 26, 2019

krishpop commented Jun 28, 2019 •

edited

alitk commented Jun 28, 2019

billtubbs commented Feb 10, 2020

ahmeda14960 commented Mar 7, 2020

tyoung-JHUAPL commented Mar 11, 2020

hmate9 commented Aug 19, 2020

krishpop commented Sep 2, 2020

varunjammula commented Nov 16, 2020

rickstaa commented Feb 23, 2021 •

edited

rickstaa commented Feb 23, 2021

registering a new environment #142

registering a new environment #142

Comments

nealmcb commented Apr 17, 2019

christopherhesse commented Jun 7, 2019

bionicles commented Jun 10, 2019

christopherhesse commented Jun 10, 2019

bionicles commented Jun 10, 2019 • edited

christopherhesse commented Jun 10, 2019

bionicles commented Jun 10, 2019

krishpop commented Jun 16, 2019 • edited

alitk commented Jun 26, 2019

krishpop commented Jun 28, 2019 • edited

alitk commented Jun 28, 2019

billtubbs commented Feb 10, 2020

ahmeda14960 commented Mar 7, 2020

tyoung-JHUAPL commented Mar 11, 2020

hmate9 commented Aug 19, 2020

krishpop commented Sep 2, 2020

varunjammula commented Nov 16, 2020

rickstaa commented Feb 23, 2021 • edited

rickstaa commented Feb 23, 2021

bionicles commented Jun 10, 2019 •

edited

krishpop commented Jun 16, 2019 •

edited

krishpop commented Jun 28, 2019 •

edited

rickstaa commented Feb 23, 2021 •

edited