better support for custom environments

[zach-nervana]

At the moment, the following steps are required to use an environment not included with coach:

1. implement an environment which matches coach custom environment type
2. import your environment inside of environments/__init__.py
3. a new entry should be added to the EnvTypes enum mapping the environment name to the wrapper's class name, also in environments/__init__.py
4. a new configuration class should be implemented for defining the environment's parameters and added to configurations.py
5. a new preset should be added to presets.py

these steps are from here: http://coach.nervanasys.com/contributing/add_env/index.html

Many of these steps require modifying coach internal files. In order to track these changes in a version control system, coach itself must be forked and modified. This is inconvenient for multiple reasons:

1. users can not simply run pip install coach --upgrade to get upgrades.
2. users must continually rebase their changes on top of the latest coach changes and resolve any merge conflicts that occur.
3. users may have environments defined in existing projects and git repositories. with the current set up, they will need to continue committing to their existing repositories and also make sure their coach fork matches their environment repository, and make separate commits to the environment wrapper if necessary.

Additionally, this pattern of modifying coaches internals in order to add environments or run experiments is not very new user-friendly.

Here is an example from rllab documentation:

from rllab.algos.trpo import TRPO
from rllab.baselines.linear_feature_baseline import LinearFeatureBaseline
from examples.point_env import PointEnv
from rllab.envs.normalized_env import normalize
from rllab.policies.gaussian_mlp_policy import GaussianMLPPolicy

env = normalize(PointEnv())
policy = GaussianMLPPolicy(
    env_spec=env.spec,
)
baseline = LinearFeatureBaseline(env_spec=env.spec)
algo = TRPO(
    env=env,
    policy=policy,
    baseline=baseline,
)
algo.train()

The user is required to create a new environment wrapper following their environment pattern similar to our step 1. After that however, the user only needs to write a simple python script with a few imports and some configuration.

Looking through coach.py, it looks like it shouldn't be terribly difficult to provide a similar interface, though I am not entirely sure. What do you guys think?

[itaicaspi-intel]

Thanks Zach, I think that this is a good suggestion.
Maybe something along the lines of Gym's register environment?
To allow using Coach within other scripts, it would probably also be good to have a better python interface to Coach, such as the one you cited from rllab.

[jtoy]

agree, would love this. I wanted to try using this with https://github.com/jtoy/sensenet as its openai compatible, but there were too many files to modify.

[zach-nervana]

I added a new method for specifying which environment to use in the case that the environment follows the interface provided by openai gym. In a preset, you can now specify the environment via: env.level = 'package.module:ClassName'. This means that new environments and be integrated more easily. The only place where coach needs modified to work with new environments now is presets.py. This change was completed in #41.

I will leave this issue open because it would still be nice to configure coach without modifying presets.py. Also, this fix is only helpful in the case that your environment is a gym environment.

[jtoy]

@zach-nervana Great, I'll test it out and let you know.

[saddy001]

@zach-nervana could you update the docs (https://nervanasystems.github.io/coach/contributing/add_env/index.html) to reflect your change?

Thanks,

[itaicaspi-intel]

thanks @saddy001, it is now updated in https://nervanasystems.github.io/coach/contributing/add_env/index.html as you suggested.
the environment integration process was simplified as well, and we also have a step by step tutorial notebook to accompany the documentation (https://github.com/NervanaSystems/coach/blob/master/tutorials/2.%20Adding%20an%20Environment.ipynb)