Make Monitor optional

[toslunar]

Fix #166.

[muupan]

/test

[pfn-ci-bot]

Successfully created a job for commit 55776dd:

• Dashboard for commit 55776dd

[toslunar]

/test

[pfn-ci-bot]

Successfully created a job for commit afff3d5:

• Dashboard for commit afff3d5

[muupan]

/test

[pfn-ci-bot]

Successfully created a job for commit afff3d5:

• Dashboard for commit afff3d5

[muupan]

/test

[pfn-ci-bot]

Successfully created a job for commit 0c21181:

• Dashboard for commit 0c21181

[muupan]

/test

[pfn-ci-bot]

Successfully created a job for commit 391648c:

• Dashboard for commit 391648c

[muupan]

/test

[pfn-ci-bot]

Successfully created a job for commit 466f63a:

• Dashboard for commit 466f63a

[muupan]

I confirmed that this PR works as follows.
Before this PR, gym==0.23.0 causes an error when import pfrl:

python examples/gym/train_dqn_gym.py --gpu -1 --env Pendulum-v1
Traceback (most recent call last):
  File "examples/gym/train_dqn_gym.py", line 23, in <module>
    import pfrl
  File "/Users/user/pfndev/pfrl/pfrl/__init__.py", line 20, in <module>
    from pfrl import wrappers  # NOQA
  File "/Users/user/pfndev/pfrl/pfrl/wrappers/__init__.py", line 4, in <module>
    from pfrl.wrappers.monitor import Monitor  # NOQA
  File "/Users/user/pfndev/pfrl/pfrl/wrappers/monitor.py", line 4, in <module>
    from gym.wrappers import Monitor as _GymMonitor
ImportError: cannot import name 'Monitor' from 'gym.wrappers' (/Users/user/.local/share/virtualenvs/pfrl-1dLBIXZi/lib/python3.7/site-packages/gym/wrappers/__init__.py)

After this PR, import pfrl works fine with gym==0.23.0. An error is raised only when I try to use Monitor.

python examples/gym/train_dqn_gym.py --gpu -1 --env Pendulum-v1
Output files are saved in results/466f63ad18ea725928ebf7ad22d929ec9c453dad-70cd82e5-3ec14d5f
WARNING: Since https://github.com/pfnet/pfrl/pull/112 we have started setting `eval_during_episode=True` in this script, which affects the timings of evaluation phases.
INFO:pfrl.experiments.train_agent:outdir:results/466f63ad18ea725928ebf7ad22d929ec9c453dad-70cd82e5-3ec14d5f step:200 episode:0 R:-1.157075492297421
INFO:pfrl.experiments.train_agent:statistics:[('average_q', nan), ('average_loss', nan), ('cumulative_steps', 200), ('n_updates', 0), ('rlen', 200)]
INFO:pfrl.experiments.train_agent:outdir:results/466f63ad18ea725928ebf7ad22d929ec9c453dad-70cd82e5-3ec14d5f step:400 episode:1 R:-1.397539478750692
INFO:pfrl.experiments.train_agent:statistics:[('average_q', nan), ('average_loss', nan), ('cumulative_steps', 400), ('n_updates', 0), ('rlen', 400)]
INFO:pfrl.experiments.train_agent:outdir:results/466f63ad18ea725928ebf7ad22d929ec9c453dad-70cd82e5-3ec14d5f step:600 episode:2 R:-1.6421154508871758
INFO:pfrl.experiments.train_agent:statistics:[('average_q', nan), ('average_loss', nan), ('cumulative_steps', 600), ('n_updates', 0), ('rlen', 600)]
INFO:pfrl.experiments.train_agent:outdir:results/466f63ad18ea725928ebf7ad22d929ec9c453dad-70cd82e5-3ec14d5f step:800 episode:3 R:-1.5671076399681863
...

python examples/gym/train_dqn_gym.py --gpu -1 --monitor --env Pendulum-v1
Output files are saved in results/466f63ad18ea725928ebf7ad22d929ec9c453dad-70cd82e5-17449428
Traceback (most recent call last):
  File "examples/gym/train_dqn_gym.py", line 272, in <module>
    main()
  File "examples/gym/train_dqn_gym.py", line 122, in main
    env = make_env(test=False)
  File "examples/gym/train_dqn_gym.py", line 111, in make_env
    env = pfrl.wrappers.Monitor(env, args.outdir)
  File "/Users/user/pfndev/pfrl/pfrl/wrappers/monitor.py", line 10, in __init__
    raise RuntimeError("Monitor is not available in this version of gym")

With gym==0.22.0, --monitor won't raise any error.

[toslunar]

/test

[pfn-ci-bot]

Successfully created a job for commit d0e4a4a:

• Dashboard for commit d0e4a4a

[muupan]

/test

[pfn-ci-bot]

Successfully created a job for commit 7267243:

• Dashboard for commit 7267243