Fix PGT's training

[muupan]

#317 broke PGT's training and made tests/agents_tests/test_pgt.py fail. This PR fixes it.

[muupan]

Before this PR:

pytest -xsv --disable-pytest-warnings tests/agents_tests/test_pgt.py
========================================================================================================================== test session starts ===========================================================================================================================
platform linux -- Python 3.5.2, pytest-3.8.2, py-1.6.0, pluggy-0.7.1 -- /home/fujita/.local/share/virtualenvs/chainerrl_20180827-9TPAa4mI/bin/python3
cachedir: .pytest_cache
rootdir: /home/fujita/chainerrl_20180827, inifile:
collected 4 items

tests/agents_tests/test_pgt.py::TestPGTOnContinuousABC::test_training_cpu <- tests/agents_tests/basetest_training.py FAILED

================================================================================================================================ FAILURES ================================================================================================================================
________________________________________________________________________________________________________________ TestPGTOnContinuousABC.test_training_cpu ________________________________________________________________________________________________________________

self = <test_pgt.TestPGTOnContinuousABC testMethod=test_training_cpu>

    @testing.attr.slow
    def test_training_cpu(self):
>       self._test_training(-1, steps=100000)

tests/agents_tests/basetest_training.py:93:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
tests/agents_tests/basetest_training.py:62: in _test_training
    eval_env=test_env)
chainerrl/experiments/train_agent.py:163: in train_agent_with_evaluation
    logger=logger)
chainerrl/experiments/train_agent.py:52: in train_agent
    action = agent.act_and_train(obs, r)
chainerrl/agents/pgt.py:233: in act_and_train
    self.replay_updater.update_if_necessary(self.t)
chainerrl/replay_buffer.py:524: in update_if_necessary
    self.update_func(transitions)
chainerrl/agents/pgt.py:147: in update
    [elem['state'] for elem in experiences], self.xp, self.phi)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

.0 = <list_iterator object at 0x7f734a422630>

>   [elem['state'] for elem in experiences], self.xp, self.phi)
E   TypeError: list indices must be integers or slices, not str

chainerrl/agents/pgt.py:147: TypeError
======================================================================================================================== 1 failed in 0.63 seconds ========================================================================================================================

After this PR:

pytest -xsv --disable-pytest-warnings tests/agents_tests/test_pgt.py
========================================================================================================================== test session starts ===========================================================================================================================
platform linux -- Python 3.5.2, pytest-3.8.2, py-1.6.0, pluggy-0.7.1 -- /home/fujita/.local/share/virtualenvs/chainerrl_20180827-9TPAa4mI/bin/python3
cachedir: .pytest_cache
rootdir: /home/fujita/chainerrl_20180827, inifile:
collected 4 items

tests/agents_tests/test_pgt.py::TestPGTOnContinuousABC::test_training_cpu <- tests/agents_tests/basetest_training.py Load agent from /tmp/tmp_r05mv78/agent_final
PASSED
tests/agents_tests/test_pgt.py::TestPGTOnContinuousABC::test_training_cpu_fast <- tests/agents_tests/basetest_training.py Load agent from /tmp/tmpezwy7jld/agent_final
PASSED
tests/agents_tests/test_pgt.py::TestPGTOnContinuousABC::test_training_gpu <- tests/agents_tests/basetest_training.py Load agent from /tmp/tmp1psg3ih0/agent_final
PASSED
tests/agents_tests/test_pgt.py::TestPGTOnContinuousABC::test_training_gpu_fast <- tests/agents_tests/basetest_training.py Load agent from /tmp/tmpo486ebh2/agent_final
PASSED

================================================================================================================== 4 passed, 8 warnings in 6.73 seconds ==================================================================================================================

[muupan]

imosci test this please

[ninetan]

imosCI successfully created a job for commit 62f48ef:

• Dashboard for commit 62f48ef

[prabhatnagarajan]

LGTM