Adds env_stats to record_tb_stats

[prabhatnagarajan]

Currently, environment statistics are not being incorporated into the tensorboard output.

I modified train_dqn_ale.py by adding a dummy wrapper:

import gym
class ComputeSuccessRate(gym.Wrapper):
    def __init__(self, env):
        super().__init__(env)
        self.success_record = []

    def reset(self):
        return self.env.reset()

    def step(self, action):
        obs, r, done, info = self.env.step(action)
        return obs, r, done, info

    def get_statistics(self):
        return [("success_rate", 0.75)]

    def clear_statistics(self):
        self.success_record = []

The wrapper itself is useless, but it returns environment statistics. I've confirmed they appear in tensorboard when I run an experiment withuse_tensorboard:

[prabhatnagarajan]

/test

[pfn-ci-bot]

Successfully created a job for commit 79a512e:

• Dashboard for commit 79a512e

[prabhatnagarajan]

/test

[pfn-ci-bot]

Successfully created a job for commit f57b3d2:

• Dashboard for commit f57b3d2

[justindujardin]

I would love to use this functionality for recording custom metrics. The changes look good to me!

[prabhatnagarajan]

Thanks! Glad that you find this change useful. Currently these custom statistics should actually be available in scores.txt so you should already be able to use them. Though this PR makes it visible in Tensorboard.