Training and eval env are not of the same type

[Gieted]

When executing docker-compose exec app python3 train.py
I'm getting a warning message:

/home/selfplay/.local/lib/python3.6/site-packages/stable_baselines/common/callbacks.py:287: UserWarning: Training and eval env are not of the same type<SelfPlayEnv instance> != <stable_baselines.common.vec_env.dummy_vec_env.DummyVecEnv object at 0x7f1db36e9c50>
  "{} != {}".format(self.training_env, self.eval_env))

Besides that the training works fine. Is it something I should worry about, or it's normal, and I should ignore it?

[davidADSP]

Nah that's normal - should still run fine!

[Sylvain2903]

In my Custom Environment, after some training, the Episode reward mean varies between 0.15 and 0.4, whilst the evaluation average reward is usually around -0.2. For this reason my model never updates as it seems to fail the evaluation every time despite clearly learning how to play the game. Does this have to do with this warning? How can I fix it?