disable env.reset() after every episode

[zyzhang1130]

Hi,
May I check if I would like to keep the environment as it is after each training episode, should I just comment line line 147 in main.py or should I also comment line 130? Besides what am I supposed to do if I just want to reset the agent's position but keep the environment as it is after each training episode?

Thank you.

[Kaixhin]

Commenting out line 147 would prevent the environment resetting during training. Commenting out line 130 would prevent the environment resetting during collection of data for validating Q-values. It seems that you might need to write a different environment and use a different set of functions if you want more control.

[zyzhang1130]

In general validating Q-values and training should be consistent right (if one is reset another should also)

[Kaixhin]

Yes it makes sense to keep them consistent.

[zyzhang1130]

when I commented the reset at line 130 as well, it gave me this error:
AttributeError: 'NoneType' object has no attribute 'metadata'

when it ran line 132:
next_state, _, done = env.step(np.random.randint(0, action_space))

Does this line just let agent make a random movement and what could be the possible reasons for the error?

[Kaixhin]

That line uses random actions to collect data for validating Q-values. I'm not sure why your edit is causing the error, so you will need to try and debug it yourself.