Policy loss is going up... what parameters should I look to adjust?

[theforgottenmachine]

I couldn't find an accurate answer in the docs.

This is what my tensorboard looks like. Any ideas on what I can try to adjust?

I'm using these params:
max_steps: 5.0e4
batch_size: 128
buffer_size: 2048
beta: 1.0e-2
hidden_units: 256
summary_freq: 2000
time_horizon: 64
num_layers: 2

[MarcoMeter]

Hi @theforgottenmachine
the perfect loss would be equal to 0. So in your case it is a desired behavior that the loss is going up as it is getting closer to 0.

I don't know what your environment looks like, but from my experience I doubt that 25k learning steps are sufficient for a neural net with two layers of 256 nodes. Keep the training running for much longer.

[xiaomaogy]

Hi @theforgottenmachine , we are closing this issue due to inactivity, feel free to open it if you need more help on this.

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.