Questions about scheduler update

[toooooodo]

Hi，thanks for releasing this great work. I'm a little confused about equation 8
$$\phi^{(k+1)} \leftarrow \phi^{(k)}-\gamma \nabla_{\phi^{(k)}} \log P\left(\mathbf{W}^{(k)} ; \phi^{(k)}\right)\left(\frac{1}{N_{v}} \sum_{i=1}^{N_{v}} R_{i}^{(k)}-b\right)$$
The reward $R$ is calculated from the updated temporal model $\tilde{\theta}_{0}^{(k+1)}$ on $N_v$ validation tasks, I don't quite understand why this can be used to update the scheduler as part of the loss, it seems that the scheduler is not involved in the calculation of the reward $R$?