Generating LQR Linear Control Function

[alre5639]

More of a clarification than an Issue...
Page 4 of the paper states that we initialize u to the LQR solution to help the NN get in the neighborhood of a reasonable linear control function. I am only familiar with LQR's in which we formulate a cost function in the form of the Riccati equation, and at each time step optimize for the lowest cost control(s). How does one generate the a linear control function from a traditional LQR optimization solution? Thank you