Gradient Descent Optimization

Gradient Descent Optimization (GDO) is a good method for improving a single parameter, once your estimate is close to the truth. Although our implementation supports multi-variate tuning, in practice we find it doesn't perform reliably for more than one variable, so we recommend just using this to tune a single variable. Also, GDO only converges to the minimum within the local trough that you start in, so it isn't good at finding the global minimum unless you start it in the same trough as the global minimum.

Configuration Data

This is the configuration data for the GDO, defined in the GunnsOptimGradientDescentConfigData class in gunns/core/optimization/GunnsOptimGradientDescent.hh.

• mNumVars: (must be > 0) this is the number of monte carlo variables, the model variables that are to be optimized. This should match the number of monte carlo variables you define in the simulation input file. Unlike the PSO config data, which doesn't need to know this, this GDO config needs this so that it can allocate memory sized to this number of variables.
• mMaxEpoch: (must be > 0) similar to the PSO's term, this is the number of iterations of the GDO over the entire monte carlo set.
• mPropagationGain: this scales the amount of propagation of the states along their gradient. Higher values make the state approach the minimum faster, but lead to more overshoot. The GDO, once it reaches the local minimum, will tend to oscillate back and forth across the minimum, and never actually converge on it. This 'standoff' grows with higher values this propagation gain. So although smaller values of the propagation gain takes longer to find the local minimum, it also leads to more accurate result, so there is a tradeoff.
• mInitialState: (must not be null) this is a pointer to the array of initial values for the monte carlo variables. This is how you specify the initial state to start propagating from. The GDO will find the local minimum of the trough that this initial state is in.

Outputs

The GDO outputs 3 files to the simulation folder:

• grad_state.csv: this is the best solution state from the most recent epoch, including the monte carlo variable states and cost function result. This is the optimized result for your model. This file is comma-delimited.
• grad_gradients_history.csv: this lists the state and cost of all the monte carlo gradients, for all epochs. This can be used to see the gradients and how they trend over the epochs. This file is comma-delimited.
• grad_cost_history.csv: this lists the global best cost for each epoch. This can be used to quickly verify how well the GDO is converging towards zero cost.

Recommendations

• Limit this use to only one or two monte carlo variables. In practice we've not had much success trying to optimize more than that.
• Use small values of the propagation gain if you can, because of the overshoot problem described above.
• For most model optimization problems, the PSO is the better option.