L‐BFGS‐B Optimizer Parameter Descriptions

https://lbfgspp.statr.me/doc/classLBFGSpp_1_1LBFGSBParam.html

• m (Number of Corrections)

  • Controls the number of past iterations used to approximate the inverse Hessian matrix. Larger values improve convergence but increase memory and computation time. Don'd drop below 6, but above 10-15 will result in excessive computing time
  • Default: 6

• epsilon (Absolute Tolerance)

  • Absolute threshold for convergence. The algorithm stops when the projected gradient norm is smaller than this value. ||𝑃𝑔||∞<max{𝜖𝑎𝑏𝑠,𝜖𝑟𝑒𝑙||𝑥||}, where ||𝑥|| denotes the Euclidean (L2) norm of 𝑥, and 𝑃𝑔=𝑃(𝑥−𝑔,𝑙,𝑢)−𝑥 . For higher precision we could move to (e.g., 1e-7)
  • Default: 1e-5

• epsilon_rel (Relative Tolerance)

  • Relative threshold for convergence. If your objective function values are small, prioritize epsilon_rel over epsilon. See above Formula
  • Default: 1e-5

• delta (Delta for Convergence Test)

  • Threshold for the relative change in the objective function over past iterations. The algorithm stops when the following condition is met, |𝑓𝑘−𝑑(𝑥)−𝑓𝑘(𝑥)|<𝛿⋅max(1,|𝑓𝑘(𝑥)|,|𝑓𝑘−𝑑(𝑥)|), where 𝑓𝑘(𝑥) is the current function value, and 𝑓𝑘−𝑑(𝑥) is the function value 𝑑 iterations ago (specified by the [past]
  • Default: 1e-10

• past (Distance for Delta-Based Convergence Test)

  • This parameter determines the distance 𝑑 to compute the rate of decrease of the objective function, 𝑓𝑘−𝑑(𝑥)−𝑓𝑘(𝑥), where 𝑘 is the current iteration step. If the value of this parameter is zero, the delta-based convergence test will not be performed.
  • Default: 1

• max_iterations (Maximum Iterations)

  • Maximum number of iterations allowed. Set to 0 for no limit.
  • Default: 0
  • Right now I believe we set quite a low number, might want to see how much performaance decreases with 0 (or 1000+)

• max_submin (Maximum Subspace Minimization Iterations)

  • Maximum iterations for the subspace minimization routine. Increase if the algorithm struggles to converge
  • Default: 10

• max_linesearch (Maximum Line Search Trials)

  • Maximum number of line search trials per iteration. If we see lie search failing, might want to increase (we do!)
  • Default: 20 .

• ftol (Line Search Accuracy)

  • Controls the accuracy of the line search. Must be between 0 and 0.5.
  • Default: 1e-4

• wolfe (Wolfe Condition Coefficient)

  • The coefficient for the Wolfe condition. This parameter is valid only when the line-search algorithm is used with the Wolfe condition. The default value is 0.9. This parameter should be greater the ftol parameter and smaller than 1.0. Use a smaller value (e.g., 0.8) if the line search is too aggressive.
  • Default: 0.9

• min_step (Minimum Step Length)

  • Minimum step length allowed in the line search. Keep at the default as per instructions
  • Default: 1e-20

• max_step (Maximum Step Length)

  • Maximum step length allowed in the line search. Keep at the default as per instructions
  • Default: 1e+20