Correct line search to search over gradient

[DevonMorris]

Previously, the line search operated directly over parameter space. This caused issues when there were different types for parameters vs gradients e.g. Lie Groups. This commit changes line searches to operate over gradient parameterization and incorporates gradient updates to parameters via ArgminScaledAdd.

I'm not sure sure what level of testing is needed for this but it passes all the unit tests with cargo test and I also ran the 3 line search examples and the steepest descent example

[stefan-k]

Thanks a lot for this contribution! This is a case I didn't consider. I have only a minor request in the example you provided (which is actually my mistake).

Another thing that I would appreciate if you could do is to change P in the definition of the LineSearch trait to D (here). This is obviously purely cosmetic but I think it would make it less tempting to use P again.

Edit: I don't think that there is additional testing needed.

[codecov-commenter]

Codecov Report

Patch coverage: 100.00% and no project coverage change.

Comparison is base (2457232) 93.35% compared to head (a801c6b) 93.35%.

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #354   +/-   ##
=======================================
  Coverage   93.35%   93.35%           
=======================================
  Files         117      117           
  Lines       18803    18803           
=======================================
  Hits        17554    17554           
  Misses       1249     1249           

Impacted Files	Coverage Δ
...gmin/src/solver/gradientdescent/steepestdescent.rs	95.04% <ø> (ø)
argmin/src/solver/linesearch/backtracking.rs	99.05% <100.00%> (ø)
argmin/src/solver/linesearch/hagerzhang.rs	73.89% <100.00%> (ø)
argmin/src/solver/linesearch/morethuente.rs	86.60% <100.00%> (ø)

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

[stefan-k]

Thanks! There is only one minor comment. Once this is fixed this is ready to be squashed and merged :)

[stefan-k]

Thanks for also fixing the docs. There is only one small suggestions:

Suggested change

	/// // `D` is the type of the search direction, typically the same as the parameter vector
	/// // `D` is the type of the search direction

[DevonMorris]

Should be resolved now. I also squashed all the fixup commits.

[stefan-k]

Thanks! :)