Add L-BFGS optimizer

[jppgks]

No description provided.

[github-actions]

Unit Test Results

         6 files  +    1         6 suites  +1   2h 45m 57s ⏱️ + 21m 37s
  3 382 tests +    1  3 304 ✔️ +  2    78 💤 ±  0  0 ❌  - 1 
10 146 runs  +119  9 888 ✔️ +98  258 💤 +22  0 ❌  - 1 

Results for commit d66b5c9. ± Comparison against base commit 37add66.

[dantreiman]

Looks good to me. Neat!

[tgaddair]

@jppgks we should add tests for both local and ray backends. When using the Ray backend, it should work when specifying num_workers=1, but raise an exception if num_workers>1 due to the Horovod limitation we discussed.

Based on the ray the code is written currently, I expect it will fail when calling hvd.broadcast_optimizer_state, which is what is not supported by Horovod (potentially among other things).