[Diff since v4.1.1](v4.1.1...v4.1.2)

**Merged pull requests:**
- Fix nested bullet list in optimization.md (#881) (@abhro)
- Make DiffEqFlux test more robust (#887) (@ChrisRackauckas)

**Closed issues:**
- Optimization algorithm (Adam) not found (#882)

[Diff since v4.1.0](v4.1.0...v4.1.1)

**Merged pull requests:**
- Fix typo (#878) (@dreivmeister)
- Check more retcodes from NLopt (#879) (@sumiya11)
- Bump version (#880) (@sumiya11)

**Closed issues:**
- Incorrect return value classification of FTOL_REACHED from NLopt (#862)
- Running optimizations with maps as input (#875)

[Diff since v4.0.5](v4.0.5...v4.1.0)

**Merged pull requests:**
- MOI Constraint jacobian-vector fixes (#845) (@aml5600)
- Bump codecov/codecov-action from 4 to 5 (#856) (@dependabot[bot])
- More conditionals for not calculating hessian in NLopt (#857) (@Vaibhavdixit02)
- chore: bump Optimisers (#861) (@avik-pal)
- Update OptimizationOptimisers.jl (#869) (@lsmohan)
- fix #870 (#871) (@baggepinnen)
- Update runtests.jl in OptimizationOptimisers (#872) (@Vaibhavdixit02)

**Closed issues:**
- lbfgsb issue  (#855)
- Error in trying to use Optimization.jl for LSTM training based on Lux.jl (#860)
- Better error message when gradient is required but not available (#865)
- `maxtime` errors for Evolutionary.jl (#870)
- Expose step size parameters for AutoFiniteDiff (#874)

[Diff since v4.0.4](v4.0.4...v4.0.5)

**Merged pull requests:**
- Update `_check_and_convert_maxiters` to ensure rounding (#849) (@Vaibhavdixit02)

**Closed issues:**
- Should `maxiters` and `epochs` be disambiguated? (#842)
- iterating through a DataLoader is still broken (#843)
- `epochs` calculation should have a floor or ceil to avoid this (#848)

[Diff since v4.0.3](v4.0.3...v4.0.4)

**Merged pull requests:**
- fix: match iterations count in Sophia (#841) (@avik-pal)
- Update links to algorithms in optim.md (#846) (@AbdAlazezAhmed)
- MOI vector lambda and iteration fixes in Optimisers (#847) (@Vaibhavdixit02)

[Diff since v4.0.2](v4.0.2...v4.0.3)

**Merged pull requests:**
- feat: make MLUtils into a weakdep & suppport MLDataDevices (#827) (@avik-pal)
- fix: no printing "dkjht" (#828) (@avik-pal)
- Optimisers iteration count should be calculated with epochs and data both (#830) (@Vaibhavdixit02)
- NLopt: Reuse constraint evaluations (#832) (@Vaibhavdixit02)
- Update index.md (#836) (@Vaibhavdixit02)
- Update optimisers extensions and tests (#838) (@Vaibhavdixit02)

**Closed issues:**
- Multi objective optimization (#18)
- `PolyOpt` only accept functions without any extra inputs (#728)
- Disassociate batching from the `data` arg to `solve` and remove `data` (#776)
- Move Sophia from OptimizationOptimisers to `src/` (#783)
- Support for `MOI.eval_constraint_jacobian_transpose_product` (#808)
- docs: index.md: broken html after v3.26 (#834)
- OptimizationFunction cannot return multiple values (#839)

[Diff since v4.0.1](v4.0.1...v4.0.2)

[Diff since v4.0.0](v4.0.0...v4.0.1)

[Diff since v3.28.0](v3.28.0...v4.0.0)

**Merged pull requests:**
- [WIP] Updates for OptimizationBase v2 (#789) (@Vaibhavdixit02)
- Add constraints support for NLopt (#799) (@Vaibhavdixit02)
- Add constraints support for NOMAD (#817) (@Vaibhavdixit02)
- Bump versions (#818) (@Vaibhavdixit02)
- docs updates (#819) (@Vaibhavdixit02)

[Diff since v3.27.0](v3.27.0...v3.28.0)

**Merged pull requests:**
- CompatHelper: bump compat for OptimizationPRIMA to 0.2 for package docs, (keep existing compat) (#786) (@github-actions[bot])
- Updating solver __solve function for MOO (#787) (@ParasPuneetSingh)
- New tutorials and docs (#788) (@Vaibhavdixit02)
- Add a conversion mechanism between NLPModel to OptimizationProblem (#792) (@alonsoC1s)
- CompatHelper: add new compat entry for SymbolicAnalysis at version 0.2 for package docs, (keep existing compat) (#794) (@github-actions[bot])
- CompatHelper: add new compat entry for Symbolics at version 6 for package docs, (keep existing compat) (#796) (@github-actions[bot])
- CompatHelper: bump compat for Symbolics to 6 for package OptimizationMOI, (keep existing compat) (#797) (@github-actions[bot])
- CompatHelper: add new compat entry for SymbolicAnalysis at version 0.3 for package docs, (keep existing compat) (#800) (@github-actions[bot])
- Add remake docs (#801) (@Vaibhavdixit02)
- CompatHelper: add new compat entry for NLPModelsTest at version 0.10 for package docs, (keep existing compat) (#803) (@github-actions[bot])
- CompatHelper: add new compat entry for NLPModels at version 0.21 for package docs, (keep existing compat) (#804) (@github-actions[bot])
- Update __loss function for MOO using OptimizationMetaheuristics.jl (#805) (@ParasPuneetSingh)
- refactor: remove dependency on Pkg (#807) (@SebastianM-C)
- Added precompilation for nonnegative least squares (#809) (@arismavridis)
- call cons_vjp if available (#811) (@baggepinnen)

**Closed issues:**
- Add an abstraction for chaining optimizers  (#78)
- Maxiters not respected (#335)
- Segfault with NLopt (#344)
- Parallelize local optimizations with MultistartOptimization (#377)
- Add more polyalgorithms such as mixture of global (BBO/CMAES/...) and first order and second order optimizers (#523)
- Downstream Compat bumps (#669)
- Adding a thin wrapper to NLPModels (#790)
- maxtime documentation doesn't mention units (seconds) (#795)
- Optimization.LBFGS fails with MethodError: no method matching take(::Base.Iterators.Cycle{Tuple{OptimizationBase.NullData}}, ::Nothing) (#798)