Add optimization notes and src files: convex, global, local, and constrained optimization

[Copilot]

Adds four new optimization topics under notes/8_optimization/ and src/8_optimization/, following the existing project structure (notes, implementation, tests, example notebooks).

New notes

• convex_optimization.md — Convex sets/functions, Jensen's inequality, KKT conditions, active-set QP algorithm, worked QP example, convex vs non-convex comparison
• global_optimization.md — Grid search (curse of dimensionality), simulated annealing, genetic algorithms, worked Rastrigin example comparing global vs local methods
• local_optimization.md — Gradient descent, Newton's method, BFGS with convergence rates, worked Rosenbrock example from multiple initial guesses
• constrained_optimization.md — LP formulation, Lagrangian/KKT theory, simplex method, worked resource allocation LP and equality-constrained Lagrange multiplier example

New src files

Each topic follows the existing implementation/tests/examples directory pattern:

• convex_optimization/ — quadratic_programming(), is_convex_function() implementations, 10 tests, example notebook with QP solving and convex vs non-convex visualization
• global_optimization/ — grid_search(), simulated_annealing(), genetic_algorithm() implementations, 12 tests, example notebook with Rastrigin function comparison
• local_optimization/ — gradient_descent_optimize(), newtons_method_optimize(), bfgs() implementations, 14 tests, example notebook with Rosenbrock convergence paths from multiple starting points
• constrained_optimization/ — linear_programming() (simplex method), lagrange_multiplier_minimize() implementations, 10 tests, example notebook with feasible region plot and Lagrange multiplier visualization

README

• Added Optimization section with links to all four notes, implementations, and example notebooks

Original prompt

---

This section details on the original issue you should resolve

<issue_title>todo</issue_title>
<issue_description>Convex Optimization

• Define convex functions & sets
• Solve a quadratic programming example
• Visualize convex vs non-convex objective functions

Global Optimization*

• Demonstrate brute-force/grid search
• Use stochastic/global methods (e.g., simulated annealing, genetic algorithms)
• Compare global vs local solutions on a multimodal function

Local Optimization

• Gradient descent on a simple function (quadratic, Rosenbrock)
• Newton’s method / quasi-Newton (BFGS)
• Visualize convergence paths starting from different initial guesses

Constrained Optimization

• Linear programming example (diet problem, resource allocation)
• Nonlinear constraints (inequalities/equalities) with Lagrange multipliers
• Use solver (e.g., scipy.optimize.minimize with constraints)
• Plot feasible region and optimum</issue_description>

<agent_instructions>add new notes in style of previous notes</agent_instructions>

Comments on the Issue (you are @copilot in this section)

• Fixes todo #17

---

🔒 GitHub Advanced Security automatically protects Copilot coding agent pull requests. You can protect all pull requests by enabling Advanced Security for your repositories. Learn more about Advanced Security.

[djeada]

@copilot add respective src files same as for other projects

[Copilot]

@copilot add respective src files same as for other projects

Added src/8_optimization/ with the same structure as other projects (implementation + tests + examples) in 9826303:

• convex_optimization/ — quadratic_programming(), is_convex_function() + 10 tests + notebook with QP example and convex vs non-convex visualization
• global_optimization/ — grid_search(), simulated_annealing(), genetic_algorithm() + 12 tests + notebook with Rastrigin function comparison
• local_optimization/ — gradient_descent_optimize(), newtons_method_optimize(), bfgs() + 14 tests + notebook with Rosenbrock convergence paths
• constrained_optimization/ — linear_programming() (simplex), lagrange_multiplier_minimize() + 10 tests + notebook with feasible region plot

All 46 new tests pass alongside the existing 550. README updated with implementation and example links.