Improve quadratic duality in doc

docs/src/background/duality.md

@@ -124,12 +124,55 @@ details.
 
 ## Dual for problems with quadratic functions
 
-Given a problem with quadratic functions:
+### Quadratic Programs (QPs)
+
+For quadratic programs with only affine conic constraints,
 ```math
 \begin{align*}
 & \min_{x \in \mathbb{R}^n} & \frac{1}{2}x^TQ_0x + a_0^T x + b_0
 \\
-& \;\;\text{s.t.} & \frac{1}{2}x^TQ_ix + a_i^T x + b_i & \in \mathcal{C}_i & i = 1 \ldots m
+& \;\;\text{s.t.} & A_i x + b_i & \in \mathcal{C}_i & i = 1 \ldots m.
+\end{align*}
+```
+with cones ``\mathcal{C}_i \subseteq \mathbb{R}^{m_i}`` for ``i = 1, \ldots, m``, consider the Lagrangian function
+```math
+L(x, y) = \frac{1}{2}x^TQ_0x + a_0^T x + b_0 - \sum_{i = 1}^m y_i^T (A_i x + b_i).
+```
+Let ``z(y)`` denote ``\sum_{i = 1}^m A_i^T y_i - a_0``, the Lagrangian can be rewritten as
+```math
+L(x, y) = \frac{1}{2}{x}^TQ_0x - z(y)^T x + b_0 - \sum_{i = 1}^m y_i^T b_i.
+```
+
+The condition ``\nabla_x L(x, y) = 0`` gives
+```math
+0 = \nabla_x L(x, y) = Q_0x + a_0 - \sum_{i = 1}^m y_i^T b_i
+```
+which gives ``Q_0x = z(y)``.
+This allows to obtain that
+```math
+\min_{x \in \mathbb{R}^n} L(x, y) = -\frac{1}{2}x^TQ_0x + b_0 - \sum_{i = 1}^m y_i^T b_i
+```
+so the dual problem is
+```math
+\max_{y_i \in \mathcal{C}_i^*} \min_{x \in \mathbb{R}^n} -\frac{1}{2}x^TQ_0x + b_0 - \sum_{i = 1}^m y_i^T b_i.
+```
+If ``Q_0`` is invertible, we have ``x = Q_0^{-1}z(y)`` hence
+```math
+\min_{x \in \mathbb{R}^n} L(x, y) = -\frac{1}{2}z(y)^TQ_0^{-1}z(y) + b_0 - \sum_{i = 1}^m y_i^T b_i
+```
+so the dual problem is
+```math
+\max_{y_i \in \mathcal{C}_i^*} -\frac{1}{2}z(y)^TQ_0^{-1}z(y) + b_0 - \sum_{i = 1}^m y_i^T b_i.
+```
+
+### Quadratically Constrained Quadratic Programs (QCQPs)
+
+Given a problem with both quadratic function and quadratic objectives:
+```math
+\begin{align*}
+& \min_{x \in \mathbb{R}^n} & \frac{1}{2}x^TQ_0x + a_0^T x + b_0
+\\
+& \;\;\text{s.t.} & \frac{1}{2}x^TQ_ix + a_i^T x + b_i & \in \mathcal{C}_i & i = 1 \ldots m.
 \end{align*}
 ```
 with cones ``\mathcal{C}_i \subseteq \mathbb{R}`` for ``i = 1 \ldots m``, consider the Lagrangian function
@@ -149,35 +192,14 @@ A pair of primal-dual variables $(x^\star, y^\star)$ is optimal if
   ```math
   \max_{y_i \in \mathcal{C}_i^*} L(x^\star, y).
   ```
-  That is, for all ``i = 1, \ldots, m``, ``\frac{1}{2}x^TQ_ix + a_i^T x + b_i`` is
-  either zero or in the normal cone of ``\mathcal{C}_i^*`` at ``y^\star``.
-  For instance, if ``\mathcal{C}_i`` is ``\{ x \in \mathbb{R} : x \le 0 \}``, this means that
-  if ``\frac{1}{2}x^TQ_ix + a_i^T x + b_i`` is nonzero at ``x^\star`` then ``y_i^\star = 0``.
-  This is the classical complementary slackness condition.
+  That is, for all ``i = 1, \ldots, m``, ``\frac{1}{2}x^TQ_ix + a_i^T x + b_i``
+  is either zero or in the
+  [normal cone](https://en.wikipedia.org/wiki/Normal_cone) of
+  ``\mathcal{C}_i^*`` at ``y^\star``. For instance, if ``\mathcal{C}_i`` is
+  ``\{ z \in \mathbb{R} : z \le 0 \}``, this means that if
+  ``\frac{1}{2}x^TQ_ix + a_i^T x + b_i`` is nonzero at ``x^\star`` then
+  ``y_i^\star = 0``. This is the classical complementary slackness condition.
 
 If ``\mathcal{C}_i`` is a vector set, the discussion remains valid with
 ``y_i(\frac{1}{2}x^TQ_ix + a_i^T x + b_i)`` replaced with the scalar product
 between ``y_i`` and the vector of scalar-valued quadratic functions.
-
-!!! note
-    For quadratic programs with only affine constraints, the optimality condition
-    ``\nabla_x L(x, y^\star) = 0`` can be simplified as follows:
-    ```math
-    0 = \nabla_x L(x, y^\star) = Q_0x + a_0 - \sum_{i = 1}^m y_i^\star a_i
-    ```
-    which gives
-    ```math
-    Q_0x = \sum_{i = 1}^m y_i^\star a_i - a_0 .
-    ```
-    The Lagrangian function
-    ```math
-    L(x, y) = \frac{1}{2}x^TQ_0x + a_0^T x + b_0 - \sum_{i = 1}^m y_i (a_i^T x + b_i)
-    ```
-    can be rewritten as
-    ```math
-    L(x, y) = \frac{1}{2}x^TQ_0x - (\sum_{i = 1}^m y_i a_i^T - a_0^T) x + b_0 - \sum_{i = 1}^m y_i (a_i^T x + b_i)
-    ```
-    which, using the optimality condition ``\nabla_x L(x, y^\star) = 0``, can be simplified as
-    ```math
-    L(x, y) = -\frac{1}{2}x^TQ_0x + b_0 - \sum_{i = 1}^m y_i (a_i^T x + b_i)
-    ```