+
+
+
+
+Multivariate Calculus
+Let a function \(f\) be defined as \(f:\R^d\to\R\). This will be used to reference each multivariate function described in this article.
+Lines
+
+- A line in \(\R^d\) is a subset of \(\R^d\)
+- A line through point \(u\in\R^d\) along the vector \(x\in\R^d\) is given by \(\{x\in\R^d: x= u +\alpha v, \alpha\in\R\}\)
+- A line through two points \(u,u'\in\R^d\) is given by \(\{x\in\R^d:x=u+\alpha(u-u'), \alpha\in\R\}\)
+
+Hyper Plane
+
+- A hyperplane of \(d-1\) dimensions \(\subseteq R^d\).
+- A hyperplane perpendicular to a vector \(w\in\R^d\) with a value \(b\in\R\) is given by \(\{x\in\R^d:w^Tx=b\} = \{x\in\R^d:\sum_{i=1}^dw_ix_i=b\}\)
+
+Partial Derivatives
+The partial derivative of \(f\) is defined as the derivative of \(f\) with respect to one of the variables, keeping the other variables constant, i.e.,
+$$
+\cfrac{\partial f}{\partial x_i}(v) = lim_{\alpha\to{0}}\cfrac{f(v+\alpha e_i)-f(v)}{\alpha}
+$$
+Here \(e_i\) is the \(i^{th}\) unit vector in \(\R^d\)
+Let \(f:\R^d\to\R\) and \(f=1x_1+2x_2+\dots nx_n\)
+Then the partial derivative of \(f\)
+- with respect to the variable \(x_1\) is
+$$
+\cfrac{\partial f}{\partial x_1} = 1
+$$
+- with respect to the variable \(x_2\) is
+$$
+\cfrac{\partial f}{\partial x_2} = 2
+\\ \\vdots
+$$
+and so on
+Gradients
+Let \(f:\R^d\to\R\) be defined.
+Then \(\cfrac{\partial f}{\partial x} = \begin{bmatrix}\cfrac{\partial f}{\partial x_1},\cfrac{\partial f}{\partial x_2}, \cfrac{\partial f}{\partial x_3}, \cfrac{\partial f}{\partial x_4}, \cfrac{\partial f}{\partial x_5}, \dots, \cfrac{\partial f}{\partial x_d}, \end{bmatrix}\)
+Hence the gradient is equal to \([\cfrac{\partial f}{\partial x}]^T\) , i.e.,
+$$
+\cfrac{\triangledown f}{\triangledown x} = \begin{bmatrix}
+\cfrac{\partial f}{\partial x_1}\\ \
+\cfrac{\partial f}{\partial x_2}\ \ \
+\vdots\\ \\
+\cfrac{\partial f}{\partial x_n}
+\end{bmatrix}
+$$
+Linear Approximations and Gradients
+Let \(f\) be a function defined from \(\R^d\to\R\)
+Then the linear approximation of \(f\) at a vector \(v\in\R^d\) is given by,
+\[
+L_v[f](x)= f(x) = f(v) + \triangledown f(v)^T(x-v)
+\]
+Gradients and Tangent Planes
+The graph of \(L_v[f]\) is a plane that is tangent to the graph of \(f\) at the point \((v, f(v))\)
+Gradients and Contours
+The gradient of \(f\) evaluated at \(v\) is \(\perp\) to the level set of \(f.\) Mathematically,
+\[
+\triangledown f(v) \perp \{x\in\R^d:f(x)=f(v)\}
+\]
+Proof
+$$
+{x\in\R^d:f(x)=f(v)}\newline
+\ \newline
+\rightarrow{x\in\R^d:L_vf=f(v)}\newline
+\ \newline
+\rightarrow{x\in\R^d:f(v)+\triangledown f(v)^T(x-v)=f(v)}\newline
+\ \newline
+\rightarrow {x\in\R^d:\triangledown f(v)^Tx=\triangledown f(v)^Tv}\newline
+\ \newline
+\text{Comparing with } {x\in\R^d:W^Tx=b}, \text{where W is }\perp\text{to the plane}\newline\ \newline
+\therefore \triangledown f(v) \perp {x\in\R^d:L_vf=f(v)}
+$$
+Directional Derivative
+The directional derivative of the function \(f\) at the point \(v\) along the direction of \(u\) is given by,
+\[
+D_u[f](v) = lim_{\alpha\to 0} \cfrac{f(v+\alpha u)-f(v)}{\alpha}\newline\ \newline=lim_{\alpha\to 0} \cfrac{L_v[f](v)-f(v)}{\alpha}\newline\ \newline=lim_{\alpha\to 0} \cfrac{f(v)+\triangledown f(v)^T(\alpha u)-f(v)}{\alpha}\newline\ \newline=\triangledown f(v)^Tu
+\]
+Cauchy-Schwarz Inequality
+Let \(a,b\in\R^d\). Then the equality states that
+\[
+-||a||.||b|| \leq a^Tb \leq ||a||.||b||
+\]
+Points to Note
+
+- \(-||a||.||b|| = a^Tb\), if and only if, \(a=\alpha b\) and \(a<0\)
+- \(||a||.||b|| = a^Tb\), if and only if, \(a=\alpha b\) and \(a>0\)
+
+Direction of steepest ascent
+For a function \(f\), the direction of a unit vector \(u\) such that it maximises the directional derivative of \(f\) along \(u\) is \(u=\triangledown f(v)\), i.e., for steepest ascent \(u\) should be in direction of \(\triangledown f\)
+Direction of steepest descent
+For a function \(f\), the direction of a unit vector \(u\) such that it minimises the directional derivative of \(f\) along \(u\) is \(u=-\triangledown f(v)\), i.e., for steepest descent \(u\) should be in direction opposite to \(\triangledown f\)
+
+
+
+
+
+
+
+
+
+
+
+
+
+