68_pairwise_ops.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### [Pairwise Ops](https://scikit-learn.org/stable/modules/metrics.html)\n",
    "\n",
    "- **Distance** are functions `d(a,b)` such that `d(a,b)<d(a,c)` if a & b are \"more similar\" than a & c. Two identical objects have a zero distance. One of the most common examples is Euclidean distance. \n",
    "\n",
    "- **Kernels** are measures of similarity. `s(a,b)`>`s(a,c)` if a & b are more similar than a & c. Kernels must be [positive semi-definite](https://mathworld.wolfram.com/PositiveSemidefiniteMatrix.html).\n",
    "\n",
    "- There are multiple ways to convert between distance metrics & similarity measures such as kernels. Let $D$ = distance & $S$ = kernel.\n",
    "\n",
    "    - `S=np.exp(-D*gamma)`; one way to choose gamma is 1/num_features.\n",
    "    - `S=1/D(/np.max(D))`\n",
    "    \n",
    "    \n",
    "- `pairwise_distances` measures the row vectors of X & Y. If Y is omitted the pairwise distances of the row vectors of X are calculated. \n",
    "\n",
    "- `pairwise_kernels` calculates the kernel between X and Y using different kernel functions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[ 4.  2.]\n",
      " [ 7.  5.]\n",
      " [12. 10.]] \n",
      "\n",
      "[[0. 3. 8.]\n",
      " [3. 0. 5.]\n",
      " [8. 5. 0.]] \n",
      "\n",
      "[[ 2.  7.]\n",
      " [ 3. 11.]\n",
      " [ 5. 18.]] \n",
      "\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "from sklearn.metrics import pairwise_distances\n",
    "from sklearn.metrics.pairwise import pairwise_kernels\n",
    "\n",
    "X = np.array([[2, 3], [3, 5], [5, 8]])\n",
    "Y = np.array([[1, 0], [2, 1]])\n",
    "\n",
    "print(pairwise_distances(X, \n",
    "                         Y, \n",
    "                         metric='manhattan'),\"\\n\")\n",
    "print(pairwise_distances(X, \n",
    "                         metric='manhattan'),\"\\n\")\n",
    "print(pairwise_kernels(X, \n",
    "                       Y, \n",
    "                       metric='linear'),\"\\n\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### [Cosine Similarity](https://scikit-learn.org/stable/modules/generated/sklearn.metrics.pairwise.cosine_similarity.html#sklearn.metrics.pairwise.cosine_similarity)\n",
    "\n",
    "- `Cosine Similarity` finds the *L2-normalized dot product* of vectors. Euclidean L2 normalization projects vectors onto a unit sphere - their dot product is the *cosine of the angle between the points defined by the vectors*.\n",
    "\n",
    "- $k(x, y) = \\frac{x y^\\top}{\\|x\\| \\|y\\|}$\n",
    "\n",
    "- Popular for computing document similarity with tf-idf vectors."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### [Linear Kernel](https://scikit-learn.org/stable/modules/generated/sklearn.metrics.pairwise.linear_kernel.html#sklearn.metrics.pairwise.linear_kernel)\n",
    "\n",
    "- A linear kernel is a special case of a *polynomial kernel* with `degree=1` and `coef0=0`.\n",
    "\n",
    "- $k(x, y) = x^\\top y$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### [Polynomial Kernel](https://scikit-learn.org/stable/modules/generated/sklearn.metrics.pairwise.polynomial_kernel.html#sklearn.metrics.pairwise.polynomial_kernel)\n",
    "\n",
    "- Computes a d-degree polynomial kernel that represents the similarity between two vectors. It includes the similiarity under the same dimension & across dimensions (this accounts for feature interaction.)\n",
    "\n",
    "- $k(x, y) = (\\gamma x^\\top y +c_0)^d$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### [Sigmoid Kernel](https://scikit-learn.org/stable/modules/generated/sklearn.metrics.pairwise.sigmoid_kernel.html#sklearn.metrics.pairwise.sigmoid_kernel)\n",
    "\n",
    "- Also known as the hyperbolic tangent & is often used as an activation function in neural nets.\n",
    "\n",
    "- $k(x, y) = \\tanh( \\gamma x^\\top y + c_0)$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### [RBF Kernel](https://scikit-learn.org/stable/modules/generated/sklearn.metrics.pairwise.rbf_kernel.html#sklearn.metrics.pairwise.rbf_kernel) \n",
    "\n",
    "- $k(x, y) = \\exp( -\\gamma \\| x-y \\|^2)$\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### [Laplacian Kernel](https://scikit-learn.org/stable/modules/generated/sklearn.metrics.pairwise.laplacian_kernel.html#sklearn.metrics.pairwise.laplacian_kernel)\n",
    "\n",
    "$k(x, y) = \\exp( -\\gamma \\| x-y \\|_1)$\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### [Chi-Squared Kernel](https://scikit-learn.org/stable/modules/generated/sklearn.metrics.pairwise.chi2_kernel.html#sklearn.metrics.pairwise.chi2_kernel)\n",
    "\n",
    "- Very popular for training nonlinear SVMs for computer vision.\n",
    "\n",
    "- $k(x, y) = \\exp \\left (-\\gamma \\sum_i \\frac{(x[i] - y[i]) ^ 2}{x[i] + y[i]} \\right )$"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python [conda env:working] *",
   "language": "python",
   "name": "conda-env-working-py"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}