July update

akkefa · Sep 2, 2023 · 830bb79 · 830bb79
1 parent 01b042f
commit 830bb79
Show file tree

Hide file tree

Showing 11 changed files with 493 additions and 2,313 deletions.
diff --git a/README.md b/README.md
@@ -12,7 +12,6 @@
 
 [ml-notes.akkefa.com](ml-notes.akkefa.com)
 
-
 #### Docs build
 
 ```bash

diff --git a/docs/algebra/intro.ipynb b/docs/algebra/intro.ipynb
diff --git a/docs/calculus.rst b/docs/calculus.rst
diff --git a/docs/calculus/intro.ipynb b/docs/calculus/intro.ipynb
diff --git a/docs/deep_learning/graph.ipynb b/docs/deep_learning/graph.ipynb
diff --git a/docs/graph/graph_equations.ipynb b/docs/graph/graph_equations.ipynb
@@ -0,0 +1,62 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Graph Equations\n",
+    "\n",
+    "Explaining the graph neural networks equations\n",
+    "\n",
+    "## GCN layer\n",
+    "\n",
+    "Semi-Supervised Classification with Graph Convolutional Networks\n",
+    "https://arxiv.org/pdf/1609.02907.pdf\n",
+    "\n",
+    "$$\n",
+    "h_i^{(l+1)}=\\sigma\\left(\\sum_{j \\in \\mathcal{N}_i} \\frac{1}{c_{i j}} h_j^{(l)} W^{(l)}\\right)\n",
+    "$$\n",
+    "\n",
+    "where  𝐖(ℓ+)  denotes a trainable weight matrix of shape [num_output_features, num_input_features] and  𝑐𝑤,𝑣  refers to a fixed normalization coefficient for each edge.\n",
+    "\n",
+    "PyG implements this layer via GCNConv"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "notes",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.11.4"
+  },
+  "orig_nbformat": 4
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}
diff --git a/docs/graph/graph_neural_networks.ipynb b/docs/graph/graph_neural_networks.ipynb
@@ -9,8 +9,6 @@
     "\n",
     "Graph Neural Networks (GNNs) are a class of neural networks that operate on graphs. They are a powerful tool for solving problems in domains such as social network analysis, recommender systems, and combinatorial optimization.\n",
     "\n",
-    "You can read about Graph data structures /algorithms/graphs.html#graphs-data-structure section.\n",
-    "\n",
     "## Node Representations\n",
     "\n",
     "The goal of a GNN is to learn a function that maps a graph to a representation of its nodes. This representation can then be used for various downstream tasks, such as node classification, link prediction, and clustering.\n",

diff --git a/docs/graph/introduction.ipynb b/docs/graph/introduction.ipynb
@@ -295,6 +295,25 @@
     ":align: center\n",
     "```\n",
     "\n",
+    "### Homogeneous graph\n",
+    "\n",
+    "Homogeneous graph consists of one type of nodes and edges, and heterogeneous graph has multiple types of nodes or edges.\n",
+    "\n",
+    "An example of a homogeneous graph is an online social network with nodes representing people and edges representing friendship. means we have same node for all types of node and similary all edges are same.\n",
+    "\n",
+    "\n",
+    "### Heterogeneous graph\n",
+    "\n",
+    "Heterogeneous graphs come with different types of information attached to nodes and edges. Thus, a single node or edge feature tensor cannot hold all node or edge features of the whole graph, due to differences in type and dimensionality.\n",
+    "\n",
+    "A graph with two or more types of node and/or two or more types of edge is called heterogeneous. An online social network with edges of different types, say ‘friendship’ and ‘co-worker’, between nodes of ‘person’ type is an example of a heterogeneous\n",
+    "\n",
+    "\n",
+    "```{image} https://miro.medium.com/v2/resize:fit:4800/format:jpg/1*DfoOgPPusJAUm_kSN8O_mA.png\n",
+    ":alt: Homogeneous graph\n",
+    ":width: 80%\n",
+    ":align: center\n",
+    "```\n",
     "\n",
     "## Node degrees\n",
     "\n",

diff --git a/docs/index.rst b/docs/index.rst
@@ -18,7 +18,7 @@ Contents
 .. toctree::
    :maxdepth: 1
 
-   mathematical_notation.md
+   ml_notation.ipynb
 
 .. toctree::
    :caption: Probability
@@ -37,6 +37,11 @@ Contents
    probability/confidence_interval
    probability/hypothesis_testing
 
+.. toctree::
+   :caption: Calculus
+   :maxdepth: 1
+
+   calculus/intro
 
 .. toctree::
    :caption: Algebra
@@ -52,13 +57,11 @@ Contents
    deep_learning/tensor
    deep_learning/loss_functions
    deep_learning/evaluation_metrics
-   deep_learning/graph
 
 .. toctree::
    :caption: Mathematics
    :maxdepth: 1
 
-   calculus
    linear_algebra
    statistics
 
@@ -78,6 +81,7 @@ Contents
 
    graph/introduction
    graph/graph_neural_networks
+   graph/graph_equations
 
 .. toctree::
    :caption: Learn Pytorch

diff --git a/docs/mathematical_notation.md b/docs/mathematical_notation.md
diff --git a/docs/ml_notation.ipynb b/docs/ml_notation.ipynb
@@ -0,0 +1,128 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# ML Notation / Equations\n",
+    "\n",
+    "## Notation\n",
+    "\n",
+    "```{list-table}\n",
+    ":widths: 20 70 10\n",
+    ":header-rows: 1\n",
+    ":align: \"center\"\n",
+    "\n",
+    "* - Symbol\n",
+    "  - Formula\n",
+    "  - Explained\n",
+    "* - $\\mu$\n",
+    "  - $\\sum_{x} k P(X=x) = \\int_{-\\infty}^{\\infty} x f(x) d x$\n",
+    "  - [🔗](expected-value)\n",
+    "* - $V(X)$ or $\\sigma^2$ \n",
+    "  - $E[(X - E[X])^2] = E[(X - \\mu)^2]  = E[X^2] - E[X]^2$\n",
+    "  - [🔗](variance-link)\n",
+    "* - $\\sigma$\n",
+    "  - $\\sqrt{V(X)}$\n",
+    "  - Standard deviation\n",
+    "* - $Cov(X,Y)$ \n",
+    "  - Covariance of X and Y\n",
+    "  - Covariance of X and Y\n",
+    "* - $\\bar{X}$\n",
+    "  - The sample \n",
+    "  - The sample mean is an average value\n",
+    "* - $\\delta$\n",
+    "  - $\\delta(v)$\n",
+    "  - Activation fucntions, sigmoid, relu, etc.\n",
+    "\n",
+    "```\n",
+    "\n",
+    "\n",
+    "## Equations\n",
+    "\n",
+    "### Cosine Similarity\n",
+    "Cosine similarity is a metric used to measure the similarity between two vectors in a multi-dimensional space.\n",
+    "Cosine similarity measures the cosine of the angle between two non-zero vectors in an n-dimensional space.\n",
+    "\n",
+    "Formula = dot product / normalized sum of squares\n",
+    "\n",
+    "$$\n",
+    "\\text{cos}(x,y) = \\frac{x \\cdot y}{\\sqrt{x^2} \\cdot \\sqrt{y^2}} = \\frac{\\sum_{i=1}^n A_i B_i}{\\sqrt{\\sum_{i=1}^n A_i^2} \\sqrt{\\sum_{i=1}^n B_i^2}}\n",
+    "$$\n",
+    "\n",
+    "#### Properties\n",
+    "\n",
+    "- **Scale Invariance** Cosine similarity is scale-invariant, meaning it is not affected by the magnitude of the vectors, only by their orientations.\n",
+    "- One hot and multi hot vectors easily.\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import torch\n",
+    "from torch.nn import functional as F"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 23,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "tensor(0.7071)\n",
+      "tensor(0.7071)\n",
+      "tensor(0.7071)\n",
+      "tensor(0.7071)\n"
+     ]
+    }
+   ],
+   "source": [
+    "v1 = torch.tensor([0, 0, 1], dtype=torch.float32)\n",
+    "v2 = torch.tensor([0, 1, 1],dtype=torch.float32)\n",
+    "\n",
+    "print(F.cosine_similarity(v1, v2 , dim=0))\n",
+    "\n",
+    "print(F.normalize(v1, dim=0) @ F.normalize(v2, dim=0))\n",
+    "\n",
+    "print(torch.norm(v1) / torch.norm(v2))\n",
+    "\n",
+    "print( torch.matmul(v1, v2.T) / ( torch.sqrt( torch.sum(v1 ** 2)) * torch.sqrt( torch.sum(v2 ** 2))) )\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "notes",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.11.4"
+  },
+  "orig_nbformat": 4
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}