tutorials/broadcasting.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Broadcasting"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Copyright (c) Meta Platforms, Inc. and affiliates.**\n",
    "\n",
    "This source code is licensed under the MIT license found in the\n",
    "LICENSE file in the root directory of this source tree."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## Introduction\n",
    "\n",
    "This notebook will discuss how broadcasting works with tensors. Broadcasting is a process of applying an operation to two tensors of different sizes and having data replicated so the sizes of the tensor match and the operation can be applied."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## Housekeeping\n",
    "\n",
    "This notebook uses `api.jar` from the diffkt project.<br>\n",
    "`@file:DependsOn(\"...\")` tells the Kotlin Jupyter notebook the path to a jar that it needs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "pycharm": {
     "is_executing": true,
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "@file:DependsOn(\"../kotlin/api/build/libs/api.jar\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## Imports"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "import org.diffkt.*\n",
    "import java.util.Arrays"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## Broadcasting\n",
    "\n",
    "Normally when we apply an operation between two tensors, such as addition, we assume the two tensors have the same size and the operation is applied element by element. Below is an example of element by element addition."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "x = [[1.0, 2.0], [3.0, 4.0]]\n",
      "y = [[1.0, 1.0], [1.0, 1.0]]\n",
      "\n",
      "x + y = [[2.0, 3.0], [4.0, 5.0]]\n"
     ]
    }
   ],
   "source": [
    "// Element by element tensor addition\n",
    "\n",
    "val x = tensorOf(1f, 2f, 3f, 4f).reshape(2,2)\n",
    "val y = tensorOf(1f, 1f, 1f, 1f).reshape(2,2)\n",
    "\n",
    "val z = x + y\n",
    "\n",
    "println(\"x = ${x}\")\n",
    "println(\"y = ${y}\")\n",
    "println(\"\")\n",
    "println(\"x + y = ${z}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "With broadcasting the tensors do not have to be the same size. Broadcasting can be a difficult topic to understand but the broadcasting in **DiffKt** is similar to the broadcasting in Python. With broadcasting, the data in the smaller sized tensor is replicated to match the size of the larger tensor. In the below example, `x` is a 2x2 tensor and `y` is a 2x1 tensor. In this case `y` shares the first dimension with `x`. The data is replicated so the second dimension of `y` matches `x`, then the addition operation is applied element by element."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "x = [[1.0, 2.0], [3.0, 4.0]]\n",
      "y = [1.0, 1.0]\n",
      "\n",
      "x + y = [[2.0, 3.0], [4.0, 5.0]]\n"
     ]
    }
   ],
   "source": [
    "// Simple broadcasting example\n",
    "\n",
    "val x = tensorOf(1f, 2f, 3f, 4f).reshape(2,2)\n",
    "val y = tensorOf(1f, 1f)\n",
    "\n",
    "val z = x + y\n",
    " \n",
    "println(\"x = ${x}\")\n",
    "println(\"y = ${y}\")\n",
    "println(\"\")\n",
    "println(\"x + y = ${z}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "This example is a bit more complicated. `x` is a 1x3 row tensor. `y` is a 3x1 column tensor. Since the two tensors do not have a common dimension, each tensor is replicated such that `x` and `y` will have the same dimensions. For `x`, each row is replicated to produce a 3x3 tensor. For `y`, each column is replicated to produce a 3x3 tensor. Lastly, after `x`and `y` are replicated, the addition occurs element by element on the replicated tensors."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "x = [[1.0, 2.0, 3.0]]\n",
      "y = [[1.0], [2.0], [3.0]]\n",
      "\n",
      "x + y = [[2.0, 3.0, 4.0], [3.0, 4.0, 5.0], [4.0, 5.0, 6.0]]\n"
     ]
    }
   ],
   "source": [
    "// Complex broadcasting example\n",
    "\n",
    "val x= tensorOf(1f, 2f, 3f).reshape(1,3)\n",
    "val y = tensorOf(1f, 2f, 3f).reshape(3, 1)\n",
    "\n",
    "val z = x + y\n",
    "\n",
    "println(\"x = ${x}\")\n",
    "println(\"y = ${y}\")\n",
    "println(\"\")\n",
    "println(\"x + y = ${z}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## The End"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Kotlin",
   "language": "kotlin",
   "name": "kotlin"
  },
  "language_info": {
   "codemirror_mode": "text/x-kotlin",
   "file_extension": ".kt",
   "mimetype": "text/x-kotlin",
   "name": "kotlin",
   "nbconvert_exporter": "",
   "pygments_lexer": "kotlin",
   "version": "1.7.0-dev-3303"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}