week13/4-2.张量的数学运算.ipynb

{
 "metadata": {
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.0-final"
  },
  "orig_nbformat": 2,
  "kernelspec": {
   "name": "Python 3.7.0 64-bit ('pytorch1.6': conda)",
   "display_name": "Python 3.7.0 64-bit ('pytorch1.6': conda)",
   "metadata": {
    "interpreter": {
     "hash": "32e20fa419d82ae44b3cd2548dcb635120fe40ded07472dbee097e90a87506aa"
    }
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2,
 "cells": [
  {
   "source": [
    "# 4-2 张量的数学运算\n",
    "张量的操作主要包括张量的结构操作和数学运算。\n",
    "\n",
    "数学运算主要有：\n",
    "* 标量运算\n",
    "* 向量运算\n",
    "* 矩阵运算\n",
    "* 广播机制"
   ],
   "cell_type": "markdown",
   "metadata": {}
  },
  {
   "source": [
    "## 一、标量运算"
   ],
   "cell_type": "markdown",
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch\n",
    "import numpy as np"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "tensor([[ 6.,  8.],\n        [ 4., 12.]])"
     },
     "metadata": {},
     "execution_count": 2
    }
   ],
   "source": [
    "a = torch.tensor([[1., 2], [-3, 4]])\n",
    "b = torch.tensor([[5., 6], [7, 8]])\n",
    "a + b  # 运算符重载"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "tensor([[ -4.,  -4.],\n        [-10.,  -4.]])"
     },
     "metadata": {},
     "execution_count": 3
    }
   ],
   "source": [
    "a-b"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "tensor([[  5.,  12.],\n        [-21.,  32.]])"
     },
     "metadata": {},
     "execution_count": 4
    }
   ],
   "source": [
    "a*b"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "tensor([[ 0.2000,  0.3333],\n        [-0.4286,  0.5000]])"
     },
     "metadata": {},
     "execution_count": 5
    }
   ],
   "source": [
    "a/b"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "tensor([[ 1.,  4.],\n        [ 9., 16.]])"
     },
     "metadata": {},
     "execution_count": 6
    }
   ],
   "source": [
    "a**2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "tensor([[1.0000, 1.4142],\n        [   nan, 2.0000]])"
     },
     "metadata": {},
     "execution_count": 7
    }
   ],
   "source": [
    "a**(0.5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "tensor([[1., 2.],\n        [-0., 1.]])"
     },
     "metadata": {},
     "execution_count": 8
    }
   ],
   "source": [
    "a%3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "tensor([[ 0.,  0.],\n        [-1.,  1.]])"
     },
     "metadata": {},
     "execution_count": 9
    }
   ],
   "source": [
    "a // 3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "tensor([[False,  True],\n        [False,  True]])"
     },
     "metadata": {},
     "execution_count": 10
    }
   ],
   "source": [
    "a >= 2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "tensor([[False,  True],\n        [False, False]])"
     },
     "metadata": {},
     "execution_count": 11
    }
   ],
   "source": [
    "(a>=2) & (a<=3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "output_type": "error",
     "ename": "RuntimeError",
     "evalue": "Boolean value of Tensor with more than one value is ambiguous",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mRuntimeError\u001b[0m                              Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-12-a0089161864d>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0;34m(\u001b[0m\u001b[0ma\u001b[0m\u001b[0;34m>=\u001b[0m\u001b[0;36m2\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mand\u001b[0m \u001b[0;34m(\u001b[0m\u001b[0ma\u001b[0m\u001b[0;34m<=\u001b[0m\u001b[0;36m3\u001b[0m\u001b[0;34m)\u001b[0m   \u001b[0;31m# 直接用and不能处理多布尔\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;31mRuntimeError\u001b[0m: Boolean value of Tensor with more than one value is ambiguous"
     ]
    }
   ],
   "source": [
    "(a>=2) and (a<=3)   # 直接用and不能处理多布尔"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "tensor([[True, True],\n        [True, True]])"
     },
     "metadata": {},
     "execution_count": 13
    }
   ],
   "source": [
    "(a >= 2) | (a<=3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "tensor([[False, False],\n        [False, False]])"
     },
     "metadata": {},
     "execution_count": 14
    }
   ],
   "source": [
    "a == 5"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "tensor([[1.0000, 1.4142],\n        [   nan, 2.0000]])"
     },
     "metadata": {},
     "execution_count": 15
    }
   ],
   "source": [
    "torch.sqrt(a)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": "tensor([12., 21.])\n"
    }
   ],
   "source": [
    "a = torch.tensor([1.0,8.0])\n",
    "b = torch.tensor([5.0,6.0])\n",
    "c = torch.tensor([6.0,7.0])\n",
    "\n",
    "d = a+b+c\n",
    "print(d)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": "tensor([5., 8.])\n"
    }
   ],
   "source": [
    "print(torch.max(a,b))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": "tensor([1., 6.])\n"
    }
   ],
   "source": [
    "print(torch.min(a,b))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": "tensor([ 3., -3.])\ntensor([ 2., -3.])\ntensor([ 3., -2.])\ntensor([ 2., -2.])\n"
    }
   ],
   "source": [
    "x = torch.tensor([2.6,-2.7])\n",
    "\n",
    "# 注意, 这里的结果其实还是float\n",
    "print(torch.round(x)) #保留整数部分，四舍五入\n",
    "print(torch.floor(x)) #保留整数部分，向下归整\n",
    "print(torch.ceil(x))  #保留整数部分，向上归整\n",
    "print(torch.trunc(x)) #保留整数部分，向0归整"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": "tensor([ 0.6000, -0.7000])\ntensor([0.6000, 1.3000])\n"
    }
   ],
   "source": [
    "x = torch.tensor([2.6,-2.7])\n",
    "print(torch.fmod(x, 2)) #作除法取余数 \n",
    "print(torch.remainder(x, 2)) #作除法取剩余的部分，结果恒正"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": "tensor([ 0.9000, -0.8000,  1.0000, -1.0000,  0.7000])\ntensor([  0.9000,  -0.8000,   1.0000, -20.0000,   0.7000])\n"
    }
   ],
   "source": [
    "# 幅值裁剪\n",
    "x = torch.tensor([0.9,-0.8,100.0,-20.0,0.7])\n",
    "y = torch.clamp(x,min=-1,max = 1)\n",
    "z = torch.clamp(x,max = 1)\n",
    "print(y)\n",
    "print(z)"
   ]
  },
  {
   "source": [
    "## 二、向量运算\n",
    "向量运算符只在特定轴上运算, 将一个向量映射到一个标量或者另一个向量"
   ],
   "cell_type": "markdown",
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": "tensor(45.)\ntensor(5.)\ntensor(9.)\ntensor(1.)\ntensor(362880.)\ntensor(7.5000)\ntensor(2.7386)\ntensor(5.)\n"
    }
   ],
   "source": [
    "# 统计值\n",
    "a = torch.arange(1, 10).float()\n",
    "print(torch.sum(a))\n",
    "print(torch.mean(a))\n",
    "print(torch.max(a))\n",
    "print(torch.min(a))\n",
    "print(torch.prod(a))  # 累乘\n",
    "print(torch.var(a))  # 方差\n",
    "print(torch.std(a))  # 标准差\n",
    "print(torch.median(a))  # 中位数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": "tensor([[1., 2., 3.],\n        [4., 5., 6.],\n        [7., 8., 9.]])\ntorch.return_types.max(\nvalues=tensor([7., 8., 9.]),\nindices=tensor([2, 2, 2]))\ntorch.return_types.max(\nvalues=tensor([3., 6., 9.]),\nindices=tensor([2, 2, 2]))\n"
    }
   ],
   "source": [
    "# 指定维度计算统计值\n",
    "b = a.view(3, 3)\n",
    "print(b)\n",
    "print(torch.max(b, dim=0))  # 按dim=0遍历找到最大的, 也就是列最大\n",
    "print(torch.max(b, dim=1))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": "tensor([[[ 1,  2,  3],\n         [ 4,  5,  6],\n         [ 7,  8,  9]],\n\n        [[10, 11, 12],\n         [13, 14, 15],\n         [16, 17, 18]],\n\n        [[19, 20, 21],\n         [22, 23, 24],\n         [25, 26, 27]]])\ntorch.return_types.max(\nvalues=tensor([[19, 20, 21],\n        [22, 23, 24],\n        [25, 26, 27]]),\nindices=tensor([[2, 2, 2],\n        [2, 2, 2],\n        [2, 2, 2]]))\n===================\ntorch.return_types.max(\nvalues=tensor([[ 7,  8,  9],\n        [16, 17, 18],\n        [25, 26, 27]]),\nindices=tensor([[2, 2, 2],\n        [2, 2, 2],\n        [2, 2, 2]]))\n===================\ntorch.return_types.max(\nvalues=tensor([[ 3,  6,  9],\n        [12, 15, 18],\n        [21, 24, 27]]),\nindices=tensor([[2, 2, 2],\n        [2, 2, 2],\n        [2, 2, 2]]))\n"
    }
   ],
   "source": [
    "c = torch.arange(1, 28)\n",
    "c_ = c.view(3, 3, 3)\n",
    "print(c_)\n",
    "print(torch.max(c_, dim=0))\n",
    "print(\"===================\")\n",
    "print(torch.max(c_, dim=1))\n",
    "print(\"===================\")\n",
    "print(torch.max(c_, dim=2))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "tensor([[19, 20, 21],\n        [22, 23, 24],\n        [25, 26, 27]])"
     },
     "metadata": {},
     "execution_count": 29
    }
   ],
   "source": [
    "c_[2,...]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": "tensor([ 1,  3,  6, 10, 15, 21, 28, 36, 45])\ntensor([     1,      2,      6,     24,    120,    720,   5040,  40320, 362880])\ntensor([1, 2, 3, 4, 5, 6, 7, 8, 9])\ntensor([0, 1, 2, 3, 4, 5, 6, 7, 8])\ntorch.return_types.cummin(\nvalues=tensor([1, 1, 1, 1, 1, 1, 1, 1, 1]),\nindices=tensor([0, 0, 0, 0, 0, 0, 0, 0, 0]))\n"
    }
   ],
   "source": [
    "# cum扫描\n",
    "a = torch.arange(1, 10)\n",
    "print(torch.cumsum(a, 0))\n",
    "print(torch.cumprod(a, 0))\n",
    "print(torch.cummax(a, 0).values)\n",
    "print(torch.cummax(a, 0).indices)\n",
    "print(torch.cummin(a, 0))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": "torch.return_types.topk(\nvalues=tensor([[9., 8., 7.],\n        [5., 6., 4.]]),\nindices=tensor([[0, 0, 0],\n        [2, 2, 2]])) \n\ntorch.return_types.topk(\nvalues=tensor([[9., 8.],\n        [3., 2.],\n        [6., 5.]]),\nindices=tensor([[0, 1],\n        [1, 2],\n        [1, 0]])) \n\ntorch.return_types.sort(\nvalues=tensor([[7., 8., 9.],\n        [1., 2., 3.],\n        [4., 5., 6.]]),\nindices=tensor([[2, 1, 0],\n        [0, 2, 1],\n        [2, 0, 1]])) \n\n"
    }
   ],
   "source": [
    "# torch.sort和torch.topk可以对张量排序\n",
    "a = torch.tensor([[9, 8, 7], [1, 3, 2], [5, 6, 4]]).float()\n",
    "print(torch.topk(a, 2, dim=0), \"\\n\")\n",
    "print(torch.topk(a, 2, dim=1), \"\\n\")\n",
    "print(torch.sort(a, dim=1), \"\\n\")\n",
    "\n",
    "# 利用topk就可以实现Pytorch的KNN算法"
   ]
  },
  {
   "source": [
    "## 三、矩阵运算\n",
    "矩阵运算包括: 矩阵乘法、转置、逆矩阵、矩阵范数、行列式、特征值、矩阵分解"
   ],
   "cell_type": "markdown",
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": "tensor([[2, 4],\n        [6, 8]])\n"
    }
   ],
   "source": [
    "# 矩阵乘法\n",
    "a = torch.tensor([[1, 2], [3, 4]])\n",
    "b = torch.tensor([[2, 0], [0, 2]])\n",
    "print(a @ b)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": "tensor([[1., 3.],\n        [2., 4.]])\n"
    }
   ],
   "source": [
    "# 矩阵转置\n",
    "a = torch.tensor([[1.0, 2], [3, 4]])\n",
    "print(a.t())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": "tensor([[-2.0000,  1.0000],\n        [ 1.5000, -0.5000]])\n"
    }
   ],
   "source": [
    "# 矩阵逆, 必须为浮点类型\n",
    "a = torch.tensor([[1., 2], [3, 4]])\n",
    "print(torch.inverse(a))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": "tensor(5.)\n"
    }
   ],
   "source": [
    "# 矩阵求trace\n",
    "a = torch.tensor([[1., 2], [3, 4]])\n",
    "print(torch.trace(a))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": "tensor(5.4772)\n"
    }
   ],
   "source": [
    "# 矩阵范数\n",
    "a = torch.tensor([[1., 2], [3, 4]])\n",
    "print(torch.norm(a))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": "tensor(-2.0000)\n"
    }
   ],
   "source": [
    "# 矩阵行列式\n",
    "a = torch.tensor([[1., 2], [3, 4]])\n",
    "print(torch.det(a))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": "torch.return_types.eig(\neigenvalues=tensor([[ 2.5000,  2.7839],\n        [ 2.5000, -2.7839]]),\neigenvectors=tensor([[ 0.2535, -0.4706],\n        [ 0.8452,  0.0000]]))\n"
    }
   ],
   "source": [
    "# 矩阵特征值和特征向量\n",
    "a = torch.tensor([[1., 2], [-5, 4]], dtype=torch.float)\n",
    "print(torch.eig(a, eigenvectors=True))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": "tensor([[-0.3162, -0.9487],\n        [-0.9487,  0.3162]]) \n\ntensor([[-3.1623, -4.4272],\n        [ 0.0000, -0.6325]]) \n\ntensor([[1.0000, 2.0000],\n        [3.0000, 4.0000]])\n"
    }
   ],
   "source": [
    "# 矩阵QR分解, 将一个方阵分解为一个正交矩阵q和一个上三角矩阵r\n",
    "a = torch.tensor([[1., 2], [3, 4]])\n",
    "q, r = torch.qr(a)\n",
    "print(q, \"\\n\")\n",
    "print(r, \"\\n\")\n",
    "print(q@r)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": "tensor([[-0.2298,  0.8835],\n        [-0.5247,  0.2408],\n        [-0.8196, -0.4019]]) \n\ntensor([9.5255, 0.5143]) \n\ntensor([[-0.6196, -0.7849],\n        [-0.7849,  0.6196]]) \n\ntensor([[1.0000, 2.0000],\n        [3.0000, 4.0000],\n        [5.0000, 6.0000]])\n"
    }
   ],
   "source": [
    "#矩阵svd分解\n",
    "#svd分解可以将任意一个矩阵分解为一个正交矩阵u,一个对角阵s和一个正交矩阵v.t()的乘积\n",
    "#svd常用于矩阵压缩和降维\n",
    "a=torch.tensor([[1.0,2.0],[3.0,4.0],[5.0,6.0]])\n",
    "\n",
    "u,s,v = torch.svd(a)\n",
    "\n",
    "print(u,\"\\n\")\n",
    "print(s,\"\\n\")\n",
    "print(v,\"\\n\")\n",
    "\n",
    "print(u@torch.diag(s)@v.t())"
   ]
  },
  {
   "source": [
    "## 四、广播机制\n",
    "Pytorch的广播规则和numpy是一样的:\n",
    "\n",
    "1. 如果张量的维度不同, 将维度较小的张量进行扩展, 知道两个张量维度一样\n",
    "\n",
    "2. 如果两个张量在某个维度长度一样, 或者其中一个张量在该维度的长度为1, 那么我们就说这两个张量在该维度上是相容的\n",
    "\n",
    "3. 如果两个张量在所有维度上都是相容的, 它们就能使用广播\n",
    "\n",
    "4. 广播之后, 每个维度的长度将取两个张量在该维度长度的较大值\n",
    "\n",
    "5. 在任何一个维度上, 如果一个张量的长度为1, 另一个张量的长度大于1, 那么该维度上就好像是对第一个张量进行了复制"
   ],
   "cell_type": "markdown",
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": "tensor([[1, 2, 3],\n        [2, 3, 4],\n        [3, 4, 5]])\n"
    }
   ],
   "source": [
    "a = torch.tensor([1, 2, 3])\n",
    "b = torch.tensor([[0, 0, 0], [1, 1, 1], [2, 2, 2]])\n",
    "print(b + a)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": "tensor([[1, 2, 3],\n        [1, 2, 3],\n        [1, 2, 3]]) \n\ntensor([[0, 0, 0],\n        [1, 1, 1],\n        [2, 2, 2]]) \n\ntensor([[1, 2, 3],\n        [2, 3, 4],\n        [3, 4, 5]])\n"
    }
   ],
   "source": [
    "a_broad,b_broad = torch.broadcast_tensors(a,b)\n",
    "print(a_broad,\"\\n\")\n",
    "print(b_broad,\"\\n\")\n",
    "print(a_broad + b_broad) "
   ]
  }
 ]
}