saving-a-basic-fastai-model-in-onnx.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "98d53c05"
   },
   "source": [
    "## Saving a Cats v Dogs Model in ONNX"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Cloned from Fast.ai Course22, https://github.com/fastai/course22/blob/master/02-saving-a-basic-fastai-model.ipynb\n",
    "\n",
    "Builds upon the Fast.ai Cats v Dogs notebook to save the model in ONNX.  The notebook shows how easy it is to save a Fast.ai model to ONNX.  Once the model is in ONNX, it can be converted to other machine learning frameworks like Elixir's Axon, https://github.com/elixir-nx/axon_onnx"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "_kg_hide-input": true,
    "_kg_hide-output": true,
    "execution": {
     "iopub.execute_input": "2022-05-03T05:51:37.949032Z",
     "iopub.status.busy": "2022-05-03T05:51:37.948558Z",
     "iopub.status.idle": "2022-05-03T05:51:59.531217Z",
     "shell.execute_reply": "2022-05-03T05:51:59.530294Z",
     "shell.execute_reply.started": "2022-05-03T05:51:37.948947Z"
    },
    "id": "evvA0fqvSblq",
    "outputId": "ba21b811-767c-459a-ccdf-044758720a55"
   },
   "outputs": [],
   "source": [
    "# Make sure we've got the latest version of fastai:\n",
    "# !pip install -Uqq fastai"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First, import all the stuff we need from fastai:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-05-03T05:51:59.534478Z",
     "iopub.status.busy": "2022-05-03T05:51:59.533878Z",
     "iopub.status.idle": "2022-05-03T05:52:02.177975Z",
     "shell.execute_reply": "2022-05-03T05:52:02.177267Z",
     "shell.execute_reply.started": "2022-05-03T05:51:59.534432Z"
    },
    "id": "44eb0ad3"
   },
   "outputs": [],
   "source": [
    "from fastai.vision.all import *"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Download and decompress our dataset, which is pictures of dogs and cats:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-05-03T05:52:02.180691Z",
     "iopub.status.busy": "2022-05-03T05:52:02.180192Z",
     "iopub.status.idle": "2022-05-03T05:53:02.465242Z",
     "shell.execute_reply": "2022-05-03T05:53:02.464516Z",
     "shell.execute_reply.started": "2022-05-03T05:52:02.180651Z"
    }
   },
   "outputs": [],
   "source": [
    "path = untar_data(URLs.PETS)/'images'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We need a way to label our images as dogs or cats. In this dataset, pictures of cats are given a filename that starts with a capital letter:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-05-03T05:53:02.467572Z",
     "iopub.status.busy": "2022-05-03T05:53:02.467289Z",
     "iopub.status.idle": "2022-05-03T05:53:02.474701Z",
     "shell.execute_reply": "2022-05-03T05:53:02.474109Z",
     "shell.execute_reply.started": "2022-05-03T05:53:02.467536Z"
    },
    "id": "44eb0ad3"
   },
   "outputs": [],
   "source": [
    "def is_cat(x): return x[0].isupper() "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we can create our `DataLoaders`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-05-03T05:53:02.476084Z",
     "iopub.status.busy": "2022-05-03T05:53:02.475754Z",
     "iopub.status.idle": "2022-05-03T05:53:06.703777Z",
     "shell.execute_reply": "2022-05-03T05:53:06.703023Z",
     "shell.execute_reply.started": "2022-05-03T05:53:02.476052Z"
    },
    "id": "44eb0ad3"
   },
   "outputs": [],
   "source": [
    "dls = ImageDataLoaders.from_name_func('.',\n",
    "    get_image_files(path), valid_pct=0.2, seed=42,\n",
    "    label_func=is_cat,\n",
    "    item_tfms=Resize(192))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "... and train our model, a resnet18 (to keep it small and fast):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-05-03T05:53:28.093059Z",
     "iopub.status.busy": "2022-05-03T05:53:28.092381Z"
    },
    "id": "c107f724",
    "outputId": "fcc1de68-7c8b-43f5-b9eb-fcdb0773ef07"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/ml3/miniconda3/envs/fastai_22/lib/python3.10/site-packages/torchvision/models/_utils.py:135: UserWarning: Using 'weights' as positional parameter(s) is deprecated since 0.13 and will be removed in 0.15. Please use keyword parameter(s) instead.\n",
      "  warnings.warn(\n",
      "/home/ml3/miniconda3/envs/fastai_22/lib/python3.10/site-packages/torchvision/models/_utils.py:223: UserWarning: Arguments other than a weight enum or `None` for 'weights' are deprecated since 0.13 and will be removed in 0.15. The current behavior is equivalent to passing `weights=ResNet18_Weights.IMAGENET1K_V1`. You can also use `weights=ResNet18_Weights.DEFAULT` to get the most up-to-date weights.\n",
      "  warnings.warn(msg)\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "<style>\n",
       "    /* Turns off some styling */\n",
       "    progress {\n",
       "        /* gets rid of default border in Firefox and Opera. */\n",
       "        border: none;\n",
       "        /* Needs to be in here for Safari polyfill so background images work as expected. */\n",
       "        background-size: auto;\n",
       "    }\n",
       "    progress:not([value]), progress:not([value])::-webkit-progress-bar {\n",
       "        background: repeating-linear-gradient(45deg, #7e7e7e, #7e7e7e 10px, #5c5c5c 10px, #5c5c5c 20px);\n",
       "    }\n",
       "    .progress-bar-interrupted, .progress-bar-interrupted::-webkit-progress-bar {\n",
       "        background: #F44336;\n",
       "    }\n",
       "</style>\n"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: left;\">\n",
       "      <th>epoch</th>\n",
       "      <th>train_loss</th>\n",
       "      <th>valid_loss</th>\n",
       "      <th>error_rate</th>\n",
       "      <th>time</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>0</td>\n",
       "      <td>0.197886</td>\n",
       "      <td>0.060117</td>\n",
       "      <td>0.018945</td>\n",
       "      <td>00:13</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "\n",
       "<style>\n",
       "    /* Turns off some styling */\n",
       "    progress {\n",
       "        /* gets rid of default border in Firefox and Opera. */\n",
       "        border: none;\n",
       "        /* Needs to be in here for Safari polyfill so background images work as expected. */\n",
       "        background-size: auto;\n",
       "    }\n",
       "    progress:not([value]), progress:not([value])::-webkit-progress-bar {\n",
       "        background: repeating-linear-gradient(45deg, #7e7e7e, #7e7e7e 10px, #5c5c5c 10px, #5c5c5c 20px);\n",
       "    }\n",
       "    .progress-bar-interrupted, .progress-bar-interrupted::-webkit-progress-bar {\n",
       "        background: #F44336;\n",
       "    }\n",
       "</style>\n"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: left;\">\n",
       "      <th>epoch</th>\n",
       "      <th>train_loss</th>\n",
       "      <th>valid_loss</th>\n",
       "      <th>error_rate</th>\n",
       "      <th>time</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>0</td>\n",
       "      <td>0.068240</td>\n",
       "      <td>0.037473</td>\n",
       "      <td>0.012855</td>\n",
       "      <td>00:10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>0.040520</td>\n",
       "      <td>0.031718</td>\n",
       "      <td>0.010149</td>\n",
       "      <td>00:10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>2</td>\n",
       "      <td>0.037268</td>\n",
       "      <td>0.043240</td>\n",
       "      <td>0.012855</td>\n",
       "      <td>00:10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3</td>\n",
       "      <td>0.035163</td>\n",
       "      <td>0.063045</td>\n",
       "      <td>0.014885</td>\n",
       "      <td>00:10</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "No improvement since epoch 1: early stopping\n"
     ]
    }
   ],
   "source": [
    "learn = vision_learner(dls, resnet18, metrics=error_rate)\n",
    "learn.fine_tune(10, cbs=EarlyStoppingCallback(patience=2))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we can export our trained `Learner`. This contains all the information needed to run the model:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Path('models/cats_v_dogs.pth')"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "learn.save('cats_v_dogs')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<fastai.learner.Learner at 0x7fad983c1f00>"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "learn.load('cats_v_dogs')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Sequential(\n",
       "  (0): Sequential(\n",
       "    (0): Conv2d(3, 64, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False)\n",
       "    (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    (2): ReLU(inplace=True)\n",
       "    (3): MaxPool2d(kernel_size=3, stride=2, padding=1, dilation=1, ceil_mode=False)\n",
       "    (4): Sequential(\n",
       "      (0): BasicBlock(\n",
       "        (conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "        (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu): ReLU(inplace=True)\n",
       "        (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "        (bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      )\n",
       "      (1): BasicBlock(\n",
       "        (conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "        (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu): ReLU(inplace=True)\n",
       "        (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "        (bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      )\n",
       "    )\n",
       "    (5): Sequential(\n",
       "      (0): BasicBlock(\n",
       "        (conv1): Conv2d(64, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)\n",
       "        (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu): ReLU(inplace=True)\n",
       "        (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "        (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (downsample): Sequential(\n",
       "          (0): Conv2d(64, 128, kernel_size=(1, 1), stride=(2, 2), bias=False)\n",
       "          (1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        )\n",
       "      )\n",
       "      (1): BasicBlock(\n",
       "        (conv1): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "        (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu): ReLU(inplace=True)\n",
       "        (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "        (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      )\n",
       "    )\n",
       "    (6): Sequential(\n",
       "      (0): BasicBlock(\n",
       "        (conv1): Conv2d(128, 256, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)\n",
       "        (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu): ReLU(inplace=True)\n",
       "        (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "        (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (downsample): Sequential(\n",
       "          (0): Conv2d(128, 256, kernel_size=(1, 1), stride=(2, 2), bias=False)\n",
       "          (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        )\n",
       "      )\n",
       "      (1): BasicBlock(\n",
       "        (conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "        (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu): ReLU(inplace=True)\n",
       "        (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "        (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      )\n",
       "    )\n",
       "    (7): Sequential(\n",
       "      (0): BasicBlock(\n",
       "        (conv1): Conv2d(256, 512, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)\n",
       "        (bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu): ReLU(inplace=True)\n",
       "        (conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "        (bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (downsample): Sequential(\n",
       "          (0): Conv2d(256, 512, kernel_size=(1, 1), stride=(2, 2), bias=False)\n",
       "          (1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        )\n",
       "      )\n",
       "      (1): BasicBlock(\n",
       "        (conv1): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "        (bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "        (relu): ReLU(inplace=True)\n",
       "        (conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)\n",
       "        (bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "      )\n",
       "    )\n",
       "  )\n",
       "  (1): Sequential(\n",
       "    (0): AdaptiveConcatPool2d(\n",
       "      (ap): AdaptiveAvgPool2d(output_size=1)\n",
       "      (mp): AdaptiveMaxPool2d(output_size=1)\n",
       "    )\n",
       "    (1): Flatten(full=False)\n",
       "    (2): BatchNorm1d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    (3): Dropout(p=0.25, inplace=False)\n",
       "    (4): Linear(in_features=1024, out_features=512, bias=False)\n",
       "    (5): ReLU(inplace=True)\n",
       "    (6): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    (7): Dropout(p=0.5, inplace=False)\n",
       "    (8): Linear(in_features=512, out_features=2, bias=False)\n",
       "  )\n",
       ")"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model = learn.model\n",
    "model.cuda()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch\n",
    "import torchvision"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "dummy_input = torch.randn(10, 3, 224, 224, device=\"cuda\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Exported graph: graph(%actual_input_1 : Float(10, 3, 224, 224, strides=[150528, 50176, 224, 1], requires_grad=0, device=cuda:0),\n",
      "      %1.2.weight : Float(1024, strides=[1], requires_grad=1, device=cuda:0),\n",
      "      %1.2.bias : Float(1024, strides=[1], requires_grad=1, device=cuda:0),\n",
      "      %1.2.running_mean : Float(1024, strides=[1], requires_grad=0, device=cuda:0),\n",
      "      %1.2.running_var : Float(1024, strides=[1], requires_grad=0, device=cuda:0),\n",
      "      %1.6.weight : Float(512, strides=[1], requires_grad=1, device=cuda:0),\n",
      "      %1.6.bias : Float(512, strides=[1], requires_grad=1, device=cuda:0),\n",
      "      %1.6.running_mean : Float(512, strides=[1], requires_grad=0, device=cuda:0),\n",
      "      %1.6.running_var : Float(512, strides=[1], requires_grad=0, device=cuda:0),\n",
      "      %onnx::Conv_218 : Float(64, 3, 7, 7, strides=[147, 49, 7, 1], requires_grad=0, device=cuda:0),\n",
      "      %onnx::Conv_219 : Float(64, strides=[1], requires_grad=0, device=cuda:0),\n",
      "      %onnx::Conv_221 : Float(64, 64, 3, 3, strides=[576, 9, 3, 1], requires_grad=0, device=cuda:0),\n",
      "      %onnx::Conv_222 : Float(64, strides=[1], requires_grad=0, device=cuda:0),\n",
      "      %onnx::Conv_224 : Float(64, 64, 3, 3, strides=[576, 9, 3, 1], requires_grad=0, device=cuda:0),\n",
      "      %onnx::Conv_225 : Float(64, strides=[1], requires_grad=0, device=cuda:0),\n",
      "      %onnx::Conv_227 : Float(64, 64, 3, 3, strides=[576, 9, 3, 1], requires_grad=0, device=cuda:0),\n",
      "      %onnx::Conv_228 : Float(64, strides=[1], requires_grad=0, device=cuda:0),\n",
      "      %onnx::Conv_230 : Float(64, 64, 3, 3, strides=[576, 9, 3, 1], requires_grad=0, device=cuda:0),\n",
      "      %onnx::Conv_231 : Float(64, strides=[1], requires_grad=0, device=cuda:0),\n",
      "      %onnx::Conv_233 : Float(128, 64, 3, 3, strides=[576, 9, 3, 1], requires_grad=0, device=cuda:0),\n",
      "      %onnx::Conv_234 : Float(128, strides=[1], requires_grad=0, device=cuda:0),\n",
      "      %onnx::Conv_236 : Float(128, 128, 3, 3, strides=[1152, 9, 3, 1], requires_grad=0, device=cuda:0),\n",
      "      %onnx::Conv_237 : Float(128, strides=[1], requires_grad=0, device=cuda:0),\n",
      "      %onnx::Conv_239 : Float(128, 64, 1, 1, strides=[64, 1, 1, 1], requires_grad=0, device=cuda:0),\n",
      "      %onnx::Conv_240 : Float(128, strides=[1], requires_grad=0, device=cuda:0),\n",
      "      %onnx::Conv_242 : Float(128, 128, 3, 3, strides=[1152, 9, 3, 1], requires_grad=0, device=cuda:0),\n",
      "      %onnx::Conv_243 : Float(128, strides=[1], requires_grad=0, device=cuda:0),\n",
      "      %onnx::Conv_245 : Float(128, 128, 3, 3, strides=[1152, 9, 3, 1], requires_grad=0, device=cuda:0),\n",
      "      %onnx::Conv_246 : Float(128, strides=[1], requires_grad=0, device=cuda:0),\n",
      "      %onnx::Conv_248 : Float(256, 128, 3, 3, strides=[1152, 9, 3, 1], requires_grad=0, device=cuda:0),\n",
      "      %onnx::Conv_249 : Float(256, strides=[1], requires_grad=0, device=cuda:0),\n",
      "      %onnx::Conv_251 : Float(256, 256, 3, 3, strides=[2304, 9, 3, 1], requires_grad=0, device=cuda:0),\n",
      "      %onnx::Conv_252 : Float(256, strides=[1], requires_grad=0, device=cuda:0),\n",
      "      %onnx::Conv_254 : Float(256, 128, 1, 1, strides=[128, 1, 1, 1], requires_grad=0, device=cuda:0),\n",
      "      %onnx::Conv_255 : Float(256, strides=[1], requires_grad=0, device=cuda:0),\n",
      "      %onnx::Conv_257 : Float(256, 256, 3, 3, strides=[2304, 9, 3, 1], requires_grad=0, device=cuda:0),\n",
      "      %onnx::Conv_258 : Float(256, strides=[1], requires_grad=0, device=cuda:0),\n",
      "      %onnx::Conv_260 : Float(256, 256, 3, 3, strides=[2304, 9, 3, 1], requires_grad=0, device=cuda:0),\n",
      "      %onnx::Conv_261 : Float(256, strides=[1], requires_grad=0, device=cuda:0),\n",
      "      %onnx::Conv_263 : Float(512, 256, 3, 3, strides=[2304, 9, 3, 1], requires_grad=0, device=cuda:0),\n",
      "      %onnx::Conv_264 : Float(512, strides=[1], requires_grad=0, device=cuda:0),\n",
      "      %onnx::Conv_266 : Float(512, 512, 3, 3, strides=[4608, 9, 3, 1], requires_grad=0, device=cuda:0),\n",
      "      %onnx::Conv_267 : Float(512, strides=[1], requires_grad=0, device=cuda:0),\n",
      "      %onnx::Conv_269 : Float(512, 256, 1, 1, strides=[256, 1, 1, 1], requires_grad=0, device=cuda:0),\n",
      "      %onnx::Conv_270 : Float(512, strides=[1], requires_grad=0, device=cuda:0),\n",
      "      %onnx::Conv_272 : Float(512, 512, 3, 3, strides=[4608, 9, 3, 1], requires_grad=0, device=cuda:0),\n",
      "      %onnx::Conv_273 : Float(512, strides=[1], requires_grad=0, device=cuda:0),\n",
      "      %onnx::Conv_275 : Float(512, 512, 3, 3, strides=[4608, 9, 3, 1], requires_grad=0, device=cuda:0),\n",
      "      %onnx::Conv_276 : Float(512, strides=[1], requires_grad=0, device=cuda:0),\n",
      "      %onnx::Reshape_279 : Long(2, strides=[1], requires_grad=0, device=cpu),\n",
      "      %onnx::MatMul_280 : Float(1024, 512, strides=[1, 1024], requires_grad=0, device=cuda:0),\n",
      "      %onnx::MatMul_281 : Float(512, 2, strides=[1, 512], requires_grad=0, device=cuda:0)):\n",
      "  %input.4 : Float(10, 64, 112, 112, strides=[802816, 12544, 112, 1], requires_grad=1, device=cuda:0) = onnx::Conv[dilations=[1, 1], group=1, kernel_shape=[7, 7], pads=[3, 3, 3, 3], strides=[2, 2], onnx_name=\"Conv_0\"](%actual_input_1, %onnx::Conv_218, %onnx::Conv_219) # /home/ml3/miniconda3/envs/fastai_22/lib/python3.10/site-packages/torch/nn/modules/conv.py:452:0\n",
      "  %onnx::MaxPool_135 : Float(10, 64, 112, 112, strides=[802816, 12544, 112, 1], requires_grad=1, device=cuda:0) = onnx::Relu[onnx_name=\"Relu_1\"](%input.4) # /home/ml3/miniconda3/envs/fastai_22/lib/python3.10/site-packages/torch/nn/functional.py:1453:0\n",
      "  %input.8 : Float(10, 64, 56, 56, strides=[200704, 3136, 56, 1], requires_grad=1, device=cuda:0) = onnx::MaxPool[ceil_mode=0, kernel_shape=[3, 3], pads=[1, 1, 1, 1], strides=[2, 2], onnx_name=\"MaxPool_2\"](%onnx::MaxPool_135) # /home/ml3/miniconda3/envs/fastai_22/lib/python3.10/site-packages/torch/nn/functional.py:780:0\n",
      "  %input.16 : Float(10, 64, 56, 56, strides=[200704, 3136, 56, 1], requires_grad=1, device=cuda:0) = onnx::Conv[dilations=[1, 1], group=1, kernel_shape=[3, 3], pads=[1, 1, 1, 1], strides=[1, 1], onnx_name=\"Conv_3\"](%input.8, %onnx::Conv_221, %onnx::Conv_222) # /home/ml3/miniconda3/envs/fastai_22/lib/python3.10/site-packages/torch/nn/modules/conv.py:452:0\n",
      "  %onnx::Conv_139 : Float(10, 64, 56, 56, strides=[200704, 3136, 56, 1], requires_grad=1, device=cuda:0) = onnx::Relu[onnx_name=\"Relu_4\"](%input.16) # /home/ml3/miniconda3/envs/fastai_22/lib/python3.10/site-packages/torch/nn/functional.py:1453:0\n",
      "  %onnx::Add_223 : Float(10, 64, 56, 56, strides=[200704, 3136, 56, 1], requires_grad=1, device=cuda:0) = onnx::Conv[dilations=[1, 1], group=1, kernel_shape=[3, 3], pads=[1, 1, 1, 1], strides=[1, 1], onnx_name=\"Conv_5\"](%onnx::Conv_139, %onnx::Conv_224, %onnx::Conv_225) # /home/ml3/miniconda3/envs/fastai_22/lib/python3.10/site-packages/torch/nn/modules/conv.py:452:0\n",
      "  %onnx::Relu_142 : Float(10, 64, 56, 56, strides=[200704, 3136, 56, 1], requires_grad=1, device=cuda:0) = onnx::Add[onnx_name=\"Add_6\"](%onnx::Add_223, %input.8) # /home/ml3/miniconda3/envs/fastai_22/lib/python3.10/site-packages/torchvision/models/resnet.py:96:0\n",
      "  %input.24 : Float(10, 64, 56, 56, strides=[200704, 3136, 56, 1], requires_grad=1, device=cuda:0) = onnx::Relu[onnx_name=\"Relu_7\"](%onnx::Relu_142) # /home/ml3/miniconda3/envs/fastai_22/lib/python3.10/site-packages/torch/nn/functional.py:1453:0\n",
      "  %input.32 : Float(10, 64, 56, 56, strides=[200704, 3136, 56, 1], requires_grad=1, device=cuda:0) = onnx::Conv[dilations=[1, 1], group=1, kernel_shape=[3, 3], pads=[1, 1, 1, 1], strides=[1, 1], onnx_name=\"Conv_8\"](%input.24, %onnx::Conv_227, %onnx::Conv_228) # /home/ml3/miniconda3/envs/fastai_22/lib/python3.10/site-packages/torch/nn/modules/conv.py:452:0\n",
      "  %onnx::Conv_146 : Float(10, 64, 56, 56, strides=[200704, 3136, 56, 1], requires_grad=1, device=cuda:0) = onnx::Relu[onnx_name=\"Relu_9\"](%input.32) # /home/ml3/miniconda3/envs/fastai_22/lib/python3.10/site-packages/torch/nn/functional.py:1453:0\n",
      "  %onnx::Add_229 : Float(10, 64, 56, 56, strides=[200704, 3136, 56, 1], requires_grad=1, device=cuda:0) = onnx::Conv[dilations=[1, 1], group=1, kernel_shape=[3, 3], pads=[1, 1, 1, 1], strides=[1, 1], onnx_name=\"Conv_10\"](%onnx::Conv_146, %onnx::Conv_230, %onnx::Conv_231) # /home/ml3/miniconda3/envs/fastai_22/lib/python3.10/site-packages/torch/nn/modules/conv.py:452:0\n",
      "  %onnx::Relu_149 : Float(10, 64, 56, 56, strides=[200704, 3136, 56, 1], requires_grad=1, device=cuda:0) = onnx::Add[onnx_name=\"Add_11\"](%onnx::Add_229, %input.24) # /home/ml3/miniconda3/envs/fastai_22/lib/python3.10/site-packages/torchvision/models/resnet.py:96:0\n",
      "  %input.40 : Float(10, 64, 56, 56, strides=[200704, 3136, 56, 1], requires_grad=1, device=cuda:0) = onnx::Relu[onnx_name=\"Relu_12\"](%onnx::Relu_149) # /home/ml3/miniconda3/envs/fastai_22/lib/python3.10/site-packages/torch/nn/functional.py:1453:0\n",
      "  %input.48 : Float(10, 128, 28, 28, strides=[100352, 784, 28, 1], requires_grad=1, device=cuda:0) = onnx::Conv[dilations=[1, 1], group=1, kernel_shape=[3, 3], pads=[1, 1, 1, 1], strides=[2, 2], onnx_name=\"Conv_13\"](%input.40, %onnx::Conv_233, %onnx::Conv_234) # /home/ml3/miniconda3/envs/fastai_22/lib/python3.10/site-packages/torch/nn/modules/conv.py:452:0\n",
      "  %onnx::Conv_153 : Float(10, 128, 28, 28, strides=[100352, 784, 28, 1], requires_grad=1, device=cuda:0) = onnx::Relu[onnx_name=\"Relu_14\"](%input.48) # /home/ml3/miniconda3/envs/fastai_22/lib/python3.10/site-packages/torch/nn/functional.py:1453:0\n",
      "  %onnx::Add_235 : Float(10, 128, 28, 28, strides=[100352, 784, 28, 1], requires_grad=1, device=cuda:0) = onnx::Conv[dilations=[1, 1], group=1, kernel_shape=[3, 3], pads=[1, 1, 1, 1], strides=[1, 1], onnx_name=\"Conv_15\"](%onnx::Conv_153, %onnx::Conv_236, %onnx::Conv_237) # /home/ml3/miniconda3/envs/fastai_22/lib/python3.10/site-packages/torch/nn/modules/conv.py:452:0\n",
      "  %onnx::Add_238 : Float(10, 128, 28, 28, strides=[100352, 784, 28, 1], requires_grad=1, device=cuda:0) = onnx::Conv[dilations=[1, 1], group=1, kernel_shape=[1, 1], pads=[0, 0, 0, 0], strides=[2, 2], onnx_name=\"Conv_16\"](%input.40, %onnx::Conv_239, %onnx::Conv_240) # /home/ml3/miniconda3/envs/fastai_22/lib/python3.10/site-packages/torch/nn/modules/conv.py:452:0\n",
      "  %onnx::Relu_158 : Float(10, 128, 28, 28, strides=[100352, 784, 28, 1], requires_grad=1, device=cuda:0) = onnx::Add[onnx_name=\"Add_17\"](%onnx::Add_235, %onnx::Add_238) # /home/ml3/miniconda3/envs/fastai_22/lib/python3.10/site-packages/torchvision/models/resnet.py:96:0\n",
      "  %input.60 : Float(10, 128, 28, 28, strides=[100352, 784, 28, 1], requires_grad=1, device=cuda:0) = onnx::Relu[onnx_name=\"Relu_18\"](%onnx::Relu_158) # /home/ml3/miniconda3/envs/fastai_22/lib/python3.10/site-packages/torch/nn/functional.py:1453:0\n",
      "  %input.68 : Float(10, 128, 28, 28, strides=[100352, 784, 28, 1], requires_grad=1, device=cuda:0) = onnx::Conv[dilations=[1, 1], group=1, kernel_shape=[3, 3], pads=[1, 1, 1, 1], strides=[1, 1], onnx_name=\"Conv_19\"](%input.60, %onnx::Conv_242, %onnx::Conv_243) # /home/ml3/miniconda3/envs/fastai_22/lib/python3.10/site-packages/torch/nn/modules/conv.py:452:0\n",
      "  %onnx::Conv_162 : Float(10, 128, 28, 28, strides=[100352, 784, 28, 1], requires_grad=1, device=cuda:0) = onnx::Relu[onnx_name=\"Relu_20\"](%input.68) # /home/ml3/miniconda3/envs/fastai_22/lib/python3.10/site-packages/torch/nn/functional.py:1453:0\n",
      "  %onnx::Add_244 : Float(10, 128, 28, 28, strides=[100352, 784, 28, 1], requires_grad=1, device=cuda:0) = onnx::Conv[dilations=[1, 1], group=1, kernel_shape=[3, 3], pads=[1, 1, 1, 1], strides=[1, 1], onnx_name=\"Conv_21\"](%onnx::Conv_162, %onnx::Conv_245, %onnx::Conv_246) # /home/ml3/miniconda3/envs/fastai_22/lib/python3.10/site-packages/torch/nn/modules/conv.py:452:0\n",
      "  %onnx::Relu_165 : Float(10, 128, 28, 28, strides=[100352, 784, 28, 1], requires_grad=1, device=cuda:0) = onnx::Add[onnx_name=\"Add_22\"](%onnx::Add_244, %input.60) # /home/ml3/miniconda3/envs/fastai_22/lib/python3.10/site-packages/torchvision/models/resnet.py:96:0\n",
      "  %input.76 : Float(10, 128, 28, 28, strides=[100352, 784, 28, 1], requires_grad=1, device=cuda:0) = onnx::Relu[onnx_name=\"Relu_23\"](%onnx::Relu_165) # /home/ml3/miniconda3/envs/fastai_22/lib/python3.10/site-packages/torch/nn/functional.py:1453:0\n",
      "  %input.84 : Float(10, 256, 14, 14, strides=[50176, 196, 14, 1], requires_grad=1, device=cuda:0) = onnx::Conv[dilations=[1, 1], group=1, kernel_shape=[3, 3], pads=[1, 1, 1, 1], strides=[2, 2], onnx_name=\"Conv_24\"](%input.76, %onnx::Conv_248, %onnx::Conv_249) # /home/ml3/miniconda3/envs/fastai_22/lib/python3.10/site-packages/torch/nn/modules/conv.py:452:0\n",
      "  %onnx::Conv_169 : Float(10, 256, 14, 14, strides=[50176, 196, 14, 1], requires_grad=1, device=cuda:0) = onnx::Relu[onnx_name=\"Relu_25\"](%input.84) # /home/ml3/miniconda3/envs/fastai_22/lib/python3.10/site-packages/torch/nn/functional.py:1453:0\n",
      "  %onnx::Add_250 : Float(10, 256, 14, 14, strides=[50176, 196, 14, 1], requires_grad=1, device=cuda:0) = onnx::Conv[dilations=[1, 1], group=1, kernel_shape=[3, 3], pads=[1, 1, 1, 1], strides=[1, 1], onnx_name=\"Conv_26\"](%onnx::Conv_169, %onnx::Conv_251, %onnx::Conv_252) # /home/ml3/miniconda3/envs/fastai_22/lib/python3.10/site-packages/torch/nn/modules/conv.py:452:0\n",
      "  %onnx::Add_253 : Float(10, 256, 14, 14, strides=[50176, 196, 14, 1], requires_grad=1, device=cuda:0) = onnx::Conv[dilations=[1, 1], group=1, kernel_shape=[1, 1], pads=[0, 0, 0, 0], strides=[2, 2], onnx_name=\"Conv_27\"](%input.76, %onnx::Conv_254, %onnx::Conv_255) # /home/ml3/miniconda3/envs/fastai_22/lib/python3.10/site-packages/torch/nn/modules/conv.py:452:0\n",
      "  %onnx::Relu_174 : Float(10, 256, 14, 14, strides=[50176, 196, 14, 1], requires_grad=1, device=cuda:0) = onnx::Add[onnx_name=\"Add_28\"](%onnx::Add_250, %onnx::Add_253) # /home/ml3/miniconda3/envs/fastai_22/lib/python3.10/site-packages/torchvision/models/resnet.py:96:0\n",
      "  %input.96 : Float(10, 256, 14, 14, strides=[50176, 196, 14, 1], requires_grad=1, device=cuda:0) = onnx::Relu[onnx_name=\"Relu_29\"](%onnx::Relu_174) # /home/ml3/miniconda3/envs/fastai_22/lib/python3.10/site-packages/torch/nn/functional.py:1453:0\n",
      "  %input.104 : Float(10, 256, 14, 14, strides=[50176, 196, 14, 1], requires_grad=1, device=cuda:0) = onnx::Conv[dilations=[1, 1], group=1, kernel_shape=[3, 3], pads=[1, 1, 1, 1], strides=[1, 1], onnx_name=\"Conv_30\"](%input.96, %onnx::Conv_257, %onnx::Conv_258) # /home/ml3/miniconda3/envs/fastai_22/lib/python3.10/site-packages/torch/nn/modules/conv.py:452:0\n",
      "  %onnx::Conv_178 : Float(10, 256, 14, 14, strides=[50176, 196, 14, 1], requires_grad=1, device=cuda:0) = onnx::Relu[onnx_name=\"Relu_31\"](%input.104) # /home/ml3/miniconda3/envs/fastai_22/lib/python3.10/site-packages/torch/nn/functional.py:1453:0\n",
      "  %onnx::Add_259 : Float(10, 256, 14, 14, strides=[50176, 196, 14, 1], requires_grad=1, device=cuda:0) = onnx::Conv[dilations=[1, 1], group=1, kernel_shape=[3, 3], pads=[1, 1, 1, 1], strides=[1, 1], onnx_name=\"Conv_32\"](%onnx::Conv_178, %onnx::Conv_260, %onnx::Conv_261) # /home/ml3/miniconda3/envs/fastai_22/lib/python3.10/site-packages/torch/nn/modules/conv.py:452:0\n",
      "  %onnx::Relu_181 : Float(10, 256, 14, 14, strides=[50176, 196, 14, 1], requires_grad=1, device=cuda:0) = onnx::Add[onnx_name=\"Add_33\"](%onnx::Add_259, %input.96) # /home/ml3/miniconda3/envs/fastai_22/lib/python3.10/site-packages/torchvision/models/resnet.py:96:0\n",
      "  %input.112 : Float(10, 256, 14, 14, strides=[50176, 196, 14, 1], requires_grad=1, device=cuda:0) = onnx::Relu[onnx_name=\"Relu_34\"](%onnx::Relu_181) # /home/ml3/miniconda3/envs/fastai_22/lib/python3.10/site-packages/torch/nn/functional.py:1453:0\n",
      "  %input.120 : Float(10, 512, 7, 7, strides=[25088, 49, 7, 1], requires_grad=1, device=cuda:0) = onnx::Conv[dilations=[1, 1], group=1, kernel_shape=[3, 3], pads=[1, 1, 1, 1], strides=[2, 2], onnx_name=\"Conv_35\"](%input.112, %onnx::Conv_263, %onnx::Conv_264) # /home/ml3/miniconda3/envs/fastai_22/lib/python3.10/site-packages/torch/nn/modules/conv.py:452:0\n",
      "  %onnx::Conv_185 : Float(10, 512, 7, 7, strides=[25088, 49, 7, 1], requires_grad=1, device=cuda:0) = onnx::Relu[onnx_name=\"Relu_36\"](%input.120) # /home/ml3/miniconda3/envs/fastai_22/lib/python3.10/site-packages/torch/nn/functional.py:1453:0\n",
      "  %onnx::Add_265 : Float(10, 512, 7, 7, strides=[25088, 49, 7, 1], requires_grad=1, device=cuda:0) = onnx::Conv[dilations=[1, 1], group=1, kernel_shape=[3, 3], pads=[1, 1, 1, 1], strides=[1, 1], onnx_name=\"Conv_37\"](%onnx::Conv_185, %onnx::Conv_266, %onnx::Conv_267) # /home/ml3/miniconda3/envs/fastai_22/lib/python3.10/site-packages/torch/nn/modules/conv.py:452:0\n",
      "  %onnx::Add_268 : Float(10, 512, 7, 7, strides=[25088, 49, 7, 1], requires_grad=1, device=cuda:0) = onnx::Conv[dilations=[1, 1], group=1, kernel_shape=[1, 1], pads=[0, 0, 0, 0], strides=[2, 2], onnx_name=\"Conv_38\"](%input.112, %onnx::Conv_269, %onnx::Conv_270) # /home/ml3/miniconda3/envs/fastai_22/lib/python3.10/site-packages/torch/nn/modules/conv.py:452:0\n",
      "  %onnx::Relu_190 : Float(10, 512, 7, 7, strides=[25088, 49, 7, 1], requires_grad=1, device=cuda:0) = onnx::Add[onnx_name=\"Add_39\"](%onnx::Add_265, %onnx::Add_268) # /home/ml3/miniconda3/envs/fastai_22/lib/python3.10/site-packages/torchvision/models/resnet.py:96:0\n",
      "  %input.132 : Float(10, 512, 7, 7, strides=[25088, 49, 7, 1], requires_grad=1, device=cuda:0) = onnx::Relu[onnx_name=\"Relu_40\"](%onnx::Relu_190) # /home/ml3/miniconda3/envs/fastai_22/lib/python3.10/site-packages/torch/nn/functional.py:1453:0\n",
      "  %input.140 : Float(10, 512, 7, 7, strides=[25088, 49, 7, 1], requires_grad=1, device=cuda:0) = onnx::Conv[dilations=[1, 1], group=1, kernel_shape=[3, 3], pads=[1, 1, 1, 1], strides=[1, 1], onnx_name=\"Conv_41\"](%input.132, %onnx::Conv_272, %onnx::Conv_273) # /home/ml3/miniconda3/envs/fastai_22/lib/python3.10/site-packages/torch/nn/modules/conv.py:452:0\n",
      "  %onnx::Conv_194 : Float(10, 512, 7, 7, strides=[25088, 49, 7, 1], requires_grad=1, device=cuda:0) = onnx::Relu[onnx_name=\"Relu_42\"](%input.140) # /home/ml3/miniconda3/envs/fastai_22/lib/python3.10/site-packages/torch/nn/functional.py:1453:0\n",
      "  %onnx::Add_274 : Float(10, 512, 7, 7, strides=[25088, 49, 7, 1], requires_grad=1, device=cuda:0) = onnx::Conv[dilations=[1, 1], group=1, kernel_shape=[3, 3], pads=[1, 1, 1, 1], strides=[1, 1], onnx_name=\"Conv_43\"](%onnx::Conv_194, %onnx::Conv_275, %onnx::Conv_276) # /home/ml3/miniconda3/envs/fastai_22/lib/python3.10/site-packages/torch/nn/modules/conv.py:452:0\n",
      "  %onnx::Relu_197 : Float(10, 512, 7, 7, strides=[25088, 49, 7, 1], requires_grad=1, device=cuda:0) = onnx::Add[onnx_name=\"Add_44\"](%onnx::Add_274, %input.132) # /home/ml3/miniconda3/envs/fastai_22/lib/python3.10/site-packages/torchvision/models/resnet.py:96:0\n",
      "  %input.148 : Float(10, 512, 7, 7, strides=[25088, 49, 7, 1], requires_grad=1, device=cuda:0) = onnx::Relu[onnx_name=\"Relu_45\"](%onnx::Relu_197) # /home/ml3/miniconda3/envs/fastai_22/lib/python3.10/site-packages/torch/nn/functional.py:1453:0\n",
      "  %onnx::Concat_199 : Float(10, 512, 1, 1, strides=[512, 1, 1, 1], requires_grad=1, device=cuda:0) = onnx::MaxPool[kernel_shape=[7, 7], pads=[0, 0, 0, 0], strides=[7, 7], onnx_name=\"MaxPool_46\"](%input.148) # /home/ml3/miniconda3/envs/fastai_22/lib/python3.10/site-packages/torch/nn/functional.py:1117:0\n",
      "  %onnx::Concat_200 : Float(10, 512, 1, 1, strides=[512, 1, 1, 1], requires_grad=1, device=cuda:0) = onnx::GlobalAveragePool[onnx_name=\"GlobalAveragePool_47\"](%input.148) # /home/ml3/miniconda3/envs/fastai_22/lib/python3.10/site-packages/torch/nn/functional.py:1213:0\n",
      "  %onnx::Reshape_201 : Float(10, 1024, 1, 1, strides=[1024, 1, 1, 1], requires_grad=1, device=cuda:0) = onnx::Concat[axis=1, onnx_name=\"Concat_48\"](%onnx::Concat_199, %onnx::Concat_200) # /home/ml3/miniconda3/envs/fastai_22/lib/python3.10/site-packages/fastai/layers.py:128:0\n",
      "  %x : Float(10, 1024, strides=[1024, 1], requires_grad=1, device=cuda:0) = onnx::Reshape[onnx_name=\"Reshape_49\"](%onnx::Reshape_201, %onnx::Reshape_279) # /home/ml3/miniconda3/envs/fastai_22/lib/python3.10/site-packages/fastai/layers.py:74:0\n",
      "  %ret : Float(10, 1024, strides=[1024, 1], requires_grad=1, device=cuda:0) = onnx::BatchNormalization[epsilon=1.0000000000000001e-05, momentum=0.90000000000000002, onnx_name=\"BatchNormalization_50\"](%x, %1.2.weight, %1.2.bias, %1.2.running_mean, %1.2.running_var) # /home/ml3/miniconda3/envs/fastai_22/lib/python3.10/site-packages/torch/nn/functional.py:2423:0\n",
      "  %ret.3 : Float(10, 512, strides=[512, 1], requires_grad=1, device=cuda:0) = onnx::MatMul[onnx_name=\"MatMul_51\"](%ret, %onnx::MatMul_280) # /home/ml3/miniconda3/envs/fastai_22/lib/python3.10/site-packages/torch/_tensor.py:1117:0\n",
      "  %onnx::BatchNormalization_213 : Float(10, 512, strides=[512, 1], requires_grad=1, device=cuda:0) = onnx::Relu[onnx_name=\"Relu_52\"](%ret.3) # /home/ml3/miniconda3/envs/fastai_22/lib/python3.10/site-packages/torch/nn/functional.py:1453:0\n",
      "  %ret.7 : Float(10, 512, strides=[512, 1], requires_grad=1, device=cuda:0) = onnx::BatchNormalization[epsilon=1.0000000000000001e-05, momentum=0.90000000000000002, onnx_name=\"BatchNormalization_53\"](%onnx::BatchNormalization_213, %1.6.weight, %1.6.bias, %1.6.running_mean, %1.6.running_var) # /home/ml3/miniconda3/envs/fastai_22/lib/python3.10/site-packages/torch/nn/functional.py:2423:0\n",
      "  %output1 : Float(10, 2, strides=[2, 1], requires_grad=1, device=cuda:0) = onnx::MatMul[onnx_name=\"MatMul_54\"](%ret.7, %onnx::MatMul_281) # /home/ml3/miniconda3/envs/fastai_22/lib/python3.10/site-packages/torch/_tensor.py:1117:0\n",
      "  return (%output1)\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# From https://pytorch.org/docs/stable/onnx.html#example-alexnet-from-pytorch-to-onnx\n",
    "\n",
    "# Providing input and output names sets the display names for values\n",
    "# within the model's graph. Setting these does not change the semantics\n",
    "# of the graph; it is only for readability.\n",
    "#\n",
    "# The inputs to the network consist of the flat list of inputs (i.e.\n",
    "# the values you would pass to the forward() method) followed by the\n",
    "# flat list of parameters. You can partially specify names, i.e. provide\n",
    "# a list here shorter than the number of inputs to the model, and we will\n",
    "# only set that subset of names, starting from the beginning.\n",
    "\n",
    "\n",
    "input_names = [ \"actual_input_1\" ] + [ \"learned_%d\" % i for i in range(16) ]\n",
    "output_names = [ \"output1\" ]\n",
    "\n",
    "torch.onnx.export(model, dummy_input, \"onnx/cats_v_dogs.onnx\", verbose=True, input_names=input_names, output_names=output_names)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can now download the ONNX file and import it into a framework of your choice."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}