docs/tutorials/junos_parsing.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Parsing (JUNOS)\n",
    "\n",
    "One of the features ntc_rosetta supports is parsing native configuration and turning into data modelled after YANG models. For that purpose ntc_rosetta leverages [yangify](https://github.com/networktocode/yangify) and builds on top of it to make it more consumable.\n",
    "\n",
    "ntc_rosetta introduces the concept of \"drivers\". Drivers are objects that implements the parsing and translation of a given YANG model for a particular NOS. For instance, if you wanted to parse IOS configuration and convert it into data that follows the openconfig model you would load the corresponding driver like this:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "from ntc_rosetta import get_driver\n",
    "\n",
    "junos = get_driver(\"junos\", \"openconfig\")\n",
    "junos_driver = junos()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The same processor can also translate the given model to native configuration.\n",
    "\n",
    "Now, let's see how we can use this driver to parse IOS configuration and turn it into an Openconfig model. First, let's load some IOS configuration:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "with open(\"data/junos/dev_conf.xml\", \"r\") as f:\n",
    "    config = f.read()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<configuration>\n",
      "        <interfaces>\n",
      "            <interface>\n",
      "                <name>xe-0/0/1</name>\n",
      "                <unit>\n",
      "                    <name>0</name>\n",
      "                    <family>\n",
      "                        <ethernet-switching>\n",
      "                            <interface-mode>access</interface-mode>\n",
      "                            <vlan>\n",
      "                                <members>10</members>\n",
      "                            </vlan>\n",
      "                        </ethernet-switching>\n",
      "                    </family>\n",
      "                </unit>\n",
      "            </interface>\n",
      "            <interface>\n",
      "                <name>xe-0/0/3</name>\n",
      "                <unit>\n",
      "                    <name>0</name>\n",
      "                    <family>\n",
      "                        <ethernet-switching>\n",
      "                            <interface-mode>trunk</interface-mode>\n",
      "                            <vlan>\n",
      "                                <members>10</members>\n",
      "                                <members>20</members>\n",
      "                            </vlan>\n",
      "                        </ethernet-switching>\n",
      "                    </family>\n",
      "                </unit>\n",
      "            </interface>\n",
      "            <interface>\n",
      "                <name>xe-0/0/4</name>\n",
      "                <unit>\n",
      "                    <name>0</name>\n",
      "                    <family>\n",
      "                        <ethernet-switching>\n",
      "                            <interface-mode>trunk</interface-mode>\n",
      "                            <vlan>\n",
      "                                <members>VLAN-100</members>\n",
      "                                <members>VLAN-200</members>\n",
      "                            </vlan>\n",
      "                        </ethernet-switching>\n",
      "                    </family>\n",
      "                </unit>\n",
      "            </interface>\n",
      "            <interface>\n",
      "                <name>xe-0/0/5</name>\n",
      "                <unit>\n",
      "                    <name>0</name>\n",
      "                    <family>\n",
      "                        <ethernet-switching>\n",
      "                            <interface-mode>access</interface-mode>\n",
      "                            <vlan>\n",
      "                                <members>VLAN-100</members>\n",
      "                            </vlan>\n",
      "                        </ethernet-switching>\n",
      "                    </family>\n",
      "                </unit>\n",
      "            </interface>\n",
      "        </interfaces>\n",
      "    <vlans>\n",
      "            <vlan>\n",
      "                <name>default</name>\n",
      "                <vlan-id>1</vlan-id>\n",
      "            </vlan>\n",
      "            <vlan>\n",
      "                <name>prod</name>\n",
      "                <vlan-id>20</vlan-id>\n",
      "            </vlan>\n",
      "            <vlan inactive=\"inactive\">\n",
      "                <vlan-id>10</vlan-id>\n",
      "            </vlan>\n",
      "            <vlan>\n",
      "                <name>VLAN-100</name>\n",
      "                <vlan-id>100</vlan-id>\n",
      "            </vlan>\n",
      "            <vlan>\n",
      "                <name>VLAN-200</name>\n",
      "                <vlan-id>200</vlan-id>\n",
      "            </vlan>\n",
      "    </vlans>\n",
      "</configuration>\n",
      "\n"
     ]
    }
   ],
   "source": [
    "print(config)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Once the configuration is loaded, you need to parse it. The parser has some conventions you have to be aware of, for instance, when parsing configuration, it's going to expect you pass a `native` argument with a dictionary where the key `dev_conf` is the native configuration:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "parsed = junos_driver.parse(native={\"dev_conf\": config})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "That's literally all you have to do parse the native configuration and turn it into structured data. We can check the result by dumping the `parsed.raw_value()`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\n",
      "    \"openconfig-interfaces:interfaces\": {\n",
      "        \"interface\": [\n",
      "            {\n",
      "                \"name\": \"xe-0/0/1\",\n",
      "                \"config\": {\n",
      "                    \"name\": \"xe-0/0/1\",\n",
      "                    \"type\": \"iana-if-type:ethernetCsmacd\",\n",
      "                    \"enabled\": true\n",
      "                },\n",
      "                \"subinterfaces\": {\n",
      "                    \"subinterface\": [\n",
      "                        {\n",
      "                            \"index\": 0,\n",
      "                            \"config\": {\n",
      "                                \"index\": 0\n",
      "                            }\n",
      "                        }\n",
      "                    ]\n",
      "                },\n",
      "                \"openconfig-if-ethernet:ethernet\": {\n",
      "                    \"openconfig-vlan:switched-vlan\": {\n",
      "                        \"config\": {\n",
      "                            \"interface-mode\": \"ACCESS\",\n",
      "                            \"access-vlan\": 10\n",
      "                        }\n",
      "                    }\n",
      "                }\n",
      "            },\n",
      "            {\n",
      "                \"name\": \"xe-0/0/3\",\n",
      "                \"config\": {\n",
      "                    \"name\": \"xe-0/0/3\",\n",
      "                    \"type\": \"iana-if-type:ethernetCsmacd\",\n",
      "                    \"enabled\": true\n",
      "                },\n",
      "                \"subinterfaces\": {\n",
      "                    \"subinterface\": [\n",
      "                        {\n",
      "                            \"index\": 0,\n",
      "                            \"config\": {\n",
      "                                \"index\": 0\n",
      "                            }\n",
      "                        }\n",
      "                    ]\n",
      "                },\n",
      "                \"openconfig-if-ethernet:ethernet\": {\n",
      "                    \"openconfig-vlan:switched-vlan\": {\n",
      "                        \"config\": {\n",
      "                            \"interface-mode\": \"TRUNK\",\n",
      "                            \"trunk-vlans\": [\n",
      "                                10,\n",
      "                                20\n",
      "                            ]\n",
      "                        }\n",
      "                    }\n",
      "                }\n",
      "            },\n",
      "            {\n",
      "                \"name\": \"xe-0/0/4\",\n",
      "                \"config\": {\n",
      "                    \"name\": \"xe-0/0/4\",\n",
      "                    \"type\": \"iana-if-type:ethernetCsmacd\",\n",
      "                    \"enabled\": true\n",
      "                },\n",
      "                \"subinterfaces\": {\n",
      "                    \"subinterface\": [\n",
      "                        {\n",
      "                            \"index\": 0,\n",
      "                            \"config\": {\n",
      "                                \"index\": 0\n",
      "                            }\n",
      "                        }\n",
      "                    ]\n",
      "                },\n",
      "                \"openconfig-if-ethernet:ethernet\": {\n",
      "                    \"openconfig-vlan:switched-vlan\": {\n",
      "                        \"config\": {\n",
      "                            \"interface-mode\": \"TRUNK\",\n",
      "                            \"trunk-vlans\": [\n",
      "                                100,\n",
      "                                200\n",
      "                            ]\n",
      "                        }\n",
      "                    }\n",
      "                }\n",
      "            },\n",
      "            {\n",
      "                \"name\": \"xe-0/0/5\",\n",
      "                \"config\": {\n",
      "                    \"name\": \"xe-0/0/5\",\n",
      "                    \"type\": \"iana-if-type:ethernetCsmacd\",\n",
      "                    \"enabled\": true\n",
      "                },\n",
      "                \"subinterfaces\": {\n",
      "                    \"subinterface\": [\n",
      "                        {\n",
      "                            \"index\": 0,\n",
      "                            \"config\": {\n",
      "                                \"index\": 0\n",
      "                            }\n",
      "                        }\n",
      "                    ]\n",
      "                },\n",
      "                \"openconfig-if-ethernet:ethernet\": {\n",
      "                    \"openconfig-vlan:switched-vlan\": {\n",
      "                        \"config\": {\n",
      "                            \"interface-mode\": \"ACCESS\",\n",
      "                            \"access-vlan\": 100\n",
      "                        }\n",
      "                    }\n",
      "                }\n",
      "            }\n",
      "        ]\n",
      "    },\n",
      "    \"openconfig-network-instance:network-instances\": {\n",
      "        \"network-instance\": [\n",
      "            {\n",
      "                \"name\": \"default\",\n",
      "                \"config\": {\n",
      "                    \"name\": \"default\"\n",
      "                },\n",
      "                \"vlans\": {\n",
      "                    \"vlan\": [\n",
      "                        {\n",
      "                            \"vlan-id\": 1,\n",
      "                            \"config\": {\n",
      "                                \"vlan-id\": 1,\n",
      "                                \"name\": \"default\",\n",
      "                                \"status\": \"ACTIVE\"\n",
      "                            }\n",
      "                        },\n",
      "                        {\n",
      "                            \"vlan-id\": 20,\n",
      "                            \"config\": {\n",
      "                                \"vlan-id\": 20,\n",
      "                                \"name\": \"prod\",\n",
      "                                \"status\": \"ACTIVE\"\n",
      "                            }\n",
      "                        },\n",
      "                        {\n",
      "                            \"vlan-id\": 10,\n",
      "                            \"config\": {\n",
      "                                \"vlan-id\": 10,\n",
      "                                \"status\": \"SUSPENDED\"\n",
      "                            }\n",
      "                        },\n",
      "                        {\n",
      "                            \"vlan-id\": 100,\n",
      "                            \"config\": {\n",
      "                                \"vlan-id\": 100,\n",
      "                                \"name\": \"VLAN-100\",\n",
      "                                \"status\": \"ACTIVE\"\n",
      "                            }\n",
      "                        },\n",
      "                        {\n",
      "                            \"vlan-id\": 200,\n",
      "                            \"config\": {\n",
      "                                \"vlan-id\": 200,\n",
      "                                \"name\": \"VLAN-200\",\n",
      "                                \"status\": \"ACTIVE\"\n",
      "                            }\n",
      "                        }\n",
      "                    ]\n",
      "                }\n",
      "            }\n",
      "        ]\n",
      "    }\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "import json\n",
    "print(json.dumps(parsed.raw_value(), indent=4))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "ntc_rosetta, also let's you parse some parts of the model, however, you need to be aware that might break the validation of the object:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "error: [/openconfig-network-instance:network-instances/network-instance/0/name] instance-required\n"
     ]
    }
   ],
   "source": [
    "from yangson.exceptions import SemanticError\n",
    "try:\n",
    "    parsed_vlans = junos_driver.parse(\n",
    "        native={\"dev_conf\": config},\n",
    "        include=[\n",
    "            \"/openconfig-network-instance:network-instances/network-instance/vlans\",        \n",
    "        ]\n",
    "    )\n",
    "except SemanticError as e:\n",
    "    print(f\"error: {e}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can workaround this in two ways:\n",
    "1. By disabling the validation of the object\n",
    "2. By parsing all the necessary elements to make the object compliant.\n",
    "\n",
    "You can disable the validation of the object by passing `validate=False`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\n",
      "    \"openconfig-network-instance:network-instances\": {\n",
      "        \"network-instance\": [\n",
      "            {\n",
      "                \"name\": \"default\",\n",
      "                \"vlans\": {\n",
      "                    \"vlan\": [\n",
      "                        {\n",
      "                            \"vlan-id\": 1,\n",
      "                            \"config\": {\n",
      "                                \"vlan-id\": 1,\n",
      "                                \"name\": \"default\",\n",
      "                                \"status\": \"ACTIVE\"\n",
      "                            }\n",
      "                        },\n",
      "                        {\n",
      "                            \"vlan-id\": 20,\n",
      "                            \"config\": {\n",
      "                                \"vlan-id\": 20,\n",
      "                                \"name\": \"prod\",\n",
      "                                \"status\": \"ACTIVE\"\n",
      "                            }\n",
      "                        },\n",
      "                        {\n",
      "                            \"vlan-id\": 10,\n",
      "                            \"config\": {\n",
      "                                \"vlan-id\": 10,\n",
      "                                \"status\": \"SUSPENDED\"\n",
      "                            }\n",
      "                        },\n",
      "                        {\n",
      "                            \"vlan-id\": 100,\n",
      "                            \"config\": {\n",
      "                                \"vlan-id\": 100,\n",
      "                                \"name\": \"VLAN-100\",\n",
      "                                \"status\": \"ACTIVE\"\n",
      "                            }\n",
      "                        },\n",
      "                        {\n",
      "                            \"vlan-id\": 200,\n",
      "                            \"config\": {\n",
      "                                \"vlan-id\": 200,\n",
      "                                \"name\": \"VLAN-200\",\n",
      "                                \"status\": \"ACTIVE\"\n",
      "                            }\n",
      "                        }\n",
      "                    ]\n",
      "                }\n",
      "            }\n",
      "        ]\n",
      "    }\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "parsed_vlans = junos_driver.parse(\n",
    "    native={\"dev_conf\": config},\n",
    "    validate=False,\n",
    "    include=[\n",
    "        \"/openconfig-network-instance:network-instances/network-instance/vlans\",        \n",
    "    ]\n",
    ")\n",
    "print(json.dumps(parsed_vlans.raw_value(), indent=4))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And you can make sure your object is valid by passing the list of elements that are needed to make the object compliant:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\n",
      "    \"openconfig-network-instance:network-instances\": {\n",
      "        \"network-instance\": [\n",
      "            {\n",
      "                \"name\": \"default\",\n",
      "                \"config\": {\n",
      "                    \"name\": \"default\"\n",
      "                },\n",
      "                \"vlans\": {\n",
      "                    \"vlan\": [\n",
      "                        {\n",
      "                            \"vlan-id\": 1,\n",
      "                            \"config\": {\n",
      "                                \"vlan-id\": 1,\n",
      "                                \"name\": \"default\",\n",
      "                                \"status\": \"ACTIVE\"\n",
      "                            }\n",
      "                        },\n",
      "                        {\n",
      "                            \"vlan-id\": 20,\n",
      "                            \"config\": {\n",
      "                                \"vlan-id\": 20,\n",
      "                                \"name\": \"prod\",\n",
      "                                \"status\": \"ACTIVE\"\n",
      "                            }\n",
      "                        },\n",
      "                        {\n",
      "                            \"vlan-id\": 10,\n",
      "                            \"config\": {\n",
      "                                \"vlan-id\": 10,\n",
      "                                \"status\": \"SUSPENDED\"\n",
      "                            }\n",
      "                        },\n",
      "                        {\n",
      "                            \"vlan-id\": 100,\n",
      "                            \"config\": {\n",
      "                                \"vlan-id\": 100,\n",
      "                                \"name\": \"VLAN-100\",\n",
      "                                \"status\": \"ACTIVE\"\n",
      "                            }\n",
      "                        },\n",
      "                        {\n",
      "                            \"vlan-id\": 200,\n",
      "                            \"config\": {\n",
      "                                \"vlan-id\": 200,\n",
      "                                \"name\": \"VLAN-200\",\n",
      "                                \"status\": \"ACTIVE\"\n",
      "                            }\n",
      "                        }\n",
      "                    ]\n",
      "                }\n",
      "            }\n",
      "        ]\n",
      "    }\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "parsed_vlans = junos_driver.parse(\n",
    "    native={\"dev_conf\": config},\n",
    "    include=[\n",
    "        \"/openconfig-network-instance:network-instances/network-instance/name\",\n",
    "        \"/openconfig-network-instance:network-instances/network-instance/config\",\n",
    "        \"/openconfig-network-instance:network-instances/network-instance/vlans\",        \n",
    "    ]\n",
    ")\n",
    "print(json.dumps(parsed_vlans.raw_value(), indent=4))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}