tutorials/How to read yaml files.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "9da924b8",
   "metadata": {
    "toc": true
   },
   "source": [
    "<h1>Table of Contents<span class=\"tocSkip\"></span></h1>\n",
    "<div class=\"toc\"><ul class=\"toc-item\"><li><span><a href=\"#Introduction\" data-toc-modified-id=\"Introduction-1\"><span class=\"toc-item-num\">1&nbsp;&nbsp;</span>Introduction</a></span></li><li><span><a href=\"#What-is-an-YAML-file?\" data-toc-modified-id=\"What-is-an-YAML-file?-2\"><span class=\"toc-item-num\">2&nbsp;&nbsp;</span>What is an YAML file?</a></span></li><li><span><a href=\"#Imports\" data-toc-modified-id=\"Imports-3\"><span class=\"toc-item-num\">3&nbsp;&nbsp;</span>Imports</a></span></li><li><span><a href=\"#Define-data\" data-toc-modified-id=\"Define-data-4\"><span class=\"toc-item-num\">4&nbsp;&nbsp;</span>Define data</a></span></li><li><span><a href=\"#Write-YAML-file\" data-toc-modified-id=\"Write-YAML-file-5\"><span class=\"toc-item-num\">5&nbsp;&nbsp;</span>Write YAML file</a></span></li><li><span><a href=\"#Read-YAML-file\" data-toc-modified-id=\"Read-YAML-file-6\"><span class=\"toc-item-num\">6&nbsp;&nbsp;</span>Read YAML file</a></span></li><li><span><a href=\"#Sanity-check-write+read\" data-toc-modified-id=\"Sanity-check-write+read-7\"><span class=\"toc-item-num\">7&nbsp;&nbsp;</span>Sanity check write+read</a></span></li><li><span><a href=\"#OmegaConf\" data-toc-modified-id=\"OmegaConf-8\"><span class=\"toc-item-num\">8&nbsp;&nbsp;</span><code>OmegaConf</code></a></span></li><li><span><a href=\"#References\" data-toc-modified-id=\"References-9\"><span class=\"toc-item-num\">9&nbsp;&nbsp;</span>References</a></span></li></ul></div>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "84b00ec3",
   "metadata": {},
   "source": [
    "# Introduction\n",
    "<hr style = \"border:2px solid black\" ></hr>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9179d0b6",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-warning\">\n",
    "<font color=black>\n",
    "\n",
    "**What?** How to read an YMAL file\n",
    "\n",
    "</font>\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f386383b",
   "metadata": {},
   "source": [
    "# What is an YAML file?\n",
    "<hr style = \"border:2px solid black\" ></hr>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "725f1e87",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-info\">\n",
    "<font color=black>\n",
    "\n",
    "- It is a human-readable data-serialization language. It is commonly used for configuration files and in applications where data is being stored or transmitted.\n",
    "- Both JSON and YAML are developed to provide a human-readable data interchange format. \n",
    "- The YAML is realized as a superset of JSON format.\n",
    "- It means that we can parse JSON using a YAML parser. Although the practical implementation of this theory is little tricky.\n",
    "- Be aware thath YAML is an extremely bad choice for any configuration file because it's wildly unpredictable.\n",
    "\n",
    "</font>\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e6e321af",
   "metadata": {},
   "source": [
    "# Imports\n",
    "<hr style = \"border:2px solid black\" ></hr>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "beadbfd4",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-02-20T10:52:38.665412Z",
     "start_time": "2023-02-20T10:52:38.638783Z"
    }
   },
   "outputs": [],
   "source": [
    "import yaml\n",
    "import io"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "99750f04",
   "metadata": {},
   "source": [
    "# Define data\n",
    "<hr style = \"border:2px solid black\" ></hr>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "58bb0538",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-02-20T10:53:06.198316Z",
     "start_time": "2023-02-20T10:53:06.194699Z"
    }
   },
   "outputs": [],
   "source": [
    "# Define data\n",
    "data = {\n",
    "    'a list': [\n",
    "        1, \n",
    "        42, \n",
    "        3.141, \n",
    "        1337, \n",
    "        'help', \n",
    "        u'€'\n",
    "    ],\n",
    "    'a string': 'bla',\n",
    "    'another dict': {\n",
    "        'foo': 'bar',\n",
    "        'key': 'value',\n",
    "        'the answer': 42\n",
    "    }\n",
    "}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "120c58a0",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-02-20T10:53:10.647641Z",
     "start_time": "2023-02-20T10:53:10.638525Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'a list': [1, 42, 3.141, 1337, 'help', '€'],\n",
       " 'a string': 'bla',\n",
       " 'another dict': {'foo': 'bar', 'key': 'value', 'the answer': 42}}"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b37b5067",
   "metadata": {},
   "source": [
    "# Write YAML file\n",
    "<hr style = \"border:2px solid black\" ></hr>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "c3a3f131",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-02-20T10:53:30.270166Z",
     "start_time": "2023-02-20T10:53:30.266139Z"
    }
   },
   "outputs": [],
   "source": [
    "# Write data\n",
    "with io.open('./data/data.yaml', 'w', encoding='utf8') as outfile:\n",
    "    yaml.dump(data, outfile, default_flow_style=False, allow_unicode=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e160d212",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-02-20T10:44:56.098759Z",
     "start_time": "2023-02-20T10:44:56.094276Z"
    }
   },
   "source": [
    "# Read YAML file\n",
    "<hr style = \"border:2px solid black\" ></hr>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "d3a6cab7",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-02-20T10:54:00.372812Z",
     "start_time": "2023-02-20T10:54:00.367616Z"
    }
   },
   "outputs": [],
   "source": [
    "# Read YAML file\n",
    "with open(\"./data/data.yaml\", 'r') as stream:\n",
    "    data_loaded = yaml.safe_load(stream)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cc2092a4",
   "metadata": {},
   "source": [
    "# Sanity check write+read\n",
    "<hr style = \"border:2px solid black\" ></hr>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "af5c64be",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-02-20T10:54:14.674879Z",
     "start_time": "2023-02-20T10:54:14.671266Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data == data_loaded"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e520133d",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-02-23T10:01:17.428010Z",
     "start_time": "2023-02-23T10:01:17.424431Z"
    }
   },
   "source": [
    "# `OmegaConf`\n",
    "<hr style = \"border:2px solid black\" ></hr>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "62687fe5",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-info\">\n",
    "<font color=black>\n",
    "\n",
    "- Hydra operates on top of OmegaConf, which is a YAML based hierarchical configuration system, with support for merging configurations from multiple sources (files, CLI argument, environment variables) providing a consistent API regardless of how the configuration was created.\n",
    "\n",
    "</font>\n",
    "</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "c4d1fef3",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-02-23T10:02:44.628632Z",
     "start_time": "2023-02-23T10:02:44.621616Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'a list': [1, 42, 3.141, 1337, 'help', '€'], 'a string': 'bla', 'another dict': {'foo': 'bar', 'key': 'value', 'the answer': 42}}\n",
      "bla\n"
     ]
    }
   ],
   "source": [
    "from omegaconf import OmegaConf\n",
    "\n",
    "# Loading\n",
    "config = OmegaConf.load('./data/data.yaml')\n",
    "\n",
    "# Accessing\n",
    "print(config)\n",
    "print(config[\"a string\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d7dd6171",
   "metadata": {},
   "source": [
    "# References\n",
    "<hr style = \"border:2px solid black\" ></hr>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "08841ecb",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-warning\">\n",
    "<font color=black>\n",
    "\n",
    "- https://stackoverflow.com/questions/1773805/how-can-i-parse-a-yaml-file-in-python\n",
    "- https://github.com/omry/omegaconf\n",
    "\n",
    "</font>\n",
    "</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a0e1e631",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "trainingAI",
   "language": "python",
   "name": "trainingai"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.7"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": true,
   "toc_position": {
    "height": "calc(100% - 180px)",
    "left": "10px",
    "top": "150px",
    "width": "209.35px"
   },
   "toc_section_display": true,
   "toc_window_display": true
  },
  "varInspector": {
   "cols": {
    "lenName": 16,
    "lenType": 16,
    "lenVar": 40
   },
   "kernels_config": {
    "python": {
     "delete_cmd_postfix": "",
     "delete_cmd_prefix": "del ",
     "library": "var_list.py",
     "varRefreshCmd": "print(var_dic_list())"
    },
    "r": {
     "delete_cmd_postfix": ") ",
     "delete_cmd_prefix": "rm(",
     "library": "var_list.r",
     "varRefreshCmd": "cat(var_dic_list()) "
    }
   },
   "types_to_exclude": [
    "module",
    "function",
    "builtin_function_or_method",
    "instance",
    "_Feature"
   ],
   "window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}