tutorials/data_expectations.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "source": [
    "# YData Quality - Data Expectations Tutorial\n",
    "Time-to-Value: 8 minutes\n",
    "\n",
    "This notebook provides a tutorial for the ydata_quality package integration of the Great Expectations library for managing data expectations.\n",
    "\n",
    "**Structure:**\n",
    "\n",
    "1. A data expectations introduction\n",
    "2. Load example dataset\n",
    "3. Instantiate the Data Quality engine\n",
    "4. Run the quality checks\n",
    "5. Assess the warnings\n",
    "6. (Extra) Detailed overview"
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "## A data expectations introduction\n",
    "### What are data expectations?\n",
    "Detecting inconsistencies or even errors in data can sometimes be a trivial task, but surely this is far from being the norm. Many times this task requires minucious inspection of lots of data structures or advanced domain knowledge that allows a user to confidently label any shortcoming.\n",
    "\n",
    "Consider __[Test-Driven-Development](https://en.wikipedia.org/wiki/Test-driven_development)__ (TDD) for a moment. In a TDD process, software requirements are realized into test cases before the development of the software itself. Software changes are constantly ran against these test cases in order to, hopefully, detect any sort of problem that might occur. A full software pipeline can be tested in this fashion to establish a quality assurance protocol, warrant a green light for a production push, and supporting refactorizations.\n",
    "\n",
    "But what about data? What if you could generalize domain knowledge, and generally expected data behaviour, into the datasets you manipulate, either internally sourced or from third parties? In fact many teams already do this in one way or another, but generally resorting to ad-hoc and hard to generalize processes. Taking the lesson from TDD, if we could develop a set of verifiable tests that work just like software test cases we would also get the same benefits.\n",
    "\n",
    "**Data Expectation** is the name we use for unit tests applied to data, to define an expectation about data is to develop a unit test that asserts a certain property about the data and provides an actionable output in any deviation.\n",
    "\n",
    "### What is Great Expectations?\n",
    "__[Great Expectations](https://greatexpectations.io/)__ is a Python tool for creating and running data expectations suite, allowing you to validate, profile your data, automate report creation in the form of HTML documents and store validation logs. Great Expectations offers a wide range of built-in expectations but also allows you to define custom expectations that better fit to your needs.\n",
    "\n",
    "### How can I leverage my Great Expectations project with YData Quality?\n",
    "It's simple!\n",
    "\n",
    "1. Locate the validations directory of your Great Expectations project, which should be under the *uncommitted* directory. There you will find a set of folders, one for each validation run that you executed.\n",
    "2. Choose a validation run to which you would like to get more insight, and copy the path to the json file.\n",
    "3. Instantiate a DataExpectationsReporter engine and run evaluate by providing the json file path.\n",
    "\n",
    "Congratulations you are all set!\n"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "source": [
    "import pandas as pd\n",
    "\n",
    "from ydata_quality.data_expectations import DataExpectationsReporter"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "## Load the example dataset and path to Great Expectations validation run\n",
    "We will use a demo from the GE tutorials. The taxi ride dataset and a log from a validation run on this dataset."
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "source": [
    "# This is the DataFrame used in the demo from GE tutorials\n",
    "df = pd.read_csv('../datasets/original/taxi_yellow_tripdata_sample_2019-01.csv')\n",
    "\n",
    "# This is a sample json log taken from a validation run\n",
    "results_json_path = '../datasets/original/taxi_long.json'"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "## Create the engine\n",
    "Each engine contains the checks and tests for each suite."
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "source": [
    "der = DataExpectationsReporter()"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "### Full Evaluation\n",
    "The easiest way to assess the data quality analysis is to run `.evaluate()` which returns a dictionary with outputs of operation performed. "
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "source": [
    "results = der.evaluate(results_json_path, df)"
   ],
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "\u001b[1mWarnings:\u001b[0m\n",
      "\tTOTAL: 9 warning(s)\n",
      "\t\u001b[1m\u001b[38;5;11mPriority 2\u001b[0m: 2 warning(s)\n",
      "\t\u001b[1m\u001b[38;5;69mPriority 3\u001b[0m: 7 warning(s)\n",
      "\n",
      "\n",
      "\u001b[38;5;11m\u001b[1mPriority 2\u001b[0m - \u001b[1musage allowed, limited human intelligibility\u001b[0m:\n",
      "\t\u001b[38;5;11m*\u001b[0m \u001b[1m[DATA EXPECTATIONS\u001b[0m - \u001b[4mOVERALL ASSESSMENT]\u001b[0m 10 expectations have failed, which is more than the implied absolute threshold of 0 failed expectations.\n",
      "\t\u001b[38;5;11m*\u001b[0m \u001b[1m[DATA EXPECTATIONS\u001b[0m - \u001b[4mCOVERAGE FRACTION]\u001b[0m The provided DataFrame has a total expectation coverage of 11% of its columns, which is below the expected coverage of 75%.\n",
      "\u001b[38;5;69m\u001b[1mPriority 3\u001b[0m - \u001b[1mminor impact, aesthetic\u001b[0m:\n",
      "\t\u001b[38;5;69m*\u001b[0m \u001b[1m[DATA EXPECTATIONS\u001b[0m - \u001b[4mEXPECTATION ASSESSMENT - VALUE BETWEEN]\u001b[0m Column passenger_count - The observed value is outside of the expected range.\n",
      "\t- The observed value is -100% deviated from the nearest bound of the expected range.\n",
      "\t\u001b[38;5;69m*\u001b[0m \u001b[1m[DATA EXPECTATIONS\u001b[0m - \u001b[4mEXPECTATION ASSESSMENT - VALUE BETWEEN]\u001b[0m Column trip_distance - The observed value is outside of the expected range.\n",
      "\t- The observed value is 5% deviated from the nearest bound of the expected range.\n",
      "\t\u001b[38;5;69m*\u001b[0m \u001b[1m[DATA EXPECTATIONS\u001b[0m - \u001b[4mEXPECTATION ASSESSMENT - VALUE BETWEEN]\u001b[0m Column trip_distance - The observed value is outside of the expected range.\n",
      "\t- The observed value is 1% deviated from the nearest bound of the expected range.\n",
      "\t\u001b[38;5;69m*\u001b[0m \u001b[1m[DATA EXPECTATIONS\u001b[0m - \u001b[4mEXPECTATION ASSESSMENT - VALUE BETWEEN]\u001b[0m Column trip_distance - The observed value is outside of the expected range.\n",
      "\t- The observed value is -35% deviated from the nearest bound of the expected range.\n",
      "\t\u001b[38;5;69m*\u001b[0m \u001b[1m[DATA EXPECTATIONS\u001b[0m - \u001b[4mEXPECTATION ASSESSMENT - VALUE BETWEEN]\u001b[0m Column passenger_count - The observed value is outside of the expected range.\n",
      "\t- The observed value is -14% deviated from the nearest bound of the expected range.\n",
      "\t\u001b[38;5;69m*\u001b[0m \u001b[1m[DATA EXPECTATIONS\u001b[0m - \u001b[4mEXPECTATION ASSESSMENT - VALUE BETWEEN]\u001b[0m Column trip_distance - The observed value is outside of the expected range.\n",
      "\t- The observed value is 3% deviated from the nearest bound of the expected range.\n",
      "\t\u001b[38;5;69m*\u001b[0m \u001b[1m[DATA EXPECTATIONS\u001b[0m - \u001b[4mEXPECTATION ASSESSMENT - VALUE BETWEEN]\u001b[0m Column passenger_count - The observed value is outside of the expected range.\n",
      "\t- The observed value is 17% deviated from the nearest bound of the expected range.\n"
     ]
    },
    {
     "output_type": "stream",
     "name": "stderr",
     "text": [
      "/home/francisco/Documents/git/ydata-quality/.venv/lib/python3.8/site-packages/pandas/core/internals/blocks.py:1002: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray.\n",
      "  arr_value = np.array(value)\n"
     ]
    }
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "## Check the status\n",
    "After running the data quality checks, you can check the warnings for each individual operation over the GE validation log. The warnings are sorted by priority and have additional details that can provide better insights for Data Scientists."
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "source": [
    "warnings = der.get_warnings()\n",
    "warnings[0]"
   ],
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": [
       "QualityWarning(category='Data Expectations', test='Overall Assessment', description='10 expectations have failed, which is more than the implied absolute threshold of 0 failed expectations.', priority=<Priority.P2: 2>, data={'Failed expectation indexes': [2, 4, 6, 7, 9, 12, 13, 14, 15, 17]})"
      ]
     },
     "metadata": {},
     "execution_count": 5
    }
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "## Full Test Suite\n",
    "In this section, you will find a detailed overview of the available tests in the data expectations module of ydata_quality. These are all run with the `evaluate` method, which centralizes input arguments and produces specific outputs in the returned results dictionary, structured by test."
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "source": [
    "# Results object structure\n",
    "list(results.keys())"
   ],
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": [
       "['Coverage Fraction', 'Overall Assessment', 'Expectation Level Assessment']"
      ]
     },
     "metadata": {},
     "execution_count": 6
    }
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "### Overall assessment\n",
    "\n",
    "This method controls for errors in the expectation suite level.\n",
    "It receives your results_json_path and 2 optional arguments.\n",
    "The default is a 0 failed expectations tolerance. You can configure this threshold using one of two arguments:\n",
    "1. An integer for the maximum number of expectations you tolerate as failures, error_tol\n",
    "\n",
    "or\n",
    "\n",
    "\n",
    "2. The fraction of expectations you tolerate as failures, rel_error_tol\n",
    "\n",
    "Any number of failed expectations greater than the one implied by these arguments will store a warning that will be part of the report. The data object of this warning consists in a dictionary with a value consisting in the list of indexes of the failed expectations, according to your expectation suite. The same list is stored in the returned results of `evaluate`. Note that the IDs are not arbitrary, they obey the order implied by your expectation suite, according to zero-based numbering."
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "source": [
    "failed_expectations_ids = results['Overall Assessment']\n",
    "failed_expectations_ids"
   ],
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": [
       "[2, 4, 6, 7, 9, 12, 13, 14, 15, 17]"
      ]
     },
     "metadata": {},
     "execution_count": 7
    }
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "### Coverage fraction\n",
    "\n",
    "This method controls for total expectation coverage of your dataset.\n",
    "It receives your results_json_path, the dataset for which you ran your validation and 1 optional argument.\n",
    "By default the engine expects a minimum coverage of 70% of your columns by the expectation suite. Coverage is considered only in specific column expectations, table expectations like __[the list of table expectations](https://docs.greatexpectations.io/docs/reference/glossary_of_expectations#table-shape)__ are not considered here. You can callibrate your desired minimum coverage by providing the argument minimum_coverage as a fraction.\n",
    "\n",
    "Any coverage inferior to the one implied by these arguments will store a warning that will be part of the report. The data object of this warning is the set of columns of your dataset that are not covered by the expectation suite. The method stores in the results object the fraction of columns of your dataset that are covered by at least one expectation.\n",
    "\n",
    "Additionally, if there are any expectations in your expectation suite meant for a column which cannot be found on the provided dataset, an exception will be raised which will be captured by the `evaluate` method."
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "source": [
    "coverage_fraction = results['Coverage Fraction']\n",
    "coverage_fraction"
   ],
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": [
       "0.1111111111111111"
      ]
     },
     "metadata": {},
     "execution_count": 8
    }
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "### Expectation level assessment\n",
    "\n",
    "This method checks the success of your expectation suite at the expectation level.\n",
    "It receives one argument, your results_json_path.\n",
    "\n",
    "\n",
    "It stores no warnings directly but depending on the failing expectations, it may call private methods to further digest the stored information. These expectation specific methods can store warnings for you to have some additional insight into what is wrong.\n",
    "\n",
    "\n",
    "After running, a tuple will be returned with the following contents:\n",
    "1. A report containing the status for all your expectations, succesful or not and with additional information on the failed expectations. Further information for interpreting the error metrics is provided in the stored warnings description.\n",
    "2. A dense representation of the expectations, including: \n",
    "    * Results format;\n",
    "    * Success status;\n",
    "    * Expectation type;\n",
    "    * Flag indicating if it is a table expectation;\n",
    "    * General kwargs;\n",
    "    * Column kwargs."
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "source": [
    "expectations_report, expectations_dense = results['Expectation Level Assessment']\n",
    "expectations_report"
   ],
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": [
       "                                     Expectation type Successful?  \\\n",
       "0          expect_table_columns_to_match_ordered_list        True   \n",
       "1                expect_table_row_count_to_be_between        True   \n",
       "2                     expect_column_min_to_be_between       False   \n",
       "3                     expect_column_max_to_be_between        True   \n",
       "4                    expect_column_mean_to_be_between       False   \n",
       "5                  expect_column_median_to_be_between        True   \n",
       "6         expect_column_quantile_values_to_be_between       False   \n",
       "7                   expect_column_values_to_be_in_set       False   \n",
       "8                 expect_column_values_to_not_be_null        True   \n",
       "9   expect_column_proportion_of_unique_values_to_b...       False   \n",
       "10            expect_column_values_to_be_in_type_list        True   \n",
       "11                    expect_column_min_to_be_between        True   \n",
       "12                    expect_column_max_to_be_between       False   \n",
       "13                   expect_column_mean_to_be_between       False   \n",
       "14                 expect_column_median_to_be_between       False   \n",
       "15        expect_column_quantile_values_to_be_between       False   \n",
       "16                expect_column_values_to_not_be_null        True   \n",
       "17  expect_column_proportion_of_unique_values_to_b...       False   \n",
       "18            expect_column_values_to_be_in_type_list        True   \n",
       "\n",
       "                 Error metric(s)  \n",
       "0                           None  \n",
       "1                           None  \n",
       "2                   (None, -1.0)  \n",
       "3                           None  \n",
       "4   (None, -0.13610333418172574)  \n",
       "5                           None  \n",
       "6                           None  \n",
       "7                           None  \n",
       "8                           None  \n",
       "9    (None, 0.16666666666666677)  \n",
       "10                          None  \n",
       "11                          None  \n",
       "12   (None, -0.3525452976704055)  \n",
       "13   (None, 0.03264019346203025)  \n",
       "14   (None, 0.04575163398692814)  \n",
       "15                          None  \n",
       "16                          None  \n",
       "17  (None, 0.013513513513513431)  \n",
       "18                          None  "
      ],
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Expectation type</th>\n",
       "      <th>Successful?</th>\n",
       "      <th>Error metric(s)</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>expect_table_columns_to_match_ordered_list</td>\n",
       "      <td>True</td>\n",
       "      <td>None</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>expect_table_row_count_to_be_between</td>\n",
       "      <td>True</td>\n",
       "      <td>None</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>expect_column_min_to_be_between</td>\n",
       "      <td>False</td>\n",
       "      <td>(None, -1.0)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>expect_column_max_to_be_between</td>\n",
       "      <td>True</td>\n",
       "      <td>None</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>expect_column_mean_to_be_between</td>\n",
       "      <td>False</td>\n",
       "      <td>(None, -0.13610333418172574)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>expect_column_median_to_be_between</td>\n",
       "      <td>True</td>\n",
       "      <td>None</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>expect_column_quantile_values_to_be_between</td>\n",
       "      <td>False</td>\n",
       "      <td>None</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>expect_column_values_to_be_in_set</td>\n",
       "      <td>False</td>\n",
       "      <td>None</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>expect_column_values_to_not_be_null</td>\n",
       "      <td>True</td>\n",
       "      <td>None</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>expect_column_proportion_of_unique_values_to_b...</td>\n",
       "      <td>False</td>\n",
       "      <td>(None, 0.16666666666666677)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>expect_column_values_to_be_in_type_list</td>\n",
       "      <td>True</td>\n",
       "      <td>None</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>expect_column_min_to_be_between</td>\n",
       "      <td>True</td>\n",
       "      <td>None</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>expect_column_max_to_be_between</td>\n",
       "      <td>False</td>\n",
       "      <td>(None, -0.3525452976704055)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>expect_column_mean_to_be_between</td>\n",
       "      <td>False</td>\n",
       "      <td>(None, 0.03264019346203025)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>expect_column_median_to_be_between</td>\n",
       "      <td>False</td>\n",
       "      <td>(None, 0.04575163398692814)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>expect_column_quantile_values_to_be_between</td>\n",
       "      <td>False</td>\n",
       "      <td>None</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>expect_column_values_to_not_be_null</td>\n",
       "      <td>True</td>\n",
       "      <td>None</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>expect_column_proportion_of_unique_values_to_b...</td>\n",
       "      <td>False</td>\n",
       "      <td>(None, 0.013513513513513431)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>expect_column_values_to_be_in_type_list</td>\n",
       "      <td>True</td>\n",
       "      <td>None</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ]
     },
     "metadata": {},
     "execution_count": 9
    }
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "source": [
    "# Retrieve a dense representation of an expectation\n",
    "expectations_dense[0]"
   ],
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": [
       "{'results_format': 'BASIC+',\n",
       " 'success': True,\n",
       " 'type': 'expect_table_columns_to_match_ordered_list',\n",
       " 'kwargs': {'column_list': ['vendor_id',\n",
       "   'pickup_datetime',\n",
       "   'dropoff_datetime',\n",
       "   'passenger_count',\n",
       "   'trip_distance',\n",
       "   'rate_code_id',\n",
       "   'store_and_fwd_flag',\n",
       "   'pickup_location_id',\n",
       "   'dropoff_location_id',\n",
       "   'payment_type',\n",
       "   'fare_amount',\n",
       "   'extra',\n",
       "   'mta_tax',\n",
       "   'tip_amount',\n",
       "   'tolls_amount',\n",
       "   'improvement_surcharge',\n",
       "   'total_amount',\n",
       "   'congestion_surcharge']},\n",
       " 'result': {'observed_value': ['vendor_id',\n",
       "   'pickup_datetime',\n",
       "   'dropoff_datetime',\n",
       "   'passenger_count',\n",
       "   'trip_distance',\n",
       "   'rate_code_id',\n",
       "   'store_and_fwd_flag',\n",
       "   'pickup_location_id',\n",
       "   'dropoff_location_id',\n",
       "   'payment_type',\n",
       "   'fare_amount',\n",
       "   'extra',\n",
       "   'mta_tax',\n",
       "   'tip_amount',\n",
       "   'tolls_amount',\n",
       "   'improvement_surcharge',\n",
       "   'total_amount',\n",
       "   'congestion_surcharge']},\n",
       " 'is_table_expectation': True,\n",
       " 'column_kwargs': {'column_list': ['vendor_id',\n",
       "   'pickup_datetime',\n",
       "   'dropoff_datetime',\n",
       "   'passenger_count',\n",
       "   'trip_distance',\n",
       "   'rate_code_id',\n",
       "   'store_and_fwd_flag',\n",
       "   'pickup_location_id',\n",
       "   'dropoff_location_id',\n",
       "   'payment_type',\n",
       "   'fare_amount',\n",
       "   'extra',\n",
       "   'mta_tax',\n",
       "   'tip_amount',\n",
       "   'tolls_amount',\n",
       "   'improvement_surcharge',\n",
       "   'total_amount',\n",
       "   'congestion_surcharge']}}"
      ]
     },
     "metadata": {},
     "execution_count": 10
    }
   ],
   "metadata": {}
  }
 ],
 "metadata": {
  "interpreter": {
   "hash": "cdc2bce73c2a9ac283f602628cabf735dbe06c4ee87a7849fc5f3d1177c8f304"
  },
  "kernelspec": {
   "name": "python3",
   "display_name": "Python 3.8.10 64-bit ('.venv': venv)"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.10"
  },
  "metadata": {
   "interpreter": {
    "hash": "cdc2bce73c2a9ac283f602628cabf735dbe06c4ee87a7849fc5f3d1177c8f304"
   }
  },
  "orig_nbformat": 2
 },
 "nbformat": 4,
 "nbformat_minor": 2
}