strm_batch_mode_demo.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "Pp285DLTODKe",
    "outputId": "037bc06d-76c8-43bd-e9e8-8367c422bbfd",
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "import boto3\n",
    "import argparse\n",
    "import pandas as pd\n",
    "import random\n",
    "import string\n",
    "import time\n",
    "from datetime import datetime\n",
    "from tqdm import tqdm\n",
    "import io\n",
    "import math\n",
    "import re"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "l7dtw0vmrepK",
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "# Creating a batch job\n",
    "\n",
    "1. Sign up to STRM Privacy\n",
    "2. Create a schema and corresponding contract or use a publicly available one\n",
    "3. Create a Data Connector (we use an AWS S3 bucket)\n",
    "4. Generate (or use real) data\n",
    "5. Send data to the input data connector\n",
    "6. Run batch job and write to output data connector (possibly same bucket as input)\n",
    "7. Fetch data from data connector and inspect"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "rmUfycISsMh2",
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## 1. Sign up to STRM Privacy\n",
    "See the [authentication docs](https://docs.strmprivacy.io/docs/latest/quickstart/authentication-cli.html) to sign up to STRM Privacy.\n",
    "\n",
    "## 2. Create Schema and Contract\n",
    "See [schemas and contract docs](https://docs.strmprivacy.io/docs/latest/concepts/schemas-and-contracts.html)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "unp0JqHNOWna",
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## 3. Create Data Connector\n",
    "\n",
    "In this tutorial we use an AWS S3 bucket. Either use the [console](https://console.strmprivacy.io/data-connectors) to create a data connector for the bucket, or follow the steps from the quickstart [docs](https://docs.strmprivacy.io/docs/latest/quickstart/batch-exporter/).\n",
    "\n",
    "The `s3.json` should follow this structure:\n",
    "\n",
    "```json\n",
    "{ \n",
    "    \"AccessKey\": {\n",
    "        \"UserName\": \"your-username\",\n",
    "        \"AccessKeyId\": \"***\",\n",
    "        \"Status\": \"Active\",\n",
    "        \"SecretAccessKey\": \"***\",\n",
    "        \"CreateDate\": \"***\"\n",
    "    }\n",
    "}\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "4FfNvaUIOxdP",
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## 4. Generate data\n",
    "\n",
    "We have generated some sample data. There are three pii-fields: `Email`, `PrivateFieldA` and `PrivateFieldB`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>SessionId</th>\n",
       "      <th>UserName</th>\n",
       "      <th>Timestamp</th>\n",
       "      <th>Email</th>\n",
       "      <th>PublicFieldA</th>\n",
       "      <th>PublicFieldB</th>\n",
       "      <th>PrivateFieldA</th>\n",
       "      <th>PrivateFieldB</th>\n",
       "      <th>PrivacyPlane</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>session_1</td>\n",
       "      <td>strm_demo_user</td>\n",
       "      <td>2022-02-06 06.43.12.962083:+0100</td>\n",
       "      <td>ibthtfnf@mbeito.com</td>\n",
       "      <td>PublicFieldA_ZIyrjcBj</td>\n",
       "      <td>0.020183</td>\n",
       "      <td>PrivateFieldA_kHmSNlAn</td>\n",
       "      <td>279909288129237</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>session_1</td>\n",
       "      <td>strm_demo_user</td>\n",
       "      <td>2022-02-06 11.43.36.340336:+0100</td>\n",
       "      <td>ifutwmxu@sllcja.com</td>\n",
       "      <td>PublicFieldA_pBznLzVx</td>\n",
       "      <td>0.261294</td>\n",
       "      <td>PrivateFieldA_EwVAJxxF</td>\n",
       "      <td>304221592396358</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>session_2</td>\n",
       "      <td>strm_demo_user</td>\n",
       "      <td>2022-02-06 16.43.10.365912:+0100</td>\n",
       "      <td>ilmcedhf@sxdsyh.com</td>\n",
       "      <td>PublicFieldA_SyVIWPEH</td>\n",
       "      <td>0.632461</td>\n",
       "      <td>PrivateFieldA_tLwMlNoz</td>\n",
       "      <td>384570554091913</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>session_3</td>\n",
       "      <td>strm_demo_user</td>\n",
       "      <td>2022-02-06 21.43.02.232553:+0100</td>\n",
       "      <td>vjmorzkn@pfsmul.com</td>\n",
       "      <td>PublicFieldA_qvUYaCPR</td>\n",
       "      <td>0.920910</td>\n",
       "      <td>PrivateFieldA_yRxNLMRa</td>\n",
       "      <td>66897108978672</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>session_3</td>\n",
       "      <td>strm_demo_user</td>\n",
       "      <td>2022-02-07 02.43.10.174305:+0100</td>\n",
       "      <td>sxsoyvkd@gtggzt.com</td>\n",
       "      <td>PublicFieldA_oBodytpb</td>\n",
       "      <td>0.802284</td>\n",
       "      <td>PrivateFieldA_jDwasBFI</td>\n",
       "      <td>668450493294117</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>session_4</td>\n",
       "      <td>strm_demo_user</td>\n",
       "      <td>2022-02-07 07.43.06.975069:+0100</td>\n",
       "      <td>sxpmqguk@pvrazs.com</td>\n",
       "      <td>PublicFieldA_eaPWZEao</td>\n",
       "      <td>0.701134</td>\n",
       "      <td>PrivateFieldA_UrotTNQH</td>\n",
       "      <td>488952946713404</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>session_5</td>\n",
       "      <td>strm_demo_user</td>\n",
       "      <td>2022-02-07 12.43.10.145600:+0100</td>\n",
       "      <td>dhrmhyzd@arhrpv.com</td>\n",
       "      <td>PublicFieldA_ZEiDwfZH</td>\n",
       "      <td>0.681264</td>\n",
       "      <td>PrivateFieldA_ZxZuogOc</td>\n",
       "      <td>167594023188503</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>session_5</td>\n",
       "      <td>strm_demo_user</td>\n",
       "      <td>2022-02-07 17.43.30.268479:+0100</td>\n",
       "      <td>wbkztlnr@zumcpf.com</td>\n",
       "      <td>PublicFieldA_JSyWvVoh</td>\n",
       "      <td>0.212249</td>\n",
       "      <td>PrivateFieldA_SOjxTSGC</td>\n",
       "      <td>852564768616528</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>session_6</td>\n",
       "      <td>strm_demo_user</td>\n",
       "      <td>2022-02-07 22.43.26.284456:+0100</td>\n",
       "      <td>dzdbvaly@hbtblw.com</td>\n",
       "      <td>PublicFieldA_EEXuECdg</td>\n",
       "      <td>0.368563</td>\n",
       "      <td>PrivateFieldA_ZUexCLVS</td>\n",
       "      <td>56735494626097</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>session_7</td>\n",
       "      <td>strm_demo_user</td>\n",
       "      <td>2022-02-08 03.42.50.399490:+0100</td>\n",
       "      <td>iavumtgg@navmrw.com</td>\n",
       "      <td>PublicFieldA_qwTBCmmE</td>\n",
       "      <td>0.921930</td>\n",
       "      <td>PrivateFieldA_UMHXHyxL</td>\n",
       "      <td>226976555491828</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   SessionId        UserName                         Timestamp  \\\n",
       "0  session_1  strm_demo_user  2022-02-06 06.43.12.962083:+0100   \n",
       "1  session_1  strm_demo_user  2022-02-06 11.43.36.340336:+0100   \n",
       "2  session_2  strm_demo_user  2022-02-06 16.43.10.365912:+0100   \n",
       "3  session_3  strm_demo_user  2022-02-06 21.43.02.232553:+0100   \n",
       "4  session_3  strm_demo_user  2022-02-07 02.43.10.174305:+0100   \n",
       "5  session_4  strm_demo_user  2022-02-07 07.43.06.975069:+0100   \n",
       "6  session_5  strm_demo_user  2022-02-07 12.43.10.145600:+0100   \n",
       "7  session_5  strm_demo_user  2022-02-07 17.43.30.268479:+0100   \n",
       "8  session_6  strm_demo_user  2022-02-07 22.43.26.284456:+0100   \n",
       "9  session_7  strm_demo_user  2022-02-08 03.42.50.399490:+0100   \n",
       "\n",
       "                 Email           PublicFieldA  PublicFieldB  \\\n",
       "0  ibthtfnf@mbeito.com  PublicFieldA_ZIyrjcBj      0.020183   \n",
       "1  ifutwmxu@sllcja.com  PublicFieldA_pBznLzVx      0.261294   \n",
       "2  ilmcedhf@sxdsyh.com  PublicFieldA_SyVIWPEH      0.632461   \n",
       "3  vjmorzkn@pfsmul.com  PublicFieldA_qvUYaCPR      0.920910   \n",
       "4  sxsoyvkd@gtggzt.com  PublicFieldA_oBodytpb      0.802284   \n",
       "5  sxpmqguk@pvrazs.com  PublicFieldA_eaPWZEao      0.701134   \n",
       "6  dhrmhyzd@arhrpv.com  PublicFieldA_ZEiDwfZH      0.681264   \n",
       "7  wbkztlnr@zumcpf.com  PublicFieldA_JSyWvVoh      0.212249   \n",
       "8  dzdbvaly@hbtblw.com  PublicFieldA_EEXuECdg      0.368563   \n",
       "9  iavumtgg@navmrw.com  PublicFieldA_qwTBCmmE      0.921930   \n",
       "\n",
       "            PrivateFieldA    PrivateFieldB  PrivacyPlane  \n",
       "0  PrivateFieldA_kHmSNlAn  279909288129237             2  \n",
       "1  PrivateFieldA_EwVAJxxF  304221592396358             2  \n",
       "2  PrivateFieldA_tLwMlNoz  384570554091913             1  \n",
       "3  PrivateFieldA_yRxNLMRa   66897108978672             0  \n",
       "4  PrivateFieldA_jDwasBFI  668450493294117             0  \n",
       "5  PrivateFieldA_UrotTNQH  488952946713404             1  \n",
       "6  PrivateFieldA_ZxZuogOc  167594023188503             2  \n",
       "7  PrivateFieldA_SOjxTSGC  852564768616528             0  \n",
       "8  PrivateFieldA_ZUexCLVS   56735494626097             1  \n",
       "9  PrivateFieldA_UMHXHyxL  226976555491828             0  "
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.read_csv('./data/input.csv')\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "hKbMKQbbO8A3",
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## 5. Write data to input data connector\n",
    "\n",
    "Send the data to the S3 bucket."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "id": "aYGcE_vxOzEb",
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "class AwsProperties(object):\n",
    "    aws_access_key_id = '***'\n",
    "    aws_secret_access_key = '***'\n",
    "    region = 'eu-central-1'\n",
    "    bucket = 'databert' "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "id": "JTZMEkCgO6H6",
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "AWS = AwsProperties()\n",
    "s3 = boto3.resource(\n",
    "      service_name='s3',\n",
    "      region_name=AWS.region,\n",
    "      aws_access_key_id=AWS.aws_access_key_id,\n",
    "      aws_secret_access_key=AWS.aws_secret_access_key\n",
    "    )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "id": "LMKMTQhGPB5p",
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "resp = s3.Object('databert', 'databert-demo.csv').put(Body=open('./databert-demo.csv', 'rb'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Rxjl9fAjHxKw",
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## 6. Run batch job\n",
    "We'll be running a batch job from the CLI. We need to pass the configuration of the batch job as an argument. An example configuration can be found below.\n",
    "\n",
    "\n",
    "```json\n",
    "{\n",
    "    \"ref\": {\n",
    "      \"billing_id\": \"your_billing_id\"\n",
    "    },\n",
    "    \"source_data\": {\n",
    "      \"data_connector_ref\": {\n",
    "        \"billing_id\": \"your_billing_id\",\n",
    "        \"name\": \"databert-demo\"\n",
    "      },\n",
    "      \"file_name\": \"databert-demo.csv\",\n",
    "      \"data_type\": {\n",
    "        \"csv\": {\n",
    "          \"charset\": \"UTF-8\"\n",
    "        }\n",
    "      }\n",
    "    },\n",
    "    \"consent\": {\n",
    "      \"default_consent_levels\": [\n",
    "        0\n",
    "      ],\n",
    "      \"consent_level_extractor\": {\n",
    "        \"field\": \"PrivacyPlane\",\n",
    "        \"field_patterns\": {\n",
    "          \"1\": {\n",
    "            \"consent_levels\": [\n",
    "              1\n",
    "            ]\n",
    "          },\n",
    "          \"2\": {\n",
    "            \"consent_levels\": [\n",
    "              2\n",
    "            ]\n",
    "          }\n",
    "        }\n",
    "      }\n",
    "    },\n",
    "    \"encryption\": {\n",
    "      \"timestamp_config\": {\n",
    "        \"field\": \"Timestamp\",\n",
    "        \"format\": \"yyyy-MM-dd HH.mm.ss.nnnnnn:Z\",\n",
    "        \"default_time_zone\": {\n",
    "          \"id\": \"UTC\"\n",
    "        }\n",
    "      },\n",
    "      \"batch_job_group_id\": null\n",
    "    },\n",
    "    \"event_contract_ref\": {\n",
    "      \"handle\": \"databert-handle\",\n",
    "      \"name\": \"batch_job_public\",\n",
    "      \"version\": \"1.0.0\"\n",
    "    },\n",
    "    \"encrypted_data\": {\n",
    "      \"target\": {\n",
    "        \"data_connector_ref\": {\n",
    "          \"billing_id\": \"your_billing_id\",\n",
    "          \"name\": \"databert-demo\"\n",
    "        },\n",
    "        \"data_type\": {\n",
    "          \"csv\": {\n",
    "            \"charset\": \"UTF-8\"\n",
    "          }\n",
    "        },\n",
    "        \"file_name\": \"databert-demo-encrypted.csv\"\n",
    "      }\n",
    "    },\n",
    "    \"encryption_keys_data\": {\n",
    "      \"target\": {\n",
    "        \"data_connector_ref\": {\n",
    "          \"billing_id\": \"your_billing_id\",\n",
    "          \"name\": \"databert-demo\"\n",
    "        },\n",
    "        \"data_type\": {\n",
    "          \"csv\": {\n",
    "            \"charset\": \"UTF-8\"\n",
    "          }\n",
    "        },\n",
    "        \"file_name\": \"databert-demo-encryption-keys.csv\"\n",
    "      }\n",
    "    },\n",
    "    \"derived_data\": [\n",
    "      {\n",
    "        \"target\": {\n",
    "          \"data_connector_ref\": {\n",
    "            \"billing_id\": \"your_billing_id\",\n",
    "            \"name\": \"databert-demo\"\n",
    "          },\n",
    "          \"data_type\": {\n",
    "            \"csv\": {\n",
    "              \"charset\": \"UTF-8\"\n",
    "            }\n",
    "          },\n",
    "          \"file_name\": \"databert-demo-derived.csv\"\n",
    "        },\n",
    "        \"consent_levels\": [\n",
    "          2\n",
    "        ],\n",
    "        \"consent_level_type\": \"CUMULATIVE\",\n",
    "        \"masked_fields\": {\n",
    "          \"field_patterns\": {\n",
    "            \"databert-handle/batch_job_public/1.0.0\": {\n",
    "              \"field_patterns\": [\n",
    "                \"Email\",\n",
    "                \"UserName\"\n",
    "              ]\n",
    "            }\n",
    "          }\n",
    "        }\n",
    "      }\n",
    "    ]\n",
    "  }\n",
    "```\n",
    "\n",
    "Now call the batch job: `strm create batch-job -F batch-job.json`\n",
    "\n",
    "Wait for the job to finish. Status can be checked via `strm list batch-jobs`\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "DaYvJHsUPQlu",
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "## 7. Fetch and explore Data\n",
    "\n",
    "Now we fetch the encrypted and derived data from the data connector.\n",
    "We expect from our data contract that the pii-fields `Email`, `PrivateFieldA` and `PrivateFieldB` are encrypted in the encrypted file.\n",
    "We expect that the fields `Email` and `UserName` are masked with a hash in the derived file. Let's investigate: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "id": "PHbbin8QPSGK"
   },
   "outputs": [],
   "source": [
    "# Get objects from bucket\n",
    "encrypted = s3.Object('databert', 'databert-demo-encrypted.csv').get()\n",
    "df_encrypted = pd.read_csv(io.BytesIO(encrypted['Body'].read()))\n",
    "\n",
    "encryption_keys = s3.Object('databert', 'databert-demo-encryption-keys.csv').get()\n",
    "df_encryption_keys = pd.read_csv(io.BytesIO(encryption_keys['Body'].read()))\n",
    "\n",
    "derived = s3.Object('databert', 'databert-demo-derived.csv').get()\n",
    "df_derived = pd.read_csv(io.BytesIO(derived['Body'].read()))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 395
    },
    "id": "Ds3RI9N1PDNn",
    "outputId": "176507db-affc-4ba1-d37e-1fd52fc66755"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>SessionId</th>\n",
       "      <th>UserName</th>\n",
       "      <th>Email</th>\n",
       "      <th>PublicFieldA</th>\n",
       "      <th>PublicFieldB</th>\n",
       "      <th>PrivateFieldA</th>\n",
       "      <th>PrivateFieldB</th>\n",
       "      <th>PrivacyPlane</th>\n",
       "      <th>strmMeta.eventContractRef</th>\n",
       "      <th>strmMeta.nonce</th>\n",
       "      <th>strmMeta.timestamp</th>\n",
       "      <th>strmMeta.keyLink</th>\n",
       "      <th>strmMeta.billingId</th>\n",
       "      <th>strmMeta.consentLevels</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>session_1</td>\n",
       "      <td>strm_demo_user</td>\n",
       "      <td>AX+sVreQHft41CYozNAalRpuzy00lJanLD4RI5d9Z/8+Kf...</td>\n",
       "      <td>PublicFieldA_ZIyrjcBj</td>\n",
       "      <td>0.020183</td>\n",
       "      <td>AX+sVrd6wrGKZbnNl1AXhmrp7K00DgWKP6V0cD3BflciW3...</td>\n",
       "      <td>AX+sVrecjlZ4KBfZDI9gjMIoOX/lCyFsL2mbcPIBfUJzsgCR</td>\n",
       "      <td>2</td>\n",
       "      <td>databert-handle/batch_job_public/1.0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>1644126192000</td>\n",
       "      <td>843ba12a-d9c4-4b53-ae99-9fa1c8adc80e</td>\n",
       "      <td>databert986673817</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>session_1</td>\n",
       "      <td>strm_demo_user</td>\n",
       "      <td>AX+sVrceKUWi3pbDRblf9UjqX+zMIb1AAu8bpVRAJCemA7...</td>\n",
       "      <td>PublicFieldA_pBznLzVx</td>\n",
       "      <td>0.261294</td>\n",
       "      <td>AX+sVrdJqRHCqZuhLsDsuZQXW9Gv3TckI3MTc7WNo8PjZx...</td>\n",
       "      <td>AX+sVrfoC8CbYIzUoPkIf8ITKr4RL9/+0RHlDk6QolWoUTbl</td>\n",
       "      <td>2</td>\n",
       "      <td>databert-handle/batch_job_public/1.0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>1644144216000</td>\n",
       "      <td>843ba12a-d9c4-4b53-ae99-9fa1c8adc80e</td>\n",
       "      <td>databert986673817</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>session_2</td>\n",
       "      <td>strm_demo_user</td>\n",
       "      <td>AX+sVrc7xTqPpID3FaUvhGR3oYCKragNbn1cBP7hxNJheV...</td>\n",
       "      <td>PublicFieldA_SyVIWPEH</td>\n",
       "      <td>0.632461</td>\n",
       "      <td>AX+sVrfQZ6pb8ESQ6i3cAEd0FpmfAOy8uO4WyhQDj3+k/5...</td>\n",
       "      <td>AX+sVrcP4RGKM86jOgm7rdYe2WpJjNOyr0RVLinOSqTg7N4d</td>\n",
       "      <td>1</td>\n",
       "      <td>databert-handle/batch_job_public/1.0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>1644162190000</td>\n",
       "      <td>843ba12a-d9c4-4b53-ae99-9fa1c8adc80e</td>\n",
       "      <td>databert986673817</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>session_3</td>\n",
       "      <td>strm_demo_user</td>\n",
       "      <td>AX+sVrcLXAtDrTcXyaVezGK87seVrO+q/Y1QthUEqHcDhq...</td>\n",
       "      <td>PublicFieldA_qvUYaCPR</td>\n",
       "      <td>0.920910</td>\n",
       "      <td>AX+sVrcmwsqzKiuS5Z6rO90G9Gl1ank2n6f8Icz4wYG9TC...</td>\n",
       "      <td>AX+sVrfDmTFRseI4mwllkPjdAJ6jrLTfdYbf6IcLyS69PVQ=</td>\n",
       "      <td>0</td>\n",
       "      <td>databert-handle/batch_job_public/1.0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>1644180182000</td>\n",
       "      <td>843ba12a-d9c4-4b53-ae99-9fa1c8adc80e</td>\n",
       "      <td>databert986673817</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>session_3</td>\n",
       "      <td>strm_demo_user</td>\n",
       "      <td>AT1NfVOKYWkt99iRfFEx5a1LidWiq3UBnZiyuNRrJCuk5V...</td>\n",
       "      <td>PublicFieldA_oBodytpb</td>\n",
       "      <td>0.802284</td>\n",
       "      <td>AT1NfVPiAXhJ4KtY5N2ucOfMHoMGQTRPpp3qwNNKx8a2nG...</td>\n",
       "      <td>AT1NfVOpF+3ibfe1sHHav/NnkS7RyjV3avfofWajGC9ZY718</td>\n",
       "      <td>0</td>\n",
       "      <td>databert-handle/batch_job_public/1.0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>1644198190000</td>\n",
       "      <td>1ceae097-c760-42af-9212-144d78944b0e</td>\n",
       "      <td>databert986673817</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>session_4</td>\n",
       "      <td>strm_demo_user</td>\n",
       "      <td>AT1NfVNQi2SeLwsLHgi0qKX2sf8T0RuyRNYBxc6vELqWfP...</td>\n",
       "      <td>PublicFieldA_eaPWZEao</td>\n",
       "      <td>0.701134</td>\n",
       "      <td>AT1NfVMuSuRKuRpJj0ouzPBuL7UKTdwK466Y2HQTJzBmjU...</td>\n",
       "      <td>AT1NfVNQIBt246u3xBx6xo9tDZJJQV30BgICuZr/KDIua06x</td>\n",
       "      <td>1</td>\n",
       "      <td>databert-handle/batch_job_public/1.0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>1644216186000</td>\n",
       "      <td>1ceae097-c760-42af-9212-144d78944b0e</td>\n",
       "      <td>databert986673817</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>session_5</td>\n",
       "      <td>strm_demo_user</td>\n",
       "      <td>AT1NfVPf5IHajoUlCHWlyHKhGsW/i7QZihDfj/mpEJAA71...</td>\n",
       "      <td>PublicFieldA_ZEiDwfZH</td>\n",
       "      <td>0.681264</td>\n",
       "      <td>AT1NfVNMCky18pGM/uSFYTlesThB9QALXTAGPeNFXnumxj...</td>\n",
       "      <td>AT1NfVMiCY2z8fOCoYenb0CnWW9kAtbjaP4+O6VlIRUaXK1I</td>\n",
       "      <td>2</td>\n",
       "      <td>databert-handle/batch_job_public/1.0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>1644234190000</td>\n",
       "      <td>1ceae097-c760-42af-9212-144d78944b0e</td>\n",
       "      <td>databert986673817</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>session_5</td>\n",
       "      <td>strm_demo_user</td>\n",
       "      <td>AT1NfVNY/30jPRNg50lzqJ1AZRdggxODYrcTuIH+ZoIGCK...</td>\n",
       "      <td>PublicFieldA_JSyWvVoh</td>\n",
       "      <td>0.212249</td>\n",
       "      <td>AT1NfVNdQICmqLKtjNwRPIDOzDeUAkYX4Ov9VZnpiSHKdU...</td>\n",
       "      <td>AT1NfVM2+hjPhDrpgmNwBIO203V1QWZiK5+0Vck9JD6tJmAh</td>\n",
       "      <td>0</td>\n",
       "      <td>databert-handle/batch_job_public/1.0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>1644252210000</td>\n",
       "      <td>1ceae097-c760-42af-9212-144d78944b0e</td>\n",
       "      <td>databert986673817</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>session_6</td>\n",
       "      <td>strm_demo_user</td>\n",
       "      <td>AT1NfVNTKkIbxrMlKnJpY3GnRvdBFYlUmtwd124mkpG3/x...</td>\n",
       "      <td>PublicFieldA_EEXuECdg</td>\n",
       "      <td>0.368563</td>\n",
       "      <td>AT1NfVO2e4S/9zCy1heXMwF/L+m+2x5A3XW5QHzE1XJ/XY...</td>\n",
       "      <td>AT1NfVPkPwoGgtjhJ8HtltDQQfbRus7skGlv4n3Ui9RzIvI=</td>\n",
       "      <td>1</td>\n",
       "      <td>databert-handle/batch_job_public/1.0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>1644270206000</td>\n",
       "      <td>1ceae097-c760-42af-9212-144d78944b0e</td>\n",
       "      <td>databert986673817</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>session_7</td>\n",
       "      <td>strm_demo_user</td>\n",
       "      <td>AWUGvEGwxWtWfrYflkdDEMnpm5gXTKj+Q+uObgP8iBGY4O...</td>\n",
       "      <td>PublicFieldA_qwTBCmmE</td>\n",
       "      <td>0.921930</td>\n",
       "      <td>AWUGvEH36tmpm5MVCKLLVCy2/fgeyJHxBg4IqgWdHpnovb...</td>\n",
       "      <td>AWUGvEH8mmqemMAPsKw8+MD4NtWQv66je5dulzsUnaaXHH8c</td>\n",
       "      <td>0</td>\n",
       "      <td>databert-handle/batch_job_public/1.0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>1644288170000</td>\n",
       "      <td>ea388588-06dc-4e90-8d9e-1651021b5ebe</td>\n",
       "      <td>databert986673817</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   SessionId        UserName  \\\n",
       "0  session_1  strm_demo_user   \n",
       "1  session_1  strm_demo_user   \n",
       "2  session_2  strm_demo_user   \n",
       "3  session_3  strm_demo_user   \n",
       "4  session_3  strm_demo_user   \n",
       "5  session_4  strm_demo_user   \n",
       "6  session_5  strm_demo_user   \n",
       "7  session_5  strm_demo_user   \n",
       "8  session_6  strm_demo_user   \n",
       "9  session_7  strm_demo_user   \n",
       "\n",
       "                                               Email           PublicFieldA  \\\n",
       "0  AX+sVreQHft41CYozNAalRpuzy00lJanLD4RI5d9Z/8+Kf...  PublicFieldA_ZIyrjcBj   \n",
       "1  AX+sVrceKUWi3pbDRblf9UjqX+zMIb1AAu8bpVRAJCemA7...  PublicFieldA_pBznLzVx   \n",
       "2  AX+sVrc7xTqPpID3FaUvhGR3oYCKragNbn1cBP7hxNJheV...  PublicFieldA_SyVIWPEH   \n",
       "3  AX+sVrcLXAtDrTcXyaVezGK87seVrO+q/Y1QthUEqHcDhq...  PublicFieldA_qvUYaCPR   \n",
       "4  AT1NfVOKYWkt99iRfFEx5a1LidWiq3UBnZiyuNRrJCuk5V...  PublicFieldA_oBodytpb   \n",
       "5  AT1NfVNQi2SeLwsLHgi0qKX2sf8T0RuyRNYBxc6vELqWfP...  PublicFieldA_eaPWZEao   \n",
       "6  AT1NfVPf5IHajoUlCHWlyHKhGsW/i7QZihDfj/mpEJAA71...  PublicFieldA_ZEiDwfZH   \n",
       "7  AT1NfVNY/30jPRNg50lzqJ1AZRdggxODYrcTuIH+ZoIGCK...  PublicFieldA_JSyWvVoh   \n",
       "8  AT1NfVNTKkIbxrMlKnJpY3GnRvdBFYlUmtwd124mkpG3/x...  PublicFieldA_EEXuECdg   \n",
       "9  AWUGvEGwxWtWfrYflkdDEMnpm5gXTKj+Q+uObgP8iBGY4O...  PublicFieldA_qwTBCmmE   \n",
       "\n",
       "   PublicFieldB                                      PrivateFieldA  \\\n",
       "0      0.020183  AX+sVrd6wrGKZbnNl1AXhmrp7K00DgWKP6V0cD3BflciW3...   \n",
       "1      0.261294  AX+sVrdJqRHCqZuhLsDsuZQXW9Gv3TckI3MTc7WNo8PjZx...   \n",
       "2      0.632461  AX+sVrfQZ6pb8ESQ6i3cAEd0FpmfAOy8uO4WyhQDj3+k/5...   \n",
       "3      0.920910  AX+sVrcmwsqzKiuS5Z6rO90G9Gl1ank2n6f8Icz4wYG9TC...   \n",
       "4      0.802284  AT1NfVPiAXhJ4KtY5N2ucOfMHoMGQTRPpp3qwNNKx8a2nG...   \n",
       "5      0.701134  AT1NfVMuSuRKuRpJj0ouzPBuL7UKTdwK466Y2HQTJzBmjU...   \n",
       "6      0.681264  AT1NfVNMCky18pGM/uSFYTlesThB9QALXTAGPeNFXnumxj...   \n",
       "7      0.212249  AT1NfVNdQICmqLKtjNwRPIDOzDeUAkYX4Ov9VZnpiSHKdU...   \n",
       "8      0.368563  AT1NfVO2e4S/9zCy1heXMwF/L+m+2x5A3XW5QHzE1XJ/XY...   \n",
       "9      0.921930  AWUGvEH36tmpm5MVCKLLVCy2/fgeyJHxBg4IqgWdHpnovb...   \n",
       "\n",
       "                                      PrivateFieldB  PrivacyPlane  \\\n",
       "0  AX+sVrecjlZ4KBfZDI9gjMIoOX/lCyFsL2mbcPIBfUJzsgCR             2   \n",
       "1  AX+sVrfoC8CbYIzUoPkIf8ITKr4RL9/+0RHlDk6QolWoUTbl             2   \n",
       "2  AX+sVrcP4RGKM86jOgm7rdYe2WpJjNOyr0RVLinOSqTg7N4d             1   \n",
       "3  AX+sVrfDmTFRseI4mwllkPjdAJ6jrLTfdYbf6IcLyS69PVQ=             0   \n",
       "4  AT1NfVOpF+3ibfe1sHHav/NnkS7RyjV3avfofWajGC9ZY718             0   \n",
       "5  AT1NfVNQIBt246u3xBx6xo9tDZJJQV30BgICuZr/KDIua06x             1   \n",
       "6  AT1NfVMiCY2z8fOCoYenb0CnWW9kAtbjaP4+O6VlIRUaXK1I             2   \n",
       "7  AT1NfVM2+hjPhDrpgmNwBIO203V1QWZiK5+0Vck9JD6tJmAh             0   \n",
       "8  AT1NfVPkPwoGgtjhJ8HtltDQQfbRus7skGlv4n3Ui9RzIvI=             1   \n",
       "9  AWUGvEH8mmqemMAPsKw8+MD4NtWQv66je5dulzsUnaaXHH8c             0   \n",
       "\n",
       "                strmMeta.eventContractRef  strmMeta.nonce  strmMeta.timestamp  \\\n",
       "0  databert-handle/batch_job_public/1.0.0               0       1644126192000   \n",
       "1  databert-handle/batch_job_public/1.0.0               0       1644144216000   \n",
       "2  databert-handle/batch_job_public/1.0.0               0       1644162190000   \n",
       "3  databert-handle/batch_job_public/1.0.0               0       1644180182000   \n",
       "4  databert-handle/batch_job_public/1.0.0               0       1644198190000   \n",
       "5  databert-handle/batch_job_public/1.0.0               0       1644216186000   \n",
       "6  databert-handle/batch_job_public/1.0.0               0       1644234190000   \n",
       "7  databert-handle/batch_job_public/1.0.0               0       1644252210000   \n",
       "8  databert-handle/batch_job_public/1.0.0               0       1644270206000   \n",
       "9  databert-handle/batch_job_public/1.0.0               0       1644288170000   \n",
       "\n",
       "                       strmMeta.keyLink strmMeta.billingId  \\\n",
       "0  843ba12a-d9c4-4b53-ae99-9fa1c8adc80e  databert986673817   \n",
       "1  843ba12a-d9c4-4b53-ae99-9fa1c8adc80e  databert986673817   \n",
       "2  843ba12a-d9c4-4b53-ae99-9fa1c8adc80e  databert986673817   \n",
       "3  843ba12a-d9c4-4b53-ae99-9fa1c8adc80e  databert986673817   \n",
       "4  1ceae097-c760-42af-9212-144d78944b0e  databert986673817   \n",
       "5  1ceae097-c760-42af-9212-144d78944b0e  databert986673817   \n",
       "6  1ceae097-c760-42af-9212-144d78944b0e  databert986673817   \n",
       "7  1ceae097-c760-42af-9212-144d78944b0e  databert986673817   \n",
       "8  1ceae097-c760-42af-9212-144d78944b0e  databert986673817   \n",
       "9  ea388588-06dc-4e90-8d9e-1651021b5ebe  databert986673817   \n",
       "\n",
       "   strmMeta.consentLevels  \n",
       "0                       2  \n",
       "1                       2  \n",
       "2                       1  \n",
       "3                       0  \n",
       "4                       0  \n",
       "5                       1  \n",
       "6                       2  \n",
       "7                       0  \n",
       "8                       1  \n",
       "9                       0  "
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_encrypted"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 661
    },
    "id": "3ucSwwdBPVxA",
    "outputId": "328d0dbe-1b12-41a9-c1e3-403633fc406f"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>SessionId</th>\n",
       "      <th>UserName</th>\n",
       "      <th>Email</th>\n",
       "      <th>PublicFieldA</th>\n",
       "      <th>PublicFieldB</th>\n",
       "      <th>PrivateFieldA</th>\n",
       "      <th>PrivateFieldB</th>\n",
       "      <th>PrivacyPlane</th>\n",
       "      <th>strmMeta.eventContractRef</th>\n",
       "      <th>strmMeta.nonce</th>\n",
       "      <th>strmMeta.timestamp</th>\n",
       "      <th>strmMeta.keyLink</th>\n",
       "      <th>strmMeta.billingId</th>\n",
       "      <th>strmMeta.consentLevels</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>session_1</td>\n",
       "      <td>788d082a29fd07f61e1df95bbe98ef9f</td>\n",
       "      <td>4d0602d8bfe14089fb14cdef7d8380be</td>\n",
       "      <td>PublicFieldA_ZIyrjcBj</td>\n",
       "      <td>0.020183</td>\n",
       "      <td>PrivateFieldA_kHmSNlAn</td>\n",
       "      <td>279909288129237</td>\n",
       "      <td>2</td>\n",
       "      <td>databert-handle/batch_job_public/1.0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>1644126192000</td>\n",
       "      <td>843ba12a-d9c4-4b53-ae99-9fa1c8adc80e</td>\n",
       "      <td>databert986673817</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>session_1</td>\n",
       "      <td>788d082a29fd07f61e1df95bbe98ef9f</td>\n",
       "      <td>89a133907a42f4926a7ac68f45872c95</td>\n",
       "      <td>PublicFieldA_pBznLzVx</td>\n",
       "      <td>0.261294</td>\n",
       "      <td>PrivateFieldA_EwVAJxxF</td>\n",
       "      <td>304221592396358</td>\n",
       "      <td>2</td>\n",
       "      <td>databert-handle/batch_job_public/1.0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>1644144216000</td>\n",
       "      <td>843ba12a-d9c4-4b53-ae99-9fa1c8adc80e</td>\n",
       "      <td>databert986673817</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>session_5</td>\n",
       "      <td>788d082a29fd07f61e1df95bbe98ef9f</td>\n",
       "      <td>cf3276704ba63f44ea326a06d010884a</td>\n",
       "      <td>PublicFieldA_ZEiDwfZH</td>\n",
       "      <td>0.681264</td>\n",
       "      <td>PrivateFieldA_ZxZuogOc</td>\n",
       "      <td>167594023188503</td>\n",
       "      <td>2</td>\n",
       "      <td>databert-handle/batch_job_public/1.0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>1644234190000</td>\n",
       "      <td>1ceae097-c760-42af-9212-144d78944b0e</td>\n",
       "      <td>databert986673817</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   SessionId                          UserName  \\\n",
       "0  session_1  788d082a29fd07f61e1df95bbe98ef9f   \n",
       "1  session_1  788d082a29fd07f61e1df95bbe98ef9f   \n",
       "2  session_5  788d082a29fd07f61e1df95bbe98ef9f   \n",
       "\n",
       "                              Email           PublicFieldA  PublicFieldB  \\\n",
       "0  4d0602d8bfe14089fb14cdef7d8380be  PublicFieldA_ZIyrjcBj      0.020183   \n",
       "1  89a133907a42f4926a7ac68f45872c95  PublicFieldA_pBznLzVx      0.261294   \n",
       "2  cf3276704ba63f44ea326a06d010884a  PublicFieldA_ZEiDwfZH      0.681264   \n",
       "\n",
       "            PrivateFieldA    PrivateFieldB  PrivacyPlane  \\\n",
       "0  PrivateFieldA_kHmSNlAn  279909288129237             2   \n",
       "1  PrivateFieldA_EwVAJxxF  304221592396358             2   \n",
       "2  PrivateFieldA_ZxZuogOc  167594023188503             2   \n",
       "\n",
       "                strmMeta.eventContractRef  strmMeta.nonce  strmMeta.timestamp  \\\n",
       "0  databert-handle/batch_job_public/1.0.0               0       1644126192000   \n",
       "1  databert-handle/batch_job_public/1.0.0               0       1644144216000   \n",
       "2  databert-handle/batch_job_public/1.0.0               0       1644234190000   \n",
       "\n",
       "                       strmMeta.keyLink strmMeta.billingId  \\\n",
       "0  843ba12a-d9c4-4b53-ae99-9fa1c8adc80e  databert986673817   \n",
       "1  843ba12a-d9c4-4b53-ae99-9fa1c8adc80e  databert986673817   \n",
       "2  1ceae097-c760-42af-9212-144d78944b0e  databert986673817   \n",
       "\n",
       "   strmMeta.consentLevels  \n",
       "0                       2  \n",
       "1                       2  \n",
       "2                       2  "
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_derived"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 143
    },
    "id": "6EkdHWenPXI-",
    "outputId": "3f2a615e-9fb1-423b-99b9-352051a79251"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>keyLink</th>\n",
       "      <th>encryptionKey</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>843ba12a-d9c4-4b53-ae99-9fa1c8adc80e</td>\n",
       "      <td>{\"primaryKeyId\":2142000823,\"key\":[{\"keyData\":{...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1ceae097-c760-42af-9212-144d78944b0e</td>\n",
       "      <td>{\"primaryKeyId\":1028488531,\"key\":[{\"keyData\":{...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>ea388588-06dc-4e90-8d9e-1651021b5ebe</td>\n",
       "      <td>{\"primaryKeyId\":1694940225,\"key\":[{\"keyData\":{...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                keyLink  \\\n",
       "0  843ba12a-d9c4-4b53-ae99-9fa1c8adc80e   \n",
       "1  1ceae097-c760-42af-9212-144d78944b0e   \n",
       "2  ea388588-06dc-4e90-8d9e-1651021b5ebe   \n",
       "\n",
       "                                       encryptionKey  \n",
       "0  {\"primaryKeyId\":2142000823,\"key\":[{\"keyData\":{...  \n",
       "1  {\"primaryKeyId\":1028488531,\"key\":[{\"keyData\":{...  \n",
       "2  {\"primaryKeyId\":1694940225,\"key\":[{\"keyData\":{...  "
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_encryption_keys"
   ]
  }
 ],
 "metadata": {
  "colab": {
   "collapsed_sections": [],
   "name": "strm_batch_jobs.ipynb",
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.12"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}