AWS-GP-demo-2.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Greenplum Database  Concepts Explained - Part 2\n",
    "\n",
    "This is Part 2 of Greenplum Database  Concepts Explained, ***Basic Table Functions***. \n",
    "- If you missed Part 1 (*Setup, Describe Input Dataset & Data Loading*) or wish to repeat, then click [here](AWS-GP-demo-1.ipynb)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "import os, re\n",
    "from IPython.display import display_html\n",
    "\n",
    "import pygments.lexers\n",
    "from pygments import highlight\n",
    "from pygments.formatters import HtmlFormatter\n",
    "\n",
    "CONNECTION_STRING = os.getenv('AWSGPDBCONN')\n",
    "\n",
    "cs = re.match('^postgresql:\\/\\/(\\S+):(\\S+)@(\\S+):(\\S+)\\/(\\S+)$', CONNECTION_STRING)\n",
    "\n",
    "DB_USER   = cs.group(1)\n",
    "DB_PWD    = cs.group(2)\n",
    "DB_SERVER = cs.group(3)\n",
    "DB_PORT   = cs.group(4)\n",
    "DB_NAME   = cs.group(5)\n",
    "\n",
    "%reload_ext sql\n",
    "%sql $CONNECTION_STRING"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4. Basic Table Functions\n",
    "\n",
    "### 4.1. DESCRIBE *demo.amzn_reviews* table using psql utility (`\\d <table name>`)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<!DOCTYPE html PUBLIC \"-//W3C//DTD HTML 4.01//EN\"\n",
       "   \"http://www.w3.org/TR/html4/strict.dtd\">\n",
       "<!--\n",
       "generated by Pygments <http://pygments.org>\n",
       "Copyright 2006-2019 by the Pygments team.\n",
       "Licensed under the BSD license, see LICENSE for details.\n",
       "-->\n",
       "<html>\n",
       "<head>\n",
       "  <title></title>\n",
       "  <meta http-equiv=\"content-type\" content=\"text/html; charset=UTF-8\">\n",
       "  <style type=\"text/css\">\n",
       "/*\n",
       "generated by Pygments <http://pygments.org>\n",
       "Copyright 2006-2019 by the Pygments team.\n",
       "Licensed under the BSD license, see LICENSE for details.\n",
       "*/\n",
       "td.linenos { background-color: #f0f0f0; padding-right: 10px; }\n",
       "span.lineno { background-color: #f0f0f0; padding: 0 5px 0 5px; }\n",
       "pre { line-height: 125%; }\n",
       "body .hll { background-color: #ffffcc }\n",
       "body  { background: #ffffff; }\n",
       "body .c { color: #888888 } /* Comment */\n",
       "body .err { color: #FF0000; background-color: #FFAAAA } /* Error */\n",
       "body .k { color: #008800; font-weight: bold } /* Keyword */\n",
       "body .o { color: #333333 } /* Operator */\n",
       "body .ch { color: #888888 } /* Comment.Hashbang */\n",
       "body .cm { color: #888888 } /* Comment.Multiline */\n",
       "body .cp { color: #557799 } /* Comment.Preproc */\n",
       "body .cpf { color: #888888 } /* Comment.PreprocFile */\n",
       "body .c1 { color: #888888 } /* Comment.Single */\n",
       "body .cs { color: #cc0000; font-weight: bold } /* Comment.Special */\n",
       "body .gd { color: #A00000 } /* Generic.Deleted */\n",
       "body .ge { font-style: italic } /* Generic.Emph */\n",
       "body .gr { color: #FF0000 } /* Generic.Error */\n",
       "body .gh { color: #000080; font-weight: bold } /* Generic.Heading */\n",
       "body .gi { color: #00A000 } /* Generic.Inserted */\n",
       "body .go { color: #888888 } /* Generic.Output */\n",
       "body .gp { color: #c65d09; font-weight: bold } /* Generic.Prompt */\n",
       "body .gs { font-weight: bold } /* Generic.Strong */\n",
       "body .gu { color: #800080; font-weight: bold } /* Generic.Subheading */\n",
       "body .gt { color: #0044DD } /* Generic.Traceback */\n",
       "body .kc { color: #008800; font-weight: bold } /* Keyword.Constant */\n",
       "body .kd { color: #008800; font-weight: bold } /* Keyword.Declaration */\n",
       "body .kn { color: #008800; font-weight: bold } /* Keyword.Namespace */\n",
       "body .kp { color: #003388; font-weight: bold } /* Keyword.Pseudo */\n",
       "body .kr { color: #008800; font-weight: bold } /* Keyword.Reserved */\n",
       "body .kt { color: #333399; font-weight: bold } /* Keyword.Type */\n",
       "body .m { color: #6600EE; font-weight: bold } /* Literal.Number */\n",
       "body .s { background-color: #fff0f0 } /* Literal.String */\n",
       "body .na { color: #0000CC } /* Name.Attribute */\n",
       "body .nb { color: #007020 } /* Name.Builtin */\n",
       "body .nc { color: #BB0066; font-weight: bold } /* Name.Class */\n",
       "body .no { color: #003366; font-weight: bold } /* Name.Constant */\n",
       "body .nd { color: #555555; font-weight: bold } /* Name.Decorator */\n",
       "body .ni { color: #880000; font-weight: bold } /* Name.Entity */\n",
       "body .ne { color: #FF0000; font-weight: bold } /* Name.Exception */\n",
       "body .nf { color: #0066BB; font-weight: bold } /* Name.Function */\n",
       "body .nl { color: #997700; font-weight: bold } /* Name.Label */\n",
       "body .nn { color: #0e84b5; font-weight: bold } /* Name.Namespace */\n",
       "body .nt { color: #007700 } /* Name.Tag */\n",
       "body .nv { color: #996633 } /* Name.Variable */\n",
       "body .ow { color: #000000; font-weight: bold } /* Operator.Word */\n",
       "body .w { color: #bbbbbb } /* Text.Whitespace */\n",
       "body .mb { color: #6600EE; font-weight: bold } /* Literal.Number.Bin */\n",
       "body .mf { color: #6600EE; font-weight: bold } /* Literal.Number.Float */\n",
       "body .mh { color: #005588; font-weight: bold } /* Literal.Number.Hex */\n",
       "body .mi { color: #0000DD; font-weight: bold } /* Literal.Number.Integer */\n",
       "body .mo { color: #4400EE; font-weight: bold } /* Literal.Number.Oct */\n",
       "body .sa { background-color: #fff0f0 } /* Literal.String.Affix */\n",
       "body .sb { background-color: #fff0f0 } /* Literal.String.Backtick */\n",
       "body .sc { color: #0044DD } /* Literal.String.Char */\n",
       "body .dl { background-color: #fff0f0 } /* Literal.String.Delimiter */\n",
       "body .sd { color: #DD4422 } /* Literal.String.Doc */\n",
       "body .s2 { background-color: #fff0f0 } /* Literal.String.Double */\n",
       "body .se { color: #666666; font-weight: bold; background-color: #fff0f0 } /* Literal.String.Escape */\n",
       "body .sh { background-color: #fff0f0 } /* Literal.String.Heredoc */\n",
       "body .si { background-color: #eeeeee } /* Literal.String.Interpol */\n",
       "body .sx { color: #DD2200; background-color: #fff0f0 } /* Literal.String.Other */\n",
       "body .sr { color: #000000; background-color: #fff0ff } /* Literal.String.Regex */\n",
       "body .s1 { background-color: #fff0f0 } /* Literal.String.Single */\n",
       "body .ss { color: #AA6600 } /* Literal.String.Symbol */\n",
       "body .bp { color: #007020 } /* Name.Builtin.Pseudo */\n",
       "body .fm { color: #0066BB; font-weight: bold } /* Name.Function.Magic */\n",
       "body .vc { color: #336699 } /* Name.Variable.Class */\n",
       "body .vg { color: #dd7700; font-weight: bold } /* Name.Variable.Global */\n",
       "body .vi { color: #3333BB } /* Name.Variable.Instance */\n",
       "body .vm { color: #996633 } /* Name.Variable.Magic */\n",
       "body .il { color: #0000DD; font-weight: bold } /* Literal.Number.Integer.Long */\n",
       "\n",
       "  </style>\n",
       "</head>\n",
       "<body>\n",
       "<h2></h2>\n",
       "\n",
       "<div class=\"highlight\"><pre><span></span><span class=\"kp\">\\d</span> <span class=\"ss\">demo.amzn_reviews</span>\n",
       "</pre></div>\n",
       "</body>\n",
       "</html>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<table border=\"1\">  <caption>Table &quot;demo.amzn_reviews&quot;</caption>  <tr>    <th align=\"center\">Column</th>    <th align=\"center\">Type</th>    <th align=\"center\">Collation</th>    <th align=\"center\">Nullable</th>    <th align=\"center\">Default</th>  </tr>  <tr valign=\"top\">    <td align=\"left\">marketplace</td>    <td align=\"left\">text</td>    <td align=\"left\">&nbsp; </td>    <td align=\"left\">&nbsp; </td>    <td align=\"left\">&nbsp; </td>  </tr>  <tr valign=\"top\">    <td align=\"left\">customer_id</td>    <td align=\"left\">bigint</td>    <td align=\"left\">&nbsp; </td>    <td align=\"left\">&nbsp; </td>    <td align=\"left\">&nbsp; </td>  </tr>  <tr valign=\"top\">    <td align=\"left\">review_id</td>    <td align=\"left\">text</td>    <td align=\"left\">&nbsp; </td>    <td align=\"left\">&nbsp; </td>    <td align=\"left\">&nbsp; </td>  </tr>  <tr valign=\"top\">    <td align=\"left\">product_id</td>    <td align=\"left\">text</td>    <td align=\"left\">&nbsp; </td>    <td align=\"left\">&nbsp; </td>    <td align=\"left\">&nbsp; </td>  </tr>  <tr valign=\"top\">    <td align=\"left\">product_parent</td>    <td align=\"left\">bigint</td>    <td align=\"left\">&nbsp; </td>    <td align=\"left\">&nbsp; </td>    <td align=\"left\">&nbsp; </td>  </tr>  <tr valign=\"top\">    <td align=\"left\">product_title</td>    <td align=\"left\">text</td>    <td align=\"left\">&nbsp; </td>    <td align=\"left\">&nbsp; </td>    <td align=\"left\">&nbsp; </td>  </tr>  <tr valign=\"top\">    <td align=\"left\">product_category</td>    <td align=\"left\">text</td>    <td align=\"left\">&nbsp; </td>    <td align=\"left\">&nbsp; </td>    <td align=\"left\">&nbsp; </td>  </tr>  <tr valign=\"top\">    <td align=\"left\">star_rating</td>    <td align=\"left\">integer</td>    <td align=\"left\">&nbsp; </td>    <td align=\"left\">&nbsp; </td>    <td align=\"left\">&nbsp; </td>  </tr>  <tr valign=\"top\">    <td align=\"left\">helpful_votes</td>    <td align=\"left\">integer</td>    <td align=\"left\">&nbsp; </td>    <td align=\"left\">&nbsp; </td>    <td align=\"left\">&nbsp; </td>  </tr>  <tr valign=\"top\">    <td align=\"left\">total_votes</td>    <td align=\"left\">integer</td>    <td align=\"left\">&nbsp; </td>    <td align=\"left\">&nbsp; </td>    <td align=\"left\">&nbsp; </td>  </tr>  <tr valign=\"top\">    <td align=\"left\">vine</td>    <td align=\"left\">text</td>    <td align=\"left\">&nbsp; </td>    <td align=\"left\">&nbsp; </td>    <td align=\"left\">&nbsp; </td>  </tr>  <tr valign=\"top\">    <td align=\"left\">verified_purchase</td>    <td align=\"left\">text</td>    <td align=\"left\">&nbsp; </td>    <td align=\"left\">&nbsp; </td>    <td align=\"left\">&nbsp; </td>  </tr>  <tr valign=\"top\">    <td align=\"left\">review_headline</td>    <td align=\"left\">text</td>    <td align=\"left\">&nbsp; </td>    <td align=\"left\">&nbsp; </td>    <td align=\"left\">&nbsp; </td>  </tr>  <tr valign=\"top\">    <td align=\"left\">review_body</td>    <td align=\"left\">text</td>    <td align=\"left\">&nbsp; </td>    <td align=\"left\">&nbsp; </td>    <td align=\"left\">&nbsp; </td>  </tr>  <tr valign=\"top\">    <td align=\"left\">review_date</td>    <td align=\"left\">date</td>    <td align=\"left\">&nbsp; </td>    <td align=\"left\">&nbsp; </td>    <td align=\"left\">&nbsp; </td>  </tr></table>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "sqlfilecode = !pygmentize -f html -O full,style=colorful -l psql script/4-1-psql-describe-amzn-reviews.sql\n",
    "\n",
    "display_html('\\n'.join(sqlfilecode), raw=True)\n",
    "\n",
    "psql_out = !cat script/4-1-psql-describe-amzn-reviews.sql | psql -H $CONNECTION_STRING\n",
    "\n",
    "display_html(''.join(psql_out), raw=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4.2. DESCRIBE *demo.amzn_reviews* table using `information_schema` catalog table."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<!DOCTYPE html PUBLIC \"-//W3C//DTD HTML 4.01//EN\"\n",
       "   \"http://www.w3.org/TR/html4/strict.dtd\">\n",
       "<!--\n",
       "generated by Pygments <http://pygments.org>\n",
       "Copyright 2006-2019 by the Pygments team.\n",
       "Licensed under the BSD license, see LICENSE for details.\n",
       "-->\n",
       "<html>\n",
       "<head>\n",
       "  <title></title>\n",
       "  <meta http-equiv=\"content-type\" content=\"text/html; charset=UTF-8\">\n",
       "  <style type=\"text/css\">\n",
       "/*\n",
       "generated by Pygments <http://pygments.org>\n",
       "Copyright 2006-2019 by the Pygments team.\n",
       "Licensed under the BSD license, see LICENSE for details.\n",
       "*/\n",
       "td.linenos { background-color: #f0f0f0; padding-right: 10px; }\n",
       "span.lineno { background-color: #f0f0f0; padding: 0 5px 0 5px; }\n",
       "pre { line-height: 125%; }\n",
       "body .hll { background-color: #ffffcc }\n",
       "body  { background: #ffffff; }\n",
       "body .c { color: #888888 } /* Comment */\n",
       "body .err { color: #FF0000; background-color: #FFAAAA } /* Error */\n",
       "body .k { color: #008800; font-weight: bold } /* Keyword */\n",
       "body .o { color: #333333 } /* Operator */\n",
       "body .ch { color: #888888 } /* Comment.Hashbang */\n",
       "body .cm { color: #888888 } /* Comment.Multiline */\n",
       "body .cp { color: #557799 } /* Comment.Preproc */\n",
       "body .cpf { color: #888888 } /* Comment.PreprocFile */\n",
       "body .c1 { color: #888888 } /* Comment.Single */\n",
       "body .cs { color: #cc0000; font-weight: bold } /* Comment.Special */\n",
       "body .gd { color: #A00000 } /* Generic.Deleted */\n",
       "body .ge { font-style: italic } /* Generic.Emph */\n",
       "body .gr { color: #FF0000 } /* Generic.Error */\n",
       "body .gh { color: #000080; font-weight: bold } /* Generic.Heading */\n",
       "body .gi { color: #00A000 } /* Generic.Inserted */\n",
       "body .go { color: #888888 } /* Generic.Output */\n",
       "body .gp { color: #c65d09; font-weight: bold } /* Generic.Prompt */\n",
       "body .gs { font-weight: bold } /* Generic.Strong */\n",
       "body .gu { color: #800080; font-weight: bold } /* Generic.Subheading */\n",
       "body .gt { color: #0044DD } /* Generic.Traceback */\n",
       "body .kc { color: #008800; font-weight: bold } /* Keyword.Constant */\n",
       "body .kd { color: #008800; font-weight: bold } /* Keyword.Declaration */\n",
       "body .kn { color: #008800; font-weight: bold } /* Keyword.Namespace */\n",
       "body .kp { color: #003388; font-weight: bold } /* Keyword.Pseudo */\n",
       "body .kr { color: #008800; font-weight: bold } /* Keyword.Reserved */\n",
       "body .kt { color: #333399; font-weight: bold } /* Keyword.Type */\n",
       "body .m { color: #6600EE; font-weight: bold } /* Literal.Number */\n",
       "body .s { background-color: #fff0f0 } /* Literal.String */\n",
       "body .na { color: #0000CC } /* Name.Attribute */\n",
       "body .nb { color: #007020 } /* Name.Builtin */\n",
       "body .nc { color: #BB0066; font-weight: bold } /* Name.Class */\n",
       "body .no { color: #003366; font-weight: bold } /* Name.Constant */\n",
       "body .nd { color: #555555; font-weight: bold } /* Name.Decorator */\n",
       "body .ni { color: #880000; font-weight: bold } /* Name.Entity */\n",
       "body .ne { color: #FF0000; font-weight: bold } /* Name.Exception */\n",
       "body .nf { color: #0066BB; font-weight: bold } /* Name.Function */\n",
       "body .nl { color: #997700; font-weight: bold } /* Name.Label */\n",
       "body .nn { color: #0e84b5; font-weight: bold } /* Name.Namespace */\n",
       "body .nt { color: #007700 } /* Name.Tag */\n",
       "body .nv { color: #996633 } /* Name.Variable */\n",
       "body .ow { color: #000000; font-weight: bold } /* Operator.Word */\n",
       "body .w { color: #bbbbbb } /* Text.Whitespace */\n",
       "body .mb { color: #6600EE; font-weight: bold } /* Literal.Number.Bin */\n",
       "body .mf { color: #6600EE; font-weight: bold } /* Literal.Number.Float */\n",
       "body .mh { color: #005588; font-weight: bold } /* Literal.Number.Hex */\n",
       "body .mi { color: #0000DD; font-weight: bold } /* Literal.Number.Integer */\n",
       "body .mo { color: #4400EE; font-weight: bold } /* Literal.Number.Oct */\n",
       "body .sa { background-color: #fff0f0 } /* Literal.String.Affix */\n",
       "body .sb { background-color: #fff0f0 } /* Literal.String.Backtick */\n",
       "body .sc { color: #0044DD } /* Literal.String.Char */\n",
       "body .dl { background-color: #fff0f0 } /* Literal.String.Delimiter */\n",
       "body .sd { color: #DD4422 } /* Literal.String.Doc */\n",
       "body .s2 { background-color: #fff0f0 } /* Literal.String.Double */\n",
       "body .se { color: #666666; font-weight: bold; background-color: #fff0f0 } /* Literal.String.Escape */\n",
       "body .sh { background-color: #fff0f0 } /* Literal.String.Heredoc */\n",
       "body .si { background-color: #eeeeee } /* Literal.String.Interpol */\n",
       "body .sx { color: #DD2200; background-color: #fff0f0 } /* Literal.String.Other */\n",
       "body .sr { color: #000000; background-color: #fff0ff } /* Literal.String.Regex */\n",
       "body .s1 { background-color: #fff0f0 } /* Literal.String.Single */\n",
       "body .ss { color: #AA6600 } /* Literal.String.Symbol */\n",
       "body .bp { color: #007020 } /* Name.Builtin.Pseudo */\n",
       "body .fm { color: #0066BB; font-weight: bold } /* Name.Function.Magic */\n",
       "body .vc { color: #336699 } /* Name.Variable.Class */\n",
       "body .vg { color: #dd7700; font-weight: bold } /* Name.Variable.Global */\n",
       "body .vi { color: #3333BB } /* Name.Variable.Instance */\n",
       "body .vm { color: #996633 } /* Name.Variable.Magic */\n",
       "body .il { color: #0000DD; font-weight: bold } /* Literal.Number.Integer.Long */\n",
       "\n",
       "  </style>\n",
       "</head>\n",
       "<body>\n",
       "<h2></h2>\n",
       "\n",
       "<div class=\"highlight\"><pre><span></span><span class=\"k\">SELECT</span> <span class=\"o\">*</span> \n",
       "<span class=\"k\">FROM</span> <span class=\"n\">information_schema</span><span class=\"mf\">.</span><span class=\"n\">COLUMNS</span> \n",
       "<span class=\"k\">WHERE</span> <span class=\"n\">TABLE_NAME</span> <span class=\"o\">=</span> <span class=\"s1\">&#39;amzn_reviews&#39;</span><span class=\"p\">;</span>\n",
       "</pre></div>\n",
       "</body>\n",
       "</html>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "15 rows affected.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<table>\n",
       "    <tr>\n",
       "        <th>table_catalog</th>\n",
       "        <th>table_schema</th>\n",
       "        <th>table_name</th>\n",
       "        <th>column_name</th>\n",
       "        <th>ordinal_position</th>\n",
       "        <th>column_default</th>\n",
       "        <th>is_nullable</th>\n",
       "        <th>data_type</th>\n",
       "        <th>character_maximum_length</th>\n",
       "        <th>character_octet_length</th>\n",
       "        <th>numeric_precision</th>\n",
       "        <th>numeric_precision_radix</th>\n",
       "        <th>numeric_scale</th>\n",
       "        <th>datetime_precision</th>\n",
       "        <th>interval_type</th>\n",
       "        <th>interval_precision</th>\n",
       "        <th>character_set_catalog</th>\n",
       "        <th>character_set_schema</th>\n",
       "        <th>character_set_name</th>\n",
       "        <th>collation_catalog</th>\n",
       "        <th>collation_schema</th>\n",
       "        <th>collation_name</th>\n",
       "        <th>domain_catalog</th>\n",
       "        <th>domain_schema</th>\n",
       "        <th>domain_name</th>\n",
       "        <th>udt_catalog</th>\n",
       "        <th>udt_schema</th>\n",
       "        <th>udt_name</th>\n",
       "        <th>scope_catalog</th>\n",
       "        <th>scope_schema</th>\n",
       "        <th>scope_name</th>\n",
       "        <th>maximum_cardinality</th>\n",
       "        <th>dtd_identifier</th>\n",
       "        <th>is_self_referencing</th>\n",
       "        <th>is_identity</th>\n",
       "        <th>identity_generation</th>\n",
       "        <th>identity_start</th>\n",
       "        <th>identity_increment</th>\n",
       "        <th>identity_maximum</th>\n",
       "        <th>identity_minimum</th>\n",
       "        <th>identity_cycle</th>\n",
       "        <th>is_generated</th>\n",
       "        <th>generation_expression</th>\n",
       "        <th>is_updatable</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td>dev</td>\n",
       "        <td>demo</td>\n",
       "        <td>amzn_reviews</td>\n",
       "        <td>review_date</td>\n",
       "        <td>15</td>\n",
       "        <td>None</td>\n",
       "        <td>YES</td>\n",
       "        <td>date</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>0</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>dev</td>\n",
       "        <td>pg_catalog</td>\n",
       "        <td>date</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>15</td>\n",
       "        <td>NO</td>\n",
       "        <td>NO</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>NEVER</td>\n",
       "        <td>None</td>\n",
       "        <td>YES</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td>dev</td>\n",
       "        <td>demo</td>\n",
       "        <td>amzn_reviews</td>\n",
       "        <td>marketplace</td>\n",
       "        <td>1</td>\n",
       "        <td>None</td>\n",
       "        <td>YES</td>\n",
       "        <td>text</td>\n",
       "        <td>None</td>\n",
       "        <td>1073741824</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>dev</td>\n",
       "        <td>pg_catalog</td>\n",
       "        <td>text</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>1</td>\n",
       "        <td>NO</td>\n",
       "        <td>NO</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>NEVER</td>\n",
       "        <td>None</td>\n",
       "        <td>YES</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td>dev</td>\n",
       "        <td>demo</td>\n",
       "        <td>amzn_reviews</td>\n",
       "        <td>review_id</td>\n",
       "        <td>3</td>\n",
       "        <td>None</td>\n",
       "        <td>YES</td>\n",
       "        <td>text</td>\n",
       "        <td>None</td>\n",
       "        <td>1073741824</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>dev</td>\n",
       "        <td>pg_catalog</td>\n",
       "        <td>text</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>3</td>\n",
       "        <td>NO</td>\n",
       "        <td>NO</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>NEVER</td>\n",
       "        <td>None</td>\n",
       "        <td>YES</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td>dev</td>\n",
       "        <td>demo</td>\n",
       "        <td>amzn_reviews</td>\n",
       "        <td>product_id</td>\n",
       "        <td>4</td>\n",
       "        <td>None</td>\n",
       "        <td>YES</td>\n",
       "        <td>text</td>\n",
       "        <td>None</td>\n",
       "        <td>1073741824</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>dev</td>\n",
       "        <td>pg_catalog</td>\n",
       "        <td>text</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>4</td>\n",
       "        <td>NO</td>\n",
       "        <td>NO</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>NEVER</td>\n",
       "        <td>None</td>\n",
       "        <td>YES</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td>dev</td>\n",
       "        <td>demo</td>\n",
       "        <td>amzn_reviews</td>\n",
       "        <td>product_title</td>\n",
       "        <td>6</td>\n",
       "        <td>None</td>\n",
       "        <td>YES</td>\n",
       "        <td>text</td>\n",
       "        <td>None</td>\n",
       "        <td>1073741824</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>dev</td>\n",
       "        <td>pg_catalog</td>\n",
       "        <td>text</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>6</td>\n",
       "        <td>NO</td>\n",
       "        <td>NO</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>NEVER</td>\n",
       "        <td>None</td>\n",
       "        <td>YES</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td>dev</td>\n",
       "        <td>demo</td>\n",
       "        <td>amzn_reviews</td>\n",
       "        <td>product_category</td>\n",
       "        <td>7</td>\n",
       "        <td>None</td>\n",
       "        <td>YES</td>\n",
       "        <td>text</td>\n",
       "        <td>None</td>\n",
       "        <td>1073741824</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>dev</td>\n",
       "        <td>pg_catalog</td>\n",
       "        <td>text</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>7</td>\n",
       "        <td>NO</td>\n",
       "        <td>NO</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>NEVER</td>\n",
       "        <td>None</td>\n",
       "        <td>YES</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td>dev</td>\n",
       "        <td>demo</td>\n",
       "        <td>amzn_reviews</td>\n",
       "        <td>vine</td>\n",
       "        <td>11</td>\n",
       "        <td>None</td>\n",
       "        <td>YES</td>\n",
       "        <td>text</td>\n",
       "        <td>None</td>\n",
       "        <td>1073741824</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>dev</td>\n",
       "        <td>pg_catalog</td>\n",
       "        <td>text</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>11</td>\n",
       "        <td>NO</td>\n",
       "        <td>NO</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>NEVER</td>\n",
       "        <td>None</td>\n",
       "        <td>YES</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td>dev</td>\n",
       "        <td>demo</td>\n",
       "        <td>amzn_reviews</td>\n",
       "        <td>verified_purchase</td>\n",
       "        <td>12</td>\n",
       "        <td>None</td>\n",
       "        <td>YES</td>\n",
       "        <td>text</td>\n",
       "        <td>None</td>\n",
       "        <td>1073741824</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>dev</td>\n",
       "        <td>pg_catalog</td>\n",
       "        <td>text</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>12</td>\n",
       "        <td>NO</td>\n",
       "        <td>NO</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>NEVER</td>\n",
       "        <td>None</td>\n",
       "        <td>YES</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td>dev</td>\n",
       "        <td>demo</td>\n",
       "        <td>amzn_reviews</td>\n",
       "        <td>review_headline</td>\n",
       "        <td>13</td>\n",
       "        <td>None</td>\n",
       "        <td>YES</td>\n",
       "        <td>text</td>\n",
       "        <td>None</td>\n",
       "        <td>1073741824</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>dev</td>\n",
       "        <td>pg_catalog</td>\n",
       "        <td>text</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>13</td>\n",
       "        <td>NO</td>\n",
       "        <td>NO</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>NEVER</td>\n",
       "        <td>None</td>\n",
       "        <td>YES</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td>dev</td>\n",
       "        <td>demo</td>\n",
       "        <td>amzn_reviews</td>\n",
       "        <td>review_body</td>\n",
       "        <td>14</td>\n",
       "        <td>None</td>\n",
       "        <td>YES</td>\n",
       "        <td>text</td>\n",
       "        <td>None</td>\n",
       "        <td>1073741824</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>dev</td>\n",
       "        <td>pg_catalog</td>\n",
       "        <td>text</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>14</td>\n",
       "        <td>NO</td>\n",
       "        <td>NO</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>NEVER</td>\n",
       "        <td>None</td>\n",
       "        <td>YES</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td>dev</td>\n",
       "        <td>demo</td>\n",
       "        <td>amzn_reviews</td>\n",
       "        <td>star_rating</td>\n",
       "        <td>8</td>\n",
       "        <td>None</td>\n",
       "        <td>YES</td>\n",
       "        <td>integer</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>32</td>\n",
       "        <td>2</td>\n",
       "        <td>0</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>dev</td>\n",
       "        <td>pg_catalog</td>\n",
       "        <td>int4</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>8</td>\n",
       "        <td>NO</td>\n",
       "        <td>NO</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>NEVER</td>\n",
       "        <td>None</td>\n",
       "        <td>YES</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td>dev</td>\n",
       "        <td>demo</td>\n",
       "        <td>amzn_reviews</td>\n",
       "        <td>helpful_votes</td>\n",
       "        <td>9</td>\n",
       "        <td>None</td>\n",
       "        <td>YES</td>\n",
       "        <td>integer</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>32</td>\n",
       "        <td>2</td>\n",
       "        <td>0</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>dev</td>\n",
       "        <td>pg_catalog</td>\n",
       "        <td>int4</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>9</td>\n",
       "        <td>NO</td>\n",
       "        <td>NO</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>NEVER</td>\n",
       "        <td>None</td>\n",
       "        <td>YES</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td>dev</td>\n",
       "        <td>demo</td>\n",
       "        <td>amzn_reviews</td>\n",
       "        <td>total_votes</td>\n",
       "        <td>10</td>\n",
       "        <td>None</td>\n",
       "        <td>YES</td>\n",
       "        <td>integer</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>32</td>\n",
       "        <td>2</td>\n",
       "        <td>0</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>dev</td>\n",
       "        <td>pg_catalog</td>\n",
       "        <td>int4</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>10</td>\n",
       "        <td>NO</td>\n",
       "        <td>NO</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>NEVER</td>\n",
       "        <td>None</td>\n",
       "        <td>YES</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td>dev</td>\n",
       "        <td>demo</td>\n",
       "        <td>amzn_reviews</td>\n",
       "        <td>customer_id</td>\n",
       "        <td>2</td>\n",
       "        <td>None</td>\n",
       "        <td>YES</td>\n",
       "        <td>bigint</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>64</td>\n",
       "        <td>2</td>\n",
       "        <td>0</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>dev</td>\n",
       "        <td>pg_catalog</td>\n",
       "        <td>int8</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>2</td>\n",
       "        <td>NO</td>\n",
       "        <td>NO</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>NEVER</td>\n",
       "        <td>None</td>\n",
       "        <td>YES</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td>dev</td>\n",
       "        <td>demo</td>\n",
       "        <td>amzn_reviews</td>\n",
       "        <td>product_parent</td>\n",
       "        <td>5</td>\n",
       "        <td>None</td>\n",
       "        <td>YES</td>\n",
       "        <td>bigint</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>64</td>\n",
       "        <td>2</td>\n",
       "        <td>0</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>dev</td>\n",
       "        <td>pg_catalog</td>\n",
       "        <td>int8</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>5</td>\n",
       "        <td>NO</td>\n",
       "        <td>NO</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>NEVER</td>\n",
       "        <td>None</td>\n",
       "        <td>YES</td>\n",
       "    </tr>\n",
       "</table>"
      ],
      "text/plain": [
       "[('dev', 'demo', 'amzn_reviews', 'review_date', 15, None, 'YES', 'date', None, None, None, None, None, 0, None, None, None, None, None, None, None, None, None, None, None, 'dev', 'pg_catalog', 'date', None, None, None, None, '15', 'NO', 'NO', None, None, None, None, None, None, 'NEVER', None, 'YES'),\n",
       " ('dev', 'demo', 'amzn_reviews', 'marketplace', 1, None, 'YES', 'text', None, 1073741824, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, 'dev', 'pg_catalog', 'text', None, None, None, None, '1', 'NO', 'NO', None, None, None, None, None, None, 'NEVER', None, 'YES'),\n",
       " ('dev', 'demo', 'amzn_reviews', 'review_id', 3, None, 'YES', 'text', None, 1073741824, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, 'dev', 'pg_catalog', 'text', None, None, None, None, '3', 'NO', 'NO', None, None, None, None, None, None, 'NEVER', None, 'YES'),\n",
       " ('dev', 'demo', 'amzn_reviews', 'product_id', 4, None, 'YES', 'text', None, 1073741824, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, 'dev', 'pg_catalog', 'text', None, None, None, None, '4', 'NO', 'NO', None, None, None, None, None, None, 'NEVER', None, 'YES'),\n",
       " ('dev', 'demo', 'amzn_reviews', 'product_title', 6, None, 'YES', 'text', None, 1073741824, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, 'dev', 'pg_catalog', 'text', None, None, None, None, '6', 'NO', 'NO', None, None, None, None, None, None, 'NEVER', None, 'YES'),\n",
       " ('dev', 'demo', 'amzn_reviews', 'product_category', 7, None, 'YES', 'text', None, 1073741824, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, 'dev', 'pg_catalog', 'text', None, None, None, None, '7', 'NO', 'NO', None, None, None, None, None, None, 'NEVER', None, 'YES'),\n",
       " ('dev', 'demo', 'amzn_reviews', 'vine', 11, None, 'YES', 'text', None, 1073741824, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, 'dev', 'pg_catalog', 'text', None, None, None, None, '11', 'NO', 'NO', None, None, None, None, None, None, 'NEVER', None, 'YES'),\n",
       " ('dev', 'demo', 'amzn_reviews', 'verified_purchase', 12, None, 'YES', 'text', None, 1073741824, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, 'dev', 'pg_catalog', 'text', None, None, None, None, '12', 'NO', 'NO', None, None, None, None, None, None, 'NEVER', None, 'YES'),\n",
       " ('dev', 'demo', 'amzn_reviews', 'review_headline', 13, None, 'YES', 'text', None, 1073741824, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, 'dev', 'pg_catalog', 'text', None, None, None, None, '13', 'NO', 'NO', None, None, None, None, None, None, 'NEVER', None, 'YES'),\n",
       " ('dev', 'demo', 'amzn_reviews', 'review_body', 14, None, 'YES', 'text', None, 1073741824, None, None, None, None, None, None, None, None, None, None, None, None, None, None, None, 'dev', 'pg_catalog', 'text', None, None, None, None, '14', 'NO', 'NO', None, None, None, None, None, None, 'NEVER', None, 'YES'),\n",
       " ('dev', 'demo', 'amzn_reviews', 'star_rating', 8, None, 'YES', 'integer', None, None, 32, 2, 0, None, None, None, None, None, None, None, None, None, None, None, None, 'dev', 'pg_catalog', 'int4', None, None, None, None, '8', 'NO', 'NO', None, None, None, None, None, None, 'NEVER', None, 'YES'),\n",
       " ('dev', 'demo', 'amzn_reviews', 'helpful_votes', 9, None, 'YES', 'integer', None, None, 32, 2, 0, None, None, None, None, None, None, None, None, None, None, None, None, 'dev', 'pg_catalog', 'int4', None, None, None, None, '9', 'NO', 'NO', None, None, None, None, None, None, 'NEVER', None, 'YES'),\n",
       " ('dev', 'demo', 'amzn_reviews', 'total_votes', 10, None, 'YES', 'integer', None, None, 32, 2, 0, None, None, None, None, None, None, None, None, None, None, None, None, 'dev', 'pg_catalog', 'int4', None, None, None, None, '10', 'NO', 'NO', None, None, None, None, None, None, 'NEVER', None, 'YES'),\n",
       " ('dev', 'demo', 'amzn_reviews', 'customer_id', 2, None, 'YES', 'bigint', None, None, 64, 2, 0, None, None, None, None, None, None, None, None, None, None, None, None, 'dev', 'pg_catalog', 'int8', None, None, None, None, '2', 'NO', 'NO', None, None, None, None, None, None, 'NEVER', None, 'YES'),\n",
       " ('dev', 'demo', 'amzn_reviews', 'product_parent', 5, None, 'YES', 'bigint', None, None, 64, 2, 0, None, None, None, None, None, None, None, None, None, None, None, None, 'dev', 'pg_catalog', 'int8', None, None, None, None, '5', 'NO', 'NO', None, None, None, None, None, None, 'NEVER', None, 'YES')]"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sqlfilecode = !pygmentize -f html -O full,style=colorful -l postgres script/4-2-gp-describe-amzn-reviews.sql\n",
    "\n",
    "display_html('\\n'.join(sqlfilecode), raw=True)\n",
    "\n",
    "query = !cat script/4-2-gp-describe-amzn-reviews.sql\n",
    "\n",
    "%sql $DB_USER@$DB_SERVER {''.join(query)}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4.3. Retrieve a sample of the _demo.amzn_reviews_ table data (10 rows)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<!DOCTYPE html PUBLIC \"-//W3C//DTD HTML 4.01//EN\"\n",
       "   \"http://www.w3.org/TR/html4/strict.dtd\">\n",
       "<!--\n",
       "generated by Pygments <http://pygments.org>\n",
       "Copyright 2006-2019 by the Pygments team.\n",
       "Licensed under the BSD license, see LICENSE for details.\n",
       "-->\n",
       "<html>\n",
       "<head>\n",
       "  <title></title>\n",
       "  <meta http-equiv=\"content-type\" content=\"text/html; charset=UTF-8\">\n",
       "  <style type=\"text/css\">\n",
       "/*\n",
       "generated by Pygments <http://pygments.org>\n",
       "Copyright 2006-2019 by the Pygments team.\n",
       "Licensed under the BSD license, see LICENSE for details.\n",
       "*/\n",
       "td.linenos { background-color: #f0f0f0; padding-right: 10px; }\n",
       "span.lineno { background-color: #f0f0f0; padding: 0 5px 0 5px; }\n",
       "pre { line-height: 125%; }\n",
       "body .hll { background-color: #ffffcc }\n",
       "body  { background: #ffffff; }\n",
       "body .c { color: #888888 } /* Comment */\n",
       "body .err { color: #FF0000; background-color: #FFAAAA } /* Error */\n",
       "body .k { color: #008800; font-weight: bold } /* Keyword */\n",
       "body .o { color: #333333 } /* Operator */\n",
       "body .ch { color: #888888 } /* Comment.Hashbang */\n",
       "body .cm { color: #888888 } /* Comment.Multiline */\n",
       "body .cp { color: #557799 } /* Comment.Preproc */\n",
       "body .cpf { color: #888888 } /* Comment.PreprocFile */\n",
       "body .c1 { color: #888888 } /* Comment.Single */\n",
       "body .cs { color: #cc0000; font-weight: bold } /* Comment.Special */\n",
       "body .gd { color: #A00000 } /* Generic.Deleted */\n",
       "body .ge { font-style: italic } /* Generic.Emph */\n",
       "body .gr { color: #FF0000 } /* Generic.Error */\n",
       "body .gh { color: #000080; font-weight: bold } /* Generic.Heading */\n",
       "body .gi { color: #00A000 } /* Generic.Inserted */\n",
       "body .go { color: #888888 } /* Generic.Output */\n",
       "body .gp { color: #c65d09; font-weight: bold } /* Generic.Prompt */\n",
       "body .gs { font-weight: bold } /* Generic.Strong */\n",
       "body .gu { color: #800080; font-weight: bold } /* Generic.Subheading */\n",
       "body .gt { color: #0044DD } /* Generic.Traceback */\n",
       "body .kc { color: #008800; font-weight: bold } /* Keyword.Constant */\n",
       "body .kd { color: #008800; font-weight: bold } /* Keyword.Declaration */\n",
       "body .kn { color: #008800; font-weight: bold } /* Keyword.Namespace */\n",
       "body .kp { color: #003388; font-weight: bold } /* Keyword.Pseudo */\n",
       "body .kr { color: #008800; font-weight: bold } /* Keyword.Reserved */\n",
       "body .kt { color: #333399; font-weight: bold } /* Keyword.Type */\n",
       "body .m { color: #6600EE; font-weight: bold } /* Literal.Number */\n",
       "body .s { background-color: #fff0f0 } /* Literal.String */\n",
       "body .na { color: #0000CC } /* Name.Attribute */\n",
       "body .nb { color: #007020 } /* Name.Builtin */\n",
       "body .nc { color: #BB0066; font-weight: bold } /* Name.Class */\n",
       "body .no { color: #003366; font-weight: bold } /* Name.Constant */\n",
       "body .nd { color: #555555; font-weight: bold } /* Name.Decorator */\n",
       "body .ni { color: #880000; font-weight: bold } /* Name.Entity */\n",
       "body .ne { color: #FF0000; font-weight: bold } /* Name.Exception */\n",
       "body .nf { color: #0066BB; font-weight: bold } /* Name.Function */\n",
       "body .nl { color: #997700; font-weight: bold } /* Name.Label */\n",
       "body .nn { color: #0e84b5; font-weight: bold } /* Name.Namespace */\n",
       "body .nt { color: #007700 } /* Name.Tag */\n",
       "body .nv { color: #996633 } /* Name.Variable */\n",
       "body .ow { color: #000000; font-weight: bold } /* Operator.Word */\n",
       "body .w { color: #bbbbbb } /* Text.Whitespace */\n",
       "body .mb { color: #6600EE; font-weight: bold } /* Literal.Number.Bin */\n",
       "body .mf { color: #6600EE; font-weight: bold } /* Literal.Number.Float */\n",
       "body .mh { color: #005588; font-weight: bold } /* Literal.Number.Hex */\n",
       "body .mi { color: #0000DD; font-weight: bold } /* Literal.Number.Integer */\n",
       "body .mo { color: #4400EE; font-weight: bold } /* Literal.Number.Oct */\n",
       "body .sa { background-color: #fff0f0 } /* Literal.String.Affix */\n",
       "body .sb { background-color: #fff0f0 } /* Literal.String.Backtick */\n",
       "body .sc { color: #0044DD } /* Literal.String.Char */\n",
       "body .dl { background-color: #fff0f0 } /* Literal.String.Delimiter */\n",
       "body .sd { color: #DD4422 } /* Literal.String.Doc */\n",
       "body .s2 { background-color: #fff0f0 } /* Literal.String.Double */\n",
       "body .se { color: #666666; font-weight: bold; background-color: #fff0f0 } /* Literal.String.Escape */\n",
       "body .sh { background-color: #fff0f0 } /* Literal.String.Heredoc */\n",
       "body .si { background-color: #eeeeee } /* Literal.String.Interpol */\n",
       "body .sx { color: #DD2200; background-color: #fff0f0 } /* Literal.String.Other */\n",
       "body .sr { color: #000000; background-color: #fff0ff } /* Literal.String.Regex */\n",
       "body .s1 { background-color: #fff0f0 } /* Literal.String.Single */\n",
       "body .ss { color: #AA6600 } /* Literal.String.Symbol */\n",
       "body .bp { color: #007020 } /* Name.Builtin.Pseudo */\n",
       "body .fm { color: #0066BB; font-weight: bold } /* Name.Function.Magic */\n",
       "body .vc { color: #336699 } /* Name.Variable.Class */\n",
       "body .vg { color: #dd7700; font-weight: bold } /* Name.Variable.Global */\n",
       "body .vi { color: #3333BB } /* Name.Variable.Instance */\n",
       "body .vm { color: #996633 } /* Name.Variable.Magic */\n",
       "body .il { color: #0000DD; font-weight: bold } /* Literal.Number.Integer.Long */\n",
       "\n",
       "  </style>\n",
       "</head>\n",
       "<body>\n",
       "<h2></h2>\n",
       "\n",
       "<div class=\"highlight\"><pre><span></span><span class=\"k\">SELECT</span> <span class=\"o\">*</span> \n",
       "<span class=\"k\">FROM</span> <span class=\"n\">demo</span><span class=\"mf\">.</span><span class=\"n\">amzn_reviews</span> \n",
       "<span class=\"k\">ORDER</span> <span class=\"k\">BY</span> <span class=\"n\">RANDOM</span><span class=\"p\">()</span> \n",
       "<span class=\"k\">LIMIT</span> <span class=\"mf\">10</span><span class=\"p\">;</span>\n",
       "</pre></div>\n",
       "</body>\n",
       "</html>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0 rows affected.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<table>\n",
       "    <tr>\n",
       "        <th>marketplace</th>\n",
       "        <th>customer_id</th>\n",
       "        <th>review_id</th>\n",
       "        <th>product_id</th>\n",
       "        <th>product_parent</th>\n",
       "        <th>product_title</th>\n",
       "        <th>product_category</th>\n",
       "        <th>star_rating</th>\n",
       "        <th>helpful_votes</th>\n",
       "        <th>total_votes</th>\n",
       "        <th>vine</th>\n",
       "        <th>verified_purchase</th>\n",
       "        <th>review_headline</th>\n",
       "        <th>review_body</th>\n",
       "        <th>review_date</th>\n",
       "    </tr>\n",
       "</table>"
      ],
      "text/plain": [
       "[]"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sqlfilecode = !pygmentize -f html -O full,style=colorful -l postgres script/4-3-select-sample-amzn-reviews.sql\n",
    "\n",
    "display_html('\\n'.join(sqlfilecode), raw=True)\n",
    "\n",
    "query = !cat script/4-3-select-sample-amzn-reviews.sql\n",
    "\n",
    "%sql $DB_USER@$DB_SERVER {''.join(query)}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4.4. Show *demo.amzn_reviews* table data distribution across segments:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<!DOCTYPE html PUBLIC \"-//W3C//DTD HTML 4.01//EN\"\n",
       "   \"http://www.w3.org/TR/html4/strict.dtd\">\n",
       "<!--\n",
       "generated by Pygments <http://pygments.org>\n",
       "Copyright 2006-2019 by the Pygments team.\n",
       "Licensed under the BSD license, see LICENSE for details.\n",
       "-->\n",
       "<html>\n",
       "<head>\n",
       "  <title></title>\n",
       "  <meta http-equiv=\"content-type\" content=\"text/html; charset=UTF-8\">\n",
       "  <style type=\"text/css\">\n",
       "/*\n",
       "generated by Pygments <http://pygments.org>\n",
       "Copyright 2006-2019 by the Pygments team.\n",
       "Licensed under the BSD license, see LICENSE for details.\n",
       "*/\n",
       "td.linenos { background-color: #f0f0f0; padding-right: 10px; }\n",
       "span.lineno { background-color: #f0f0f0; padding: 0 5px 0 5px; }\n",
       "pre { line-height: 125%; }\n",
       "body .hll { background-color: #ffffcc }\n",
       "body  { background: #ffffff; }\n",
       "body .c { color: #888888 } /* Comment */\n",
       "body .err { color: #FF0000; background-color: #FFAAAA } /* Error */\n",
       "body .k { color: #008800; font-weight: bold } /* Keyword */\n",
       "body .o { color: #333333 } /* Operator */\n",
       "body .ch { color: #888888 } /* Comment.Hashbang */\n",
       "body .cm { color: #888888 } /* Comment.Multiline */\n",
       "body .cp { color: #557799 } /* Comment.Preproc */\n",
       "body .cpf { color: #888888 } /* Comment.PreprocFile */\n",
       "body .c1 { color: #888888 } /* Comment.Single */\n",
       "body .cs { color: #cc0000; font-weight: bold } /* Comment.Special */\n",
       "body .gd { color: #A00000 } /* Generic.Deleted */\n",
       "body .ge { font-style: italic } /* Generic.Emph */\n",
       "body .gr { color: #FF0000 } /* Generic.Error */\n",
       "body .gh { color: #000080; font-weight: bold } /* Generic.Heading */\n",
       "body .gi { color: #00A000 } /* Generic.Inserted */\n",
       "body .go { color: #888888 } /* Generic.Output */\n",
       "body .gp { color: #c65d09; font-weight: bold } /* Generic.Prompt */\n",
       "body .gs { font-weight: bold } /* Generic.Strong */\n",
       "body .gu { color: #800080; font-weight: bold } /* Generic.Subheading */\n",
       "body .gt { color: #0044DD } /* Generic.Traceback */\n",
       "body .kc { color: #008800; font-weight: bold } /* Keyword.Constant */\n",
       "body .kd { color: #008800; font-weight: bold } /* Keyword.Declaration */\n",
       "body .kn { color: #008800; font-weight: bold } /* Keyword.Namespace */\n",
       "body .kp { color: #003388; font-weight: bold } /* Keyword.Pseudo */\n",
       "body .kr { color: #008800; font-weight: bold } /* Keyword.Reserved */\n",
       "body .kt { color: #333399; font-weight: bold } /* Keyword.Type */\n",
       "body .m { color: #6600EE; font-weight: bold } /* Literal.Number */\n",
       "body .s { background-color: #fff0f0 } /* Literal.String */\n",
       "body .na { color: #0000CC } /* Name.Attribute */\n",
       "body .nb { color: #007020 } /* Name.Builtin */\n",
       "body .nc { color: #BB0066; font-weight: bold } /* Name.Class */\n",
       "body .no { color: #003366; font-weight: bold } /* Name.Constant */\n",
       "body .nd { color: #555555; font-weight: bold } /* Name.Decorator */\n",
       "body .ni { color: #880000; font-weight: bold } /* Name.Entity */\n",
       "body .ne { color: #FF0000; font-weight: bold } /* Name.Exception */\n",
       "body .nf { color: #0066BB; font-weight: bold } /* Name.Function */\n",
       "body .nl { color: #997700; font-weight: bold } /* Name.Label */\n",
       "body .nn { color: #0e84b5; font-weight: bold } /* Name.Namespace */\n",
       "body .nt { color: #007700 } /* Name.Tag */\n",
       "body .nv { color: #996633 } /* Name.Variable */\n",
       "body .ow { color: #000000; font-weight: bold } /* Operator.Word */\n",
       "body .w { color: #bbbbbb } /* Text.Whitespace */\n",
       "body .mb { color: #6600EE; font-weight: bold } /* Literal.Number.Bin */\n",
       "body .mf { color: #6600EE; font-weight: bold } /* Literal.Number.Float */\n",
       "body .mh { color: #005588; font-weight: bold } /* Literal.Number.Hex */\n",
       "body .mi { color: #0000DD; font-weight: bold } /* Literal.Number.Integer */\n",
       "body .mo { color: #4400EE; font-weight: bold } /* Literal.Number.Oct */\n",
       "body .sa { background-color: #fff0f0 } /* Literal.String.Affix */\n",
       "body .sb { background-color: #fff0f0 } /* Literal.String.Backtick */\n",
       "body .sc { color: #0044DD } /* Literal.String.Char */\n",
       "body .dl { background-color: #fff0f0 } /* Literal.String.Delimiter */\n",
       "body .sd { color: #DD4422 } /* Literal.String.Doc */\n",
       "body .s2 { background-color: #fff0f0 } /* Literal.String.Double */\n",
       "body .se { color: #666666; font-weight: bold; background-color: #fff0f0 } /* Literal.String.Escape */\n",
       "body .sh { background-color: #fff0f0 } /* Literal.String.Heredoc */\n",
       "body .si { background-color: #eeeeee } /* Literal.String.Interpol */\n",
       "body .sx { color: #DD2200; background-color: #fff0f0 } /* Literal.String.Other */\n",
       "body .sr { color: #000000; background-color: #fff0ff } /* Literal.String.Regex */\n",
       "body .s1 { background-color: #fff0f0 } /* Literal.String.Single */\n",
       "body .ss { color: #AA6600 } /* Literal.String.Symbol */\n",
       "body .bp { color: #007020 } /* Name.Builtin.Pseudo */\n",
       "body .fm { color: #0066BB; font-weight: bold } /* Name.Function.Magic */\n",
       "body .vc { color: #336699 } /* Name.Variable.Class */\n",
       "body .vg { color: #dd7700; font-weight: bold } /* Name.Variable.Global */\n",
       "body .vi { color: #3333BB } /* Name.Variable.Instance */\n",
       "body .vm { color: #996633 } /* Name.Variable.Magic */\n",
       "body .il { color: #0000DD; font-weight: bold } /* Literal.Number.Integer.Long */\n",
       "\n",
       "  </style>\n",
       "</head>\n",
       "<body>\n",
       "<h2></h2>\n",
       "\n",
       "<div class=\"highlight\"><pre><span></span><span class=\"k\">SELECT</span> <span class=\"n\">gp_segment_id</span><span class=\"p\">,</span> \n",
       "  <span class=\"n\">count</span><span class=\"p\">(</span><span class=\"o\">*</span><span class=\"p\">)</span> \n",
       "<span class=\"k\">FROM</span>\n",
       "  <span class=\"n\">demo</span><span class=\"mf\">.</span><span class=\"n\">amzn_reviews</span> \n",
       "<span class=\"k\">GROUP</span> <span class=\"k\">BY</span> <span class=\"mf\">1</span> \n",
       "<span class=\"k\">ORDER</span> <span class=\"k\">BY</span> <span class=\"mf\">1</span><span class=\"p\">;</span>\n",
       "</pre></div>\n",
       "</body>\n",
       "</html>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0 rows affected.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<table>\n",
       "    <tr>\n",
       "        <th>gp_segment_id</th>\n",
       "        <th>count</th>\n",
       "    </tr>\n",
       "</table>"
      ],
      "text/plain": [
       "[]"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sqlfilecode = !pygmentize -f html -O full,style=colorful -l postgres script/4-4-data-distrib-amzn-reviews.sql\n",
    "\n",
    "display_html('\\n'.join(sqlfilecode), raw=True)\n",
    "\n",
    "query = !cat script/4-4-data-distrib-amzn-reviews.sql\n",
    "\n",
    "%sql $DB_USER@$DB_SERVER {''.join(query)}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4.5. *demo.amzn_reviews* Table Size and Disk Space Usage\n",
    "\n",
    "#### 4.5.1. Using PostgreSQL System Administration Functions (PG 8.4)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<!DOCTYPE html PUBLIC \"-//W3C//DTD HTML 4.01//EN\"\n",
       "   \"http://www.w3.org/TR/html4/strict.dtd\">\n",
       "<!--\n",
       "generated by Pygments <http://pygments.org>\n",
       "Copyright 2006-2019 by the Pygments team.\n",
       "Licensed under the BSD license, see LICENSE for details.\n",
       "-->\n",
       "<html>\n",
       "<head>\n",
       "  <title></title>\n",
       "  <meta http-equiv=\"content-type\" content=\"text/html; charset=UTF-8\">\n",
       "  <style type=\"text/css\">\n",
       "/*\n",
       "generated by Pygments <http://pygments.org>\n",
       "Copyright 2006-2019 by the Pygments team.\n",
       "Licensed under the BSD license, see LICENSE for details.\n",
       "*/\n",
       "td.linenos { background-color: #f0f0f0; padding-right: 10px; }\n",
       "span.lineno { background-color: #f0f0f0; padding: 0 5px 0 5px; }\n",
       "pre { line-height: 125%; }\n",
       "body .hll { background-color: #ffffcc }\n",
       "body  { background: #ffffff; }\n",
       "body .c { color: #888888 } /* Comment */\n",
       "body .err { color: #FF0000; background-color: #FFAAAA } /* Error */\n",
       "body .k { color: #008800; font-weight: bold } /* Keyword */\n",
       "body .o { color: #333333 } /* Operator */\n",
       "body .ch { color: #888888 } /* Comment.Hashbang */\n",
       "body .cm { color: #888888 } /* Comment.Multiline */\n",
       "body .cp { color: #557799 } /* Comment.Preproc */\n",
       "body .cpf { color: #888888 } /* Comment.PreprocFile */\n",
       "body .c1 { color: #888888 } /* Comment.Single */\n",
       "body .cs { color: #cc0000; font-weight: bold } /* Comment.Special */\n",
       "body .gd { color: #A00000 } /* Generic.Deleted */\n",
       "body .ge { font-style: italic } /* Generic.Emph */\n",
       "body .gr { color: #FF0000 } /* Generic.Error */\n",
       "body .gh { color: #000080; font-weight: bold } /* Generic.Heading */\n",
       "body .gi { color: #00A000 } /* Generic.Inserted */\n",
       "body .go { color: #888888 } /* Generic.Output */\n",
       "body .gp { color: #c65d09; font-weight: bold } /* Generic.Prompt */\n",
       "body .gs { font-weight: bold } /* Generic.Strong */\n",
       "body .gu { color: #800080; font-weight: bold } /* Generic.Subheading */\n",
       "body .gt { color: #0044DD } /* Generic.Traceback */\n",
       "body .kc { color: #008800; font-weight: bold } /* Keyword.Constant */\n",
       "body .kd { color: #008800; font-weight: bold } /* Keyword.Declaration */\n",
       "body .kn { color: #008800; font-weight: bold } /* Keyword.Namespace */\n",
       "body .kp { color: #003388; font-weight: bold } /* Keyword.Pseudo */\n",
       "body .kr { color: #008800; font-weight: bold } /* Keyword.Reserved */\n",
       "body .kt { color: #333399; font-weight: bold } /* Keyword.Type */\n",
       "body .m { color: #6600EE; font-weight: bold } /* Literal.Number */\n",
       "body .s { background-color: #fff0f0 } /* Literal.String */\n",
       "body .na { color: #0000CC } /* Name.Attribute */\n",
       "body .nb { color: #007020 } /* Name.Builtin */\n",
       "body .nc { color: #BB0066; font-weight: bold } /* Name.Class */\n",
       "body .no { color: #003366; font-weight: bold } /* Name.Constant */\n",
       "body .nd { color: #555555; font-weight: bold } /* Name.Decorator */\n",
       "body .ni { color: #880000; font-weight: bold } /* Name.Entity */\n",
       "body .ne { color: #FF0000; font-weight: bold } /* Name.Exception */\n",
       "body .nf { color: #0066BB; font-weight: bold } /* Name.Function */\n",
       "body .nl { color: #997700; font-weight: bold } /* Name.Label */\n",
       "body .nn { color: #0e84b5; font-weight: bold } /* Name.Namespace */\n",
       "body .nt { color: #007700 } /* Name.Tag */\n",
       "body .nv { color: #996633 } /* Name.Variable */\n",
       "body .ow { color: #000000; font-weight: bold } /* Operator.Word */\n",
       "body .w { color: #bbbbbb } /* Text.Whitespace */\n",
       "body .mb { color: #6600EE; font-weight: bold } /* Literal.Number.Bin */\n",
       "body .mf { color: #6600EE; font-weight: bold } /* Literal.Number.Float */\n",
       "body .mh { color: #005588; font-weight: bold } /* Literal.Number.Hex */\n",
       "body .mi { color: #0000DD; font-weight: bold } /* Literal.Number.Integer */\n",
       "body .mo { color: #4400EE; font-weight: bold } /* Literal.Number.Oct */\n",
       "body .sa { background-color: #fff0f0 } /* Literal.String.Affix */\n",
       "body .sb { background-color: #fff0f0 } /* Literal.String.Backtick */\n",
       "body .sc { color: #0044DD } /* Literal.String.Char */\n",
       "body .dl { background-color: #fff0f0 } /* Literal.String.Delimiter */\n",
       "body .sd { color: #DD4422 } /* Literal.String.Doc */\n",
       "body .s2 { background-color: #fff0f0 } /* Literal.String.Double */\n",
       "body .se { color: #666666; font-weight: bold; background-color: #fff0f0 } /* Literal.String.Escape */\n",
       "body .sh { background-color: #fff0f0 } /* Literal.String.Heredoc */\n",
       "body .si { background-color: #eeeeee } /* Literal.String.Interpol */\n",
       "body .sx { color: #DD2200; background-color: #fff0f0 } /* Literal.String.Other */\n",
       "body .sr { color: #000000; background-color: #fff0ff } /* Literal.String.Regex */\n",
       "body .s1 { background-color: #fff0f0 } /* Literal.String.Single */\n",
       "body .ss { color: #AA6600 } /* Literal.String.Symbol */\n",
       "body .bp { color: #007020 } /* Name.Builtin.Pseudo */\n",
       "body .fm { color: #0066BB; font-weight: bold } /* Name.Function.Magic */\n",
       "body .vc { color: #336699 } /* Name.Variable.Class */\n",
       "body .vg { color: #dd7700; font-weight: bold } /* Name.Variable.Global */\n",
       "body .vi { color: #3333BB } /* Name.Variable.Instance */\n",
       "body .vm { color: #996633 } /* Name.Variable.Magic */\n",
       "body .il { color: #0000DD; font-weight: bold } /* Literal.Number.Integer.Long */\n",
       "\n",
       "  </style>\n",
       "</head>\n",
       "<body>\n",
       "<h2></h2>\n",
       "\n",
       "<div class=\"highlight\"><pre><span></span><span class=\"k\">SELECT</span> <span class=\"s1\">&#39;demo&#39;</span> <span class=\"k\">AS</span> <span class=\"n\">schemaname</span><span class=\"p\">,</span> <span class=\"s1\">&#39;amzn_reviews&#39;</span> <span class=\"k\">AS</span> <span class=\"n\">tablename</span><span class=\"p\">,</span> <span class=\"n\">pg_size_pretty</span><span class=\"p\">(</span><span class=\"n\">pg_relation_size</span><span class=\"p\">(</span><span class=\"s1\">&#39;demo.amzn_reviews&#39;</span><span class=\"p\">))</span> <span class=\"k\">AS</span> <span class=\"n\">size</span><span class=\"p\">,</span> <span class=\"s1\">&#39;Disk space used by the table or index.&#39;</span> <span class=\"k\">AS</span> <span class=\"k\">level</span> \n",
       "<span class=\"k\">UNION</span> <span class=\"k\">ALL</span> \n",
       "<span class=\"k\">SELECT</span> <span class=\"s1\">&#39;demo&#39;</span> <span class=\"k\">AS</span> <span class=\"n\">schemaname</span><span class=\"p\">,</span> <span class=\"s1\">&#39;amzn_reviews&#39;</span> <span class=\"k\">AS</span> <span class=\"n\">tablename</span><span class=\"p\">,</span> <span class=\"n\">pg_size_pretty</span><span class=\"p\">(</span><span class=\"n\">pg_total_relation_size</span><span class=\"p\">(</span><span class=\"s1\">&#39;demo.amzn_reviews&#39;</span><span class=\"p\">))</span> <span class=\"k\">AS</span> <span class=\"n\">size</span><span class=\"p\">,</span> <span class=\"s1\">&#39;Total disk space used by the table, including indexes and toasted data.&#39;</span> <span class=\"k\">AS</span> <span class=\"k\">level</span><span class=\"p\">;</span>\n",
       "</pre></div>\n",
       "</body>\n",
       "</html>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2 rows affected.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<table>\n",
       "    <tr>\n",
       "        <th>schemaname</th>\n",
       "        <th>tablename</th>\n",
       "        <th>size</th>\n",
       "        <th>level</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td>demo</td>\n",
       "        <td>amzn_reviews</td>\n",
       "        <td>0 bytes</td>\n",
       "        <td>Disk space used by the table or index.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td>demo</td>\n",
       "        <td>amzn_reviews</td>\n",
       "        <td>2336 kB</td>\n",
       "        <td>Total disk space used by the table, including indexes and toasted data.</td>\n",
       "    </tr>\n",
       "</table>"
      ],
      "text/plain": [
       "[('demo', 'amzn_reviews', '0 bytes', 'Disk space used by the table or index.'),\n",
       " ('demo', 'amzn_reviews', '2336 kB', 'Total disk space used by the table, including indexes and toasted data.')]"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sqlfilecode = !pygmentize -f html -O full,style=colorful -l postgres script/4-5-1-object-size-and-disk-space.sql\n",
    "\n",
    "display_html('\\n'.join(sqlfilecode), raw=True)\n",
    "\n",
    "query = !cat script/4-5-1-object-size-and-disk-space.sql\n",
    "\n",
    "%sql $DB_USER@$DB_SERVER {''.join(query)}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 4.5.2. Using the `gp_toolkit` Administrative Schema (Greenplum 5.x)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<!DOCTYPE html PUBLIC \"-//W3C//DTD HTML 4.01//EN\"\n",
       "   \"http://www.w3.org/TR/html4/strict.dtd\">\n",
       "<!--\n",
       "generated by Pygments <http://pygments.org>\n",
       "Copyright 2006-2019 by the Pygments team.\n",
       "Licensed under the BSD license, see LICENSE for details.\n",
       "-->\n",
       "<html>\n",
       "<head>\n",
       "  <title></title>\n",
       "  <meta http-equiv=\"content-type\" content=\"text/html; charset=UTF-8\">\n",
       "  <style type=\"text/css\">\n",
       "/*\n",
       "generated by Pygments <http://pygments.org>\n",
       "Copyright 2006-2019 by the Pygments team.\n",
       "Licensed under the BSD license, see LICENSE for details.\n",
       "*/\n",
       "td.linenos { background-color: #f0f0f0; padding-right: 10px; }\n",
       "span.lineno { background-color: #f0f0f0; padding: 0 5px 0 5px; }\n",
       "pre { line-height: 125%; }\n",
       "body .hll { background-color: #ffffcc }\n",
       "body  { background: #ffffff; }\n",
       "body .c { color: #888888 } /* Comment */\n",
       "body .err { color: #FF0000; background-color: #FFAAAA } /* Error */\n",
       "body .k { color: #008800; font-weight: bold } /* Keyword */\n",
       "body .o { color: #333333 } /* Operator */\n",
       "body .ch { color: #888888 } /* Comment.Hashbang */\n",
       "body .cm { color: #888888 } /* Comment.Multiline */\n",
       "body .cp { color: #557799 } /* Comment.Preproc */\n",
       "body .cpf { color: #888888 } /* Comment.PreprocFile */\n",
       "body .c1 { color: #888888 } /* Comment.Single */\n",
       "body .cs { color: #cc0000; font-weight: bold } /* Comment.Special */\n",
       "body .gd { color: #A00000 } /* Generic.Deleted */\n",
       "body .ge { font-style: italic } /* Generic.Emph */\n",
       "body .gr { color: #FF0000 } /* Generic.Error */\n",
       "body .gh { color: #000080; font-weight: bold } /* Generic.Heading */\n",
       "body .gi { color: #00A000 } /* Generic.Inserted */\n",
       "body .go { color: #888888 } /* Generic.Output */\n",
       "body .gp { color: #c65d09; font-weight: bold } /* Generic.Prompt */\n",
       "body .gs { font-weight: bold } /* Generic.Strong */\n",
       "body .gu { color: #800080; font-weight: bold } /* Generic.Subheading */\n",
       "body .gt { color: #0044DD } /* Generic.Traceback */\n",
       "body .kc { color: #008800; font-weight: bold } /* Keyword.Constant */\n",
       "body .kd { color: #008800; font-weight: bold } /* Keyword.Declaration */\n",
       "body .kn { color: #008800; font-weight: bold } /* Keyword.Namespace */\n",
       "body .kp { color: #003388; font-weight: bold } /* Keyword.Pseudo */\n",
       "body .kr { color: #008800; font-weight: bold } /* Keyword.Reserved */\n",
       "body .kt { color: #333399; font-weight: bold } /* Keyword.Type */\n",
       "body .m { color: #6600EE; font-weight: bold } /* Literal.Number */\n",
       "body .s { background-color: #fff0f0 } /* Literal.String */\n",
       "body .na { color: #0000CC } /* Name.Attribute */\n",
       "body .nb { color: #007020 } /* Name.Builtin */\n",
       "body .nc { color: #BB0066; font-weight: bold } /* Name.Class */\n",
       "body .no { color: #003366; font-weight: bold } /* Name.Constant */\n",
       "body .nd { color: #555555; font-weight: bold } /* Name.Decorator */\n",
       "body .ni { color: #880000; font-weight: bold } /* Name.Entity */\n",
       "body .ne { color: #FF0000; font-weight: bold } /* Name.Exception */\n",
       "body .nf { color: #0066BB; font-weight: bold } /* Name.Function */\n",
       "body .nl { color: #997700; font-weight: bold } /* Name.Label */\n",
       "body .nn { color: #0e84b5; font-weight: bold } /* Name.Namespace */\n",
       "body .nt { color: #007700 } /* Name.Tag */\n",
       "body .nv { color: #996633 } /* Name.Variable */\n",
       "body .ow { color: #000000; font-weight: bold } /* Operator.Word */\n",
       "body .w { color: #bbbbbb } /* Text.Whitespace */\n",
       "body .mb { color: #6600EE; font-weight: bold } /* Literal.Number.Bin */\n",
       "body .mf { color: #6600EE; font-weight: bold } /* Literal.Number.Float */\n",
       "body .mh { color: #005588; font-weight: bold } /* Literal.Number.Hex */\n",
       "body .mi { color: #0000DD; font-weight: bold } /* Literal.Number.Integer */\n",
       "body .mo { color: #4400EE; font-weight: bold } /* Literal.Number.Oct */\n",
       "body .sa { background-color: #fff0f0 } /* Literal.String.Affix */\n",
       "body .sb { background-color: #fff0f0 } /* Literal.String.Backtick */\n",
       "body .sc { color: #0044DD } /* Literal.String.Char */\n",
       "body .dl { background-color: #fff0f0 } /* Literal.String.Delimiter */\n",
       "body .sd { color: #DD4422 } /* Literal.String.Doc */\n",
       "body .s2 { background-color: #fff0f0 } /* Literal.String.Double */\n",
       "body .se { color: #666666; font-weight: bold; background-color: #fff0f0 } /* Literal.String.Escape */\n",
       "body .sh { background-color: #fff0f0 } /* Literal.String.Heredoc */\n",
       "body .si { background-color: #eeeeee } /* Literal.String.Interpol */\n",
       "body .sx { color: #DD2200; background-color: #fff0f0 } /* Literal.String.Other */\n",
       "body .sr { color: #000000; background-color: #fff0ff } /* Literal.String.Regex */\n",
       "body .s1 { background-color: #fff0f0 } /* Literal.String.Single */\n",
       "body .ss { color: #AA6600 } /* Literal.String.Symbol */\n",
       "body .bp { color: #007020 } /* Name.Builtin.Pseudo */\n",
       "body .fm { color: #0066BB; font-weight: bold } /* Name.Function.Magic */\n",
       "body .vc { color: #336699 } /* Name.Variable.Class */\n",
       "body .vg { color: #dd7700; font-weight: bold } /* Name.Variable.Global */\n",
       "body .vi { color: #3333BB } /* Name.Variable.Instance */\n",
       "body .vm { color: #996633 } /* Name.Variable.Magic */\n",
       "body .il { color: #0000DD; font-weight: bold } /* Literal.Number.Integer.Long */\n",
       "\n",
       "  </style>\n",
       "</head>\n",
       "<body>\n",
       "<h2></h2>\n",
       "\n",
       "<div class=\"highlight\"><pre><span></span><span class=\"k\">SELECT</span> <span class=\"n\">pg_namespace</span><span class=\"mf\">.</span><span class=\"n\">nspname</span> <span class=\"k\">AS</span> <span class=\"k\">schema</span><span class=\"p\">,</span>\n",
       "\t<span class=\"n\">pg_class</span><span class=\"mf\">.</span><span class=\"n\">relname</span> <span class=\"k\">AS</span> <span class=\"n\">relation</span><span class=\"p\">,</span> \n",
       "\t<span class=\"n\">pg_size_pretty</span><span class=\"p\">(</span><span class=\"n\">sotd</span><span class=\"mf\">.</span><span class=\"n\">sotdsize</span><span class=\"o\">::</span><span class=\"nb\">BIGINT</span><span class=\"p\">)</span> <span class=\"k\">as</span> <span class=\"n\">tablesize</span><span class=\"p\">,</span> \n",
       "\t<span class=\"n\">pg_size_pretty</span><span class=\"p\">(</span><span class=\"n\">sotd</span><span class=\"mf\">.</span><span class=\"n\">sotdtoastsize</span><span class=\"o\">::</span><span class=\"nb\">BIGINT</span><span class=\"p\">)</span> <span class=\"k\">as</span> <span class=\"n\">toastsize</span><span class=\"p\">,</span> \n",
       "\t<span class=\"n\">pg_size_pretty</span><span class=\"p\">(</span><span class=\"n\">sotd</span><span class=\"mf\">.</span><span class=\"n\">sotdadditionalsize</span><span class=\"o\">::</span><span class=\"nb\">BIGINT</span><span class=\"p\">)</span> <span class=\"k\">as</span> <span class=\"n\">othersize</span><span class=\"p\">,</span> \n",
       "\t<span class=\"n\">pg_size_pretty</span><span class=\"p\">(</span><span class=\"n\">sotaid</span><span class=\"mf\">.</span><span class=\"n\">sotaidtablesize</span><span class=\"o\">::</span><span class=\"nb\">BIGINT</span><span class=\"p\">)</span> <span class=\"k\">as</span> <span class=\"n\">tabledisksize</span><span class=\"p\">,</span> \n",
       "\t<span class=\"n\">pg_size_pretty</span><span class=\"p\">(</span><span class=\"n\">sotaid</span><span class=\"mf\">.</span><span class=\"n\">sotaididxsize</span><span class=\"o\">::</span><span class=\"nb\">BIGINT</span><span class=\"p\">)</span> <span class=\"k\">as</span> <span class=\"n\">indexsize</span> \n",
       "<span class=\"k\">FROM</span> <span class=\"n\">pg_class</span> \n",
       "\t<span class=\"k\">LEFT</span> <span class=\"k\">JOIN</span> <span class=\"n\">pg_stat_user_tables</span> \n",
       "\t\t<span class=\"k\">ON</span> <span class=\"n\">pg_stat_user_tables</span><span class=\"mf\">.</span><span class=\"n\">relid</span> <span class=\"o\">=</span> <span class=\"n\">pg_class</span><span class=\"mf\">.</span><span class=\"n\">oid</span> \n",
       "\t<span class=\"k\">LEFT</span> <span class=\"k\">JOIN</span> <span class=\"n\">gp_toolkit</span><span class=\"mf\">.</span><span class=\"n\">gp_size_of_table_disk</span> <span class=\"n\">sotd</span> \n",
       "\t\t<span class=\"k\">ON</span> <span class=\"n\">sotd</span><span class=\"mf\">.</span><span class=\"n\">sotdoid</span> <span class=\"o\">=</span> <span class=\"n\">pg_class</span><span class=\"mf\">.</span><span class=\"n\">oid</span> \n",
       "\t<span class=\"k\">LEFT</span> <span class=\"k\">JOIN</span> <span class=\"n\">gp_toolkit</span><span class=\"mf\">.</span><span class=\"n\">gp_size_of_table_and_indexes_disk</span> <span class=\"n\">sotaid</span> \n",
       "\t\t<span class=\"k\">ON</span> <span class=\"n\">sotaid</span><span class=\"mf\">.</span><span class=\"n\">sotaidoid</span> <span class=\"o\">=</span> <span class=\"n\">pg_class</span><span class=\"mf\">.</span><span class=\"n\">oid</span> \n",
       "\t<span class=\"k\">LEFT</span> <span class=\"k\">JOIN</span> <span class=\"n\">pg_namespace</span> \n",
       "\t\t<span class=\"k\">ON</span> <span class=\"n\">pg_namespace</span><span class=\"mf\">.</span><span class=\"n\">oid</span> <span class=\"o\">=</span> <span class=\"n\">pg_class</span><span class=\"mf\">.</span><span class=\"n\">relnamespace</span> \n",
       "<span class=\"k\">WHERE</span> \n",
       "\t<span class=\"n\">pg_class</span><span class=\"mf\">.</span><span class=\"n\">relkind</span> <span class=\"o\">=</span> <span class=\"s1\">&#39;r&#39;</span>\n",
       "\t<span class=\"k\">AND</span> <span class=\"n\">relstorage</span> <span class=\"o\">!=</span> <span class=\"s1\">&#39;x&#39;</span> \t\t\t\t\t\t\n",
       "\t<span class=\"k\">AND</span> <span class=\"n\">pg_namespace</span><span class=\"mf\">.</span><span class=\"n\">nspname</span> <span class=\"k\">NOT</span> <span class=\"k\">IN</span> <span class=\"p\">(</span><span class=\"s1\">&#39;information_schema&#39;</span><span class=\"p\">,</span> <span class=\"s1\">&#39;madlib&#39;</span><span class=\"p\">,</span> <span class=\"s1\">&#39;pg_catalog&#39;</span><span class=\"p\">,</span> <span class=\"s1\">&#39;gptext&#39;</span><span class=\"p\">)</span> \n",
       "\t<span class=\"k\">AND</span> <span class=\"n\">pg_class</span><span class=\"mf\">.</span><span class=\"n\">relname</span> <span class=\"k\">NOT</span> <span class=\"k\">IN</span> <span class=\"p\">(</span><span class=\"s1\">&#39;spatial_ref_sys&#39;</span><span class=\"p\">)</span> \n",
       "<span class=\"k\">ORDER</span> <span class=\"k\">BY</span> <span class=\"mf\">1</span><span class=\"p\">,</span> <span class=\"mf\">2</span>\n",
       "</pre></div>\n",
       "</body>\n",
       "</html>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "21 rows affected.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<table>\n",
       "    <tr>\n",
       "        <th>schema</th>\n",
       "        <th>relation</th>\n",
       "        <th>tablesize</th>\n",
       "        <th>toastsize</th>\n",
       "        <th>othersize</th>\n",
       "        <th>tabledisksize</th>\n",
       "        <th>indexsize</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td>demo</td>\n",
       "        <td>amzn_reviews</td>\n",
       "        <td>0 bytes</td>\n",
       "        <td>2336 kB</td>\n",
       "        <td>0 bytes</td>\n",
       "        <td>2336 kB</td>\n",
       "        <td>0 bytes</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td>demo</td>\n",
       "        <td>amzn_reviews_1</td>\n",
       "        <td>86 GB</td>\n",
       "        <td>259 MB</td>\n",
       "        <td>0 bytes</td>\n",
       "        <td>86 GB</td>\n",
       "        <td>0 bytes</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td>demo</td>\n",
       "        <td>amzn_reviews_1_1</td>\n",
       "        <td>86 GB</td>\n",
       "        <td>259 MB</td>\n",
       "        <td>0 bytes</td>\n",
       "        <td>86 GB</td>\n",
       "        <td>0 bytes</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td>demo</td>\n",
       "        <td>amzn_reviews_1_2</td>\n",
       "        <td>83 GB</td>\n",
       "        <td>260 MB</td>\n",
       "        <td>4640 kB</td>\n",
       "        <td>83 GB</td>\n",
       "        <td>0 bytes</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td>demo</td>\n",
       "        <td>amzn_reviews_2</td>\n",
       "        <td>86 GB</td>\n",
       "        <td>259 MB</td>\n",
       "        <td>0 bytes</td>\n",
       "        <td>86 GB</td>\n",
       "        <td>0 bytes</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td>demo</td>\n",
       "        <td>amzn_reviews_3</td>\n",
       "        <td>86 GB</td>\n",
       "        <td>259 MB</td>\n",
       "        <td>0 bytes</td>\n",
       "        <td>86 GB</td>\n",
       "        <td>0 bytes</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td>demo</td>\n",
       "        <td>amzn_reviews_3_1</td>\n",
       "        <td>86 GB</td>\n",
       "        <td>259 MB</td>\n",
       "        <td>0 bytes</td>\n",
       "        <td>86 GB</td>\n",
       "        <td>0 bytes</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td>demo</td>\n",
       "        <td>amzn_reviews_summary</td>\n",
       "        <td>32 kB</td>\n",
       "        <td>2336 kB</td>\n",
       "        <td>0 bytes</td>\n",
       "        <td>2368 kB</td>\n",
       "        <td>0 bytes</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td>public</td>\n",
       "        <td>test100</td>\n",
       "        <td>1760 kB</td>\n",
       "        <td>0 bytes</td>\n",
       "        <td>0 bytes</td>\n",
       "        <td>1760 kB</td>\n",
       "        <td>0 bytes</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td>public</td>\n",
       "        <td>test1000</td>\n",
       "        <td>2304 kB</td>\n",
       "        <td>0 bytes</td>\n",
       "        <td>0 bytes</td>\n",
       "        <td>2304 kB</td>\n",
       "        <td>0 bytes</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td>public</td>\n",
       "        <td>test1000_summary</td>\n",
       "        <td>32 kB</td>\n",
       "        <td>2336 kB</td>\n",
       "        <td>0 bytes</td>\n",
       "        <td>2368 kB</td>\n",
       "        <td>0 bytes</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td>public</td>\n",
       "        <td>test100_summary</td>\n",
       "        <td>32 kB</td>\n",
       "        <td>2336 kB</td>\n",
       "        <td>0 bytes</td>\n",
       "        <td>2368 kB</td>\n",
       "        <td>0 bytes</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td>public</td>\n",
       "        <td>test100k</td>\n",
       "        <td>4608 kB</td>\n",
       "        <td>0 bytes</td>\n",
       "        <td>0 bytes</td>\n",
       "        <td>4608 kB</td>\n",
       "        <td>0 bytes</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td>public</td>\n",
       "        <td>test100k_summary</td>\n",
       "        <td>32 kB</td>\n",
       "        <td>2336 kB</td>\n",
       "        <td>0 bytes</td>\n",
       "        <td>2368 kB</td>\n",
       "        <td>0 bytes</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td>public</td>\n",
       "        <td>test100m</td>\n",
       "        <td>4201 MB</td>\n",
       "        <td>0 bytes</td>\n",
       "        <td>0 bytes</td>\n",
       "        <td>4201 MB</td>\n",
       "        <td>0 bytes</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td>public</td>\n",
       "        <td>test10k</td>\n",
       "        <td>2304 kB</td>\n",
       "        <td>0 bytes</td>\n",
       "        <td>0 bytes</td>\n",
       "        <td>2304 kB</td>\n",
       "        <td>0 bytes</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td>public</td>\n",
       "        <td>test10k_summary</td>\n",
       "        <td>32 kB</td>\n",
       "        <td>2336 kB</td>\n",
       "        <td>0 bytes</td>\n",
       "        <td>2368 kB</td>\n",
       "        <td>0 bytes</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td>public</td>\n",
       "        <td>test10m</td>\n",
       "        <td>421 MB</td>\n",
       "        <td>0 bytes</td>\n",
       "        <td>0 bytes</td>\n",
       "        <td>421 MB</td>\n",
       "        <td>0 bytes</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td>public</td>\n",
       "        <td>test10m_summary</td>\n",
       "        <td>32 kB</td>\n",
       "        <td>2336 kB</td>\n",
       "        <td>0 bytes</td>\n",
       "        <td>2368 kB</td>\n",
       "        <td>0 bytes</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td>public</td>\n",
       "        <td>test1m</td>\n",
       "        <td>43 MB</td>\n",
       "        <td>0 bytes</td>\n",
       "        <td>0 bytes</td>\n",
       "        <td>43 MB</td>\n",
       "        <td>0 bytes</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td>public</td>\n",
       "        <td>test1m_summary</td>\n",
       "        <td>32 kB</td>\n",
       "        <td>2336 kB</td>\n",
       "        <td>0 bytes</td>\n",
       "        <td>2368 kB</td>\n",
       "        <td>0 bytes</td>\n",
       "    </tr>\n",
       "</table>"
      ],
      "text/plain": [
       "[('demo', 'amzn_reviews', '0 bytes', '2336 kB', '0 bytes', '2336 kB', '0 bytes'),\n",
       " ('demo', 'amzn_reviews_1', '86 GB', '259 MB', '0 bytes', '86 GB', '0 bytes'),\n",
       " ('demo', 'amzn_reviews_1_1', '86 GB', '259 MB', '0 bytes', '86 GB', '0 bytes'),\n",
       " ('demo', 'amzn_reviews_1_2', '83 GB', '260 MB', '4640 kB', '83 GB', '0 bytes'),\n",
       " ('demo', 'amzn_reviews_2', '86 GB', '259 MB', '0 bytes', '86 GB', '0 bytes'),\n",
       " ('demo', 'amzn_reviews_3', '86 GB', '259 MB', '0 bytes', '86 GB', '0 bytes'),\n",
       " ('demo', 'amzn_reviews_3_1', '86 GB', '259 MB', '0 bytes', '86 GB', '0 bytes'),\n",
       " ('demo', 'amzn_reviews_summary', '32 kB', '2336 kB', '0 bytes', '2368 kB', '0 bytes'),\n",
       " ('public', 'test100', '1760 kB', '0 bytes', '0 bytes', '1760 kB', '0 bytes'),\n",
       " ('public', 'test1000', '2304 kB', '0 bytes', '0 bytes', '2304 kB', '0 bytes'),\n",
       " ('public', 'test1000_summary', '32 kB', '2336 kB', '0 bytes', '2368 kB', '0 bytes'),\n",
       " ('public', 'test100_summary', '32 kB', '2336 kB', '0 bytes', '2368 kB', '0 bytes'),\n",
       " ('public', 'test100k', '4608 kB', '0 bytes', '0 bytes', '4608 kB', '0 bytes'),\n",
       " ('public', 'test100k_summary', '32 kB', '2336 kB', '0 bytes', '2368 kB', '0 bytes'),\n",
       " ('public', 'test100m', '4201 MB', '0 bytes', '0 bytes', '4201 MB', '0 bytes'),\n",
       " ('public', 'test10k', '2304 kB', '0 bytes', '0 bytes', '2304 kB', '0 bytes'),\n",
       " ('public', 'test10k_summary', '32 kB', '2336 kB', '0 bytes', '2368 kB', '0 bytes'),\n",
       " ('public', 'test10m', '421 MB', '0 bytes', '0 bytes', '421 MB', '0 bytes'),\n",
       " ('public', 'test10m_summary', '32 kB', '2336 kB', '0 bytes', '2368 kB', '0 bytes'),\n",
       " ('public', 'test1m', '43 MB', '0 bytes', '0 bytes', '43 MB', '0 bytes'),\n",
       " ('public', 'test1m_summary', '32 kB', '2336 kB', '0 bytes', '2368 kB', '0 bytes')]"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sqlfilecode = !pygmentize -f html -O full,style=colorful -l postgres script/4-5-2-object-size-and-disk-space.sql\n",
    "\n",
    "display_html('\\n'.join(sqlfilecode), raw=True)\n",
    "\n",
    "query = !cat script/4-5-2-object-size-and-disk-space.sql\n",
    "\n",
    "%sql $DB_USER@$DB_SERVER {''.join(query)}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4.6. Check table for Data Skew\n",
    "Data skew may be caused by uneven data distribution due to the wrong choice of distribution keys or single tuple table insert or copy operations. Present at the table level, data skew, is often the root cause of poor query performance and out of memory conditions. Skewed data affects scan (read) performance, but it also affects all other query execution operations, for instance, joins and group by operations.\n",
    "\n",
    "It is very important to *validate* distributions to ensure that data is evenly distributed after the initial load. It is equally important to *continue* to validate distributions after incremental loads.\n",
    "\n",
    "The following query shows the number of rows per segment as well as the variance from the minimum and maximum numbers of rows:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<!DOCTYPE html PUBLIC \"-//W3C//DTD HTML 4.01//EN\"\n",
       "   \"http://www.w3.org/TR/html4/strict.dtd\">\n",
       "<!--\n",
       "generated by Pygments <http://pygments.org>\n",
       "Copyright 2006-2019 by the Pygments team.\n",
       "Licensed under the BSD license, see LICENSE for details.\n",
       "-->\n",
       "<html>\n",
       "<head>\n",
       "  <title></title>\n",
       "  <meta http-equiv=\"content-type\" content=\"text/html; charset=UTF-8\">\n",
       "  <style type=\"text/css\">\n",
       "/*\n",
       "generated by Pygments <http://pygments.org>\n",
       "Copyright 2006-2019 by the Pygments team.\n",
       "Licensed under the BSD license, see LICENSE for details.\n",
       "*/\n",
       "td.linenos { background-color: #f0f0f0; padding-right: 10px; }\n",
       "span.lineno { background-color: #f0f0f0; padding: 0 5px 0 5px; }\n",
       "pre { line-height: 125%; }\n",
       "body .hll { background-color: #ffffcc }\n",
       "body  { background: #ffffff; }\n",
       "body .c { color: #888888 } /* Comment */\n",
       "body .err { color: #FF0000; background-color: #FFAAAA } /* Error */\n",
       "body .k { color: #008800; font-weight: bold } /* Keyword */\n",
       "body .o { color: #333333 } /* Operator */\n",
       "body .ch { color: #888888 } /* Comment.Hashbang */\n",
       "body .cm { color: #888888 } /* Comment.Multiline */\n",
       "body .cp { color: #557799 } /* Comment.Preproc */\n",
       "body .cpf { color: #888888 } /* Comment.PreprocFile */\n",
       "body .c1 { color: #888888 } /* Comment.Single */\n",
       "body .cs { color: #cc0000; font-weight: bold } /* Comment.Special */\n",
       "body .gd { color: #A00000 } /* Generic.Deleted */\n",
       "body .ge { font-style: italic } /* Generic.Emph */\n",
       "body .gr { color: #FF0000 } /* Generic.Error */\n",
       "body .gh { color: #000080; font-weight: bold } /* Generic.Heading */\n",
       "body .gi { color: #00A000 } /* Generic.Inserted */\n",
       "body .go { color: #888888 } /* Generic.Output */\n",
       "body .gp { color: #c65d09; font-weight: bold } /* Generic.Prompt */\n",
       "body .gs { font-weight: bold } /* Generic.Strong */\n",
       "body .gu { color: #800080; font-weight: bold } /* Generic.Subheading */\n",
       "body .gt { color: #0044DD } /* Generic.Traceback */\n",
       "body .kc { color: #008800; font-weight: bold } /* Keyword.Constant */\n",
       "body .kd { color: #008800; font-weight: bold } /* Keyword.Declaration */\n",
       "body .kn { color: #008800; font-weight: bold } /* Keyword.Namespace */\n",
       "body .kp { color: #003388; font-weight: bold } /* Keyword.Pseudo */\n",
       "body .kr { color: #008800; font-weight: bold } /* Keyword.Reserved */\n",
       "body .kt { color: #333399; font-weight: bold } /* Keyword.Type */\n",
       "body .m { color: #6600EE; font-weight: bold } /* Literal.Number */\n",
       "body .s { background-color: #fff0f0 } /* Literal.String */\n",
       "body .na { color: #0000CC } /* Name.Attribute */\n",
       "body .nb { color: #007020 } /* Name.Builtin */\n",
       "body .nc { color: #BB0066; font-weight: bold } /* Name.Class */\n",
       "body .no { color: #003366; font-weight: bold } /* Name.Constant */\n",
       "body .nd { color: #555555; font-weight: bold } /* Name.Decorator */\n",
       "body .ni { color: #880000; font-weight: bold } /* Name.Entity */\n",
       "body .ne { color: #FF0000; font-weight: bold } /* Name.Exception */\n",
       "body .nf { color: #0066BB; font-weight: bold } /* Name.Function */\n",
       "body .nl { color: #997700; font-weight: bold } /* Name.Label */\n",
       "body .nn { color: #0e84b5; font-weight: bold } /* Name.Namespace */\n",
       "body .nt { color: #007700 } /* Name.Tag */\n",
       "body .nv { color: #996633 } /* Name.Variable */\n",
       "body .ow { color: #000000; font-weight: bold } /* Operator.Word */\n",
       "body .w { color: #bbbbbb } /* Text.Whitespace */\n",
       "body .mb { color: #6600EE; font-weight: bold } /* Literal.Number.Bin */\n",
       "body .mf { color: #6600EE; font-weight: bold } /* Literal.Number.Float */\n",
       "body .mh { color: #005588; font-weight: bold } /* Literal.Number.Hex */\n",
       "body .mi { color: #0000DD; font-weight: bold } /* Literal.Number.Integer */\n",
       "body .mo { color: #4400EE; font-weight: bold } /* Literal.Number.Oct */\n",
       "body .sa { background-color: #fff0f0 } /* Literal.String.Affix */\n",
       "body .sb { background-color: #fff0f0 } /* Literal.String.Backtick */\n",
       "body .sc { color: #0044DD } /* Literal.String.Char */\n",
       "body .dl { background-color: #fff0f0 } /* Literal.String.Delimiter */\n",
       "body .sd { color: #DD4422 } /* Literal.String.Doc */\n",
       "body .s2 { background-color: #fff0f0 } /* Literal.String.Double */\n",
       "body .se { color: #666666; font-weight: bold; background-color: #fff0f0 } /* Literal.String.Escape */\n",
       "body .sh { background-color: #fff0f0 } /* Literal.String.Heredoc */\n",
       "body .si { background-color: #eeeeee } /* Literal.String.Interpol */\n",
       "body .sx { color: #DD2200; background-color: #fff0f0 } /* Literal.String.Other */\n",
       "body .sr { color: #000000; background-color: #fff0ff } /* Literal.String.Regex */\n",
       "body .s1 { background-color: #fff0f0 } /* Literal.String.Single */\n",
       "body .ss { color: #AA6600 } /* Literal.String.Symbol */\n",
       "body .bp { color: #007020 } /* Name.Builtin.Pseudo */\n",
       "body .fm { color: #0066BB; font-weight: bold } /* Name.Function.Magic */\n",
       "body .vc { color: #336699 } /* Name.Variable.Class */\n",
       "body .vg { color: #dd7700; font-weight: bold } /* Name.Variable.Global */\n",
       "body .vi { color: #3333BB } /* Name.Variable.Instance */\n",
       "body .vm { color: #996633 } /* Name.Variable.Magic */\n",
       "body .il { color: #0000DD; font-weight: bold } /* Literal.Number.Integer.Long */\n",
       "\n",
       "  </style>\n",
       "</head>\n",
       "<body>\n",
       "<h2></h2>\n",
       "\n",
       "<div class=\"highlight\"><pre><span></span><span class=\"k\">SELECT</span>\n",
       "\t<span class=\"s1\">&#39;demo.amzn_reviews&#39;</span> <span class=\"k\">AS</span> <span class=\"s s-Name\">&quot;Table Name&quot;</span><span class=\"p\">,</span> \n",
       "\t<span class=\"n\">max</span><span class=\"p\">(</span><span class=\"n\">c</span><span class=\"p\">)</span> <span class=\"k\">AS</span> <span class=\"s s-Name\">&quot;Max Seg Rows&quot;</span><span class=\"p\">,</span> <span class=\"n\">min</span><span class=\"p\">(</span><span class=\"n\">c</span><span class=\"p\">)</span> <span class=\"k\">AS</span> <span class=\"s s-Name\">&quot;Min Seg Rows&quot;</span><span class=\"p\">,</span> \n",
       "\t<span class=\"p\">(</span><span class=\"n\">max</span><span class=\"p\">(</span><span class=\"n\">c</span><span class=\"p\">)</span><span class=\"o\">-</span><span class=\"n\">min</span><span class=\"p\">(</span><span class=\"n\">c</span><span class=\"p\">))</span><span class=\"o\">*</span><span class=\"mf\">100.0</span><span class=\"o\">/</span><span class=\"n\">max</span><span class=\"p\">(</span><span class=\"n\">c</span><span class=\"p\">)</span> <span class=\"k\">AS</span> <span class=\"s s-Name\">&quot;Percentage Difference Between Max &amp; Min&quot;</span> \n",
       "<span class=\"k\">FROM</span> <span class=\"p\">(</span>\n",
       "\t<span class=\"k\">SELECT</span>\n",
       "\t\t<span class=\"n\">count</span><span class=\"p\">(</span><span class=\"o\">*</span><span class=\"p\">)</span> <span class=\"n\">c</span><span class=\"p\">,</span> \n",
       "\t\t<span class=\"n\">gp_segment_id</span> \n",
       "\t<span class=\"k\">FROM</span>\n",
       "\t\t<span class=\"n\">demo</span><span class=\"mf\">.</span><span class=\"n\">amzn_reviews</span>\n",
       "\t<span class=\"k\">GROUP</span> <span class=\"k\">BY</span> <span class=\"mf\">2</span>\n",
       "<span class=\"p\">)</span> <span class=\"k\">AS</span> <span class=\"n\">a</span><span class=\"p\">;</span>\n",
       "</pre></div>\n",
       "</body>\n",
       "</html>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1 rows affected.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<table>\n",
       "    <tr>\n",
       "        <th>Table Name</th>\n",
       "        <th>Max Seg Rows</th>\n",
       "        <th>Min Seg Rows</th>\n",
       "        <th>Percentage Difference Between Max &amp; Min</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td>demo.amzn_reviews</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "        <td>None</td>\n",
       "    </tr>\n",
       "</table>"
      ],
      "text/plain": [
       "[('demo.amzn_reviews', None, None, None)]"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sqlfilecode = !pygmentize -f html -O full,style=colorful -l postgres script/4-6-1-data-skew.sql\n",
    "\n",
    "display_html('\\n'.join(sqlfilecode), raw=True)\n",
    "\n",
    "query = !cat script/4-6-1-data-skew.sql\n",
    "\n",
    "%sql $DB_USER@$DB_SERVER {''.join(query)}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `gp_toolkit` schema has two views that you can use to check for skew.\n",
    "- The `gp_toolkit.gp_skew_coefficients` view shows data distribution skew by calculating the coefficient of variation (CV) for the data stored on each segment. The `skccoeff` column shows the coefficient of variation (CV), which is calculated as the standard deviation divided by the average. It takes into account both the average and variability around the average of a data series. The lower the value, the better. Higher values indicate greater data skew."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<!DOCTYPE html PUBLIC \"-//W3C//DTD HTML 4.01//EN\"\n",
       "   \"http://www.w3.org/TR/html4/strict.dtd\">\n",
       "<!--\n",
       "generated by Pygments <http://pygments.org>\n",
       "Copyright 2006-2019 by the Pygments team.\n",
       "Licensed under the BSD license, see LICENSE for details.\n",
       "-->\n",
       "<html>\n",
       "<head>\n",
       "  <title></title>\n",
       "  <meta http-equiv=\"content-type\" content=\"text/html; charset=UTF-8\">\n",
       "  <style type=\"text/css\">\n",
       "/*\n",
       "generated by Pygments <http://pygments.org>\n",
       "Copyright 2006-2019 by the Pygments team.\n",
       "Licensed under the BSD license, see LICENSE for details.\n",
       "*/\n",
       "td.linenos { background-color: #f0f0f0; padding-right: 10px; }\n",
       "span.lineno { background-color: #f0f0f0; padding: 0 5px 0 5px; }\n",
       "pre { line-height: 125%; }\n",
       "body .hll { background-color: #ffffcc }\n",
       "body  { background: #ffffff; }\n",
       "body .c { color: #888888 } /* Comment */\n",
       "body .err { color: #FF0000; background-color: #FFAAAA } /* Error */\n",
       "body .k { color: #008800; font-weight: bold } /* Keyword */\n",
       "body .o { color: #333333 } /* Operator */\n",
       "body .ch { color: #888888 } /* Comment.Hashbang */\n",
       "body .cm { color: #888888 } /* Comment.Multiline */\n",
       "body .cp { color: #557799 } /* Comment.Preproc */\n",
       "body .cpf { color: #888888 } /* Comment.PreprocFile */\n",
       "body .c1 { color: #888888 } /* Comment.Single */\n",
       "body .cs { color: #cc0000; font-weight: bold } /* Comment.Special */\n",
       "body .gd { color: #A00000 } /* Generic.Deleted */\n",
       "body .ge { font-style: italic } /* Generic.Emph */\n",
       "body .gr { color: #FF0000 } /* Generic.Error */\n",
       "body .gh { color: #000080; font-weight: bold } /* Generic.Heading */\n",
       "body .gi { color: #00A000 } /* Generic.Inserted */\n",
       "body .go { color: #888888 } /* Generic.Output */\n",
       "body .gp { color: #c65d09; font-weight: bold } /* Generic.Prompt */\n",
       "body .gs { font-weight: bold } /* Generic.Strong */\n",
       "body .gu { color: #800080; font-weight: bold } /* Generic.Subheading */\n",
       "body .gt { color: #0044DD } /* Generic.Traceback */\n",
       "body .kc { color: #008800; font-weight: bold } /* Keyword.Constant */\n",
       "body .kd { color: #008800; font-weight: bold } /* Keyword.Declaration */\n",
       "body .kn { color: #008800; font-weight: bold } /* Keyword.Namespace */\n",
       "body .kp { color: #003388; font-weight: bold } /* Keyword.Pseudo */\n",
       "body .kr { color: #008800; font-weight: bold } /* Keyword.Reserved */\n",
       "body .kt { color: #333399; font-weight: bold } /* Keyword.Type */\n",
       "body .m { color: #6600EE; font-weight: bold } /* Literal.Number */\n",
       "body .s { background-color: #fff0f0 } /* Literal.String */\n",
       "body .na { color: #0000CC } /* Name.Attribute */\n",
       "body .nb { color: #007020 } /* Name.Builtin */\n",
       "body .nc { color: #BB0066; font-weight: bold } /* Name.Class */\n",
       "body .no { color: #003366; font-weight: bold } /* Name.Constant */\n",
       "body .nd { color: #555555; font-weight: bold } /* Name.Decorator */\n",
       "body .ni { color: #880000; font-weight: bold } /* Name.Entity */\n",
       "body .ne { color: #FF0000; font-weight: bold } /* Name.Exception */\n",
       "body .nf { color: #0066BB; font-weight: bold } /* Name.Function */\n",
       "body .nl { color: #997700; font-weight: bold } /* Name.Label */\n",
       "body .nn { color: #0e84b5; font-weight: bold } /* Name.Namespace */\n",
       "body .nt { color: #007700 } /* Name.Tag */\n",
       "body .nv { color: #996633 } /* Name.Variable */\n",
       "body .ow { color: #000000; font-weight: bold } /* Operator.Word */\n",
       "body .w { color: #bbbbbb } /* Text.Whitespace */\n",
       "body .mb { color: #6600EE; font-weight: bold } /* Literal.Number.Bin */\n",
       "body .mf { color: #6600EE; font-weight: bold } /* Literal.Number.Float */\n",
       "body .mh { color: #005588; font-weight: bold } /* Literal.Number.Hex */\n",
       "body .mi { color: #0000DD; font-weight: bold } /* Literal.Number.Integer */\n",
       "body .mo { color: #4400EE; font-weight: bold } /* Literal.Number.Oct */\n",
       "body .sa { background-color: #fff0f0 } /* Literal.String.Affix */\n",
       "body .sb { background-color: #fff0f0 } /* Literal.String.Backtick */\n",
       "body .sc { color: #0044DD } /* Literal.String.Char */\n",
       "body .dl { background-color: #fff0f0 } /* Literal.String.Delimiter */\n",
       "body .sd { color: #DD4422 } /* Literal.String.Doc */\n",
       "body .s2 { background-color: #fff0f0 } /* Literal.String.Double */\n",
       "body .se { color: #666666; font-weight: bold; background-color: #fff0f0 } /* Literal.String.Escape */\n",
       "body .sh { background-color: #fff0f0 } /* Literal.String.Heredoc */\n",
       "body .si { background-color: #eeeeee } /* Literal.String.Interpol */\n",
       "body .sx { color: #DD2200; background-color: #fff0f0 } /* Literal.String.Other */\n",
       "body .sr { color: #000000; background-color: #fff0ff } /* Literal.String.Regex */\n",
       "body .s1 { background-color: #fff0f0 } /* Literal.String.Single */\n",
       "body .ss { color: #AA6600 } /* Literal.String.Symbol */\n",
       "body .bp { color: #007020 } /* Name.Builtin.Pseudo */\n",
       "body .fm { color: #0066BB; font-weight: bold } /* Name.Function.Magic */\n",
       "body .vc { color: #336699 } /* Name.Variable.Class */\n",
       "body .vg { color: #dd7700; font-weight: bold } /* Name.Variable.Global */\n",
       "body .vi { color: #3333BB } /* Name.Variable.Instance */\n",
       "body .vm { color: #996633 } /* Name.Variable.Magic */\n",
       "body .il { color: #0000DD; font-weight: bold } /* Literal.Number.Integer.Long */\n",
       "\n",
       "  </style>\n",
       "</head>\n",
       "<body>\n",
       "<h2></h2>\n",
       "\n",
       "<div class=\"highlight\"><pre><span></span><span class=\"k\">SELECT</span>\n",
       "\t<span class=\"n\">skcc</span><span class=\"mf\">.</span><span class=\"n\">skcnamespace</span> <span class=\"k\">as</span> <span class=\"n\">schemaname</span><span class=\"p\">,</span>\n",
       "\t<span class=\"n\">skcc</span><span class=\"mf\">.</span><span class=\"n\">skcrelname</span> <span class=\"k\">as</span> <span class=\"n\">tablename</span><span class=\"p\">,</span> \n",
       "\t<span class=\"n\">skcc</span><span class=\"mf\">.</span><span class=\"n\">skccoeff</span> <span class=\"k\">as</span> <span class=\"n\">coefficient</span> \n",
       "<span class=\"k\">FROM</span>\n",
       "\t<span class=\"n\">gp_toolkit</span><span class=\"mf\">.</span><span class=\"n\">gp_skew_coefficients</span> <span class=\"n\">skcc</span> \n",
       "<span class=\"k\">WHERE</span>\n",
       "\t<span class=\"n\">skcc</span><span class=\"mf\">.</span><span class=\"n\">skcnamespace</span> <span class=\"o\">||</span> <span class=\"s1\">&#39;.&#39;</span> <span class=\"o\">||</span> <span class=\"n\">skcc</span><span class=\"mf\">.</span><span class=\"n\">skcrelname</span> <span class=\"o\">=</span> <span class=\"s1\">&#39;demo.amzn_reviews&#39;</span><span class=\"p\">;</span>\n",
       "</pre></div>\n",
       "</body>\n",
       "</html>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "ename": "DataError",
     "evalue": "(psycopg2.errors.NullValueNotAllowed) query string argument of EXECUTE is null\nCONTEXT:  PL/pgSQL function gp_toolkit.gp_skew_details(oid) line 51 at OPEN\nSQL function \"gp_skew_coefficient\" statement 1\nSQL statement \"SELECT *                     FROM\n            gp_toolkit.gp_skew_coefficient(skcoid)\"\nPL/pgSQL function gp_toolkit.__gp_skew_coefficients() line 9 at SQL statement\n\n[SQL: SELECT skcc.skcnamespace as schemaname, skcc.skcrelname as tablename, skcc.skccoeff as coefficient FROM gp_toolkit.gp_skew_coefficients skcc WHERE skcc.skcnamespace || '.' || skcc.skcrelname = 'demo.amzn_reviews' ;]\n(Background on this error at: http://sqlalche.me/e/9h9h)",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mNullValueNotAllowed\u001b[0m                       Traceback (most recent call last)",
      "\u001b[0;32m~/anaconda3/lib/python3.7/site-packages/sqlalchemy/engine/base.py\u001b[0m in \u001b[0;36m_execute_context\u001b[0;34m(self, dialect, constructor, statement, parameters, *args)\u001b[0m\n\u001b[1;32m   1245\u001b[0m                     self.dialect.do_execute(\n\u001b[0;32m-> 1246\u001b[0;31m                         \u001b[0mcursor\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mstatement\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mparameters\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mcontext\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   1247\u001b[0m                     )\n",
      "\u001b[0;32m~/anaconda3/lib/python3.7/site-packages/sqlalchemy/engine/default.py\u001b[0m in \u001b[0;36mdo_execute\u001b[0;34m(self, cursor, statement, parameters, context)\u001b[0m\n\u001b[1;32m    587\u001b[0m     \u001b[0;32mdef\u001b[0m \u001b[0mdo_execute\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mcursor\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mstatement\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mparameters\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mcontext\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mNone\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 588\u001b[0;31m         \u001b[0mcursor\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mexecute\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mstatement\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mparameters\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    589\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mNullValueNotAllowed\u001b[0m: query string argument of EXECUTE is null\nCONTEXT:  PL/pgSQL function gp_toolkit.gp_skew_details(oid) line 51 at OPEN\nSQL function \"gp_skew_coefficient\" statement 1\nSQL statement \"SELECT *                     FROM\n            gp_toolkit.gp_skew_coefficient(skcoid)\"\nPL/pgSQL function gp_toolkit.__gp_skew_coefficients() line 9 at SQL statement\n",
      "\nThe above exception was the direct cause of the following exception:\n",
      "\u001b[0;31mDataError\u001b[0m                                 Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-10-7383cd4c63cc>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[1;32m      5\u001b[0m \u001b[0mquery\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mget_ipython\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mgetoutput\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'cat script/4-6-2-data-skew.sql'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      6\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 7\u001b[0;31m \u001b[0mget_ipython\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mrun_line_magic\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'sql'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m\"$DB_USER@$DB_SERVER {''.join(query)}\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;32m~/anaconda3/lib/python3.7/site-packages/IPython/core/interactiveshell.py\u001b[0m in \u001b[0;36mrun_line_magic\u001b[0;34m(self, magic_name, line, _stack_depth)\u001b[0m\n\u001b[1;32m   2315\u001b[0m                 \u001b[0mkwargs\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m'local_ns'\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0msys\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_getframe\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mstack_depth\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mf_locals\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   2316\u001b[0m             \u001b[0;32mwith\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mbuiltin_trap\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 2317\u001b[0;31m                 \u001b[0mresult\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mfn\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   2318\u001b[0m             \u001b[0;32mreturn\u001b[0m \u001b[0mresult\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   2319\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m</root/anaconda3/lib/python3.7/site-packages/decorator.py:decorator-gen-127>\u001b[0m in \u001b[0;36mexecute\u001b[0;34m(self, line, cell, local_ns)\u001b[0m\n",
      "\u001b[0;32m~/anaconda3/lib/python3.7/site-packages/IPython/core/magic.py\u001b[0m in \u001b[0;36m<lambda>\u001b[0;34m(f, *a, **k)\u001b[0m\n\u001b[1;32m    185\u001b[0m     \u001b[0;31m# but it's overkill for just that one bit of state.\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    186\u001b[0m     \u001b[0;32mdef\u001b[0m \u001b[0mmagic_deco\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0marg\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 187\u001b[0;31m         \u001b[0mcall\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;32mlambda\u001b[0m \u001b[0mf\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m*\u001b[0m\u001b[0ma\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mk\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0mf\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0ma\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mk\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    188\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    189\u001b[0m         \u001b[0;32mif\u001b[0m \u001b[0mcallable\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0marg\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m</root/anaconda3/lib/python3.7/site-packages/decorator.py:decorator-gen-126>\u001b[0m in \u001b[0;36mexecute\u001b[0;34m(self, line, cell, local_ns)\u001b[0m\n",
      "\u001b[0;32m~/anaconda3/lib/python3.7/site-packages/IPython/core/magic.py\u001b[0m in \u001b[0;36m<lambda>\u001b[0;34m(f, *a, **k)\u001b[0m\n\u001b[1;32m    185\u001b[0m     \u001b[0;31m# but it's overkill for just that one bit of state.\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    186\u001b[0m     \u001b[0;32mdef\u001b[0m \u001b[0mmagic_deco\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0marg\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 187\u001b[0;31m         \u001b[0mcall\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;32mlambda\u001b[0m \u001b[0mf\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m*\u001b[0m\u001b[0ma\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mk\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0mf\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0ma\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mk\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    188\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    189\u001b[0m         \u001b[0;32mif\u001b[0m \u001b[0mcallable\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0marg\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/anaconda3/lib/python3.7/site-packages/sql/magic.py\u001b[0m in \u001b[0;36mexecute\u001b[0;34m(self, line, cell, local_ns)\u001b[0m\n\u001b[1;32m    215\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    216\u001b[0m         \u001b[0;32mtry\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 217\u001b[0;31m             \u001b[0mresult\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0msql\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mrun\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mrun\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mconn\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mparsed\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m\"sql\"\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0muser_ns\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    218\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    219\u001b[0m             if (\n",
      "\u001b[0;32m~/anaconda3/lib/python3.7/site-packages/sql/run.py\u001b[0m in \u001b[0;36mrun\u001b[0;34m(conn, sql, config, user_namespace)\u001b[0m\n\u001b[1;32m    365\u001b[0m             \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    366\u001b[0m                 \u001b[0mtxt\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0msqlalchemy\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msql\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mtext\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mstatement\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 367\u001b[0;31m                 \u001b[0mresult\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mconn\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msession\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mexecute\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mtxt\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0muser_namespace\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    368\u001b[0m             \u001b[0m_commit\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mconn\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mconn\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mconfig\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mconfig\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    369\u001b[0m             \u001b[0;32mif\u001b[0m \u001b[0mresult\u001b[0m \u001b[0;32mand\u001b[0m \u001b[0mconfig\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mfeedback\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/anaconda3/lib/python3.7/site-packages/sqlalchemy/engine/base.py\u001b[0m in \u001b[0;36mexecute\u001b[0;34m(self, object_, *multiparams, **params)\u001b[0m\n\u001b[1;32m    980\u001b[0m             \u001b[0;32mraise\u001b[0m \u001b[0mexc\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mObjectNotExecutableError\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mobject_\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    981\u001b[0m         \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 982\u001b[0;31m             \u001b[0;32mreturn\u001b[0m \u001b[0mmeth\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mmultiparams\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mparams\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    983\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    984\u001b[0m     \u001b[0;32mdef\u001b[0m \u001b[0m_execute_function\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mfunc\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mmultiparams\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mparams\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/anaconda3/lib/python3.7/site-packages/sqlalchemy/sql/elements.py\u001b[0m in \u001b[0;36m_execute_on_connection\u001b[0;34m(self, connection, multiparams, params)\u001b[0m\n\u001b[1;32m    291\u001b[0m     \u001b[0;32mdef\u001b[0m \u001b[0m_execute_on_connection\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mconnection\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mmultiparams\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mparams\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    292\u001b[0m         \u001b[0;32mif\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msupports_execution\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 293\u001b[0;31m             \u001b[0;32mreturn\u001b[0m \u001b[0mconnection\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_execute_clauseelement\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mmultiparams\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mparams\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    294\u001b[0m         \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    295\u001b[0m             \u001b[0;32mraise\u001b[0m \u001b[0mexc\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mObjectNotExecutableError\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/anaconda3/lib/python3.7/site-packages/sqlalchemy/engine/base.py\u001b[0m in \u001b[0;36m_execute_clauseelement\u001b[0;34m(self, elem, multiparams, params)\u001b[0m\n\u001b[1;32m   1099\u001b[0m             \u001b[0mdistilled_params\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1100\u001b[0m             \u001b[0mcompiled_sql\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1101\u001b[0;31m             \u001b[0mdistilled_params\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   1102\u001b[0m         )\n\u001b[1;32m   1103\u001b[0m         \u001b[0;32mif\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_has_events\u001b[0m \u001b[0;32mor\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mengine\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_has_events\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/anaconda3/lib/python3.7/site-packages/sqlalchemy/engine/base.py\u001b[0m in \u001b[0;36m_execute_context\u001b[0;34m(self, dialect, constructor, statement, parameters, *args)\u001b[0m\n\u001b[1;32m   1248\u001b[0m         \u001b[0;32mexcept\u001b[0m \u001b[0mBaseException\u001b[0m \u001b[0;32mas\u001b[0m \u001b[0me\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1249\u001b[0m             self._handle_dbapi_exception(\n\u001b[0;32m-> 1250\u001b[0;31m                 \u001b[0me\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mstatement\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mparameters\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mcursor\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mcontext\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   1251\u001b[0m             )\n\u001b[1;32m   1252\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/anaconda3/lib/python3.7/site-packages/sqlalchemy/engine/base.py\u001b[0m in \u001b[0;36m_handle_dbapi_exception\u001b[0;34m(self, e, statement, parameters, cursor, context)\u001b[0m\n\u001b[1;32m   1474\u001b[0m                 \u001b[0mutil\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mraise_from_cause\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mnewraise\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mexc_info\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1475\u001b[0m             \u001b[0;32melif\u001b[0m \u001b[0mshould_wrap\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1476\u001b[0;31m                 \u001b[0mutil\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mraise_from_cause\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0msqlalchemy_exception\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mexc_info\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   1477\u001b[0m             \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1478\u001b[0m                 \u001b[0mutil\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mreraise\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0mexc_info\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/anaconda3/lib/python3.7/site-packages/sqlalchemy/util/compat.py\u001b[0m in \u001b[0;36mraise_from_cause\u001b[0;34m(exception, exc_info)\u001b[0m\n\u001b[1;32m    396\u001b[0m     \u001b[0mexc_type\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mexc_value\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mexc_tb\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mexc_info\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    397\u001b[0m     \u001b[0mcause\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mexc_value\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mexc_value\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0mexception\u001b[0m \u001b[0;32melse\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 398\u001b[0;31m     \u001b[0mreraise\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mtype\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mexception\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mexception\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mtb\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mexc_tb\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mcause\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mcause\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    399\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    400\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/anaconda3/lib/python3.7/site-packages/sqlalchemy/util/compat.py\u001b[0m in \u001b[0;36mreraise\u001b[0;34m(tp, value, tb, cause)\u001b[0m\n\u001b[1;32m    150\u001b[0m             \u001b[0mvalue\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m__cause__\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mcause\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    151\u001b[0m         \u001b[0;32mif\u001b[0m \u001b[0mvalue\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m__traceback__\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0mtb\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 152\u001b[0;31m             \u001b[0;32mraise\u001b[0m \u001b[0mvalue\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mwith_traceback\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mtb\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    153\u001b[0m         \u001b[0;32mraise\u001b[0m \u001b[0mvalue\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    154\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/anaconda3/lib/python3.7/site-packages/sqlalchemy/engine/base.py\u001b[0m in \u001b[0;36m_execute_context\u001b[0;34m(self, dialect, constructor, statement, parameters, *args)\u001b[0m\n\u001b[1;32m   1244\u001b[0m                 \u001b[0;32mif\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0mevt_handled\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1245\u001b[0m                     self.dialect.do_execute(\n\u001b[0;32m-> 1246\u001b[0;31m                         \u001b[0mcursor\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mstatement\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mparameters\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mcontext\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   1247\u001b[0m                     )\n\u001b[1;32m   1248\u001b[0m         \u001b[0;32mexcept\u001b[0m \u001b[0mBaseException\u001b[0m \u001b[0;32mas\u001b[0m \u001b[0me\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/anaconda3/lib/python3.7/site-packages/sqlalchemy/engine/default.py\u001b[0m in \u001b[0;36mdo_execute\u001b[0;34m(self, cursor, statement, parameters, context)\u001b[0m\n\u001b[1;32m    586\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    587\u001b[0m     \u001b[0;32mdef\u001b[0m \u001b[0mdo_execute\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mcursor\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mstatement\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mparameters\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mcontext\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mNone\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 588\u001b[0;31m         \u001b[0mcursor\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mexecute\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mstatement\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mparameters\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    589\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    590\u001b[0m     \u001b[0;32mdef\u001b[0m \u001b[0mdo_execute_no_params\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mcursor\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mstatement\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mcontext\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mNone\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mDataError\u001b[0m: (psycopg2.errors.NullValueNotAllowed) query string argument of EXECUTE is null\nCONTEXT:  PL/pgSQL function gp_toolkit.gp_skew_details(oid) line 51 at OPEN\nSQL function \"gp_skew_coefficient\" statement 1\nSQL statement \"SELECT *                     FROM\n            gp_toolkit.gp_skew_coefficient(skcoid)\"\nPL/pgSQL function gp_toolkit.__gp_skew_coefficients() line 9 at SQL statement\n\n[SQL: SELECT skcc.skcnamespace as schemaname, skcc.skcrelname as tablename, skcc.skccoeff as coefficient FROM gp_toolkit.gp_skew_coefficients skcc WHERE skcc.skcnamespace || '.' || skcc.skcrelname = 'demo.amzn_reviews' ;]\n(Background on this error at: http://sqlalche.me/e/9h9h)"
     ]
    }
   ],
   "source": [
    "sqlfilecode = !pygmentize -f html -O full,style=colorful -l postgres script/4-6-2-data-skew.sql\n",
    "\n",
    "display_html('\\n'.join(sqlfilecode), raw=True)\n",
    "\n",
    "query = !cat script/4-6-2-data-skew.sql\n",
    "\n",
    "%sql $DB_USER@$DB_SERVER {''.join(query)}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- The `gp_toolkit.gp_skew_idle_fractions` view shows data distribution skew by calculating the percentage of the system that is idle during a table scan, which is an indicator of computational skew. The `siffraction` column shows the percentage of the system that is idle during a table scan. This is an indicator of uneven data distribution or query processing skew. For example, a value of 0.1 indicates 10% skew, a value of 0.5 indicates 50% skew, and so on. Tables that have more than 10% skew should have their distribution policies evaluated."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<!DOCTYPE html PUBLIC \"-//W3C//DTD HTML 4.01//EN\"\n",
       "   \"http://www.w3.org/TR/html4/strict.dtd\">\n",
       "<!--\n",
       "generated by Pygments <http://pygments.org>\n",
       "Copyright 2006-2019 by the Pygments team.\n",
       "Licensed under the BSD license, see LICENSE for details.\n",
       "-->\n",
       "<html>\n",
       "<head>\n",
       "  <title></title>\n",
       "  <meta http-equiv=\"content-type\" content=\"text/html; charset=UTF-8\">\n",
       "  <style type=\"text/css\">\n",
       "/*\n",
       "generated by Pygments <http://pygments.org>\n",
       "Copyright 2006-2019 by the Pygments team.\n",
       "Licensed under the BSD license, see LICENSE for details.\n",
       "*/\n",
       "td.linenos { background-color: #f0f0f0; padding-right: 10px; }\n",
       "span.lineno { background-color: #f0f0f0; padding: 0 5px 0 5px; }\n",
       "pre { line-height: 125%; }\n",
       "body .hll { background-color: #ffffcc }\n",
       "body  { background: #ffffff; }\n",
       "body .c { color: #888888 } /* Comment */\n",
       "body .err { color: #FF0000; background-color: #FFAAAA } /* Error */\n",
       "body .k { color: #008800; font-weight: bold } /* Keyword */\n",
       "body .o { color: #333333 } /* Operator */\n",
       "body .ch { color: #888888 } /* Comment.Hashbang */\n",
       "body .cm { color: #888888 } /* Comment.Multiline */\n",
       "body .cp { color: #557799 } /* Comment.Preproc */\n",
       "body .cpf { color: #888888 } /* Comment.PreprocFile */\n",
       "body .c1 { color: #888888 } /* Comment.Single */\n",
       "body .cs { color: #cc0000; font-weight: bold } /* Comment.Special */\n",
       "body .gd { color: #A00000 } /* Generic.Deleted */\n",
       "body .ge { font-style: italic } /* Generic.Emph */\n",
       "body .gr { color: #FF0000 } /* Generic.Error */\n",
       "body .gh { color: #000080; font-weight: bold } /* Generic.Heading */\n",
       "body .gi { color: #00A000 } /* Generic.Inserted */\n",
       "body .go { color: #888888 } /* Generic.Output */\n",
       "body .gp { color: #c65d09; font-weight: bold } /* Generic.Prompt */\n",
       "body .gs { font-weight: bold } /* Generic.Strong */\n",
       "body .gu { color: #800080; font-weight: bold } /* Generic.Subheading */\n",
       "body .gt { color: #0044DD } /* Generic.Traceback */\n",
       "body .kc { color: #008800; font-weight: bold } /* Keyword.Constant */\n",
       "body .kd { color: #008800; font-weight: bold } /* Keyword.Declaration */\n",
       "body .kn { color: #008800; font-weight: bold } /* Keyword.Namespace */\n",
       "body .kp { color: #003388; font-weight: bold } /* Keyword.Pseudo */\n",
       "body .kr { color: #008800; font-weight: bold } /* Keyword.Reserved */\n",
       "body .kt { color: #333399; font-weight: bold } /* Keyword.Type */\n",
       "body .m { color: #6600EE; font-weight: bold } /* Literal.Number */\n",
       "body .s { background-color: #fff0f0 } /* Literal.String */\n",
       "body .na { color: #0000CC } /* Name.Attribute */\n",
       "body .nb { color: #007020 } /* Name.Builtin */\n",
       "body .nc { color: #BB0066; font-weight: bold } /* Name.Class */\n",
       "body .no { color: #003366; font-weight: bold } /* Name.Constant */\n",
       "body .nd { color: #555555; font-weight: bold } /* Name.Decorator */\n",
       "body .ni { color: #880000; font-weight: bold } /* Name.Entity */\n",
       "body .ne { color: #FF0000; font-weight: bold } /* Name.Exception */\n",
       "body .nf { color: #0066BB; font-weight: bold } /* Name.Function */\n",
       "body .nl { color: #997700; font-weight: bold } /* Name.Label */\n",
       "body .nn { color: #0e84b5; font-weight: bold } /* Name.Namespace */\n",
       "body .nt { color: #007700 } /* Name.Tag */\n",
       "body .nv { color: #996633 } /* Name.Variable */\n",
       "body .ow { color: #000000; font-weight: bold } /* Operator.Word */\n",
       "body .w { color: #bbbbbb } /* Text.Whitespace */\n",
       "body .mb { color: #6600EE; font-weight: bold } /* Literal.Number.Bin */\n",
       "body .mf { color: #6600EE; font-weight: bold } /* Literal.Number.Float */\n",
       "body .mh { color: #005588; font-weight: bold } /* Literal.Number.Hex */\n",
       "body .mi { color: #0000DD; font-weight: bold } /* Literal.Number.Integer */\n",
       "body .mo { color: #4400EE; font-weight: bold } /* Literal.Number.Oct */\n",
       "body .sa { background-color: #fff0f0 } /* Literal.String.Affix */\n",
       "body .sb { background-color: #fff0f0 } /* Literal.String.Backtick */\n",
       "body .sc { color: #0044DD } /* Literal.String.Char */\n",
       "body .dl { background-color: #fff0f0 } /* Literal.String.Delimiter */\n",
       "body .sd { color: #DD4422 } /* Literal.String.Doc */\n",
       "body .s2 { background-color: #fff0f0 } /* Literal.String.Double */\n",
       "body .se { color: #666666; font-weight: bold; background-color: #fff0f0 } /* Literal.String.Escape */\n",
       "body .sh { background-color: #fff0f0 } /* Literal.String.Heredoc */\n",
       "body .si { background-color: #eeeeee } /* Literal.String.Interpol */\n",
       "body .sx { color: #DD2200; background-color: #fff0f0 } /* Literal.String.Other */\n",
       "body .sr { color: #000000; background-color: #fff0ff } /* Literal.String.Regex */\n",
       "body .s1 { background-color: #fff0f0 } /* Literal.String.Single */\n",
       "body .ss { color: #AA6600 } /* Literal.String.Symbol */\n",
       "body .bp { color: #007020 } /* Name.Builtin.Pseudo */\n",
       "body .fm { color: #0066BB; font-weight: bold } /* Name.Function.Magic */\n",
       "body .vc { color: #336699 } /* Name.Variable.Class */\n",
       "body .vg { color: #dd7700; font-weight: bold } /* Name.Variable.Global */\n",
       "body .vi { color: #3333BB } /* Name.Variable.Instance */\n",
       "body .vm { color: #996633 } /* Name.Variable.Magic */\n",
       "body .il { color: #0000DD; font-weight: bold } /* Literal.Number.Integer.Long */\n",
       "\n",
       "  </style>\n",
       "</head>\n",
       "<body>\n",
       "<h2></h2>\n",
       "\n",
       "<div class=\"highlight\"><pre><span></span><span class=\"k\">SELECT</span>\n",
       "\t<span class=\"n\">sif</span><span class=\"mf\">.</span><span class=\"n\">sifnamespace</span> <span class=\"k\">as</span> <span class=\"n\">schemaname</span><span class=\"p\">,</span>\n",
       "\t<span class=\"n\">sif</span><span class=\"mf\">.</span><span class=\"n\">sifrelname</span> <span class=\"k\">as</span> <span class=\"n\">tablename</span><span class=\"p\">,</span> \n",
       "\t<span class=\"n\">sif</span><span class=\"mf\">.</span><span class=\"n\">siffraction</span> <span class=\"k\">as</span> <span class=\"n\">fraction</span> \n",
       "<span class=\"k\">FROM</span>\n",
       "\t<span class=\"n\">gp_toolkit</span><span class=\"mf\">.</span><span class=\"n\">gp_skew_idle_fractions</span> <span class=\"n\">sif</span> \n",
       "<span class=\"k\">WHERE</span>\n",
       "\t<span class=\"n\">sif</span><span class=\"mf\">.</span><span class=\"n\">sifnamespace</span> <span class=\"o\">||</span> <span class=\"s1\">&#39;.&#39;</span> <span class=\"o\">||</span> <span class=\"n\">sif</span><span class=\"mf\">.</span><span class=\"n\">sifrelname</span> <span class=\"o\">=</span> <span class=\"s1\">&#39;demo.amzn_reviews&#39;</span>\n",
       "</pre></div>\n",
       "</body>\n",
       "</html>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1 rows affected.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<table>\n",
       "    <tr>\n",
       "        <th>schemaname</th>\n",
       "        <th>tablename</th>\n",
       "        <th>fraction</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "        <td>demo</td>\n",
       "        <td>amzn_reviews</td>\n",
       "        <td>0</td>\n",
       "    </tr>\n",
       "</table>"
      ],
      "text/plain": [
       "[('demo', 'amzn_reviews', Decimal('0'))]"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sqlfilecode = !pygmentize -f html -O full,style=colorful -l postgres script/4-6-3-data-skew.sql\n",
    "\n",
    "display_html('\\n'.join(sqlfilecode), raw=True)\n",
    "\n",
    "query = !cat script/4-6-3-data-skew.sql\n",
    "\n",
    "%sql $DB_USER@$DB_SERVER {''.join(query)}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Continue to Part 3 of Greenplum Database  Concepts Explained; **[MPP Fundamentals and Partitioning](AWS-GP-demo-3.ipynb)**."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "celltoolbar": "Raw Cell Format",
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}