convokit/expected_context_framework/demos/scotus_orientation_demo.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Examining orientation of justice utterances in US Supreme Court oral arguments\n",
    "\n",
    "This notebook illustrates how the Expected Context Framework can be used to derive a property of utterances and terms, orientation, detailed in [this paper](https://www.cs.cornell.edu/~cristian/Orientation_files/orientation-forwards-backwards.pdf) and [this dissertation](https://tisjune.github.io/research/dissertation). Orientation quantifies the extent to which a term or utterance aims at advancing a conversation forwards or addressing backwards. We originally used it to analyze counselor utterances in crisis counseling conversations.\n",
    "Here, we demonstrate how orientation can be computed on a public dataset, transcripts of oral arguments from the US Supreme Court; for this notebook, we focus on characterizing utterances from justices. See [this dissertation](https://tisjune.github.io/research/dissertation) for more comments on the below analyses.\n",
    "\n",
    "We can draw a loose parallel between the oral argument and counseling settings, in that both involve _asymmetric_ conversations, taking part between interlocutors playing different roles (justice vs lawyer, counselor vs individual seeking help). Beyond this parallel, there are some notable differences which perhaps lead to less interpretable output in this setting: justices and counselors have very different goals, the language used in this setting is less structured than that used in the counseling setting, utterances here are much more varied in length. We encourage future work to tinker with the approach presented here, and to consider other ways to examine interactional dynamics."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import warnings\n",
    "warnings.filterwarnings('ignore')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "import math\n",
    "import os"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. Loading and preprocessing the dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from convokit import Corpus\n",
    "from convokit import download"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# OPTION 1: DOWNLOAD CORPUS \n",
    "# UNCOMMENT THESE LINES TO DOWNLOAD CORPUS\n",
    "# DATA_DIR = '<YOUR DIRECTORY>'\n",
    "# SCOTUS_CORPUS_PATH = download('supreme-corpus', data_dir=DATA_DIR)\n",
    "\n",
    "# OPTION 2: READ PREVIOUSLY-DOWNLOADED CORPUS FROM DISK\n",
    "# UNCOMMENT THIS LINE AND REPLACE WITH THE DIRECTORY WHERE THE TENNIS-CORPUS IS LOCATED\n",
    "# SCOTUS_CORPUS_PATH = '<YOUR DIRECTORY>'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "scotus_corpus = Corpus(SCOTUS_CORPUS_PATH)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of Speakers: 8979\n",
      "Number of Utterances: 1700789\n",
      "Number of Conversations: 7817\n"
     ]
    }
   ],
   "source": [
    "scotus_corpus.print_summary_stats()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We represent justice and lawyer utterances as dependency-parse arcs, which we load as preprocessed features (which we include with the data release). We also load tokenized versions of these utterances to facilitate taking word-counts."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "scotus_corpus.load_info('utterance',['arcs','tokens'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will restrict the training data we use to sufficiently long utterances; to facilitate this, we compute the number of words in each utterance:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from convokit.text_processing import TextProcessor\n",
    "wordcounter = TextProcessor(input_field='tokens', output_field='wordcount',\n",
    "           proc_fn=lambda x: len(x.split()))\n",
    "scotus_corpus = wordcounter.transform(scotus_corpus)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In applying the framework, we need to associate utterances with their replies, so we store the ID of the reply to an utterance in its metadata:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "for ut in scotus_corpus.iter_utterances(selector=lambda x: x.reply_to is not None):\n",
    "    scotus_corpus.get_utterance(ut.reply_to).meta['next_id'] = ut.id"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Constructing the training data\n",
    "\n",
    "As training data, we consider a subset of justice and lawyer utterances (labeled in the utterance metadata as `J` and `A` respectively). We ultimately wish to characterize justice utterances, using the previous and subsequent lawyer utterances as conversational context.\n",
    "\n",
    "We make the following filtering decisions:\n",
    "* We only consider lawyer utterances between 10 and 75 words long;\n",
    "* We only consider justice utterances between 10 and 50 words long, and that occurred between sufficiently long lawyer utterances."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "min_wc = 10\n",
    "max_wc = 50\n",
    "min_wc_context = 10\n",
    "max_wc_context = 75"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "for ut in scotus_corpus.iter_utterances():\n",
    "    ut.meta['is_valid_context'] = (ut.meta['speaker_type'] == 'A')\\\n",
    "        and (ut.meta['arcs'] != '')\\\n",
    "        and (ut.meta['wordcount'] >= min_wc_context)\\\n",
    "        and (ut.meta['wordcount'] <= max_wc_context)        \n",
    "for ut in scotus_corpus.iter_utterances():\n",
    "    if ('next_id' not in ut.meta) or (ut.reply_to is None): \n",
    "        ut.meta['is_valid_utt'] = False\n",
    "    else:\n",
    "        ut.meta['is_valid_utt'] = (ut.meta['speaker_type'] == 'J')\\\n",
    "            and (ut.meta['arcs'] != '')\\\n",
    "            and (ut.meta['wordcount'] >= min_wc)\\\n",
    "            and (ut.meta['wordcount'] <= max_wc)\\\n",
    "            and scotus_corpus.get_utterance(ut.meta['next_id']).meta['is_valid_context']\\\n",
    "            and scotus_corpus.get_utterance(ut.reply_to).meta['is_valid_context']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This results in ~90,000 utterances and ~370,000 context utterances:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "91924"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sum(ut.meta['is_valid_utt'] for ut in scotus_corpus.iter_utterances())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "372268"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sum(ut.meta['is_valid_context'] for ut in scotus_corpus.iter_utterances())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. Applying the Expected Context Framework"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from convokit.expected_context_framework import ColNormedTfidfTransformer, DualContextWrapper"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To apply the Expected Context Framework, we start by converting the input utterance text to an input vector representation. Here, we represent utterances in a term-document matrix that's normalized by columns (empirically, we found that this ensures that the representations derived by the framework aren't skewed by the relative frequency of utterances). We use the `ColNormedTfidfTransformer` transformer to do this.\n",
    "\n",
    "We derive different tf-idf representations (with different vocabularies and other parameters) for justice and lawyer utterances, reflecting their different roles (and hence differences in their language use)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "j_tfidf_obj = ColNormedTfidfTransformer(input_field='arcs', output_field='j_tfidf', binary=True, \n",
    "                                   min_df=250, max_df=1., max_features=2000)\n",
    "_ = j_tfidf_obj.fit(scotus_corpus, selector=lambda x: x.meta['is_valid_utt'])\n",
    "_ = j_tfidf_obj.transform(scotus_corpus, selector=lambda x: x.meta['is_valid_utt'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "a_tfidf_obj = ColNormedTfidfTransformer(input_field='arcs', output_field='a_tfidf', binary=True, \n",
    "                                   min_df=250, max_df=1., max_features=2000)\n",
    "_ = a_tfidf_obj.fit(scotus_corpus, selector=lambda x: x.meta['is_valid_context'])\n",
    "_ = a_tfidf_obj.transform(scotus_corpus, selector=lambda x: x.meta['is_valid_context'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To compute orientation, we compare characterizations of utterances with respect to their forwards context (replies) with characterizations with respect to their backwards context (predecessors). As such, we initialize  two Expected Context Models, one that relates utterances to replies and one that relates utterances to predecessors. We ensure that the forwards and backwards characterizations are comparable by initializing the second model with the first.\n",
    "\n",
    "To take care of both interlocked Expected Context Models, we use the `DualContextWrapper` transformer, which will keep track of two `ExpectedContextModelTransformer`s: one that relates utterances to predecessors (`reply_to`), and that outputs utterance-level attributes with the prefix `bk`; the other that relates utterances to replies (`next_id`) and outputs utterance-level attributes with the prefix `fw`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "dual_context_model = DualContextWrapper(context_fields=['reply_to','next_id'], output_prefixes=['bk','fw'],\n",
    "                                    vect_field='j_tfidf', context_vect_field='a_tfidf', \n",
    "                                      n_svd_dims=15,\n",
    "                                     random_state=1000)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "dual_context_model.fit(scotus_corpus, selector=lambda x: x.meta['is_valid_utt'],\n",
    "         context_selector=lambda x: x.meta['is_valid_context'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Term-level orientation\n",
    "\n",
    "We start by examining the term-level orientation statistics we've computed. For convenience the `DualContextWrapper` outputs all term-level statistics as a dataframe:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "term_df = dual_context_model.get_term_df()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We see that most terms have positive orientation. In the counseling setting, most terms have negative orientation. This difference might reflect the differing goals of counselors and justices: counselors are trained to focus heavily on empathetically addressing what an individual has said; justices are tasked with scrutinizing the arguments made by lawyers. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       " 1.0    0.700806\n",
       "-1.0    0.299194\n",
       "Name: orn, dtype: float64"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.sign(term_df.orn).value_counts(normalize=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Among high-orientation terms, we see those reflecting justices pressing the lawyers to address a point in a particular way (e.g., [is there any] difference). The low-orientation terms are somewhat harder to interpret; we note that the idea of being \"backwards-oriented\" might be somewhat ill-defined in this particular setting."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "high orientation\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>orn</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>index</th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>raised_*</th>\n",
       "      <td>0.083416</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>talking_are</th>\n",
       "      <td>0.085496</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>do&gt;*</th>\n",
       "      <td>0.086808</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>said_if</th>\n",
       "      <td>0.087252</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>agree_do</th>\n",
       "      <td>0.090253</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>is_the</th>\n",
       "      <td>0.090826</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>of_appeals</th>\n",
       "      <td>0.093167</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>brief_*</th>\n",
       "      <td>0.094093</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>suppose&gt;*</th>\n",
       "      <td>0.094863</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>is_where</th>\n",
       "      <td>0.099781</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>be_you</th>\n",
       "      <td>0.100124</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>is_issue</th>\n",
       "      <td>0.102609</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>is_really</th>\n",
       "      <td>0.106802</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>difference_any</th>\n",
       "      <td>0.107541</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>be_what</th>\n",
       "      <td>0.111437</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>make_would</th>\n",
       "      <td>0.111665</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>is_anything</th>\n",
       "      <td>0.112602</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>hear_*</th>\n",
       "      <td>0.113597</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>brief_your</th>\n",
       "      <td>0.134689</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>in_brief</th>\n",
       "      <td>0.148016</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                     orn\n",
       "index                   \n",
       "raised_*        0.083416\n",
       "talking_are     0.085496\n",
       "do>*            0.086808\n",
       "said_if         0.087252\n",
       "agree_do        0.090253\n",
       "is_the          0.090826\n",
       "of_appeals      0.093167\n",
       "brief_*         0.094093\n",
       "suppose>*       0.094863\n",
       "is_where        0.099781\n",
       "be_you          0.100124\n",
       "is_issue        0.102609\n",
       "is_really       0.106802\n",
       "difference_any  0.107541\n",
       "be_what         0.111437\n",
       "make_would      0.111665\n",
       "is_anything     0.112602\n",
       "hear_*          0.113597\n",
       "brief_your      0.134689\n",
       "in_brief        0.148016"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "low orientation\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>orn</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>index</th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>in_order</th>\n",
       "      <td>-0.068865</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>specific_*</th>\n",
       "      <td>-0.059897</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>and&gt;it</th>\n",
       "      <td>-0.058361</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>to_it</th>\n",
       "      <td>-0.056576</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>laughter_*</th>\n",
       "      <td>-0.055687</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>which&gt;*</th>\n",
       "      <td>-0.053849</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>is_for</th>\n",
       "      <td>-0.052996</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>is_which</th>\n",
       "      <td>-0.052908</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>commission_*</th>\n",
       "      <td>-0.052511</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>habeas_*</th>\n",
       "      <td>-0.052096</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>and&gt;you</th>\n",
       "      <td>-0.050960</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>available_*</th>\n",
       "      <td>-0.050422</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>claim_the</th>\n",
       "      <td>-0.049388</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>five_*</th>\n",
       "      <td>-0.049372</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>support_*</th>\n",
       "      <td>-0.048803</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>side_*</th>\n",
       "      <td>-0.047817</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>have_so</th>\n",
       "      <td>-0.047568</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>earlier_*</th>\n",
       "      <td>-0.047121</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>commission_the</th>\n",
       "      <td>-0.046870</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>find_to</th>\n",
       "      <td>-0.046081</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                     orn\n",
       "index                   \n",
       "in_order       -0.068865\n",
       "specific_*     -0.059897\n",
       "and>it         -0.058361\n",
       "to_it          -0.056576\n",
       "laughter_*     -0.055687\n",
       "which>*        -0.053849\n",
       "is_for         -0.052996\n",
       "is_which       -0.052908\n",
       "commission_*   -0.052511\n",
       "habeas_*       -0.052096\n",
       "and>you        -0.050960\n",
       "available_*    -0.050422\n",
       "claim_the      -0.049388\n",
       "five_*         -0.049372\n",
       "support_*      -0.048803\n",
       "side_*         -0.047817\n",
       "have_so        -0.047568\n",
       "earlier_*      -0.047121\n",
       "commission_the -0.046870\n",
       "find_to        -0.046081"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "print('\\nhigh orientation')\n",
    "display(term_df.sort_values('orn')[['orn']].tail(20))\n",
    "print('low orientation')\n",
    "display(term_df.sort_values('orn')[['orn']].head(20))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Sentence-level orientation\n",
    "\n",
    "We derive the same statistic at the level of _sentences_ comprising justice utterances (we could do this for utterances, but found sentences to be more interpretable). To start, we create a new corpus consisting of these sentences:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from convokit import Utterance"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "sentence_utts = []\n",
    "for ut in scotus_corpus.iter_utterances(selector=lambda x: x.meta['is_valid_utt']):\n",
    "    sents = ut.meta['arcs'].split('\\n')\n",
    "    tok_sents = ut.meta['tokens'].split('\\n')\n",
    "    for i, (sent, tok_sent) in enumerate(zip(sents, tok_sents)):\n",
    "        utt_id = ut.id + '_' + '%03d' % i\n",
    "        speaker = ut.speaker\n",
    "        text = tok_sent\n",
    "        meta = {'arcs': sent, 'utt_id': ut.id, 'speaker': ut.speaker.id}\n",
    "        sentence_utts.append(Utterance(\n",
    "                    id=utt_id, speaker=speaker, text=text,\n",
    "                    reply_to=ut.reply_to, conversation_id=ut.conversation_id,\n",
    "                    meta=meta\n",
    "                ))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "sentence_corpus = Corpus(utterances=sentence_utts)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of Speakers: 35\n",
      "Number of Utterances: 140274\n",
      "Number of Conversations: 7272\n"
     ]
    }
   ],
   "source": [
    "sentence_corpus.print_summary_stats()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "By applying the transformer, we annotate each sentence with its orientation:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "_ = j_tfidf_obj.transform(sentence_corpus)\n",
    "_ = dual_context_model.transform(sentence_corpus, selector=lambda x: x.meta['j_tfidf__n_feats'] >= 1)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "David H. Souter : Well , do you agree with me that if you do n't go to legislative intent , we have , on your reading , what may be a grammatical reading , but a very foolish statute ?\n"
     ]
    }
   ],
   "source": [
    "ut_eg_id = '20030__1_029_000'\n",
    "eg_ut = sentence_corpus.get_utterance(ut_eg_id)\n",
    "print(eg_ut.speaker.meta['name'], ':',eg_ut.text)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.02958196774118771"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "eg_ut.meta['orn']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For convenience, to inspect sentences with low or high orientation, we will load these statistics into a Pandas dataframe:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "sent_df = sentence_corpus.get_attribute_table('utterance',['orn','j_tfidf__n_feats'])\n",
    "text_df = pd.DataFrame([{'id': ut._id, 'text': ut.text, 'speaker': ut.speaker.meta['name']}\n",
    "    for ut in sentence_corpus.iter_utterances()\n",
    "]).set_index('id')\n",
    "sent_df = sent_df.join(text_df)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As with terms, the majority of sentences have positive orientation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       " 1.0    0.87056\n",
       "-1.0    0.12944\n",
       "Name: orn, dtype: float64"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.sign(sent_df.orn).value_counts(normalize=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "low_subset = sent_df[(sent_df.j_tfidf__n_feats >= 30)\n",
    "                    & (sent_df.orn < sent_df.orn.quantile(.1))].sample(10,random_state=9)\n",
    "high_subset = sent_df[(sent_df.j_tfidf__n_feats >= 30)\n",
    "                    & (sent_df.orn > sent_df.orn.quantile(.9))].sample(10,random_state=9)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Below, we print out a random sample of sentences with high and low utterances (next two cells). We note that the interpretation problems we noted at the term level remain here, and encourage future work to consider variants or alternatives of the orientation statistic on this dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "18643__2_007_000 Byron R. White orientation: 0.033653619864415174\n",
      "> Do you agree that the court below , that the Supreme Judicial Court did not hold that the state courts were required to give review of a prison disciplinary decision ?\n",
      "\n",
      "15090__1_108_000 Byron R. White orientation: 0.03345959508409968\n",
      "> Do you still would be open to the Ohio Courts to say notwithstanding the judgment turning him over to the adult Courts has been reversed by the Supreme Court of the United States .\n",
      "\n",
      "17696__2_125_000 Byron R. White orientation: 0.03340985781504202\n",
      "> Do you agree with the statement in the Court of Appeals , we hold a comprehensive major Federal action is contemplated in the Northern Great Plains and therefore , what , do you think that is enough that   some of the major Federal action is contemplated ?\n",
      "\n",
      "20872__0_022_001 Antonin Scalia orientation: 0.032146254486083214\n",
      "> In other words , did the intermediate court have any fact - finding to do which would not have been reconsidered by the Court of Military Appeals ?\n",
      "\n",
      "17783__1_045_000 Potter Stewart orientation: 0.03155276985903288\n",
      "> Well , let 's assume of a hypothetical case that all of that was ( Inaudible ) by the Supreme Court of California and that\n",
      "\n",
      "16313__4_037_000 Thurgood Marshall orientation: 0.0314680724015467\n",
      "> Well , is n't there a difference between when you passed this bill or do you admit that this bill was passed for the expressed purpose of upsetting the judgment of the District Court ?\n",
      "\n",
      "18694__1_021_000 John Paul Stevens orientation: 0.03139723796438565\n",
      "> But just to the point of whether there is anything on which they could base concern about a single theatre , at least that would be some evidence , would n't it ?\n",
      "\n",
      "15444__1_123_000 Byron R. White orientation: 0.03033936352457156\n",
      "> That 's for a different reason , but you think any criminal case in the federal court would come down in putting any difference on what the charge was , would n't you ?\n",
      "\n",
      "20030__1_029_000 David H. Souter orientation: 0.02958196774118771\n",
      "> Well , do you agree with me that if you do n't go to legislative intent , we have , on your reading , what may be a grammatical reading , but a very foolish statute ?\n",
      "\n",
      "19922__0_083_001 William H. Rehnquist orientation: 0.029011144478165662\n",
      "> Then it seems to me you are in conflict with our Martin Eby case which said that the ... the court of appeals had discretion .\n",
      "\n"
     ]
    }
   ],
   "source": [
    "for id, row in high_subset.sort_values('orn', ascending=False).iterrows():\n",
    "    print(id,row.speaker, 'orientation:',row.orn)\n",
    "    print('>', row.text)\n",
    "    print()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "16516__5_057_000 Warren E. Burger orientation: -0.0062158327698261795\n",
      "> Are you free when I put that limitation on it , are you free to offer a hypothesis as to why the Government of Cuba as not made any formal claim of act of state , but has simply depended upon a litigation position asserted by you ?\n",
      "\n",
      "13496__1_095_000 Earl Warren orientation: -0.005610162708263755\n",
      "> But what line of cases do you rely on to substantiate that point that you can sue only the enforcing agency in order to establish the unconstitutionality of the act when they have done nothing to go beyond a fair interpretation of the act ?\n",
      "\n",
      "22164__0_020_000 Anthony M. Kennedy orientation: -0.005592280465212318\n",
      "> Well then , I do n't know what effect you 're giving to the fact , as the earlier questions have indicated , that there is a structural conflict .\n",
      "\n",
      "15045__0_082_000 Earl Warren orientation: -0.005540441685332631\n",
      "> Well , why do you have to go from one border of your state way over to the middle of the state in that circuitous way in order to carve out a district ?\n",
      "\n",
      "14723__1_127_000 Felix Frankfurter orientation: -0.004934628424507426\n",
      "> What I want to know is whether by exercising the part that you attribute to the Secretary , he could subject Lehigh to the same economic burden of which it is now complained by merely extending the area .\n",
      "\n",
      "18954__0_081_000 Anthony M. Kennedy orientation: -0.004834921356991262\n",
      "> It does n't bother you that all of this inquiry is post hoc , consisting of facts that the police will never know one way or the other until after the arrest has been made ?\n",
      "\n",
      "14360__2_195_000 Felix Frankfurter orientation: -0.003264386439445599\n",
      "> But I -- but the portion of what he just read from Judge Smith 's opinion says that that , the Commission paid exclusive attention , getting the exact words , to one factor , not in fact so that we do not have to have pre - arguments .\n",
      "\n",
      "16732__1_037_000 Potter Stewart orientation: -0.002835759865154941\n",
      "> But his claim is that and was never been proven one way or the other because there has n’t been any hearing is that he was getting a good deal of advise to use euphemism from the policemen in the --\n",
      "\n",
      "16326__1_051_000 John Paul Stevens orientation: -0.002832955231349077\n",
      "> They would leverage the president of the company , so you terminate this service you are going to be get sued for three million dollars or something like that , might not have got an action ?\n",
      "\n",
      "19480__1_101_000 Sandra Day O'Connor orientation: -0.0025956561468857275\n",
      "> Well , I 'm not sure that 's accurate because apparently the Solicitor General 's speaking for the Secretary would say that states can not direct specific benefits to be provided if they are the type of benefits covered by ERISA .\n",
      "\n"
     ]
    }
   ],
   "source": [
    "for id, row in low_subset.sort_values('orn').iterrows():\n",
    "    print(id,row.speaker, 'orientation:',row.orn)\n",
    "    print('>', row.text)\n",
    "    print()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. Pipeline usage\n",
    "\n",
    "We can also apply the framework via a pipeline that handles the following:\n",
    "* processes text (via a pipeline supplied by the user; see cell below)\n",
    "* transforms text to input representation (via `ColNormedTfidfTransformer`)\n",
    "* derives framework output (via `ExpectedContextModelTransformer`)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from convokit.expected_context_framework import DualContextPipeline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# see `demo_text_pipelines.py` in this demo's directory for details\n",
    "# in short, this pipeline will compute the dependency-parse arcs we use as input features,\n",
    "# but will skip over utterances for which these attributes already exist\n",
    "from demo_text_pipelines import scotus_arc_pipeline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We initialize the pipeline with the following arguments:\n",
    "* `text_field` specifies which utterance metadata field to use as text input\n",
    "* `text_pipe` specifies the pipeline used to compute the contents of `text_field`\n",
    "* `tfidf_params` specifies the parameters to be passed into the underlying `ColNormedTfidfTransformer` object\n",
    "\n",
    "All other arguments are inherited from `DualContextWrapper`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "dual_pipe = DualContextPipeline(context_fields=['reply_to','next_id'], \n",
    "                output_prefixes=['bk','fw'], share_tfidf_models=False,\n",
    "                 text_field='arcs', text_pipe=scotus_arc_pipeline(), \n",
    "                tfidf_params={'binary': True, 'min_df': 250, 'max_features': 2000}, \n",
    "                n_svd_dims=15, random_state=1000)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "dual_pipe.fit(scotus_corpus,\n",
    "             selector=lambda x: x.meta['is_valid_utt'],\n",
    "         context_selector=lambda x: x.meta['is_valid_context'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This should produce the same output as calling the constituent steps separately."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "term_df_new = dual_pipe.get_term_df()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "high orientation\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>orn</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>index</th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>raised_*</th>\n",
       "      <td>0.083416</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>talking_are</th>\n",
       "      <td>0.085496</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>do&gt;*</th>\n",
       "      <td>0.086808</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>said_if</th>\n",
       "      <td>0.087252</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>agree_do</th>\n",
       "      <td>0.090253</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>is_the</th>\n",
       "      <td>0.090826</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>of_appeals</th>\n",
       "      <td>0.093167</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>brief_*</th>\n",
       "      <td>0.094093</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>suppose&gt;*</th>\n",
       "      <td>0.094863</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>is_where</th>\n",
       "      <td>0.099781</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>be_you</th>\n",
       "      <td>0.100124</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>is_issue</th>\n",
       "      <td>0.102609</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>is_really</th>\n",
       "      <td>0.106802</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>difference_any</th>\n",
       "      <td>0.107541</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>be_what</th>\n",
       "      <td>0.111437</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>make_would</th>\n",
       "      <td>0.111665</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>is_anything</th>\n",
       "      <td>0.112602</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>hear_*</th>\n",
       "      <td>0.113597</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>brief_your</th>\n",
       "      <td>0.134689</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>in_brief</th>\n",
       "      <td>0.148016</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                     orn\n",
       "index                   \n",
       "raised_*        0.083416\n",
       "talking_are     0.085496\n",
       "do>*            0.086808\n",
       "said_if         0.087252\n",
       "agree_do        0.090253\n",
       "is_the          0.090826\n",
       "of_appeals      0.093167\n",
       "brief_*         0.094093\n",
       "suppose>*       0.094863\n",
       "is_where        0.099781\n",
       "be_you          0.100124\n",
       "is_issue        0.102609\n",
       "is_really       0.106802\n",
       "difference_any  0.107541\n",
       "be_what         0.111437\n",
       "make_would      0.111665\n",
       "is_anything     0.112602\n",
       "hear_*          0.113597\n",
       "brief_your      0.134689\n",
       "in_brief        0.148016"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "low orientation\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>orn</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>index</th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>in_order</th>\n",
       "      <td>-0.068865</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>specific_*</th>\n",
       "      <td>-0.059897</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>and&gt;it</th>\n",
       "      <td>-0.058361</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>to_it</th>\n",
       "      <td>-0.056576</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>laughter_*</th>\n",
       "      <td>-0.055687</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>which&gt;*</th>\n",
       "      <td>-0.053849</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>is_for</th>\n",
       "      <td>-0.052996</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>is_which</th>\n",
       "      <td>-0.052908</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>commission_*</th>\n",
       "      <td>-0.052511</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>habeas_*</th>\n",
       "      <td>-0.052096</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>and&gt;you</th>\n",
       "      <td>-0.050960</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>available_*</th>\n",
       "      <td>-0.050422</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>claim_the</th>\n",
       "      <td>-0.049388</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>five_*</th>\n",
       "      <td>-0.049372</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>support_*</th>\n",
       "      <td>-0.048803</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>side_*</th>\n",
       "      <td>-0.047817</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>have_so</th>\n",
       "      <td>-0.047568</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>earlier_*</th>\n",
       "      <td>-0.047121</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>commission_the</th>\n",
       "      <td>-0.046870</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>find_to</th>\n",
       "      <td>-0.046081</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                     orn\n",
       "index                   \n",
       "in_order       -0.068865\n",
       "specific_*     -0.059897\n",
       "and>it         -0.058361\n",
       "to_it          -0.056576\n",
       "laughter_*     -0.055687\n",
       "which>*        -0.053849\n",
       "is_for         -0.052996\n",
       "is_which       -0.052908\n",
       "commission_*   -0.052511\n",
       "habeas_*       -0.052096\n",
       "and>you        -0.050960\n",
       "available_*    -0.050422\n",
       "claim_the      -0.049388\n",
       "five_*         -0.049372\n",
       "support_*      -0.048803\n",
       "side_*         -0.047817\n",
       "have_so        -0.047568\n",
       "earlier_*      -0.047121\n",
       "commission_the -0.046870\n",
       "find_to        -0.046081"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "print('\\nhigh orientation')\n",
    "display(term_df_new.sort_values('orn')[['orn']].tail(20))\n",
    "print('low orientation')\n",
    "display(term_df_new.sort_values('orn')[['orn']].head(20))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that the pipeline enables us to transform ad-hoc string input:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "eg_ut_new = dual_pipe.transform_utterance('What is the difference between these statutes?')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "orientation: 0.03682089802385846\n"
     ]
    }
   ],
   "source": [
    "print('orientation:', eg_ut_new.meta['orn'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}