notes_sLDA.ipynb

{
 "metadata": {
  "name": "notes_sLDA"
 }, 
 "nbformat": 2, 
 "worksheets": [
  {
   "cells": [
    {
     "cell_type": "markdown", 
     "source": [
      "# Running sLDA"
     ]
    }, 
    {
     "cell_type": "markdown", 
     "source": [
      "We are running an implementation of sLDA from Wang, which predicts categorical labels (consistent with our tasks so far).", 
      "", 
      "http://www.cs.cmu.edu/~chongw/slda/", 
      "", 
      "Another implementation is here but it handles continuous valued labels only?", 
      "", 
      "http://cran.r-project.org/web/packages/lda/lda.pdf", 
      "", 
      "General LDA info:", 
      "", 
      "http://www.cs.princeton.edu/~blei/topicmodeling.html"
     ]
    }, 
    {
     "cell_type": "markdown", 
     "source": [
      "The executable (after compiling according to instructions)."
     ]
    }, 
    {
     "cell_type": "code", 
     "collapsed": false, 
     "input": [
      "!slda"
     ], 
     "language": "python", 
     "outputs": [
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "usage: slda [est] [data] [label] [settings] [alpha] [k] [random/seeded/model_path] [directory]", 
        "       slda [inf] [data] [label] [settings] [model] [directory]"
       ]
      }
     ], 
     "prompt_number": 1
    }, 
    {
     "cell_type": "markdown", 
     "source": [
      "## Input data"
     ]
    }, 
    {
     "cell_type": "markdown", 
     "source": [
      "From the README:", 
      "", 
      "[data] is a file where each line is of the form:", 
      "", 
      "     [M] [term_1]:[count] [term_2]:[count] ...  [term_N]:[count]", 
      "", 
      "where [M] is the number of unique terms in the document, and the [count] associated with each term is how many times that term appeared in the document."
     ]
    }, 
    {
     "cell_type": "markdown", 
     "source": [
      "A provided sample training data file.  "
     ]
    }, 
    {
     "cell_type": "code", 
     "collapsed": false, 
     "input": [
      "!head -n 2 Wang/sample_data/Labelme/images/train-data.dat"
     ], 
     "language": "python", 
     "outputs": [
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "147 0:3 1:4 2:1 3:8 4:7 5:4 6:34 7:9 8:8 9:17 10:220 11:11 12:7 13:1 14:2 15:52 16:12 17:5 19:2 20:1 21:59 22:21 23:8 24:3 25:4 26:7 27:7 28:39 29:2 30:20 32:1 33:21 34:17 35:6 36:4 37:1 38:11 40:81 41:1 42:3 43:1 44:5 45:1 46:17 47:44 48:11 49:57 50:36 52:11 53:6 54:43 55:3 56:23 57:3 58:9 59:14 60:47 61:11 62:14 63:53 64:16 66:8 69:3 70:2 71:31 72:20 73:3 74:2 75:80 76:6 77:4 78:3 79:7 80:7 81:24 82:4 83:7 84:1 85:6 86:1 87:1 88:10 89:2 90:13 91:8 92:9 93:4 94:11 95:47 97:9 98:7 99:5 100:1 101:2 102:27 104:4 105:8 106:6 107:15 108:1 109:3 110:4 111:35 112:8 113:64 114:8 115:41 116:1 117:9 118:20 119:197 120:2 121:13 122:3 123:5 125:26 126:14 127:10 128:2 129:10 130:16 131:23 132:6 134:4 135:15 136:16 137:34 138:5 139:3 140:9 141:8 142:1 143:2 144:4 145:4 146:6 147:4 148:83 149:9 150:15 151:24 152:18 153:4 154:1 155:9 156:13 157:32", 
        "", 
        "157 0:6 1:6 2:3 3:7 4:7 5:9 6:39 7:12 8:17 9:29 10:12 11:26 12:1 13:7 14:14 15:32 16:9 17:3 18:3 19:25 20:5 21:23 22:11 23:7 24:10 25:10 26:4 27:20 28:31 29:11 30:24 31:10 32:4 33:20 34:29 35:6 36:15 37:7 38:7 39:1 40:20 41:21 42:4 43:8 44:7 45:12 46:7 47:41 48:8 49:35 50:6 51:1 52:13 53:16 54:40 55:6 56:47 57:6 58:6 59:49 60:11 61:13 62:34 63:56 64:57 65:4 66:4 67:4 68:8 69:11 70:9 71:37 72:16 73:17 74:10 75:24 76:8 77:9 78:9 79:5 80:16 81:20 82:8 83:21 84:12 85:13 86:4 87:7 88:16 89:4 90:119 91:10 92:27 93:16 94:11 95:11 97:16 98:15 99:3 100:2 101:6 102:14 103:6 104:7 105:16 106:23 107:20 108:1 109:2 110:14 111:16 112:4 113:8 114:8 115:4 116:7 117:14 118:23 119:23 120:17 121:7 122:8 123:7 124:14 125:10 126:45 127:10 128:1 129:7 130:24 131:42 132:14 133:1 134:9 135:8 136:17 137:11 138:29 139:6 140:24 141:22 142:5 143:4 144:5 145:10 146:17 147:4 148:15 149:27 150:18 151:38 152:49 153:7 154:9 155:21 156:6 157:45"
       ]
      }
     ], 
     "prompt_number": 2
    }, 
    {
     "cell_type": "markdown", 
     "source": [
      "From the README:", 
      "", 
      "[label] is a file where each line is the corresponding label for [data]. The labels must be 0, 1, ..., C-1, if we have C classes."
     ]
    }, 
    {
     "cell_type": "markdown", 
     "source": [
      "Sample training label file."
     ]
    }, 
    {
     "cell_type": "code", 
     "collapsed": false, 
     "input": [
      "!head -n 2 Wang/sample_data/Labelme/images/train-label.dat"
     ], 
     "language": "python", 
     "outputs": [
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "0", 
        "", 
        "0"
       ]
      }
     ], 
     "prompt_number": 3
    }, 
    {
     "cell_type": "markdown", 
     "source": [
      "##Estimate parameters"
     ]
    }, 
    {
     "cell_type": "code", 
     "collapsed": false, 
     "input": [
      "train_data  = 'Wang/sample_data/Labelme/images/train-data.dat'", 
      "train_label = 'Wang/sample_data/Labelme/images/train-label.dat'", 
      "settings    = 'Wang/settings.txt'", 
      "output      = 'slda_out'", 
      "initialize = 'random'", 
      "#alpha = 0.5 # doesn't work?", 
      "alpha = 1", 
      "#k = 100 # too slow, and 20 should get us to 70s in acc.", 
      "k = 20", 
      "%time !slda est $train_data $train_label $settings $alpha $k $initialize $output"
     ], 
     "language": "python", 
     "outputs": [
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "", 
        "reading data from Wang/sample_data/Labelme/images/train-data.dat"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "number of docs  : 800", 
        "number of terms : 158", 
        "number of total words : 1920800", 
        "", 
        "reading labels from Wang/sample_data/Labelme/images/train-label.dat", 
        "number of classes : 8", 
        "", 
        "alpha is fixed ...", 
        "var max iter 20", 
        "var convergence 1.00E-03", 
        "em max iter 50", 
        "em convergence 1.00E-04", 
        "L2 penalty 1.00E-02", 
        "number of topics is 20", 
        "models will be saved in slda_out", 
        "initializing ...", 
        "**** em iteration 1 ****", 
        "**** e-step ****", 
        "document 0"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 100"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 200"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 300"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 400"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 500"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 600"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 700"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "likelihood: -9748006.9562887028", 
        "**** m-step ****", 
        "maximizing ..."
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 00 -> f: -1663.380817"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 10 -> f: -1540.334096"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 20 -> f: -1491.285054"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 30 -> f: -1481.685982"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 40 -> f: -1480.277413"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "final f: -1479.075669", 
        "**** em iteration 2 ****", 
        "**** e-step ****", 
        "document 0"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 100"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 200"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 300"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 400"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 500"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 600"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 700"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "likelihood: -9630936.8880674671", 
        "**** m-step ****", 
        "maximizing ..."
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 00 -> f: -1256.449912"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 10 -> f: -1232.470027"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 20 -> f: -1226.440297"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 30 -> f: -1220.433674"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 40 -> f: -1215.163580"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "final f: -1214.779206", 
        "**** em iteration 3 ****", 
        "**** e-step ****", 
        "document 0"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 100"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 200"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 300"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 400"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 500"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 600"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 700"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "likelihood: -9515588.4603620023", 
        "**** m-step ****", 
        "maximizing ..."
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 00 -> f: -1064.857038"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 10 -> f: -988.172893"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 20 -> f: -960.458665"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 30 -> f: -938.467861"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 40 -> f: -924.795022"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "final f: -920.069577", 
        "**** em iteration 4 ****"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "**** e-step ****", 
        "document 0"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 100"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 200"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 300"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 400"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 500"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 600"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 700"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "likelihood: -9363551.4462286774", 
        "**** m-step ****", 
        "maximizing ..."
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 00 -> f: -831.774624"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 10 -> f: -811.932021"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 20 -> f: -801.111688"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 30 -> f: -795.229230"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 40 -> f: -791.444085"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "final f: -787.928890", 
        "**** em iteration 5 ****", 
        "**** e-step ****", 
        "document 0"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 100"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 200"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 300"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 400"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 500"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 600"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 700"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "likelihood: -9274271.6746991947", 
        "**** m-step ****", 
        "maximizing ..."
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 00 -> f: -755.497211"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 10 -> f: -749.971815"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 20 -> f: -747.925698"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 30 -> f: -745.933059"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 40 -> f: -743.750134"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "final f: -742.586606", 
        "**** em iteration 6 ****", 
        "**** e-step ****", 
        "document 0"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 100"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 200"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 300"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 400"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 500"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 600"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 700"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "likelihood: -9229430.9734375384", 
        "**** m-step ****", 
        "maximizing ..."
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 00 -> f: -728.364065"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 10 -> f: -724.520640"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 20 -> f: -723.060218"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 30 -> f: -721.476022"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 40 -> f: -720.818564"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "final f: -720.528978", 
        "**** em iteration 7 ****", 
        "**** e-step ****", 
        "document 0"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 100"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 200"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 300"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 400"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 500"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 600"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 700"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "likelihood: -9200674.0349101331", 
        "**** m-step ****", 
        "maximizing ..."
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 00 -> f: -710.362512"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 10 -> f: -708.483107"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 20 -> f: -707.774556"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 30 -> f: -707.135006"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 40 -> f: -706.650129"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "final f: -706.503754", 
        "**** em iteration 8 ****", 
        "**** e-step ****", 
        "document 0"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 100"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 200"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 300"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 400"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 500"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 600"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 700"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "likelihood: -9180528.5691912770", 
        "**** m-step ****", 
        "maximizing ..."
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 00 -> f: -699.419217"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 10 -> f: -698.404536"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 20 -> f: -698.001678"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 30 -> f: -697.601101"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 40 -> f: -697.446075"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "final f: -697.215056", 
        "**** em iteration 9 ****", 
        "**** e-step ****", 
        "document 0"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 100"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 200"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 300"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 400"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 500"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 600"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 700"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "likelihood: -9165833.6733603086", 
        "**** m-step ****", 
        "maximizing ..."
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 00 -> f: -692.228229"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 10 -> f: -691.419169"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 20 -> f: -691.119607"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 30 -> f: -690.697394"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 40 -> f: -690.590181"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "final f: -690.545618", 
        "**** em iteration 10 ****", 
        "**** e-step ****", 
        "document 0"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 100"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 200"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 300"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 400"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 500"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 600"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 700"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "likelihood: -9154587.3516605403", 
        "**** m-step ****", 
        "maximizing ..."
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 00 -> f: -686.012271"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 10 -> f: -685.328785"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 20 -> f: -685.198531"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 30 -> f: -685.102294"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 40 -> f: -685.040384"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "final f: -684.981469", 
        "**** em iteration 11 ****", 
        "**** e-step ****", 
        "document 0"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 100"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 200"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 300"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 400"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 500"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 600"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 700"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "likelihood: -9145556.8190778699", 
        "**** m-step ****", 
        "maximizing ..."
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 00 -> f: -680.979074"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 10 -> f: -680.487533"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 20 -> f: -680.378223"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 30 -> f: -680.295619"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 40 -> f: -680.208819"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "final f: -680.175473", 
        "**** em iteration 12 ****", 
        "**** e-step ****", 
        "document 0"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 100"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 200"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 300"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 400"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 500"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 600"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 700"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "likelihood: -9138136.4887277186", 
        "**** m-step ****", 
        "maximizing ..."
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 00 -> f: -677.210431"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 10 -> f: -676.797275"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 20 -> f: -676.658081"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 30 -> f: -676.597970"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 40 -> f: -676.549516"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "final f: -676.534683", 
        "**** em iteration 13 ****", 
        "**** e-step ****", 
        "document 0"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 100"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 200"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 300"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 400"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 500"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 600"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 700"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "likelihood: -9131827.2252566759", 
        "**** m-step ****", 
        "maximizing ..."
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 00 -> f: -674.841769"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 10 -> f: -674.512977"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 20 -> f: -674.363674"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 30 -> f: -674.327350"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 40 -> f: -674.281117"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "final f: -674.268524", 
        "**** em iteration 14 ****", 
        "**** e-step ****", 
        "document 0"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 100"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 200"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 300"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 400"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 500"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 600"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 700"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "likelihood: -9126192.8095106725", 
        "**** m-step ****", 
        "maximizing ..."
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 00 -> f: -672.296712"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 10 -> f: -672.012824"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 20 -> f: -671.886304"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 30 -> f: -671.837046"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 40 -> f: -671.799300"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "final f: -671.780297", 
        "**** em iteration 15 ****", 
        "**** e-step ****", 
        "document 0"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 100"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 200"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 300"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 400"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 500"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 600"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 700"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "likelihood: -9121168.1999670528", 
        "**** m-step ****", 
        "maximizing ..."
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 00 -> f: -669.969567"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 10 -> f: -669.680210"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 20 -> f: -669.599208"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 30 -> f: -669.544997"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 40 -> f: -669.532502"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "final f: -669.523923", 
        "**** em iteration 16 ****", 
        "**** e-step ****", 
        "document 0"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 100"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 200"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 300"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 400"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 500"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 600"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 700"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "likelihood: -9116719.2699064687", 
        "**** m-step ****", 
        "maximizing ..."
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 00 -> f: -668.293002"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 10 -> f: -668.039784"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 20 -> f: -667.965823"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 30 -> f: -667.921092"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 40 -> f: -667.910942"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "final f: -667.909812", 
        "**** em iteration 17 ****", 
        "**** e-step ****", 
        "document 0"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 100"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 200"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 300"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 400"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 500"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 600"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 700"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "likelihood: -9112722.7549859397", 
        "**** m-step ****", 
        "maximizing ..."
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 00 -> f: -666.954608"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 10 -> f: -666.718009"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 20 -> f: -666.651465"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 30 -> f: -666.604378"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 40 -> f: -666.579112"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "final f: -666.573899", 
        "**** em iteration 18 ****", 
        "**** e-step ****", 
        "document 0"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 100"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 200"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 300"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 400"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 500"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 600"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 700"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "likelihood: -9109119.8958729543", 
        "**** m-step ****", 
        "maximizing ..."
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 00 -> f: -665.922428"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 10 -> f: -665.749394"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 20 -> f: -665.677994"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 30 -> f: -665.656545"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 40 -> f: -665.643369"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "final f: -665.639895", 
        "**** em iteration 19 ****", 
        "**** e-step ****", 
        "document 0"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 100"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 200"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 300"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 400"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 500"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 600"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 700"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "likelihood: -9105968.7967952304", 
        "**** m-step ****"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "maximizing ..."
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 00 -> f: -665.477398"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 10 -> f: -665.326119"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 20 -> f: -665.265493"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 30 -> f: -665.245817"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 40 -> f: -665.229583"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "final f: -665.228868", 
        "**** em iteration 20 ****", 
        "**** e-step ****", 
        "document 0"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 100"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 200"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 300"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 400"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 500"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 600"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 700"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "likelihood: -9103126.0030853301", 
        "**** m-step ****", 
        "maximizing ..."
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 00 -> f: -664.469650"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 10 -> f: -664.314521"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 20 -> f: -664.275484"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 30 -> f: -664.260906"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 40 -> f: -664.249269"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "final f: -664.244267", 
        "**** em iteration 21 ****", 
        "**** e-step ****", 
        "document 0"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 100"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 200"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 300"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 400"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 500"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 600"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 700"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "likelihood: -9100535.9483114444", 
        "**** m-step ****", 
        "maximizing ..."
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 00 -> f: -664.003423"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 10 -> f: -663.868694"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 20 -> f: -663.835217"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 30 -> f: -663.819383"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 40 -> f: -663.811866"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "final f: -663.809479", 
        "**** em iteration 22 ****", 
        "**** e-step ****", 
        "document 0"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 100"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 200"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 300"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 400"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 500"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 600"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 700"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "likelihood: -9098292.5159986746", 
        "**** m-step ****", 
        "maximizing ..."
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 00 -> f: -664.113570"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 10 -> f: -664.007075"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 20 -> f: -663.980006"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 30 -> f: -663.970068"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 40 -> f: -663.963078"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "final f: -663.961120", 
        "**** em iteration 23 ****", 
        "**** e-step ****", 
        "document 0"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 100"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 200"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 300"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 400"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 500"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 600"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 700"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "likelihood: -9096267.0260485709", 
        "**** m-step ****", 
        "maximizing ..."
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 00 -> f: -664.131107"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 10 -> f: -664.046815"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 20 -> f: -664.013870"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 30 -> f: -664.006075"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 40 -> f: -664.002177"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "final f: -663.999762", 
        "**** em iteration 24 ****", 
        "**** e-step ****", 
        "document 0"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 100"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 200"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 300"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 400"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 500"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 600"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 700"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "likelihood: -9094468.2369568814", 
        "**** m-step ****", 
        "maximizing ..."
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 00 -> f: -664.074090"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 10 -> f: -664.005520"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 20 -> f: -663.976691"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 30 -> f: -663.967775"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 40 -> f: -663.961042"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "final f: -663.959956", 
        "**** em iteration 25 ****", 
        "**** e-step ****", 
        "document 0"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 100"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 200"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 300"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 400"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 500"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 600"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 700"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "likelihood: -9092922.8633703776", 
        "**** m-step ****", 
        "maximizing ..."
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 00 -> f: -664.146222"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 10 -> f: -664.089643"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 20 -> f: -664.058222"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 30 -> f: -664.051192"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 40 -> f: -664.047324"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "final f: -664.046393", 
        "**** em iteration 26 ****"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "**** e-step ****", 
        "document 0"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 100"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 200"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 300"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 400"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 500"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 600"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 700"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "likelihood: -9091515.9496620856", 
        "**** m-step ****", 
        "maximizing ..."
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 00 -> f: -664.314364"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 10 -> f: -664.266253"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 20 -> f: -664.243933"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 30 -> f: -664.234538"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 40 -> f: -664.228867"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "final f: -664.227264", 
        "**** em iteration 27 ****", 
        "**** e-step ****", 
        "document 0"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 100"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 200"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 300"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 400"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 500"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 600"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 700"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "likelihood: -9090272.5277456939", 
        "**** m-step ****", 
        "maximizing ..."
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 00 -> f: -664.492793"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 10 -> f: -664.443660"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 20 -> f: -664.427046"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 30 -> f: -664.422225"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 40 -> f: -664.419737"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "final f: -664.419200", 
        "**** em iteration 28 ****", 
        "**** e-step ****", 
        "document 0"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 100"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 200"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 300"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 400"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 500"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 600"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 700"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "likelihood: -9089139.3945744094", 
        "**** m-step ****", 
        "maximizing ..."
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 00 -> f: -664.649356"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 10 -> f: -664.608927"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 20 -> f: -664.597700"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 30 -> f: -664.595051"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 40 -> f: -664.591741"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "final f: -664.591074", 
        "**** em iteration 29 ****", 
        "**** e-step ****", 
        "document 0"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 100"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 200"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 300"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 400"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 500"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 600"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 700"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "likelihood: -9088140.2862774059", 
        "**** m-step ****", 
        "maximizing ..."
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 00 -> f: -664.695661"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 10 -> f: -664.664147"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 20 -> f: -664.655867"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 30 -> f: -664.648637"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 40 -> f: -664.645793"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "final f: -664.644021", 
        "**** em iteration 30 ****", 
        "**** e-step ****", 
        "document 0"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 100"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 200"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 300"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 400"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 500"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 600"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 700"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "likelihood: -9087268.0975004733", 
        "**** m-step ****", 
        "maximizing ..."
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 00 -> f: -664.770393"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 10 -> f: -664.737240"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 20 -> f: -664.731735"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 30 -> f: -664.723045"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "step: 40 -> f: -664.720224"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "final f: -664.718783", 
        "final e step document 0"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "final e step document 100"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "final e step document 200"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "final e step document 300"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "final e step document 400"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "final e step document 500"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "final e step document 600"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "final e step document 700"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "CPU times: user 5.44 s, sys: 0.55 s, total: 5.99 s"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "", 
        "Wall time: 1166.28 s"
       ]
      }
     ], 
     "prompt_number": 26
    }, 
    {
     "cell_type": "code", 
     "collapsed": false, 
     "input": [
      "1166.2/60"
     ], 
     "language": "python", 
     "outputs": [
      {
       "output_type": "pyout", 
       "prompt_number": 27, 
       "text": [
        "19.436666666666667"
       ]
      }
     ], 
     "prompt_number": 27
    }, 
    {
     "cell_type": "code", 
     "collapsed": false, 
     "input": [
      "!ls slda_out/"
     ], 
     "language": "python", 
     "outputs": [
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "010.gamma            010.model.text       020.model            030.gamma            030.model.text       final.model          likelihood.dat", 
        "010.model            020.gamma            020.model.text       030.model            final.gamma          final.model.text     word-assignments.dat"
       ]
      }
     ], 
     "prompt_number": 28
    }, 
    {
     "cell_type": "code", 
     "collapsed": false, 
     "input": [
      "!head slda_out/final.model.text | sed 6q"
     ], 
     "language": "python", 
     "outputs": [
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "alpha: 1.000000", 
        "number of topics: 20", 
        "size of vocab: 158", 
        "number of classes: 8", 
        "betas: ", 
        "-5.811285 -11.967485 -18.914221 -6.513638 -7.308558 -17.485325 -4.737131 -6.335333 -6.158160 -6.531578 -9.758288 -6.225330 -15.983147 -7.420958 -9.544016 -5.346224 -6.202065 -17.316021 -16.683645 -6.731924 -14.454849 -6.352088 -4.862735 -6.676651 -2.133869 -8.589479 -16.151304 -6.100658 -5.515120 -6.372279 -6.304289 -8.432538 -13.806954 -6.430807 -6.946295 -6.094163 -2.407369 -8.669470 -7.991627 -13.175921 -6.303987 -5.478724 -8.999448 -8.486452 -8.898187 -7.328442 -7.015964 -5.836622 -6.981790 -0.987949 -3.873769 -18.347929 -7.486795 -6.829003 -5.086598 -8.127007 -5.396060 -9.626009 -9.086355 -9.164608 -4.323623 -8.261922 -9.143617 -6.035682 -6.186491 -18.991654 -8.561136 -15.322679 -11.243548 -6.795811 -9.739773 -6.894630 -4.538295 -10.002911 -8.532067 -4.914672 -8.793383 -9.174182 -7.746222 -7.374921 -8.001740 -6.586866 -6.723496 -7.862039 -4.399132 -6.920312 -9.770824 -14.527852 -7.877004 -5.031355 -9.532180 -6.843732 -8.640296 -7.973003 -7.956802 -8.007480 -60.361463 -5.682587 -7.400962 -9.104589 -18.212717 -7.740084 -5.458851 -13.595962 -7.304395 -6.627257 -5.412624 -7.101255 -21.009302 -13.238413 -8.669999 -5.913836 -15.346631 -3.198614 -7.225454 -4.644394 -16.010196 -7.354357 -7.322840 -2.334019 -14.504019 -10.022790 -8.072066 -8.459487 -8.289934 -9.990684 -5.705173 -7.611201 -14.356285 -10.332373 -6.447641 -6.841712 -7.353309 -11.949915 -10.118405 -9.188497 -7.009836 -5.629516 -6.607716 -9.653562 -5.809436 -10.232220 -8.175217 -7.637223 -8.808883 -8.321806 -3.717843 -7.277602 -6.036518 -6.855768 -6.018941 -6.701075 -5.022462 -6.947585 -7.154787 -6.588638 -6.745715 -3.821158 "
       ]
      }
     ], 
     "prompt_number": 29
    }, 
    {
     "cell_type": "code", 
     "collapsed": false, 
     "input": [
      "!head slda_out/final.gamma | sed 2q"
     ], 
     "language": "python", 
     "outputs": [
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "137.5457049576 58.4108436712 10.6983907659 38.5357298616 109.9824764264 7.7730161953 14.9592919437 830.6986574471 100.2145620058 265.9827134377 17.0083666721 16.0686639204 1.4159489621 24.1391444999 36.3028937102 19.8441896464 417.0506211712 4.8340151467 143.8654473859 165.6693221729", 
        "88.5042387658 507.3714027368 44.7173506235 41.7852040036 148.2400710325 45.0823234536 20.0910263061 87.9644203824 258.5341636069 194.1643601731 172.1438772159 94.7629484650 2.0026171799 37.7576629550 127.9741461231 32.5109903818 233.6199272851 26.7402413235 97.2304884272 159.8025395592"
       ]
      }
     ], 
     "prompt_number": 30
    }, 
    {
     "cell_type": "code", 
     "collapsed": false, 
     "input": [
      "!head slda_out/likelihood.dat"
     ], 
     "language": "python", 
     "outputs": [
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "-9748006.9562887028\t  inf", 
        "-9630936.8880674671\t1.20096e-02", 
        "-9515588.4603620023\t1.19769e-02", 
        "-9363551.4462286774\t1.59777e-02", 
        "-9274271.6746991947\t9.53482e-03", 
        "-9229430.9734375384\t4.83496e-03", 
        "-9200674.0349101331\t3.11579e-03", 
        "-9180528.5691912770\t2.18956e-03", 
        "-9165833.6733603086\t1.60066e-03", 
        "-9154587.3516605403\t1.22698e-03"
       ]
      }
     ], 
     "prompt_number": 31
    }, 
    {
     "cell_type": "code", 
     "collapsed": false, 
     "input": [
      "!head slda_out/word-assignments.dat | sed 2q"
     ], 
     "language": "python", 
     "outputs": [
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "147 0000:07 0001:13 0002:17 0003:07 0004:19 0005:07 0006:09 0007:04 0008:07 0009:16 0010:07 0011:19 0012:06 0013:11 0014:07 0015:07 0016:04 0017:15 0019:07 0020:14 0021:16 0022:18 0023:04 0024:00 0025:04 0026:15 0027:07 0028:08 0029:04 0030:04 0032:17 0033:18 0034:16 0035:07 0036:07 0037:07 0038:03 0040:16 0041:07 0042:07 0043:07 0044:02 0045:07 0046:04 0047:16 0048:04 0049:00 0050:09 0052:03 0053:03 0054:07 0055:03 0056:19 0057:13 0058:07 0059:07 0060:07 0061:18 0062:09 0063:16 0064:07 0066:13 0069:07 0070:04 0071:08 0072:08 0073:19 0074:07 0075:09 0076:07 0077:13 0078:13 0079:03 0080:07 0081:19 0082:07 0083:19 0084:07 0085:03 0086:04 0087:14 0088:07 0089:07 0090:01 0091:16 0092:07 0093:07 0094:18 0095:07 0097:04 0098:19 0099:13 0100:15 0101:07 0102:16 0104:07 0105:16 0106:16 0107:16 0108:15 0109:06 0110:07 0111:07 0112:07 0113:07 0114:03 0115:07 0116:14 0117:07 0118:18 0119:07 0120:14 0121:07 0122:07 0123:07 0125:07 0126:16 0127:19 0128:06 0129:07 0130:07 0131:16 0132:07 0134:13 0135:07 0136:16 0137:16 0138:16 0139:07 0140:16 0141:09 0142:07 0143:09 0144:13 0145:19 0146:07 0147:03 0148:16 0149:16 0150:18 0151:16 0152:16 0153:07 0154:13 0155:04 0156:19 0157:09", 
        "157 0000:01 0001:13 0002:17 0003:11 0004:05 0005:15 0006:09 0007:04 0008:10 0009:16 0010:07 0011:04 0012:06 0013:11 0014:10 0015:08 0016:04 0017:15 0018:17 0019:01 0020:14 0021:16 0022:18 0023:04 0024:00 0025:02 0026:15 0027:01 0028:08 0029:04 0030:04 0031:14 0032:17 0033:01 0034:01 0035:10 0036:00 0037:10 0038:03 0039:15 0040:16 0041:10 0042:19 0043:05 0044:02 0045:11 0046:04 0047:01 0048:04 0049:00 0050:09 0051:06 0052:03 0053:03 0054:08 0055:03 0056:19 0057:13 0058:11 0059:01 0060:07 0061:18 0062:09 0063:01 0064:01 0065:15 0066:13 0067:15 0068:10 0069:10 0070:02 0071:08 0072:08 0073:10 0074:10 0075:09 0076:10 0077:13 0078:11 0079:03 0080:10 0081:19 0082:05 0083:10 0084:07 0085:10 0086:02 0087:14 0088:10 0089:09 0090:01 0091:04 0092:01 0093:11 0094:18 0095:09 0097:04 0098:14 0099:13 0100:15 0101:14 0102:16 0103:14 0104:05 0105:01 0106:16 0107:10 0108:15 0109:14 0110:10 0111:04 0112:15 0113:09 0114:03 0115:07 0116:14 0117:01 0118:08 0119:00 0120:14 0121:10 0122:10 0123:14 0124:14 0125:11 0126:08 0127:10 0128:06 0129:10 0130:04 0131:16 0132:10 0133:09 0134:13 0135:11 0136:01 0137:08 0138:01 0139:11 0140:01 0141:09 0142:02 0143:09 0144:13 0145:10 0146:08 0147:03 0148:14 0149:08 0150:01 0151:08 0152:08 0153:08 0154:02 0155:04 0156:04 0157:09"
       ]
      }
     ], 
     "prompt_number": 32
    }, 
    {
     "cell_type": "markdown", 
     "source": [
      "## Predict test set"
     ]
    }, 
    {
     "cell_type": "code", 
     "collapsed": false, 
     "input": [
      "test_data  = 'Wang/sample_data/Labelme/images/test-data.dat'", 
      "test_label = 'Wang/sample_data/Labelme/images/test-label.dat'", 
      "!mkdir slda_inf", 
      "%time !slda inf $test_data $test_label $settings $output/final.model slda_inf"
     ], 
     "language": "python", 
     "outputs": [
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "mkdir: slda_inf: File exists"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "", 
        "reading data from Wang/sample_data/Labelme/images/test-data.dat", 
        "number of docs  : 800", 
        "number of terms : 158", 
        "number of total words : 1920800", 
        "", 
        "reading labels from Wang/sample_data/Labelme/images/test-label.dat", 
        "number of classes : 8", 
        "", 
        "alpha is fixed ...", 
        "var max iter 20", 
        "var convergence 1.00E-03", 
        "em max iter 50", 
        "em convergence 1.00E-04", 
        "L2 penalty 1.00E-02", 
        "", 
        "results will be saved in slda_inf", 
        "document 0"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 100"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 200"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 300"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 400"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 500"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 600"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "document 700"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "average accuracy: 0.724"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "CPU times: user 0.01 s, sys: 0.01 s, total: 0.02 s"
       ]
      }, 
      {
       "output_type": "stream", 
       "stream": "stdout", 
       "text": [
        "", 
        "Wall time: 1.86 s"
       ]
      }
     ], 
     "prompt_number": 33
    }, 
    {
     "cell_type": "code", 
     "collapsed": true, 
     "input": [], 
     "language": "python", 
     "outputs": []
    }
   ]
  }
 ]
}