pytorch_version.html

<!DOCTYPE html>
<html>
<head><meta charset="utf-8" />

<title>Deep_Learning_Project-Pytorch</title>

<script src="https://cdnjs.cloudflare.com/ajax/libs/require.js/2.1.10/require.min.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/jquery/2.0.3/jquery.min.js"></script>


<style type="text/css">
    /*!
*
* Twitter Bootstrap
*
*/
/*!
 * Bootstrap v3.3.7 (http://getbootstrap.com)
 * Copyright 2011-2016 Twitter, Inc.
 * Licensed under MIT (https://github.com/twbs/bootstrap/blob/master/LICENSE)
 */
/*! normalize.css v3.0.3 | MIT License | github.com/necolas/normalize.css */
html {
  font-family: sans-serif;
  -ms-text-size-adjust: 100%;
  -webkit-text-size-adjust: 100%;
}
body {
  margin: 0;
}
article,
aside,
details,
figcaption,
figure,
footer,
header,
hgroup,
main,
menu,
nav,
section,
summary {
  display: block;
}
audio,
canvas,
progress,
video {
  display: inline-block;
  vertical-align: baseline;
}
audio:not([controls]) {
  display: none;
  height: 0;
}
[hidden],
template {
  display: none;
}
a {
  background-color: transparent;
}
a:active,
a:hover {
  outline: 0;
}
abbr[title] {
  border-bottom: 1px dotted;
}
b,
strong {
  font-weight: bold;
}
dfn {
  font-style: italic;
}
h1 {
  font-size: 2em;
  margin: 0.67em 0;
}
mark {
  background: #ff0;
  color: #000;
}
small {
  font-size: 80%;
}
sub,
sup {
  font-size: 75%;
  line-height: 0;
  position: relative;
  vertical-align: baseline;
}
sup {
  top: -0.5em;
}
sub {
  bottom: -0.25em;
}
img {
  border: 0;
}
svg:not(:root) {
  overflow: hidden;
}
figure {
  margin: 1em 40px;
}
hr {
  box-sizing: content-box;
  height: 0;
}
pre {
  overflow: auto;
}
code,
kbd,
pre,
samp {
  font-family: monospace, monospace;
  font-size: 1em;
}
button,
input,
optgroup,
select,
textarea {
  color: inherit;
  font: inherit;
  margin: 0;
}
button {
  overflow: visible;
}
button,
select {
  text-transform: none;
}
button,
html input[type="button"],
input[type="reset"],
input[type="submit"] {
  -webkit-appearance: button;
  cursor: pointer;
}
button[disabled],
html input[disabled] {
  cursor: default;
}
button::-moz-focus-inner,
input::-moz-focus-inner {
  border: 0;
  padding: 0;
}
input {
  line-height: normal;
}
input[type="checkbox"],
input[type="radio"] {
  box-sizing: border-box;
  padding: 0;
}
input[type="number"]::-webkit-inner-spin-button,
input[type="number"]::-webkit-outer-spin-button {
  height: auto;
}
input[type="search"] {
  -webkit-appearance: textfield;
  box-sizing: content-box;
}
input[type="search"]::-webkit-search-cancel-button,
input[type="search"]::-webkit-search-decoration {
  -webkit-appearance: none;
}
fieldset {
  border: 1px solid #c0c0c0;
  margin: 0 2px;
  padding: 0.35em 0.625em 0.75em;
}
legend {
  border: 0;
  padding: 0;
}
textarea {
  overflow: auto;
}
optgroup {
  font-weight: bold;
}
table {
  border-collapse: collapse;
  border-spacing: 0;
}
td,
th {
  padding: 0;
}
/*! Source: https://github.com/h5bp/html5-boilerplate/blob/master/src/css/main.css */
@media print {
  *,
  *:before,
  *:after {
    background: transparent !important;
    box-shadow: none !important;
    text-shadow: none !important;
  }
  a,
  a:visited {
    text-decoration: underline;
  }
  a[href]:after {
    content: " (" attr(href) ")";
  }
  abbr[title]:after {
    content: " (" attr(title) ")";
  }
  a[href^="#"]:after,
  a[href^="javascript:"]:after {
    content: "";
  }
  pre,
  blockquote {
    border: 1px solid #999;
    page-break-inside: avoid;
  }
  thead {
    display: table-header-group;
  }
  tr,
  img {
    page-break-inside: avoid;
  }
  img {
    max-width: 100% !important;
  }
  p,
  h2,
  h3 {
    orphans: 3;
    widows: 3;
  }
  h2,
  h3 {
    page-break-after: avoid;
  }
  .navbar {
    display: none;
  }
  .btn > .caret,
  .dropup > .btn > .caret {
    border-top-color: #000 !important;
  }
  .label {
    border: 1px solid #000;
  }
  .table {
    border-collapse: collapse !important;
  }
  .table td,
  .table th {
    background-color: #fff !important;
  }
  .table-bordered th,
  .table-bordered td {
    border: 1px solid #ddd !important;
  }
}
@font-face {
  font-family: 'Glyphicons Halflings';
  src: url('../components/bootstrap/fonts/glyphicons-halflings-regular.eot');
  src: url('../components/bootstrap/fonts/glyphicons-halflings-regular.eot?#iefix') format('embedded-opentype'), url('../components/bootstrap/fonts/glyphicons-halflings-regular.woff2') format('woff2'), url('../components/bootstrap/fonts/glyphicons-halflings-regular.woff') format('woff'), url('../components/bootstrap/fonts/glyphicons-halflings-regular.ttf') format('truetype'), url('../components/bootstrap/fonts/glyphicons-halflings-regular.svg#glyphicons_halflingsregular') format('svg');
}
.glyphicon {
  position: relative;
  top: 1px;
  display: inline-block;
  font-family: 'Glyphicons Halflings';
  font-style: normal;
  font-weight: normal;
  line-height: 1;
  -webkit-font-smoothing: antialiased;
  -moz-osx-font-smoothing: grayscale;
}
.glyphicon-asterisk:before {
  content: "\002a";
}
.glyphicon-plus:before {
  content: "\002b";
}
.glyphicon-euro:before,
.glyphicon-eur:before {
  content: "\20ac";
}
.glyphicon-minus:before {
  content: "\2212";
}
.glyphicon-cloud:before {
  content: "\2601";
}
.glyphicon-envelope:before {
  content: "\2709";
}
.glyphicon-pencil:before {
  content: "\270f";
}
.glyphicon-glass:before {
  content: "\e001";
}
.glyphicon-music:before {
  content: "\e002";
}
.glyphicon-search:before {
  content: "\e003";
}
.glyphicon-heart:before {
  content: "\e005";
}
.glyphicon-star:before {
  content: "\e006";
}
.glyphicon-star-empty:before {
  content: "\e007";
}
.glyphicon-user:before {
  content: "\e008";
}
.glyphicon-film:before {
  content: "\e009";
}
.glyphicon-th-large:before {
  content: "\e010";
}
.glyphicon-th:before {
  content: "\e011";
}
.glyphicon-th-list:before {
  content: "\e012";
}
.glyphicon-ok:before {
  content: "\e013";
}
.glyphicon-remove:before {
  content: "\e014";
}
.glyphicon-zoom-in:before {
  content: "\e015";
}
.glyphicon-zoom-out:before {
  content: "\e016";
}
.glyphicon-off:before {
  content: "\e017";
}
.glyphicon-signal:before {
  content: "\e018";
}
.glyphicon-cog:before {
  content: "\e019";
}
.glyphicon-trash:before {
  content: "\e020";
}
.glyphicon-home:before {
  content: "\e021";
}
.glyphicon-file:before {
  content: "\e022";
}
.glyphicon-time:before {
  content: "\e023";
}
.glyphicon-road:before {
  content: "\e024";
}
.glyphicon-download-alt:before {
  content: "\e025";
}
.glyphicon-download:before {
  content: "\e026";
}
.glyphicon-upload:before {
  content: "\e027";
}
.glyphicon-inbox:before {
  content: "\e028";
}
.glyphicon-play-circle:before {
  content: "\e029";
}
.glyphicon-repeat:before {
  content: "\e030";
}
.glyphicon-refresh:before {
  content: "\e031";
}
.glyphicon-list-alt:before {
  content: "\e032";
}
.glyphicon-lock:before {
  content: "\e033";
}
.glyphicon-flag:before {
  content: "\e034";
}
.glyphicon-headphones:before {
  content: "\e035";
}
.glyphicon-volume-off:before {
  content: "\e036";
}
.glyphicon-volume-down:before {
  content: "\e037";
}
.glyphicon-volume-up:before {
  content: "\e038";
}
.glyphicon-qrcode:before {
  content: "\e039";
}
.glyphicon-barcode:before {
  content: "\e040";
}
.glyphicon-tag:before {
  content: "\e041";
}
.glyphicon-tags:before {
  content: "\e042";
}
.glyphicon-book:before {
  content: "\e043";
}
.glyphicon-bookmark:before {
  content: "\e044";
}
.glyphicon-print:before {
  content: "\e045";
}
.glyphicon-camera:before {
  content: "\e046";
}
.glyphicon-font:before {
  content: "\e047";
}
.glyphicon-bold:before {
  content: "\e048";
}
.glyphicon-italic:before {
  content: "\e049";
}
.glyphicon-text-height:before {
  content: "\e050";
}
.glyphicon-text-width:before {
  content: "\e051";
}
.glyphicon-align-left:before {
  content: "\e052";
}
.glyphicon-align-center:before {
  content: "\e053";
}
.glyphicon-align-right:before {
  content: "\e054";
}
.glyphicon-align-justify:before {
  content: "\e055";
}
.glyphicon-list:before {
  content: "\e056";
}
.glyphicon-indent-left:before {
  content: "\e057";
}
.glyphicon-indent-right:before {
  content: "\e058";
}
.glyphicon-facetime-video:before {
  content: "\e059";
}
.glyphicon-picture:before {
  content: "\e060";
}
.glyphicon-map-marker:before {
  content: "\e062";
}
.glyphicon-adjust:before {
  content: "\e063";
}
.glyphicon-tint:before {
  content: "\e064";
}
.glyphicon-edit:before {
  content: "\e065";
}
.glyphicon-share:before {
  content: "\e066";
}
.glyphicon-check:before {
  content: "\e067";
}
.glyphicon-move:before {
  content: "\e068";
}
.glyphicon-step-backward:before {
  content: "\e069";
}
.glyphicon-fast-backward:before {
  content: "\e070";
}
.glyphicon-backward:before {
  content: "\e071";
}
.glyphicon-play:before {
  content: "\e072";
}
.glyphicon-pause:before {
  content: "\e073";
}
.glyphicon-stop:before {
  content: "\e074";
}
.glyphicon-forward:before {
  content: "\e075";
}
.glyphicon-fast-forward:before {
  content: "\e076";
}
.glyphicon-step-forward:before {
  content: "\e077";
}
.glyphicon-eject:before {
  content: "\e078";
}
.glyphicon-chevron-left:before {
  content: "\e079";
}
.glyphicon-chevron-right:before {
  content: "\e080";
}
.glyphicon-plus-sign:before {
  content: "\e081";
}
.glyphicon-minus-sign:before {
  content: "\e082";
}
.glyphicon-remove-sign:before {
  content: "\e083";
}
.glyphicon-ok-sign:before {
  content: "\e084";
}
.glyphicon-question-sign:before {
  content: "\e085";
}
.glyphicon-info-sign:before {
  content: "\e086";
}
.glyphicon-screenshot:before {
  content: "\e087";
}
.glyphicon-remove-circle:before {
  content: "\e088";
}
.glyphicon-ok-circle:before {
  content: "\e089";
}
.glyphicon-ban-circle:before {
  content: "\e090";
}
.glyphicon-arrow-left:before {
  content: "\e091";
}
.glyphicon-arrow-right:before {
  content: "\e092";
}
.glyphicon-arrow-up:before {
  content: "\e093";
}
.glyphicon-arrow-down:before {
  content: "\e094";
}
.glyphicon-share-alt:before {
  content: "\e095";
}
.glyphicon-resize-full:before {
  content: "\e096";
}
.glyphicon-resize-small:before {
  content: "\e097";
}
.glyphicon-exclamation-sign:before {
  content: "\e101";
}
.glyphicon-gift:before {
  content: "\e102";
}
.glyphicon-leaf:before {
  content: "\e103";
}
.glyphicon-fire:before {
  content: "\e104";
}
.glyphicon-eye-open:before {
  content: "\e105";
}
.glyphicon-eye-close:before {
  content: "\e106";
}
.glyphicon-warning-sign:before {
  content: "\e107";
}
.glyphicon-plane:before {
  content: "\e108";
}
.glyphicon-calendar:before {
  content: "\e109";
}
.glyphicon-random:before {
  content: "\e110";
}
.glyphicon-comment:before {
  content: "\e111";
}
.glyphicon-magnet:before {
  content: "\e112";
}
.glyphicon-chevron-up:before {
  content: "\e113";
}
.glyphicon-chevron-down:before {
  content: "\e114";
}
.glyphicon-retweet:before {
  content: "\e115";
}
.glyphicon-shopping-cart:before {
  content: "\e116";
}
.glyphicon-folder-close:before {
  content: "\e117";
}
.glyphicon-folder-open:before {
  content: "\e118";
}
.glyphicon-resize-vertical:before {
  content: "\e119";
}
.glyphicon-resize-horizontal:before {
  content: "\e120";
}
.glyphicon-hdd:before {
  content: "\e121";
}
.glyphicon-bullhorn:before {
  content: "\e122";
}
.glyphicon-bell:before {
  content: "\e123";
}
.glyphicon-certificate:before {
  content: "\e124";
}
.glyphicon-thumbs-up:before {
  content: "\e125";
}
.glyphicon-thumbs-down:before {
  content: "\e126";
}
.glyphicon-hand-right:before {
  content: "\e127";
}
.glyphicon-hand-left:before {
  content: "\e128";
}
.glyphicon-hand-up:before {
  content: "\e129";
}
.glyphicon-hand-down:before {
  content: "\e130";
}
.glyphicon-circle-arrow-right:before {
  content: "\e131";
}
.glyphicon-circle-arrow-left:before {
  content: "\e132";
}
.glyphicon-circle-arrow-up:before {
  content: "\e133";
}
.glyphicon-circle-arrow-down:before {
  content: "\e134";
}
.glyphicon-globe:before {
  content: "\e135";
}
.glyphicon-wrench:before {
  content: "\e136";
}
.glyphicon-tasks:before {
  content: "\e137";
}
.glyphicon-filter:before {
  content: "\e138";
}
.glyphicon-briefcase:before {
  content: "\e139";
}
.glyphicon-fullscreen:before {
  content: "\e140";
}
.glyphicon-dashboard:before {
  content: "\e141";
}
.glyphicon-paperclip:before {
  content: "\e142";
}
.glyphicon-heart-empty:before {
  content: "\e143";
}
.glyphicon-link:before {
  content: "\e144";
}
.glyphicon-phone:before {
  content: "\e145";
}
.glyphicon-pushpin:before {
  content: "\e146";
}
.glyphicon-usd:before {
  content: "\e148";
}
.glyphicon-gbp:before {
  content: "\e149";
}
.glyphicon-sort:before {
  content: "\e150";
}
.glyphicon-sort-by-alphabet:before {
  content: "\e151";
}
.glyphicon-sort-by-alphabet-alt:before {
  content: "\e152";
}
.glyphicon-sort-by-order:before {
  content: "\e153";
}
.glyphicon-sort-by-order-alt:before {
  content: "\e154";
}
.glyphicon-sort-by-attributes:before {
  content: "\e155";
}
.glyphicon-sort-by-attributes-alt:before {
  content: "\e156";
}
.glyphicon-unchecked:before {
  content: "\e157";
}
.glyphicon-expand:before {
  content: "\e158";
}
.glyphicon-collapse-down:before {
  content: "\e159";
}
.glyphicon-collapse-up:before {
  content: "\e160";
}
.glyphicon-log-in:before {
  content: "\e161";
}
.glyphicon-flash:before {
  content: "\e162";
}
.glyphicon-log-out:before {
  content: "\e163";
}
.glyphicon-new-window:before {
  content: "\e164";
}
.glyphicon-record:before {
  content: "\e165";
}
.glyphicon-save:before {
  content: "\e166";
}
.glyphicon-open:before {
  content: "\e167";
}
.glyphicon-saved:before {
  content: "\e168";
}
.glyphicon-import:before {
  content: "\e169";
}
.glyphicon-export:before {
  content: "\e170";
}
.glyphicon-send:before {
  content: "\e171";
}
.glyphicon-floppy-disk:before {
  content: "\e172";
}
.glyphicon-floppy-saved:before {
  content: "\e173";
}
.glyphicon-floppy-remove:before {
  content: "\e174";
}
.glyphicon-floppy-save:before {
  content: "\e175";
}
.glyphicon-floppy-open:before {
  content: "\e176";
}
.glyphicon-credit-card:before {
  content: "\e177";
}
.glyphicon-transfer:before {
  content: "\e178";
}
.glyphicon-cutlery:before {
  content: "\e179";
}
.glyphicon-header:before {
  content: "\e180";
}
.glyphicon-compressed:before {
  content: "\e181";
}
.glyphicon-earphone:before {
  content: "\e182";
}
.glyphicon-phone-alt:before {
  content: "\e183";
}
.glyphicon-tower:before {
  content: "\e184";
}
.glyphicon-stats:before {
  content: "\e185";
}
.glyphicon-sd-video:before {
  content: "\e186";
}
.glyphicon-hd-video:before {
  content: "\e187";
}
.glyphicon-subtitles:before {
  content: "\e188";
}
.glyphicon-sound-stereo:before {
  content: "\e189";
}
.glyphicon-sound-dolby:before {
  content: "\e190";
}
.glyphicon-sound-5-1:before {
  content: "\e191";
}
.glyphicon-sound-6-1:before {
  content: "\e192";
}
.glyphicon-sound-7-1:before {
  content: "\e193";
}
.glyphicon-copyright-mark:before {
  content: "\e194";
}
.glyphicon-registration-mark:before {
  content: "\e195";
}
.glyphicon-cloud-download:before {
  content: "\e197";
}
.glyphicon-cloud-upload:before {
  content: "\e198";
}
.glyphicon-tree-conifer:before {
  content: "\e199";
}
.glyphicon-tree-deciduous:before {
  content: "\e200";
}
.glyphicon-cd:before {
  content: "\e201";
}
.glyphicon-save-file:before {
  content: "\e202";
}
.glyphicon-open-file:before {
  content: "\e203";
}
.glyphicon-level-up:before {
  content: "\e204";
}
.glyphicon-copy:before {
  content: "\e205";
}
.glyphicon-paste:before {
  content: "\e206";
}
.glyphicon-alert:before {
  content: "\e209";
}
.glyphicon-equalizer:before {
  content: "\e210";
}
.glyphicon-king:before {
  content: "\e211";
}
.glyphicon-queen:before {
  content: "\e212";
}
.glyphicon-pawn:before {
  content: "\e213";
}
.glyphicon-bishop:before {
  content: "\e214";
}
.glyphicon-knight:before {
  content: "\e215";
}
.glyphicon-baby-formula:before {
  content: "\e216";
}
.glyphicon-tent:before {
  content: "\26fa";
}
.glyphicon-blackboard:before {
  content: "\e218";
}
.glyphicon-bed:before {
  content: "\e219";
}
.glyphicon-apple:before {
  content: "\f8ff";
}
.glyphicon-erase:before {
  content: "\e221";
}
.glyphicon-hourglass:before {
  content: "\231b";
}
.glyphicon-lamp:before {
  content: "\e223";
}
.glyphicon-duplicate:before {
  content: "\e224";
}
.glyphicon-piggy-bank:before {
  content: "\e225";
}
.glyphicon-scissors:before {
  content: "\e226";
}
.glyphicon-bitcoin:before {
  content: "\e227";
}
.glyphicon-btc:before {
  content: "\e227";
}
.glyphicon-xbt:before {
  content: "\e227";
}
.glyphicon-yen:before {
  content: "\00a5";
}
.glyphicon-jpy:before {
  content: "\00a5";
}
.glyphicon-ruble:before {
  content: "\20bd";
}
.glyphicon-rub:before {
  content: "\20bd";
}
.glyphicon-scale:before {
  content: "\e230";
}
.glyphicon-ice-lolly:before {
  content: "\e231";
}
.glyphicon-ice-lolly-tasted:before {
  content: "\e232";
}
.glyphicon-education:before {
  content: "\e233";
}
.glyphicon-option-horizontal:before {
  content: "\e234";
}
.glyphicon-option-vertical:before {
  content: "\e235";
}
.glyphicon-menu-hamburger:before {
  content: "\e236";
}
.glyphicon-modal-window:before {
  content: "\e237";
}
.glyphicon-oil:before {
  content: "\e238";
}
.glyphicon-grain:before {
  content: "\e239";
}
.glyphicon-sunglasses:before {
  content: "\e240";
}
.glyphicon-text-size:before {
  content: "\e241";
}
.glyphicon-text-color:before {
  content: "\e242";
}
.glyphicon-text-background:before {
  content: "\e243";
}
.glyphicon-object-align-top:before {
  content: "\e244";
}
.glyphicon-object-align-bottom:before {
  content: "\e245";
}
.glyphicon-object-align-horizontal:before {
  content: "\e246";
}
.glyphicon-object-align-left:before {
  content: "\e247";
}
.glyphicon-object-align-vertical:before {
  content: "\e248";
}
.glyphicon-object-align-right:before {
  content: "\e249";
}
.glyphicon-triangle-right:before {
  content: "\e250";
}
.glyphicon-triangle-left:before {
  content: "\e251";
}
.glyphicon-triangle-bottom:before {
  content: "\e252";
}
.glyphicon-triangle-top:before {
  content: "\e253";
}
.glyphicon-console:before {
  content: "\e254";
}
.glyphicon-superscript:before {
  content: "\e255";
}
.glyphicon-subscript:before {
  content: "\e256";
}
.glyphicon-menu-left:before {
  content: "\e257";
}
.glyphicon-menu-right:before {
  content: "\e258";
}
.glyphicon-menu-down:before {
  content: "\e259";
}
.glyphicon-menu-up:before {
  content: "\e260";
}
* {
  -webkit-box-sizing: border-box;
  -moz-box-sizing: border-box;
  box-sizing: border-box;
}
*:before,
*:after {
  -webkit-box-sizing: border-box;
  -moz-box-sizing: border-box;
  box-sizing: border-box;
}
html {
  font-size: 10px;
  -webkit-tap-highlight-color: rgba(0, 0, 0, 0);
}
body {
  font-family: "Helvetica Neue", Helvetica, Arial, sans-serif;
  font-size: 13px;
  line-height: 1.42857143;
  color: #000;
  background-color: #fff;
}
input,
button,
select,
textarea {
  font-family: inherit;
  font-size: inherit;
  line-height: inherit;
}
a {
  color: #337ab7;
  text-decoration: none;
}
a:hover,
a:focus {
  color: #23527c;
  text-decoration: underline;
}
a:focus {
  outline: 5px auto -webkit-focus-ring-color;
  outline-offset: -2px;
}
figure {
  margin: 0;
}
img {
  vertical-align: middle;
}
.img-responsive,
.thumbnail > img,
.thumbnail a > img,
.carousel-inner > .item > img,
.carousel-inner > .item > a > img {
  display: block;
  max-width: 100%;
  height: auto;
}
.img-rounded {
  border-radius: 3px;
}
.img-thumbnail {
  padding: 4px;
  line-height: 1.42857143;
  background-color: #fff;
  border: 1px solid #ddd;
  border-radius: 2px;
  -webkit-transition: all 0.2s ease-in-out;
  -o-transition: all 0.2s ease-in-out;
  transition: all 0.2s ease-in-out;
  display: inline-block;
  max-width: 100%;
  height: auto;
}
.img-circle {
  border-radius: 50%;
}
hr {
  margin-top: 18px;
  margin-bottom: 18px;
  border: 0;
  border-top: 1px solid #eeeeee;
}
.sr-only {
  position: absolute;
  width: 1px;
  height: 1px;
  margin: -1px;
  padding: 0;
  overflow: hidden;
  clip: rect(0, 0, 0, 0);
  border: 0;
}
.sr-only-focusable:active,
.sr-only-focusable:focus {
  position: static;
  width: auto;
  height: auto;
  margin: 0;
  overflow: visible;
  clip: auto;
}
[role="button"] {
  cursor: pointer;
}
h1,
h2,
h3,
h4,
h5,
h6,
.h1,
.h2,
.h3,
.h4,
.h5,
.h6 {
  font-family: inherit;
  font-weight: 500;
  line-height: 1.1;
  color: inherit;
}
h1 small,
h2 small,
h3 small,
h4 small,
h5 small,
h6 small,
.h1 small,
.h2 small,
.h3 small,
.h4 small,
.h5 small,
.h6 small,
h1 .small,
h2 .small,
h3 .small,
h4 .small,
h5 .small,
h6 .small,
.h1 .small,
.h2 .small,
.h3 .small,
.h4 .small,
.h5 .small,
.h6 .small {
  font-weight: normal;
  line-height: 1;
  color: #777777;
}
h1,
.h1,
h2,
.h2,
h3,
.h3 {
  margin-top: 18px;
  margin-bottom: 9px;
}
h1 small,
.h1 small,
h2 small,
.h2 small,
h3 small,
.h3 small,
h1 .small,
.h1 .small,
h2 .small,
.h2 .small,
h3 .small,
.h3 .small {
  font-size: 65%;
}
h4,
.h4,
h5,
.h5,
h6,
.h6 {
  margin-top: 9px;
  margin-bottom: 9px;
}
h4 small,
.h4 small,
h5 small,
.h5 small,
h6 small,
.h6 small,
h4 .small,
.h4 .small,
h5 .small,
.h5 .small,
h6 .small,
.h6 .small {
  font-size: 75%;
}
h1,
.h1 {
  font-size: 33px;
}
h2,
.h2 {
  font-size: 27px;
}
h3,
.h3 {
  font-size: 23px;
}
h4,
.h4 {
  font-size: 17px;
}
h5,
.h5 {
  font-size: 13px;
}
h6,
.h6 {
  font-size: 12px;
}
p {
  margin: 0 0 9px;
}
.lead {
  margin-bottom: 18px;
  font-size: 14px;
  font-weight: 300;
  line-height: 1.4;
}
@media (min-width: 768px) {
  .lead {
    font-size: 19.5px;
  }
}
small,
.small {
  font-size: 92%;
}
mark,
.mark {
  background-color: #fcf8e3;
  padding: .2em;
}
.text-left {
  text-align: left;
}
.text-right {
  text-align: right;
}
.text-center {
  text-align: center;
}
.text-justify {
  text-align: justify;
}
.text-nowrap {
  white-space: nowrap;
}
.text-lowercase {
  text-transform: lowercase;
}
.text-uppercase {
  text-transform: uppercase;
}
.text-capitalize {
  text-transform: capitalize;
}
.text-muted {
  color: #777777;
}
.text-primary {
  color: #337ab7;
}
a.text-primary:hover,
a.text-primary:focus {
  color: #286090;
}
.text-success {
  color: #3c763d;
}
a.text-success:hover,
a.text-success:focus {
  color: #2b542c;
}
.text-info {
  color: #31708f;
}
a.text-info:hover,
a.text-info:focus {
  color: #245269;
}
.text-warning {
  color: #8a6d3b;
}
a.text-warning:hover,
a.text-warning:focus {
  color: #66512c;
}
.text-danger {
  color: #a94442;
}
a.text-danger:hover,
a.text-danger:focus {
  color: #843534;
}
.bg-primary {
  color: #fff;
  background-color: #337ab7;
}
a.bg-primary:hover,
a.bg-primary:focus {
  background-color: #286090;
}
.bg-success {
  background-color: #dff0d8;
}
a.bg-success:hover,
a.bg-success:focus {
  background-color: #c1e2b3;
}
.bg-info {
  background-color: #d9edf7;
}
a.bg-info:hover,
a.bg-info:focus {
  background-color: #afd9ee;
}
.bg-warning {
  background-color: #fcf8e3;
}
a.bg-warning:hover,
a.bg-warning:focus {
  background-color: #f7ecb5;
}
.bg-danger {
  background-color: #f2dede;
}
a.bg-danger:hover,
a.bg-danger:focus {
  background-color: #e4b9b9;
}
.page-header {
  padding-bottom: 8px;
  margin: 36px 0 18px;
  border-bottom: 1px solid #eeeeee;
}
ul,
ol {
  margin-top: 0;
  margin-bottom: 9px;
}
ul ul,
ol ul,
ul ol,
ol ol {
  margin-bottom: 0;
}
.list-unstyled {
  padding-left: 0;
  list-style: none;
}
.list-inline {
  padding-left: 0;
  list-style: none;
  margin-left: -5px;
}
.list-inline > li {
  display: inline-block;
  padding-left: 5px;
  padding-right: 5px;
}
dl {
  margin-top: 0;
  margin-bottom: 18px;
}
dt,
dd {
  line-height: 1.42857143;
}
dt {
  font-weight: bold;
}
dd {
  margin-left: 0;
}
@media (min-width: 541px) {
  .dl-horizontal dt {
    float: left;
    width: 160px;
    clear: left;
    text-align: right;
    overflow: hidden;
    text-overflow: ellipsis;
    white-space: nowrap;
  }
  .dl-horizontal dd {
    margin-left: 180px;
  }
}
abbr[title],
abbr[data-original-title] {
  cursor: help;
  border-bottom: 1px dotted #777777;
}
.initialism {
  font-size: 90%;
  text-transform: uppercase;
}
blockquote {
  padding: 9px 18px;
  margin: 0 0 18px;
  font-size: inherit;
  border-left: 5px solid #eeeeee;
}
blockquote p:last-child,
blockquote ul:last-child,
blockquote ol:last-child {
  margin-bottom: 0;
}
blockquote footer,
blockquote small,
blockquote .small {
  display: block;
  font-size: 80%;
  line-height: 1.42857143;
  color: #777777;
}
blockquote footer:before,
blockquote small:before,
blockquote .small:before {
  content: '\2014 \00A0';
}
.blockquote-reverse,
blockquote.pull-right {
  padding-right: 15px;
  padding-left: 0;
  border-right: 5px solid #eeeeee;
  border-left: 0;
  text-align: right;
}
.blockquote-reverse footer:before,
blockquote.pull-right footer:before,
.blockquote-reverse small:before,
blockquote.pull-right small:before,
.blockquote-reverse .small:before,
blockquote.pull-right .small:before {
  content: '';
}
.blockquote-reverse footer:after,
blockquote.pull-right footer:after,
.blockquote-reverse small:after,
blockquote.pull-right small:after,
.blockquote-reverse .small:after,
blockquote.pull-right .small:after {
  content: '\00A0 \2014';
}
address {
  margin-bottom: 18px;
  font-style: normal;
  line-height: 1.42857143;
}
code,
kbd,
pre,
samp {
  font-family: monospace;
}
code {
  padding: 2px 4px;
  font-size: 90%;
  color: #c7254e;
  background-color: #f9f2f4;
  border-radius: 2px;
}
kbd {
  padding: 2px 4px;
  font-size: 90%;
  color: #888;
  background-color: transparent;
  border-radius: 1px;
  box-shadow: inset 0 -1px 0 rgba(0, 0, 0, 0.25);
}
kbd kbd {
  padding: 0;
  font-size: 100%;
  font-weight: bold;
  box-shadow: none;
}
pre {
  display: block;
  padding: 8.5px;
  margin: 0 0 9px;
  font-size: 12px;
  line-height: 1.42857143;
  word-break: break-all;
  word-wrap: break-word;
  color: #333333;
  background-color: #f5f5f5;
  border: 1px solid #ccc;
  border-radius: 2px;
}
pre code {
  padding: 0;
  font-size: inherit;
  color: inherit;
  white-space: pre-wrap;
  background-color: transparent;
  border-radius: 0;
}
.pre-scrollable {
  max-height: 340px;
  overflow-y: scroll;
}
.container {
  margin-right: auto;
  margin-left: auto;
  padding-left: 0px;
  padding-right: 0px;
}
@media (min-width: 768px) {
  .container {
    width: 768px;
  }
}
@media (min-width: 992px) {
  .container {
    width: 940px;
  }
}
@media (min-width: 1200px) {
  .container {
    width: 1140px;
  }
}
.container-fluid {
  margin-right: auto;
  margin-left: auto;
  padding-left: 0px;
  padding-right: 0px;
}
.row {
  margin-left: 0px;
  margin-right: 0px;
}
.col-xs-1, .col-sm-1, .col-md-1, .col-lg-1, .col-xs-2, .col-sm-2, .col-md-2, .col-lg-2, .col-xs-3, .col-sm-3, .col-md-3, .col-lg-3, .col-xs-4, .col-sm-4, .col-md-4, .col-lg-4, .col-xs-5, .col-sm-5, .col-md-5, .col-lg-5, .col-xs-6, .col-sm-6, .col-md-6, .col-lg-6, .col-xs-7, .col-sm-7, .col-md-7, .col-lg-7, .col-xs-8, .col-sm-8, .col-md-8, .col-lg-8, .col-xs-9, .col-sm-9, .col-md-9, .col-lg-9, .col-xs-10, .col-sm-10, .col-md-10, .col-lg-10, .col-xs-11, .col-sm-11, .col-md-11, .col-lg-11, .col-xs-12, .col-sm-12, .col-md-12, .col-lg-12 {
  position: relative;
  min-height: 1px;
  padding-left: 0px;
  padding-right: 0px;
}
.col-xs-1, .col-xs-2, .col-xs-3, .col-xs-4, .col-xs-5, .col-xs-6, .col-xs-7, .col-xs-8, .col-xs-9, .col-xs-10, .col-xs-11, .col-xs-12 {
  float: left;
}
.col-xs-12 {
  width: 100%;
}
.col-xs-11 {
  width: 91.66666667%;
}
.col-xs-10 {
  width: 83.33333333%;
}
.col-xs-9 {
  width: 75%;
}
.col-xs-8 {
  width: 66.66666667%;
}
.col-xs-7 {
  width: 58.33333333%;
}
.col-xs-6 {
  width: 50%;
}
.col-xs-5 {
  width: 41.66666667%;
}
.col-xs-4 {
  width: 33.33333333%;
}
.col-xs-3 {
  width: 25%;
}
.col-xs-2 {
  width: 16.66666667%;
}
.col-xs-1 {
  width: 8.33333333%;
}
.col-xs-pull-12 {
  right: 100%;
}
.col-xs-pull-11 {
  right: 91.66666667%;
}
.col-xs-pull-10 {
  right: 83.33333333%;
}
.col-xs-pull-9 {
  right: 75%;
}
.col-xs-pull-8 {
  right: 66.66666667%;
}
.col-xs-pull-7 {
  right: 58.33333333%;
}
.col-xs-pull-6 {
  right: 50%;
}
.col-xs-pull-5 {
  right: 41.66666667%;
}
.col-xs-pull-4 {
  right: 33.33333333%;
}
.col-xs-pull-3 {
  right: 25%;
}
.col-xs-pull-2 {
  right: 16.66666667%;
}
.col-xs-pull-1 {
  right: 8.33333333%;
}
.col-xs-pull-0 {
  right: auto;
}
.col-xs-push-12 {
  left: 100%;
}
.col-xs-push-11 {
  left: 91.66666667%;
}
.col-xs-push-10 {
  left: 83.33333333%;
}
.col-xs-push-9 {
  left: 75%;
}
.col-xs-push-8 {
  left: 66.66666667%;
}
.col-xs-push-7 {
  left: 58.33333333%;
}
.col-xs-push-6 {
  left: 50%;
}
.col-xs-push-5 {
  left: 41.66666667%;
}
.col-xs-push-4 {
  left: 33.33333333%;
}
.col-xs-push-3 {
  left: 25%;
}
.col-xs-push-2 {
  left: 16.66666667%;
}
.col-xs-push-1 {
  left: 8.33333333%;
}
.col-xs-push-0 {
  left: auto;
}
.col-xs-offset-12 {
  margin-left: 100%;
}
.col-xs-offset-11 {
  margin-left: 91.66666667%;
}
.col-xs-offset-10 {
  margin-left: 83.33333333%;
}
.col-xs-offset-9 {
  margin-left: 75%;
}
.col-xs-offset-8 {
  margin-left: 66.66666667%;
}
.col-xs-offset-7 {
  margin-left: 58.33333333%;
}
.col-xs-offset-6 {
  margin-left: 50%;
}
.col-xs-offset-5 {
  margin-left: 41.66666667%;
}
.col-xs-offset-4 {
  margin-left: 33.33333333%;
}
.col-xs-offset-3 {
  margin-left: 25%;
}
.col-xs-offset-2 {
  margin-left: 16.66666667%;
}
.col-xs-offset-1 {
  margin-left: 8.33333333%;
}
.col-xs-offset-0 {
  margin-left: 0%;
}
@media (min-width: 768px) {
  .col-sm-1, .col-sm-2, .col-sm-3, .col-sm-4, .col-sm-5, .col-sm-6, .col-sm-7, .col-sm-8, .col-sm-9, .col-sm-10, .col-sm-11, .col-sm-12 {
    float: left;
  }
  .col-sm-12 {
    width: 100%;
  }
  .col-sm-11 {
    width: 91.66666667%;
  }
  .col-sm-10 {
    width: 83.33333333%;
  }
  .col-sm-9 {
    width: 75%;
  }
  .col-sm-8 {
    width: 66.66666667%;
  }
  .col-sm-7 {
    width: 58.33333333%;
  }
  .col-sm-6 {
    width: 50%;
  }
  .col-sm-5 {
    width: 41.66666667%;
  }
  .col-sm-4 {
    width: 33.33333333%;
  }
  .col-sm-3 {
    width: 25%;
  }
  .col-sm-2 {
    width: 16.66666667%;
  }
  .col-sm-1 {
    width: 8.33333333%;
  }
  .col-sm-pull-12 {
    right: 100%;
  }
  .col-sm-pull-11 {
    right: 91.66666667%;
  }
  .col-sm-pull-10 {
    right: 83.33333333%;
  }
  .col-sm-pull-9 {
    right: 75%;
  }
  .col-sm-pull-8 {
    right: 66.66666667%;
  }
  .col-sm-pull-7 {
    right: 58.33333333%;
  }
  .col-sm-pull-6 {
    right: 50%;
  }
  .col-sm-pull-5 {
    right: 41.66666667%;
  }
  .col-sm-pull-4 {
    right: 33.33333333%;
  }
  .col-sm-pull-3 {
    right: 25%;
  }
  .col-sm-pull-2 {
    right: 16.66666667%;
  }
  .col-sm-pull-1 {
    right: 8.33333333%;
  }
  .col-sm-pull-0 {
    right: auto;
  }
  .col-sm-push-12 {
    left: 100%;
  }
  .col-sm-push-11 {
    left: 91.66666667%;
  }
  .col-sm-push-10 {
    left: 83.33333333%;
  }
  .col-sm-push-9 {
    left: 75%;
  }
  .col-sm-push-8 {
    left: 66.66666667%;
  }
  .col-sm-push-7 {
    left: 58.33333333%;
  }
  .col-sm-push-6 {
    left: 50%;
  }
  .col-sm-push-5 {
    left: 41.66666667%;
  }
  .col-sm-push-4 {
    left: 33.33333333%;
  }
  .col-sm-push-3 {
    left: 25%;
  }
  .col-sm-push-2 {
    left: 16.66666667%;
  }
  .col-sm-push-1 {
    left: 8.33333333%;
  }
  .col-sm-push-0 {
    left: auto;
  }
  .col-sm-offset-12 {
    margin-left: 100%;
  }
  .col-sm-offset-11 {
    margin-left: 91.66666667%;
  }
  .col-sm-offset-10 {
    margin-left: 83.33333333%;
  }
  .col-sm-offset-9 {
    margin-left: 75%;
  }
  .col-sm-offset-8 {
    margin-left: 66.66666667%;
  }
  .col-sm-offset-7 {
    margin-left: 58.33333333%;
  }
  .col-sm-offset-6 {
    margin-left: 50%;
  }
  .col-sm-offset-5 {
    margin-left: 41.66666667%;
  }
  .col-sm-offset-4 {
    margin-left: 33.33333333%;
  }
  .col-sm-offset-3 {
    margin-left: 25%;
  }
  .col-sm-offset-2 {
    margin-left: 16.66666667%;
  }
  .col-sm-offset-1 {
    margin-left: 8.33333333%;
  }
  .col-sm-offset-0 {
    margin-left: 0%;
  }
}
@media (min-width: 992px) {
  .col-md-1, .col-md-2, .col-md-3, .col-md-4, .col-md-5, .col-md-6, .col-md-7, .col-md-8, .col-md-9, .col-md-10, .col-md-11, .col-md-12 {
    float: left;
  }
  .col-md-12 {
    width: 100%;
  }
  .col-md-11 {
    width: 91.66666667%;
  }
  .col-md-10 {
    width: 83.33333333%;
  }
  .col-md-9 {
    width: 75%;
  }
  .col-md-8 {
    width: 66.66666667%;
  }
  .col-md-7 {
    width: 58.33333333%;
  }
  .col-md-6 {
    width: 50%;
  }
  .col-md-5 {
    width: 41.66666667%;
  }
  .col-md-4 {
    width: 33.33333333%;
  }
  .col-md-3 {
    width: 25%;
  }
  .col-md-2 {
    width: 16.66666667%;
  }
  .col-md-1 {
    width: 8.33333333%;
  }
  .col-md-pull-12 {
    right: 100%;
  }
  .col-md-pull-11 {
    right: 91.66666667%;
  }
  .col-md-pull-10 {
    right: 83.33333333%;
  }
  .col-md-pull-9 {
    right: 75%;
  }
  .col-md-pull-8 {
    right: 66.66666667%;
  }
  .col-md-pull-7 {
    right: 58.33333333%;
  }
  .col-md-pull-6 {
    right: 50%;
  }
  .col-md-pull-5 {
    right: 41.66666667%;
  }
  .col-md-pull-4 {
    right: 33.33333333%;
  }
  .col-md-pull-3 {
    right: 25%;
  }
  .col-md-pull-2 {
    right: 16.66666667%;
  }
  .col-md-pull-1 {
    right: 8.33333333%;
  }
  .col-md-pull-0 {
    right: auto;
  }
  .col-md-push-12 {
    left: 100%;
  }
  .col-md-push-11 {
    left: 91.66666667%;
  }
  .col-md-push-10 {
    left: 83.33333333%;
  }
  .col-md-push-9 {
    left: 75%;
  }
  .col-md-push-8 {
    left: 66.66666667%;
  }
  .col-md-push-7 {
    left: 58.33333333%;
  }
  .col-md-push-6 {
    left: 50%;
  }
  .col-md-push-5 {
    left: 41.66666667%;
  }
  .col-md-push-4 {
    left: 33.33333333%;
  }
  .col-md-push-3 {
    left: 25%;
  }
  .col-md-push-2 {
    left: 16.66666667%;
  }
  .col-md-push-1 {
    left: 8.33333333%;
  }
  .col-md-push-0 {
    left: auto;
  }
  .col-md-offset-12 {
    margin-left: 100%;
  }
  .col-md-offset-11 {
    margin-left: 91.66666667%;
  }
  .col-md-offset-10 {
    margin-left: 83.33333333%;
  }
  .col-md-offset-9 {
    margin-left: 75%;
  }
  .col-md-offset-8 {
    margin-left: 66.66666667%;
  }
  .col-md-offset-7 {
    margin-left: 58.33333333%;
  }
  .col-md-offset-6 {
    margin-left: 50%;
  }
  .col-md-offset-5 {
    margin-left: 41.66666667%;
  }
  .col-md-offset-4 {
    margin-left: 33.33333333%;
  }
  .col-md-offset-3 {
    margin-left: 25%;
  }
  .col-md-offset-2 {
    margin-left: 16.66666667%;
  }
  .col-md-offset-1 {
    margin-left: 8.33333333%;
  }
  .col-md-offset-0 {
    margin-left: 0%;
  }
}
@media (min-width: 1200px) {
  .col-lg-1, .col-lg-2, .col-lg-3, .col-lg-4, .col-lg-5, .col-lg-6, .col-lg-7, .col-lg-8, .col-lg-9, .col-lg-10, .col-lg-11, .col-lg-12 {
    float: left;
  }
  .col-lg-12 {
    width: 100%;
  }
  .col-lg-11 {
    width: 91.66666667%;
  }
  .col-lg-10 {
    width: 83.33333333%;
  }
  .col-lg-9 {
    width: 75%;
  }
  .col-lg-8 {
    width: 66.66666667%;
  }
  .col-lg-7 {
    width: 58.33333333%;
  }
  .col-lg-6 {
    width: 50%;
  }
  .col-lg-5 {
    width: 41.66666667%;
  }
  .col-lg-4 {
    width: 33.33333333%;
  }
  .col-lg-3 {
    width: 25%;
  }
  .col-lg-2 {
    width: 16.66666667%;
  }
  .col-lg-1 {
    width: 8.33333333%;
  }
  .col-lg-pull-12 {
    right: 100%;
  }
  .col-lg-pull-11 {
    right: 91.66666667%;
  }
  .col-lg-pull-10 {
    right: 83.33333333%;
  }
  .col-lg-pull-9 {
    right: 75%;
  }
  .col-lg-pull-8 {
    right: 66.66666667%;
  }
  .col-lg-pull-7 {
    right: 58.33333333%;
  }
  .col-lg-pull-6 {
    right: 50%;
  }
  .col-lg-pull-5 {
    right: 41.66666667%;
  }
  .col-lg-pull-4 {
    right: 33.33333333%;
  }
  .col-lg-pull-3 {
    right: 25%;
  }
  .col-lg-pull-2 {
    right: 16.66666667%;
  }
  .col-lg-pull-1 {
    right: 8.33333333%;
  }
  .col-lg-pull-0 {
    right: auto;
  }
  .col-lg-push-12 {
    left: 100%;
  }
  .col-lg-push-11 {
    left: 91.66666667%;
  }
  .col-lg-push-10 {
    left: 83.33333333%;
  }
  .col-lg-push-9 {
    left: 75%;
  }
  .col-lg-push-8 {
    left: 66.66666667%;
  }
  .col-lg-push-7 {
    left: 58.33333333%;
  }
  .col-lg-push-6 {
    left: 50%;
  }
  .col-lg-push-5 {
    left: 41.66666667%;
  }
  .col-lg-push-4 {
    left: 33.33333333%;
  }
  .col-lg-push-3 {
    left: 25%;
  }
  .col-lg-push-2 {
    left: 16.66666667%;
  }
  .col-lg-push-1 {
    left: 8.33333333%;
  }
  .col-lg-push-0 {
    left: auto;
  }
  .col-lg-offset-12 {
    margin-left: 100%;
  }
  .col-lg-offset-11 {
    margin-left: 91.66666667%;
  }
  .col-lg-offset-10 {
    margin-left: 83.33333333%;
  }
  .col-lg-offset-9 {
    margin-left: 75%;
  }
  .col-lg-offset-8 {
    margin-left: 66.66666667%;
  }
  .col-lg-offset-7 {
    margin-left: 58.33333333%;
  }
  .col-lg-offset-6 {
    margin-left: 50%;
  }
  .col-lg-offset-5 {
    margin-left: 41.66666667%;
  }
  .col-lg-offset-4 {
    margin-left: 33.33333333%;
  }
  .col-lg-offset-3 {
    margin-left: 25%;
  }
  .col-lg-offset-2 {
    margin-left: 16.66666667%;
  }
  .col-lg-offset-1 {
    margin-left: 8.33333333%;
  }
  .col-lg-offset-0 {
    margin-left: 0%;
  }
}
table {
  background-color: transparent;
}
caption {
  padding-top: 8px;
  padding-bottom: 8px;
  color: #777777;
  text-align: left;
}
th {
  text-align: left;
}
.table {
  width: 100%;
  max-width: 100%;
  margin-bottom: 18px;
}
.table > thead > tr > th,
.table > tbody > tr > th,
.table > tfoot > tr > th,
.table > thead > tr > td,
.table > tbody > tr > td,
.table > tfoot > tr > td {
  padding: 8px;
  line-height: 1.42857143;
  vertical-align: top;
  border-top: 1px solid #ddd;
}
.table > thead > tr > th {
  vertical-align: bottom;
  border-bottom: 2px solid #ddd;
}
.table > caption + thead > tr:first-child > th,
.table > colgroup + thead > tr:first-child > th,
.table > thead:first-child > tr:first-child > th,
.table > caption + thead > tr:first-child > td,
.table > colgroup + thead > tr:first-child > td,
.table > thead:first-child > tr:first-child > td {
  border-top: 0;
}
.table > tbody + tbody {
  border-top: 2px solid #ddd;
}
.table .table {
  background-color: #fff;
}
.table-condensed > thead > tr > th,
.table-condensed > tbody > tr > th,
.table-condensed > tfoot > tr > th,
.table-condensed > thead > tr > td,
.table-condensed > tbody > tr > td,
.table-condensed > tfoot > tr > td {
  padding: 5px;
}
.table-bordered {
  border: 1px solid #ddd;
}
.table-bordered > thead > tr > th,
.table-bordered > tbody > tr > th,
.table-bordered > tfoot > tr > th,
.table-bordered > thead > tr > td,
.table-bordered > tbody > tr > td,
.table-bordered > tfoot > tr > td {
  border: 1px solid #ddd;
}
.table-bordered > thead > tr > th,
.table-bordered > thead > tr > td {
  border-bottom-width: 2px;
}
.table-striped > tbody > tr:nth-of-type(odd) {
  background-color: #f9f9f9;
}
.table-hover > tbody > tr:hover {
  background-color: #f5f5f5;
}
table col[class*="col-"] {
  position: static;
  float: none;
  display: table-column;
}
table td[class*="col-"],
table th[class*="col-"] {
  position: static;
  float: none;
  display: table-cell;
}
.table > thead > tr > td.active,
.table > tbody > tr > td.active,
.table > tfoot > tr > td.active,
.table > thead > tr > th.active,
.table > tbody > tr > th.active,
.table > tfoot > tr > th.active,
.table > thead > tr.active > td,
.table > tbody > tr.active > td,
.table > tfoot > tr.active > td,
.table > thead > tr.active > th,
.table > tbody > tr.active > th,
.table > tfoot > tr.active > th {
  background-color: #f5f5f5;
}
.table-hover > tbody > tr > td.active:hover,
.table-hover > tbody > tr > th.active:hover,
.table-hover > tbody > tr.active:hover > td,
.table-hover > tbody > tr:hover > .active,
.table-hover > tbody > tr.active:hover > th {
  background-color: #e8e8e8;
}
.table > thead > tr > td.success,
.table > tbody > tr > td.success,
.table > tfoot > tr > td.success,
.table > thead > tr > th.success,
.table > tbody > tr > th.success,
.table > tfoot > tr > th.success,
.table > thead > tr.success > td,
.table > tbody > tr.success > td,
.table > tfoot > tr.success > td,
.table > thead > tr.success > th,
.table > tbody > tr.success > th,
.table > tfoot > tr.success > th {
  background-color: #dff0d8;
}
.table-hover > tbody > tr > td.success:hover,
.table-hover > tbody > tr > th.success:hover,
.table-hover > tbody > tr.success:hover > td,
.table-hover > tbody > tr:hover > .success,
.table-hover > tbody > tr.success:hover > th {
  background-color: #d0e9c6;
}
.table > thead > tr > td.info,
.table > tbody > tr > td.info,
.table > tfoot > tr > td.info,
.table > thead > tr > th.info,
.table > tbody > tr > th.info,
.table > tfoot > tr > th.info,
.table > thead > tr.info > td,
.table > tbody > tr.info > td,
.table > tfoot > tr.info > td,
.table > thead > tr.info > th,
.table > tbody > tr.info > th,
.table > tfoot > tr.info > th {
  background-color: #d9edf7;
}
.table-hover > tbody > tr > td.info:hover,
.table-hover > tbody > tr > th.info:hover,
.table-hover > tbody > tr.info:hover > td,
.table-hover > tbody > tr:hover > .info,
.table-hover > tbody > tr.info:hover > th {
  background-color: #c4e3f3;
}
.table > thead > tr > td.warning,
.table > tbody > tr > td.warning,
.table > tfoot > tr > td.warning,
.table > thead > tr > th.warning,
.table > tbody > tr > th.warning,
.table > tfoot > tr > th.warning,
.table > thead > tr.warning > td,
.table > tbody > tr.warning > td,
.table > tfoot > tr.warning > td,
.table > thead > tr.warning > th,
.table > tbody > tr.warning > th,
.table > tfoot > tr.warning > th {
  background-color: #fcf8e3;
}
.table-hover > tbody > tr > td.warning:hover,
.table-hover > tbody > tr > th.warning:hover,
.table-hover > tbody > tr.warning:hover > td,
.table-hover > tbody > tr:hover > .warning,
.table-hover > tbody > tr.warning:hover > th {
  background-color: #faf2cc;
}
.table > thead > tr > td.danger,
.table > tbody > tr > td.danger,
.table > tfoot > tr > td.danger,
.table > thead > tr > th.danger,
.table > tbody > tr > th.danger,
.table > tfoot > tr > th.danger,
.table > thead > tr.danger > td,
.table > tbody > tr.danger > td,
.table > tfoot > tr.danger > td,
.table > thead > tr.danger > th,
.table > tbody > tr.danger > th,
.table > tfoot > tr.danger > th {
  background-color: #f2dede;
}
.table-hover > tbody > tr > td.danger:hover,
.table-hover > tbody > tr > th.danger:hover,
.table-hover > tbody > tr.danger:hover > td,
.table-hover > tbody > tr:hover > .danger,
.table-hover > tbody > tr.danger:hover > th {
  background-color: #ebcccc;
}
.table-responsive {
  overflow-x: auto;
  min-height: 0.01%;
}
@media screen and (max-width: 767px) {
  .table-responsive {
    width: 100%;
    margin-bottom: 13.5px;
    overflow-y: hidden;
    -ms-overflow-style: -ms-autohiding-scrollbar;
    border: 1px solid #ddd;
  }
  .table-responsive > .table {
    margin-bottom: 0;
  }
  .table-responsive > .table > thead > tr > th,
  .table-responsive > .table > tbody > tr > th,
  .table-responsive > .table > tfoot > tr > th,
  .table-responsive > .table > thead > tr > td,
  .table-responsive > .table > tbody > tr > td,
  .table-responsive > .table > tfoot > tr > td {
    white-space: nowrap;
  }
  .table-responsive > .table-bordered {
    border: 0;
  }
  .table-responsive > .table-bordered > thead > tr > th:first-child,
  .table-responsive > .table-bordered > tbody > tr > th:first-child,
  .table-responsive > .table-bordered > tfoot > tr > th:first-child,
  .table-responsive > .table-bordered > thead > tr > td:first-child,
  .table-responsive > .table-bordered > tbody > tr > td:first-child,
  .table-responsive > .table-bordered > tfoot > tr > td:first-child {
    border-left: 0;
  }
  .table-responsive > .table-bordered > thead > tr > th:last-child,
  .table-responsive > .table-bordered > tbody > tr > th:last-child,
  .table-responsive > .table-bordered > tfoot > tr > th:last-child,
  .table-responsive > .table-bordered > thead > tr > td:last-child,
  .table-responsive > .table-bordered > tbody > tr > td:last-child,
  .table-responsive > .table-bordered > tfoot > tr > td:last-child {
    border-right: 0;
  }
  .table-responsive > .table-bordered > tbody > tr:last-child > th,
  .table-responsive > .table-bordered > tfoot > tr:last-child > th,
  .table-responsive > .table-bordered > tbody > tr:last-child > td,
  .table-responsive > .table-bordered > tfoot > tr:last-child > td {
    border-bottom: 0;
  }
}
fieldset {
  padding: 0;
  margin: 0;
  border: 0;
  min-width: 0;
}
legend {
  display: block;
  width: 100%;
  padding: 0;
  margin-bottom: 18px;
  font-size: 19.5px;
  line-height: inherit;
  color: #333333;
  border: 0;
  border-bottom: 1px solid #e5e5e5;
}
label {
  display: inline-block;
  max-width: 100%;
  margin-bottom: 5px;
  font-weight: bold;
}
input[type="search"] {
  -webkit-box-sizing: border-box;
  -moz-box-sizing: border-box;
  box-sizing: border-box;
}
input[type="radio"],
input[type="checkbox"] {
  margin: 4px 0 0;
  margin-top: 1px \9;
  line-height: normal;
}
input[type="file"] {
  display: block;
}
input[type="range"] {
  display: block;
  width: 100%;
}
select[multiple],
select[size] {
  height: auto;
}
input[type="file"]:focus,
input[type="radio"]:focus,
input[type="checkbox"]:focus {
  outline: 5px auto -webkit-focus-ring-color;
  outline-offset: -2px;
}
output {
  display: block;
  padding-top: 7px;
  font-size: 13px;
  line-height: 1.42857143;
  color: #555555;
}
.form-control {
  display: block;
  width: 100%;
  height: 32px;
  padding: 6px 12px;
  font-size: 13px;
  line-height: 1.42857143;
  color: #555555;
  background-color: #fff;
  background-image: none;
  border: 1px solid #ccc;
  border-radius: 2px;
  -webkit-box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075);
  box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075);
  -webkit-transition: border-color ease-in-out .15s, box-shadow ease-in-out .15s;
  -o-transition: border-color ease-in-out .15s, box-shadow ease-in-out .15s;
  transition: border-color ease-in-out .15s, box-shadow ease-in-out .15s;
}
.form-control:focus {
  border-color: #66afe9;
  outline: 0;
  -webkit-box-shadow: inset 0 1px 1px rgba(0,0,0,.075), 0 0 8px rgba(102, 175, 233, 0.6);
  box-shadow: inset 0 1px 1px rgba(0,0,0,.075), 0 0 8px rgba(102, 175, 233, 0.6);
}
.form-control::-moz-placeholder {
  color: #999;
  opacity: 1;
}
.form-control:-ms-input-placeholder {
  color: #999;
}
.form-control::-webkit-input-placeholder {
  color: #999;
}
.form-control::-ms-expand {
  border: 0;
  background-color: transparent;
}
.form-control[disabled],
.form-control[readonly],
fieldset[disabled] .form-control {
  background-color: #eeeeee;
  opacity: 1;
}
.form-control[disabled],
fieldset[disabled] .form-control {
  cursor: not-allowed;
}
textarea.form-control {
  height: auto;
}
input[type="search"] {
  -webkit-appearance: none;
}
@media screen and (-webkit-min-device-pixel-ratio: 0) {
  input[type="date"].form-control,
  input[type="time"].form-control,
  input[type="datetime-local"].form-control,
  input[type="month"].form-control {
    line-height: 32px;
  }
  input[type="date"].input-sm,
  input[type="time"].input-sm,
  input[type="datetime-local"].input-sm,
  input[type="month"].input-sm,
  .input-group-sm input[type="date"],
  .input-group-sm input[type="time"],
  .input-group-sm input[type="datetime-local"],
  .input-group-sm input[type="month"] {
    line-height: 30px;
  }
  input[type="date"].input-lg,
  input[type="time"].input-lg,
  input[type="datetime-local"].input-lg,
  input[type="month"].input-lg,
  .input-group-lg input[type="date"],
  .input-group-lg input[type="time"],
  .input-group-lg input[type="datetime-local"],
  .input-group-lg input[type="month"] {
    line-height: 45px;
  }
}
.form-group {
  margin-bottom: 15px;
}
.radio,
.checkbox {
  position: relative;
  display: block;
  margin-top: 10px;
  margin-bottom: 10px;
}
.radio label,
.checkbox label {
  min-height: 18px;
  padding-left: 20px;
  margin-bottom: 0;
  font-weight: normal;
  cursor: pointer;
}
.radio input[type="radio"],
.radio-inline input[type="radio"],
.checkbox input[type="checkbox"],
.checkbox-inline input[type="checkbox"] {
  position: absolute;
  margin-left: -20px;
  margin-top: 4px \9;
}
.radio + .radio,
.checkbox + .checkbox {
  margin-top: -5px;
}
.radio-inline,
.checkbox-inline {
  position: relative;
  display: inline-block;
  padding-left: 20px;
  margin-bottom: 0;
  vertical-align: middle;
  font-weight: normal;
  cursor: pointer;
}
.radio-inline + .radio-inline,
.checkbox-inline + .checkbox-inline {
  margin-top: 0;
  margin-left: 10px;
}
input[type="radio"][disabled],
input[type="checkbox"][disabled],
input[type="radio"].disabled,
input[type="checkbox"].disabled,
fieldset[disabled] input[type="radio"],
fieldset[disabled] input[type="checkbox"] {
  cursor: not-allowed;
}
.radio-inline.disabled,
.checkbox-inline.disabled,
fieldset[disabled] .radio-inline,
fieldset[disabled] .checkbox-inline {
  cursor: not-allowed;
}
.radio.disabled label,
.checkbox.disabled label,
fieldset[disabled] .radio label,
fieldset[disabled] .checkbox label {
  cursor: not-allowed;
}
.form-control-static {
  padding-top: 7px;
  padding-bottom: 7px;
  margin-bottom: 0;
  min-height: 31px;
}
.form-control-static.input-lg,
.form-control-static.input-sm {
  padding-left: 0;
  padding-right: 0;
}
.input-sm {
  height: 30px;
  padding: 5px 10px;
  font-size: 12px;
  line-height: 1.5;
  border-radius: 1px;
}
select.input-sm {
  height: 30px;
  line-height: 30px;
}
textarea.input-sm,
select[multiple].input-sm {
  height: auto;
}
.form-group-sm .form-control {
  height: 30px;
  padding: 5px 10px;
  font-size: 12px;
  line-height: 1.5;
  border-radius: 1px;
}
.form-group-sm select.form-control {
  height: 30px;
  line-height: 30px;
}
.form-group-sm textarea.form-control,
.form-group-sm select[multiple].form-control {
  height: auto;
}
.form-group-sm .form-control-static {
  height: 30px;
  min-height: 30px;
  padding: 6px 10px;
  font-size: 12px;
  line-height: 1.5;
}
.input-lg {
  height: 45px;
  padding: 10px 16px;
  font-size: 17px;
  line-height: 1.3333333;
  border-radius: 3px;
}
select.input-lg {
  height: 45px;
  line-height: 45px;
}
textarea.input-lg,
select[multiple].input-lg {
  height: auto;
}
.form-group-lg .form-control {
  height: 45px;
  padding: 10px 16px;
  font-size: 17px;
  line-height: 1.3333333;
  border-radius: 3px;
}
.form-group-lg select.form-control {
  height: 45px;
  line-height: 45px;
}
.form-group-lg textarea.form-control,
.form-group-lg select[multiple].form-control {
  height: auto;
}
.form-group-lg .form-control-static {
  height: 45px;
  min-height: 35px;
  padding: 11px 16px;
  font-size: 17px;
  line-height: 1.3333333;
}
.has-feedback {
  position: relative;
}
.has-feedback .form-control {
  padding-right: 40px;
}
.form-control-feedback {
  position: absolute;
  top: 0;
  right: 0;
  z-index: 2;
  display: block;
  width: 32px;
  height: 32px;
  line-height: 32px;
  text-align: center;
  pointer-events: none;
}
.input-lg + .form-control-feedback,
.input-group-lg + .form-control-feedback,
.form-group-lg .form-control + .form-control-feedback {
  width: 45px;
  height: 45px;
  line-height: 45px;
}
.input-sm + .form-control-feedback,
.input-group-sm + .form-control-feedback,
.form-group-sm .form-control + .form-control-feedback {
  width: 30px;
  height: 30px;
  line-height: 30px;
}
.has-success .help-block,
.has-success .control-label,
.has-success .radio,
.has-success .checkbox,
.has-success .radio-inline,
.has-success .checkbox-inline,
.has-success.radio label,
.has-success.checkbox label,
.has-success.radio-inline label,
.has-success.checkbox-inline label {
  color: #3c763d;
}
.has-success .form-control {
  border-color: #3c763d;
  -webkit-box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075);
  box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075);
}
.has-success .form-control:focus {
  border-color: #2b542c;
  -webkit-box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075), 0 0 6px #67b168;
  box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075), 0 0 6px #67b168;
}
.has-success .input-group-addon {
  color: #3c763d;
  border-color: #3c763d;
  background-color: #dff0d8;
}
.has-success .form-control-feedback {
  color: #3c763d;
}
.has-warning .help-block,
.has-warning .control-label,
.has-warning .radio,
.has-warning .checkbox,
.has-warning .radio-inline,
.has-warning .checkbox-inline,
.has-warning.radio label,
.has-warning.checkbox label,
.has-warning.radio-inline label,
.has-warning.checkbox-inline label {
  color: #8a6d3b;
}
.has-warning .form-control {
  border-color: #8a6d3b;
  -webkit-box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075);
  box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075);
}
.has-warning .form-control:focus {
  border-color: #66512c;
  -webkit-box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075), 0 0 6px #c0a16b;
  box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075), 0 0 6px #c0a16b;
}
.has-warning .input-group-addon {
  color: #8a6d3b;
  border-color: #8a6d3b;
  background-color: #fcf8e3;
}
.has-warning .form-control-feedback {
  color: #8a6d3b;
}
.has-error .help-block,
.has-error .control-label,
.has-error .radio,
.has-error .checkbox,
.has-error .radio-inline,
.has-error .checkbox-inline,
.has-error.radio label,
.has-error.checkbox label,
.has-error.radio-inline label,
.has-error.checkbox-inline label {
  color: #a94442;
}
.has-error .form-control {
  border-color: #a94442;
  -webkit-box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075);
  box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075);
}
.has-error .form-control:focus {
  border-color: #843534;
  -webkit-box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075), 0 0 6px #ce8483;
  box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075), 0 0 6px #ce8483;
}
.has-error .input-group-addon {
  color: #a94442;
  border-color: #a94442;
  background-color: #f2dede;
}
.has-error .form-control-feedback {
  color: #a94442;
}
.has-feedback label ~ .form-control-feedback {
  top: 23px;
}
.has-feedback label.sr-only ~ .form-control-feedback {
  top: 0;
}
.help-block {
  display: block;
  margin-top: 5px;
  margin-bottom: 10px;
  color: #404040;
}
@media (min-width: 768px) {
  .form-inline .form-group {
    display: inline-block;
    margin-bottom: 0;
    vertical-align: middle;
  }
  .form-inline .form-control {
    display: inline-block;
    width: auto;
    vertical-align: middle;
  }
  .form-inline .form-control-static {
    display: inline-block;
  }
  .form-inline .input-group {
    display: inline-table;
    vertical-align: middle;
  }
  .form-inline .input-group .input-group-addon,
  .form-inline .input-group .input-group-btn,
  .form-inline .input-group .form-control {
    width: auto;
  }
  .form-inline .input-group > .form-control {
    width: 100%;
  }
  .form-inline .control-label {
    margin-bottom: 0;
    vertical-align: middle;
  }
  .form-inline .radio,
  .form-inline .checkbox {
    display: inline-block;
    margin-top: 0;
    margin-bottom: 0;
    vertical-align: middle;
  }
  .form-inline .radio label,
  .form-inline .checkbox label {
    padding-left: 0;
  }
  .form-inline .radio input[type="radio"],
  .form-inline .checkbox input[type="checkbox"] {
    position: relative;
    margin-left: 0;
  }
  .form-inline .has-feedback .form-control-feedback {
    top: 0;
  }
}
.form-horizontal .radio,
.form-horizontal .checkbox,
.form-horizontal .radio-inline,
.form-horizontal .checkbox-inline {
  margin-top: 0;
  margin-bottom: 0;
  padding-top: 7px;
}
.form-horizontal .radio,
.form-horizontal .checkbox {
  min-height: 25px;
}
.form-horizontal .form-group {
  margin-left: 0px;
  margin-right: 0px;
}
@media (min-width: 768px) {
  .form-horizontal .control-label {
    text-align: right;
    margin-bottom: 0;
    padding-top: 7px;
  }
}
.form-horizontal .has-feedback .form-control-feedback {
  right: 0px;
}
@media (min-width: 768px) {
  .form-horizontal .form-group-lg .control-label {
    padding-top: 11px;
    font-size: 17px;
  }
}
@media (min-width: 768px) {
  .form-horizontal .form-group-sm .control-label {
    padding-top: 6px;
    font-size: 12px;
  }
}
.btn {
  display: inline-block;
  margin-bottom: 0;
  font-weight: normal;
  text-align: center;
  vertical-align: middle;
  touch-action: manipulation;
  cursor: pointer;
  background-image: none;
  border: 1px solid transparent;
  white-space: nowrap;
  padding: 6px 12px;
  font-size: 13px;
  line-height: 1.42857143;
  border-radius: 2px;
  -webkit-user-select: none;
  -moz-user-select: none;
  -ms-user-select: none;
  user-select: none;
}
.btn:focus,
.btn:active:focus,
.btn.active:focus,
.btn.focus,
.btn:active.focus,
.btn.active.focus {
  outline: 5px auto -webkit-focus-ring-color;
  outline-offset: -2px;
}
.btn:hover,
.btn:focus,
.btn.focus {
  color: #333;
  text-decoration: none;
}
.btn:active,
.btn.active {
  outline: 0;
  background-image: none;
  -webkit-box-shadow: inset 0 3px 5px rgba(0, 0, 0, 0.125);
  box-shadow: inset 0 3px 5px rgba(0, 0, 0, 0.125);
}
.btn.disabled,
.btn[disabled],
fieldset[disabled] .btn {
  cursor: not-allowed;
  opacity: 0.65;
  filter: alpha(opacity=65);
  -webkit-box-shadow: none;
  box-shadow: none;
}
a.btn.disabled,
fieldset[disabled] a.btn {
  pointer-events: none;
}
.btn-default {
  color: #333;
  background-color: #fff;
  border-color: #ccc;
}
.btn-default:focus,
.btn-default.focus {
  color: #333;
  background-color: #e6e6e6;
  border-color: #8c8c8c;
}
.btn-default:hover {
  color: #333;
  background-color: #e6e6e6;
  border-color: #adadad;
}
.btn-default:active,
.btn-default.active,
.open > .dropdown-toggle.btn-default {
  color: #333;
  background-color: #e6e6e6;
  border-color: #adadad;
}
.btn-default:active:hover,
.btn-default.active:hover,
.open > .dropdown-toggle.btn-default:hover,
.btn-default:active:focus,
.btn-default.active:focus,
.open > .dropdown-toggle.btn-default:focus,
.btn-default:active.focus,
.btn-default.active.focus,
.open > .dropdown-toggle.btn-default.focus {
  color: #333;
  background-color: #d4d4d4;
  border-color: #8c8c8c;
}
.btn-default:active,
.btn-default.active,
.open > .dropdown-toggle.btn-default {
  background-image: none;
}
.btn-default.disabled:hover,
.btn-default[disabled]:hover,
fieldset[disabled] .btn-default:hover,
.btn-default.disabled:focus,
.btn-default[disabled]:focus,
fieldset[disabled] .btn-default:focus,
.btn-default.disabled.focus,
.btn-default[disabled].focus,
fieldset[disabled] .btn-default.focus {
  background-color: #fff;
  border-color: #ccc;
}
.btn-default .badge {
  color: #fff;
  background-color: #333;
}
.btn-primary {
  color: #fff;
  background-color: #337ab7;
  border-color: #2e6da4;
}
.btn-primary:focus,
.btn-primary.focus {
  color: #fff;
  background-color: #286090;
  border-color: #122b40;
}
.btn-primary:hover {
  color: #fff;
  background-color: #286090;
  border-color: #204d74;
}
.btn-primary:active,
.btn-primary.active,
.open > .dropdown-toggle.btn-primary {
  color: #fff;
  background-color: #286090;
  border-color: #204d74;
}
.btn-primary:active:hover,
.btn-primary.active:hover,
.open > .dropdown-toggle.btn-primary:hover,
.btn-primary:active:focus,
.btn-primary.active:focus,
.open > .dropdown-toggle.btn-primary:focus,
.btn-primary:active.focus,
.btn-primary.active.focus,
.open > .dropdown-toggle.btn-primary.focus {
  color: #fff;
  background-color: #204d74;
  border-color: #122b40;
}
.btn-primary:active,
.btn-primary.active,
.open > .dropdown-toggle.btn-primary {
  background-image: none;
}
.btn-primary.disabled:hover,
.btn-primary[disabled]:hover,
fieldset[disabled] .btn-primary:hover,
.btn-primary.disabled:focus,
.btn-primary[disabled]:focus,
fieldset[disabled] .btn-primary:focus,
.btn-primary.disabled.focus,
.btn-primary[disabled].focus,
fieldset[disabled] .btn-primary.focus {
  background-color: #337ab7;
  border-color: #2e6da4;
}
.btn-primary .badge {
  color: #337ab7;
  background-color: #fff;
}
.btn-success {
  color: #fff;
  background-color: #5cb85c;
  border-color: #4cae4c;
}
.btn-success:focus,
.btn-success.focus {
  color: #fff;
  background-color: #449d44;
  border-color: #255625;
}
.btn-success:hover {
  color: #fff;
  background-color: #449d44;
  border-color: #398439;
}
.btn-success:active,
.btn-success.active,
.open > .dropdown-toggle.btn-success {
  color: #fff;
  background-color: #449d44;
  border-color: #398439;
}
.btn-success:active:hover,
.btn-success.active:hover,
.open > .dropdown-toggle.btn-success:hover,
.btn-success:active:focus,
.btn-success.active:focus,
.open > .dropdown-toggle.btn-success:focus,
.btn-success:active.focus,
.btn-success.active.focus,
.open > .dropdown-toggle.btn-success.focus {
  color: #fff;
  background-color: #398439;
  border-color: #255625;
}
.btn-success:active,
.btn-success.active,
.open > .dropdown-toggle.btn-success {
  background-image: none;
}
.btn-success.disabled:hover,
.btn-success[disabled]:hover,
fieldset[disabled] .btn-success:hover,
.btn-success.disabled:focus,
.btn-success[disabled]:focus,
fieldset[disabled] .btn-success:focus,
.btn-success.disabled.focus,
.btn-success[disabled].focus,
fieldset[disabled] .btn-success.focus {
  background-color: #5cb85c;
  border-color: #4cae4c;
}
.btn-success .badge {
  color: #5cb85c;
  background-color: #fff;
}
.btn-info {
  color: #fff;
  background-color: #5bc0de;
  border-color: #46b8da;
}
.btn-info:focus,
.btn-info.focus {
  color: #fff;
  background-color: #31b0d5;
  border-color: #1b6d85;
}
.btn-info:hover {
  color: #fff;
  background-color: #31b0d5;
  border-color: #269abc;
}
.btn-info:active,
.btn-info.active,
.open > .dropdown-toggle.btn-info {
  color: #fff;
  background-color: #31b0d5;
  border-color: #269abc;
}
.btn-info:active:hover,
.btn-info.active:hover,
.open > .dropdown-toggle.btn-info:hover,
.btn-info:active:focus,
.btn-info.active:focus,
.open > .dropdown-toggle.btn-info:focus,
.btn-info:active.focus,
.btn-info.active.focus,
.open > .dropdown-toggle.btn-info.focus {
  color: #fff;
  background-color: #269abc;
  border-color: #1b6d85;
}
.btn-info:active,
.btn-info.active,
.open > .dropdown-toggle.btn-info {
  background-image: none;
}
.btn-info.disabled:hover,
.btn-info[disabled]:hover,
fieldset[disabled] .btn-info:hover,
.btn-info.disabled:focus,
.btn-info[disabled]:focus,
fieldset[disabled] .btn-info:focus,
.btn-info.disabled.focus,
.btn-info[disabled].focus,
fieldset[disabled] .btn-info.focus {
  background-color: #5bc0de;
  border-color: #46b8da;
}
.btn-info .badge {
  color: #5bc0de;
  background-color: #fff;
}
.btn-warning {
  color: #fff;
  background-color: #f0ad4e;
  border-color: #eea236;
}
.btn-warning:focus,
.btn-warning.focus {
  color: #fff;
  background-color: #ec971f;
  border-color: #985f0d;
}
.btn-warning:hover {
  color: #fff;
  background-color: #ec971f;
  border-color: #d58512;
}
.btn-warning:active,
.btn-warning.active,
.open > .dropdown-toggle.btn-warning {
  color: #fff;
  background-color: #ec971f;
  border-color: #d58512;
}
.btn-warning:active:hover,
.btn-warning.active:hover,
.open > .dropdown-toggle.btn-warning:hover,
.btn-warning:active:focus,
.btn-warning.active:focus,
.open > .dropdown-toggle.btn-warning:focus,
.btn-warning:active.focus,
.btn-warning.active.focus,
.open > .dropdown-toggle.btn-warning.focus {
  color: #fff;
  background-color: #d58512;
  border-color: #985f0d;
}
.btn-warning:active,
.btn-warning.active,
.open > .dropdown-toggle.btn-warning {
  background-image: none;
}
.btn-warning.disabled:hover,
.btn-warning[disabled]:hover,
fieldset[disabled] .btn-warning:hover,
.btn-warning.disabled:focus,
.btn-warning[disabled]:focus,
fieldset[disabled] .btn-warning:focus,
.btn-warning.disabled.focus,
.btn-warning[disabled].focus,
fieldset[disabled] .btn-warning.focus {
  background-color: #f0ad4e;
  border-color: #eea236;
}
.btn-warning .badge {
  color: #f0ad4e;
  background-color: #fff;
}
.btn-danger {
  color: #fff;
  background-color: #d9534f;
  border-color: #d43f3a;
}
.btn-danger:focus,
.btn-danger.focus {
  color: #fff;
  background-color: #c9302c;
  border-color: #761c19;
}
.btn-danger:hover {
  color: #fff;
  background-color: #c9302c;
  border-color: #ac2925;
}
.btn-danger:active,
.btn-danger.active,
.open > .dropdown-toggle.btn-danger {
  color: #fff;
  background-color: #c9302c;
  border-color: #ac2925;
}
.btn-danger:active:hover,
.btn-danger.active:hover,
.open > .dropdown-toggle.btn-danger:hover,
.btn-danger:active:focus,
.btn-danger.active:focus,
.open > .dropdown-toggle.btn-danger:focus,
.btn-danger:active.focus,
.btn-danger.active.focus,
.open > .dropdown-toggle.btn-danger.focus {
  color: #fff;
  background-color: #ac2925;
  border-color: #761c19;
}
.btn-danger:active,
.btn-danger.active,
.open > .dropdown-toggle.btn-danger {
  background-image: none;
}
.btn-danger.disabled:hover,
.btn-danger[disabled]:hover,
fieldset[disabled] .btn-danger:hover,
.btn-danger.disabled:focus,
.btn-danger[disabled]:focus,
fieldset[disabled] .btn-danger:focus,
.btn-danger.disabled.focus,
.btn-danger[disabled].focus,
fieldset[disabled] .btn-danger.focus {
  background-color: #d9534f;
  border-color: #d43f3a;
}
.btn-danger .badge {
  color: #d9534f;
  background-color: #fff;
}
.btn-link {
  color: #337ab7;
  font-weight: normal;
  border-radius: 0;
}
.btn-link,
.btn-link:active,
.btn-link.active,
.btn-link[disabled],
fieldset[disabled] .btn-link {
  background-color: transparent;
  -webkit-box-shadow: none;
  box-shadow: none;
}
.btn-link,
.btn-link:hover,
.btn-link:focus,
.btn-link:active {
  border-color: transparent;
}
.btn-link:hover,
.btn-link:focus {
  color: #23527c;
  text-decoration: underline;
  background-color: transparent;
}
.btn-link[disabled]:hover,
fieldset[disabled] .btn-link:hover,
.btn-link[disabled]:focus,
fieldset[disabled] .btn-link:focus {
  color: #777777;
  text-decoration: none;
}
.btn-lg,
.btn-group-lg > .btn {
  padding: 10px 16px;
  font-size: 17px;
  line-height: 1.3333333;
  border-radius: 3px;
}
.btn-sm,
.btn-group-sm > .btn {
  padding: 5px 10px;
  font-size: 12px;
  line-height: 1.5;
  border-radius: 1px;
}
.btn-xs,
.btn-group-xs > .btn {
  padding: 1px 5px;
  font-size: 12px;
  line-height: 1.5;
  border-radius: 1px;
}
.btn-block {
  display: block;
  width: 100%;
}
.btn-block + .btn-block {
  margin-top: 5px;
}
input[type="submit"].btn-block,
input[type="reset"].btn-block,
input[type="button"].btn-block {
  width: 100%;
}
.fade {
  opacity: 0;
  -webkit-transition: opacity 0.15s linear;
  -o-transition: opacity 0.15s linear;
  transition: opacity 0.15s linear;
}
.fade.in {
  opacity: 1;
}
.collapse {
  display: none;
}
.collapse.in {
  display: block;
}
tr.collapse.in {
  display: table-row;
}
tbody.collapse.in {
  display: table-row-group;
}
.collapsing {
  position: relative;
  height: 0;
  overflow: hidden;
  -webkit-transition-property: height, visibility;
  transition-property: height, visibility;
  -webkit-transition-duration: 0.35s;
  transition-duration: 0.35s;
  -webkit-transition-timing-function: ease;
  transition-timing-function: ease;
}
.caret {
  display: inline-block;
  width: 0;
  height: 0;
  margin-left: 2px;
  vertical-align: middle;
  border-top: 4px dashed;
  border-top: 4px solid \9;
  border-right: 4px solid transparent;
  border-left: 4px solid transparent;
}
.dropup,
.dropdown {
  position: relative;
}
.dropdown-toggle:focus {
  outline: 0;
}
.dropdown-menu {
  position: absolute;
  top: 100%;
  left: 0;
  z-index: 1000;
  display: none;
  float: left;
  min-width: 160px;
  padding: 5px 0;
  margin: 2px 0 0;
  list-style: none;
  font-size: 13px;
  text-align: left;
  background-color: #fff;
  border: 1px solid #ccc;
  border: 1px solid rgba(0, 0, 0, 0.15);
  border-radius: 2px;
  -webkit-box-shadow: 0 6px 12px rgba(0, 0, 0, 0.175);
  box-shadow: 0 6px 12px rgba(0, 0, 0, 0.175);
  background-clip: padding-box;
}
.dropdown-menu.pull-right {
  right: 0;
  left: auto;
}
.dropdown-menu .divider {
  height: 1px;
  margin: 8px 0;
  overflow: hidden;
  background-color: #e5e5e5;
}
.dropdown-menu > li > a {
  display: block;
  padding: 3px 20px;
  clear: both;
  font-weight: normal;
  line-height: 1.42857143;
  color: #333333;
  white-space: nowrap;
}
.dropdown-menu > li > a:hover,
.dropdown-menu > li > a:focus {
  text-decoration: none;
  color: #262626;
  background-color: #f5f5f5;
}
.dropdown-menu > .active > a,
.dropdown-menu > .active > a:hover,
.dropdown-menu > .active > a:focus {
  color: #fff;
  text-decoration: none;
  outline: 0;
  background-color: #337ab7;
}
.dropdown-menu > .disabled > a,
.dropdown-menu > .disabled > a:hover,
.dropdown-menu > .disabled > a:focus {
  color: #777777;
}
.dropdown-menu > .disabled > a:hover,
.dropdown-menu > .disabled > a:focus {
  text-decoration: none;
  background-color: transparent;
  background-image: none;
  filter: progid:DXImageTransform.Microsoft.gradient(enabled = false);
  cursor: not-allowed;
}
.open > .dropdown-menu {
  display: block;
}
.open > a {
  outline: 0;
}
.dropdown-menu-right {
  left: auto;
  right: 0;
}
.dropdown-menu-left {
  left: 0;
  right: auto;
}
.dropdown-header {
  display: block;
  padding: 3px 20px;
  font-size: 12px;
  line-height: 1.42857143;
  color: #777777;
  white-space: nowrap;
}
.dropdown-backdrop {
  position: fixed;
  left: 0;
  right: 0;
  bottom: 0;
  top: 0;
  z-index: 990;
}
.pull-right > .dropdown-menu {
  right: 0;
  left: auto;
}
.dropup .caret,
.navbar-fixed-bottom .dropdown .caret {
  border-top: 0;
  border-bottom: 4px dashed;
  border-bottom: 4px solid \9;
  content: "";
}
.dropup .dropdown-menu,
.navbar-fixed-bottom .dropdown .dropdown-menu {
  top: auto;
  bottom: 100%;
  margin-bottom: 2px;
}
@media (min-width: 541px) {
  .navbar-right .dropdown-menu {
    left: auto;
    right: 0;
  }
  .navbar-right .dropdown-menu-left {
    left: 0;
    right: auto;
  }
}
.btn-group,
.btn-group-vertical {
  position: relative;
  display: inline-block;
  vertical-align: middle;
}
.btn-group > .btn,
.btn-group-vertical > .btn {
  position: relative;
  float: left;
}
.btn-group > .btn:hover,
.btn-group-vertical > .btn:hover,
.btn-group > .btn:focus,
.btn-group-vertical > .btn:focus,
.btn-group > .btn:active,
.btn-group-vertical > .btn:active,
.btn-group > .btn.active,
.btn-group-vertical > .btn.active {
  z-index: 2;
}
.btn-group .btn + .btn,
.btn-group .btn + .btn-group,
.btn-group .btn-group + .btn,
.btn-group .btn-group + .btn-group {
  margin-left: -1px;
}
.btn-toolbar {
  margin-left: -5px;
}
.btn-toolbar .btn,
.btn-toolbar .btn-group,
.btn-toolbar .input-group {
  float: left;
}
.btn-toolbar > .btn,
.btn-toolbar > .btn-group,
.btn-toolbar > .input-group {
  margin-left: 5px;
}
.btn-group > .btn:not(:first-child):not(:last-child):not(.dropdown-toggle) {
  border-radius: 0;
}
.btn-group > .btn:first-child {
  margin-left: 0;
}
.btn-group > .btn:first-child:not(:last-child):not(.dropdown-toggle) {
  border-bottom-right-radius: 0;
  border-top-right-radius: 0;
}
.btn-group > .btn:last-child:not(:first-child),
.btn-group > .dropdown-toggle:not(:first-child) {
  border-bottom-left-radius: 0;
  border-top-left-radius: 0;
}
.btn-group > .btn-group {
  float: left;
}
.btn-group > .btn-group:not(:first-child):not(:last-child) > .btn {
  border-radius: 0;
}
.btn-group > .btn-group:first-child:not(:last-child) > .btn:last-child,
.btn-group > .btn-group:first-child:not(:last-child) > .dropdown-toggle {
  border-bottom-right-radius: 0;
  border-top-right-radius: 0;
}
.btn-group > .btn-group:last-child:not(:first-child) > .btn:first-child {
  border-bottom-left-radius: 0;
  border-top-left-radius: 0;
}
.btn-group .dropdown-toggle:active,
.btn-group.open .dropdown-toggle {
  outline: 0;
}
.btn-group > .btn + .dropdown-toggle {
  padding-left: 8px;
  padding-right: 8px;
}
.btn-group > .btn-lg + .dropdown-toggle {
  padding-left: 12px;
  padding-right: 12px;
}
.btn-group.open .dropdown-toggle {
  -webkit-box-shadow: inset 0 3px 5px rgba(0, 0, 0, 0.125);
  box-shadow: inset 0 3px 5px rgba(0, 0, 0, 0.125);
}
.btn-group.open .dropdown-toggle.btn-link {
  -webkit-box-shadow: none;
  box-shadow: none;
}
.btn .caret {
  margin-left: 0;
}
.btn-lg .caret {
  border-width: 5px 5px 0;
  border-bottom-width: 0;
}
.dropup .btn-lg .caret {
  border-width: 0 5px 5px;
}
.btn-group-vertical > .btn,
.btn-group-vertical > .btn-group,
.btn-group-vertical > .btn-group > .btn {
  display: block;
  float: none;
  width: 100%;
  max-width: 100%;
}
.btn-group-vertical > .btn-group > .btn {
  float: none;
}
.btn-group-vertical > .btn + .btn,
.btn-group-vertical > .btn + .btn-group,
.btn-group-vertical > .btn-group + .btn,
.btn-group-vertical > .btn-group + .btn-group {
  margin-top: -1px;
  margin-left: 0;
}
.btn-group-vertical > .btn:not(:first-child):not(:last-child) {
  border-radius: 0;
}
.btn-group-vertical > .btn:first-child:not(:last-child) {
  border-top-right-radius: 2px;
  border-top-left-radius: 2px;
  border-bottom-right-radius: 0;
  border-bottom-left-radius: 0;
}
.btn-group-vertical > .btn:last-child:not(:first-child) {
  border-top-right-radius: 0;
  border-top-left-radius: 0;
  border-bottom-right-radius: 2px;
  border-bottom-left-radius: 2px;
}
.btn-group-vertical > .btn-group:not(:first-child):not(:last-child) > .btn {
  border-radius: 0;
}
.btn-group-vertical > .btn-group:first-child:not(:last-child) > .btn:last-child,
.btn-group-vertical > .btn-group:first-child:not(:last-child) > .dropdown-toggle {
  border-bottom-right-radius: 0;
  border-bottom-left-radius: 0;
}
.btn-group-vertical > .btn-group:last-child:not(:first-child) > .btn:first-child {
  border-top-right-radius: 0;
  border-top-left-radius: 0;
}
.btn-group-justified {
  display: table;
  width: 100%;
  table-layout: fixed;
  border-collapse: separate;
}
.btn-group-justified > .btn,
.btn-group-justified > .btn-group {
  float: none;
  display: table-cell;
  width: 1%;
}
.btn-group-justified > .btn-group .btn {
  width: 100%;
}
.btn-group-justified > .btn-group .dropdown-menu {
  left: auto;
}
[data-toggle="buttons"] > .btn input[type="radio"],
[data-toggle="buttons"] > .btn-group > .btn input[type="radio"],
[data-toggle="buttons"] > .btn input[type="checkbox"],
[data-toggle="buttons"] > .btn-group > .btn input[type="checkbox"] {
  position: absolute;
  clip: rect(0, 0, 0, 0);
  pointer-events: none;
}
.input-group {
  position: relative;
  display: table;
  border-collapse: separate;
}
.input-group[class*="col-"] {
  float: none;
  padding-left: 0;
  padding-right: 0;
}
.input-group .form-control {
  position: relative;
  z-index: 2;
  float: left;
  width: 100%;
  margin-bottom: 0;
}
.input-group .form-control:focus {
  z-index: 3;
}
.input-group-lg > .form-control,
.input-group-lg > .input-group-addon,
.input-group-lg > .input-group-btn > .btn {
  height: 45px;
  padding: 10px 16px;
  font-size: 17px;
  line-height: 1.3333333;
  border-radius: 3px;
}
select.input-group-lg > .form-control,
select.input-group-lg > .input-group-addon,
select.input-group-lg > .input-group-btn > .btn {
  height: 45px;
  line-height: 45px;
}
textarea.input-group-lg > .form-control,
textarea.input-group-lg > .input-group-addon,
textarea.input-group-lg > .input-group-btn > .btn,
select[multiple].input-group-lg > .form-control,
select[multiple].input-group-lg > .input-group-addon,
select[multiple].input-group-lg > .input-group-btn > .btn {
  height: auto;
}
.input-group-sm > .form-control,
.input-group-sm > .input-group-addon,
.input-group-sm > .input-group-btn > .btn {
  height: 30px;
  padding: 5px 10px;
  font-size: 12px;
  line-height: 1.5;
  border-radius: 1px;
}
select.input-group-sm > .form-control,
select.input-group-sm > .input-group-addon,
select.input-group-sm > .input-group-btn > .btn {
  height: 30px;
  line-height: 30px;
}
textarea.input-group-sm > .form-control,
textarea.input-group-sm > .input-group-addon,
textarea.input-group-sm > .input-group-btn > .btn,
select[multiple].input-group-sm > .form-control,
select[multiple].input-group-sm > .input-group-addon,
select[multiple].input-group-sm > .input-group-btn > .btn {
  height: auto;
}
.input-group-addon,
.input-group-btn,
.input-group .form-control {
  display: table-cell;
}
.input-group-addon:not(:first-child):not(:last-child),
.input-group-btn:not(:first-child):not(:last-child),
.input-group .form-control:not(:first-child):not(:last-child) {
  border-radius: 0;
}
.input-group-addon,
.input-group-btn {
  width: 1%;
  white-space: nowrap;
  vertical-align: middle;
}
.input-group-addon {
  padding: 6px 12px;
  font-size: 13px;
  font-weight: normal;
  line-height: 1;
  color: #555555;
  text-align: center;
  background-color: #eeeeee;
  border: 1px solid #ccc;
  border-radius: 2px;
}
.input-group-addon.input-sm {
  padding: 5px 10px;
  font-size: 12px;
  border-radius: 1px;
}
.input-group-addon.input-lg {
  padding: 10px 16px;
  font-size: 17px;
  border-radius: 3px;
}
.input-group-addon input[type="radio"],
.input-group-addon input[type="checkbox"] {
  margin-top: 0;
}
.input-group .form-control:first-child,
.input-group-addon:first-child,
.input-group-btn:first-child > .btn,
.input-group-btn:first-child > .btn-group > .btn,
.input-group-btn:first-child > .dropdown-toggle,
.input-group-btn:last-child > .btn:not(:last-child):not(.dropdown-toggle),
.input-group-btn:last-child > .btn-group:not(:last-child) > .btn {
  border-bottom-right-radius: 0;
  border-top-right-radius: 0;
}
.input-group-addon:first-child {
  border-right: 0;
}
.input-group .form-control:last-child,
.input-group-addon:last-child,
.input-group-btn:last-child > .btn,
.input-group-btn:last-child > .btn-group > .btn,
.input-group-btn:last-child > .dropdown-toggle,
.input-group-btn:first-child > .btn:not(:first-child),
.input-group-btn:first-child > .btn-group:not(:first-child) > .btn {
  border-bottom-left-radius: 0;
  border-top-left-radius: 0;
}
.input-group-addon:last-child {
  border-left: 0;
}
.input-group-btn {
  position: relative;
  font-size: 0;
  white-space: nowrap;
}
.input-group-btn > .btn {
  position: relative;
}
.input-group-btn > .btn + .btn {
  margin-left: -1px;
}
.input-group-btn > .btn:hover,
.input-group-btn > .btn:focus,
.input-group-btn > .btn:active {
  z-index: 2;
}
.input-group-btn:first-child > .btn,
.input-group-btn:first-child > .btn-group {
  margin-right: -1px;
}
.input-group-btn:last-child > .btn,
.input-group-btn:last-child > .btn-group {
  z-index: 2;
  margin-left: -1px;
}
.nav {
  margin-bottom: 0;
  padding-left: 0;
  list-style: none;
}
.nav > li {
  position: relative;
  display: block;
}
.nav > li > a {
  position: relative;
  display: block;
  padding: 10px 15px;
}
.nav > li > a:hover,
.nav > li > a:focus {
  text-decoration: none;
  background-color: #eeeeee;
}
.nav > li.disabled > a {
  color: #777777;
}
.nav > li.disabled > a:hover,
.nav > li.disabled > a:focus {
  color: #777777;
  text-decoration: none;
  background-color: transparent;
  cursor: not-allowed;
}
.nav .open > a,
.nav .open > a:hover,
.nav .open > a:focus {
  background-color: #eeeeee;
  border-color: #337ab7;
}
.nav .nav-divider {
  height: 1px;
  margin: 8px 0;
  overflow: hidden;
  background-color: #e5e5e5;
}
.nav > li > a > img {
  max-width: none;
}
.nav-tabs {
  border-bottom: 1px solid #ddd;
}
.nav-tabs > li {
  float: left;
  margin-bottom: -1px;
}
.nav-tabs > li > a {
  margin-right: 2px;
  line-height: 1.42857143;
  border: 1px solid transparent;
  border-radius: 2px 2px 0 0;
}
.nav-tabs > li > a:hover {
  border-color: #eeeeee #eeeeee #ddd;
}
.nav-tabs > li.active > a,
.nav-tabs > li.active > a:hover,
.nav-tabs > li.active > a:focus {
  color: #555555;
  background-color: #fff;
  border: 1px solid #ddd;
  border-bottom-color: transparent;
  cursor: default;
}
.nav-tabs.nav-justified {
  width: 100%;
  border-bottom: 0;
}
.nav-tabs.nav-justified > li {
  float: none;
}
.nav-tabs.nav-justified > li > a {
  text-align: center;
  margin-bottom: 5px;
}
.nav-tabs.nav-justified > .dropdown .dropdown-menu {
  top: auto;
  left: auto;
}
@media (min-width: 768px) {
  .nav-tabs.nav-justified > li {
    display: table-cell;
    width: 1%;
  }
  .nav-tabs.nav-justified > li > a {
    margin-bottom: 0;
  }
}
.nav-tabs.nav-justified > li > a {
  margin-right: 0;
  border-radius: 2px;
}
.nav-tabs.nav-justified > .active > a,
.nav-tabs.nav-justified > .active > a:hover,
.nav-tabs.nav-justified > .active > a:focus {
  border: 1px solid #ddd;
}
@media (min-width: 768px) {
  .nav-tabs.nav-justified > li > a {
    border-bottom: 1px solid #ddd;
    border-radius: 2px 2px 0 0;
  }
  .nav-tabs.nav-justified > .active > a,
  .nav-tabs.nav-justified > .active > a:hover,
  .nav-tabs.nav-justified > .active > a:focus {
    border-bottom-color: #fff;
  }
}
.nav-pills > li {
  float: left;
}
.nav-pills > li > a {
  border-radius: 2px;
}
.nav-pills > li + li {
  margin-left: 2px;
}
.nav-pills > li.active > a,
.nav-pills > li.active > a:hover,
.nav-pills > li.active > a:focus {
  color: #fff;
  background-color: #337ab7;
}
.nav-stacked > li {
  float: none;
}
.nav-stacked > li + li {
  margin-top: 2px;
  margin-left: 0;
}
.nav-justified {
  width: 100%;
}
.nav-justified > li {
  float: none;
}
.nav-justified > li > a {
  text-align: center;
  margin-bottom: 5px;
}
.nav-justified > .dropdown .dropdown-menu {
  top: auto;
  left: auto;
}
@media (min-width: 768px) {
  .nav-justified > li {
    display: table-cell;
    width: 1%;
  }
  .nav-justified > li > a {
    margin-bottom: 0;
  }
}
.nav-tabs-justified {
  border-bottom: 0;
}
.nav-tabs-justified > li > a {
  margin-right: 0;
  border-radius: 2px;
}
.nav-tabs-justified > .active > a,
.nav-tabs-justified > .active > a:hover,
.nav-tabs-justified > .active > a:focus {
  border: 1px solid #ddd;
}
@media (min-width: 768px) {
  .nav-tabs-justified > li > a {
    border-bottom: 1px solid #ddd;
    border-radius: 2px 2px 0 0;
  }
  .nav-tabs-justified > .active > a,
  .nav-tabs-justified > .active > a:hover,
  .nav-tabs-justified > .active > a:focus {
    border-bottom-color: #fff;
  }
}
.tab-content > .tab-pane {
  display: none;
}
.tab-content > .active {
  display: block;
}
.nav-tabs .dropdown-menu {
  margin-top: -1px;
  border-top-right-radius: 0;
  border-top-left-radius: 0;
}
.navbar {
  position: relative;
  min-height: 30px;
  margin-bottom: 18px;
  border: 1px solid transparent;
}
@media (min-width: 541px) {
  .navbar {
    border-radius: 2px;
  }
}
@media (min-width: 541px) {
  .navbar-header {
    float: left;
  }
}
.navbar-collapse {
  overflow-x: visible;
  padding-right: 0px;
  padding-left: 0px;
  border-top: 1px solid transparent;
  box-shadow: inset 0 1px 0 rgba(255, 255, 255, 0.1);
  -webkit-overflow-scrolling: touch;
}
.navbar-collapse.in {
  overflow-y: auto;
}
@media (min-width: 541px) {
  .navbar-collapse {
    width: auto;
    border-top: 0;
    box-shadow: none;
  }
  .navbar-collapse.collapse {
    display: block !important;
    height: auto !important;
    padding-bottom: 0;
    overflow: visible !important;
  }
  .navbar-collapse.in {
    overflow-y: visible;
  }
  .navbar-fixed-top .navbar-collapse,
  .navbar-static-top .navbar-collapse,
  .navbar-fixed-bottom .navbar-collapse {
    padding-left: 0;
    padding-right: 0;
  }
}
.navbar-fixed-top .navbar-collapse,
.navbar-fixed-bottom .navbar-collapse {
  max-height: 340px;
}
@media (max-device-width: 540px) and (orientation: landscape) {
  .navbar-fixed-top .navbar-collapse,
  .navbar-fixed-bottom .navbar-collapse {
    max-height: 200px;
  }
}
.container > .navbar-header,
.container-fluid > .navbar-header,
.container > .navbar-collapse,
.container-fluid > .navbar-collapse {
  margin-right: 0px;
  margin-left: 0px;
}
@media (min-width: 541px) {
  .container > .navbar-header,
  .container-fluid > .navbar-header,
  .container > .navbar-collapse,
  .container-fluid > .navbar-collapse {
    margin-right: 0;
    margin-left: 0;
  }
}
.navbar-static-top {
  z-index: 1000;
  border-width: 0 0 1px;
}
@media (min-width: 541px) {
  .navbar-static-top {
    border-radius: 0;
  }
}
.navbar-fixed-top,
.navbar-fixed-bottom {
  position: fixed;
  right: 0;
  left: 0;
  z-index: 1030;
}
@media (min-width: 541px) {
  .navbar-fixed-top,
  .navbar-fixed-bottom {
    border-radius: 0;
  }
}
.navbar-fixed-top {
  top: 0;
  border-width: 0 0 1px;
}
.navbar-fixed-bottom {
  bottom: 0;
  margin-bottom: 0;
  border-width: 1px 0 0;
}
.navbar-brand {
  float: left;
  padding: 6px 0px;
  font-size: 17px;
  line-height: 18px;
  height: 30px;
}
.navbar-brand:hover,
.navbar-brand:focus {
  text-decoration: none;
}
.navbar-brand > img {
  display: block;
}
@media (min-width: 541px) {
  .navbar > .container .navbar-brand,
  .navbar > .container-fluid .navbar-brand {
    margin-left: 0px;
  }
}
.navbar-toggle {
  position: relative;
  float: right;
  margin-right: 0px;
  padding: 9px 10px;
  margin-top: -2px;
  margin-bottom: -2px;
  background-color: transparent;
  background-image: none;
  border: 1px solid transparent;
  border-radius: 2px;
}
.navbar-toggle:focus {
  outline: 0;
}
.navbar-toggle .icon-bar {
  display: block;
  width: 22px;
  height: 2px;
  border-radius: 1px;
}
.navbar-toggle .icon-bar + .icon-bar {
  margin-top: 4px;
}
@media (min-width: 541px) {
  .navbar-toggle {
    display: none;
  }
}
.navbar-nav {
  margin: 3px 0px;
}
.navbar-nav > li > a {
  padding-top: 10px;
  padding-bottom: 10px;
  line-height: 18px;
}
@media (max-width: 540px) {
  .navbar-nav .open .dropdown-menu {
    position: static;
    float: none;
    width: auto;
    margin-top: 0;
    background-color: transparent;
    border: 0;
    box-shadow: none;
  }
  .navbar-nav .open .dropdown-menu > li > a,
  .navbar-nav .open .dropdown-menu .dropdown-header {
    padding: 5px 15px 5px 25px;
  }
  .navbar-nav .open .dropdown-menu > li > a {
    line-height: 18px;
  }
  .navbar-nav .open .dropdown-menu > li > a:hover,
  .navbar-nav .open .dropdown-menu > li > a:focus {
    background-image: none;
  }
}
@media (min-width: 541px) {
  .navbar-nav {
    float: left;
    margin: 0;
  }
  .navbar-nav > li {
    float: left;
  }
  .navbar-nav > li > a {
    padding-top: 6px;
    padding-bottom: 6px;
  }
}
.navbar-form {
  margin-left: 0px;
  margin-right: 0px;
  padding: 10px 0px;
  border-top: 1px solid transparent;
  border-bottom: 1px solid transparent;
  -webkit-box-shadow: inset 0 1px 0 rgba(255, 255, 255, 0.1), 0 1px 0 rgba(255, 255, 255, 0.1);
  box-shadow: inset 0 1px 0 rgba(255, 255, 255, 0.1), 0 1px 0 rgba(255, 255, 255, 0.1);
  margin-top: -1px;
  margin-bottom: -1px;
}
@media (min-width: 768px) {
  .navbar-form .form-group {
    display: inline-block;
    margin-bottom: 0;
    vertical-align: middle;
  }
  .navbar-form .form-control {
    display: inline-block;
    width: auto;
    vertical-align: middle;
  }
  .navbar-form .form-control-static {
    display: inline-block;
  }
  .navbar-form .input-group {
    display: inline-table;
    vertical-align: middle;
  }
  .navbar-form .input-group .input-group-addon,
  .navbar-form .input-group .input-group-btn,
  .navbar-form .input-group .form-control {
    width: auto;
  }
  .navbar-form .input-group > .form-control {
    width: 100%;
  }
  .navbar-form .control-label {
    margin-bottom: 0;
    vertical-align: middle;
  }
  .navbar-form .radio,
  .navbar-form .checkbox {
    display: inline-block;
    margin-top: 0;
    margin-bottom: 0;
    vertical-align: middle;
  }
  .navbar-form .radio label,
  .navbar-form .checkbox label {
    padding-left: 0;
  }
  .navbar-form .radio input[type="radio"],
  .navbar-form .checkbox input[type="checkbox"] {
    position: relative;
    margin-left: 0;
  }
  .navbar-form .has-feedback .form-control-feedback {
    top: 0;
  }
}
@media (max-width: 540px) {
  .navbar-form .form-group {
    margin-bottom: 5px;
  }
  .navbar-form .form-group:last-child {
    margin-bottom: 0;
  }
}
@media (min-width: 541px) {
  .navbar-form {
    width: auto;
    border: 0;
    margin-left: 0;
    margin-right: 0;
    padding-top: 0;
    padding-bottom: 0;
    -webkit-box-shadow: none;
    box-shadow: none;
  }
}
.navbar-nav > li > .dropdown-menu {
  margin-top: 0;
  border-top-right-radius: 0;
  border-top-left-radius: 0;
}
.navbar-fixed-bottom .navbar-nav > li > .dropdown-menu {
  margin-bottom: 0;
  border-top-right-radius: 2px;
  border-top-left-radius: 2px;
  border-bottom-right-radius: 0;
  border-bottom-left-radius: 0;
}
.navbar-btn {
  margin-top: -1px;
  margin-bottom: -1px;
}
.navbar-btn.btn-sm {
  margin-top: 0px;
  margin-bottom: 0px;
}
.navbar-btn.btn-xs {
  margin-top: 4px;
  margin-bottom: 4px;
}
.navbar-text {
  margin-top: 6px;
  margin-bottom: 6px;
}
@media (min-width: 541px) {
  .navbar-text {
    float: left;
    margin-left: 0px;
    margin-right: 0px;
  }
}
@media (min-width: 541px) {
  .navbar-left {
    float: left !important;
    float: left;
  }
  .navbar-right {
    float: right !important;
    float: right;
    margin-right: 0px;
  }
  .navbar-right ~ .navbar-right {
    margin-right: 0;
  }
}
.navbar-default {
  background-color: #f8f8f8;
  border-color: #e7e7e7;
}
.navbar-default .navbar-brand {
  color: #777;
}
.navbar-default .navbar-brand:hover,
.navbar-default .navbar-brand:focus {
  color: #5e5e5e;
  background-color: transparent;
}
.navbar-default .navbar-text {
  color: #777;
}
.navbar-default .navbar-nav > li > a {
  color: #777;
}
.navbar-default .navbar-nav > li > a:hover,
.navbar-default .navbar-nav > li > a:focus {
  color: #333;
  background-color: transparent;
}
.navbar-default .navbar-nav > .active > a,
.navbar-default .navbar-nav > .active > a:hover,
.navbar-default .navbar-nav > .active > a:focus {
  color: #555;
  background-color: #e7e7e7;
}
.navbar-default .navbar-nav > .disabled > a,
.navbar-default .navbar-nav > .disabled > a:hover,
.navbar-default .navbar-nav > .disabled > a:focus {
  color: #ccc;
  background-color: transparent;
}
.navbar-default .navbar-toggle {
  border-color: #ddd;
}
.navbar-default .navbar-toggle:hover,
.navbar-default .navbar-toggle:focus {
  background-color: #ddd;
}
.navbar-default .navbar-toggle .icon-bar {
  background-color: #888;
}
.navbar-default .navbar-collapse,
.navbar-default .navbar-form {
  border-color: #e7e7e7;
}
.navbar-default .navbar-nav > .open > a,
.navbar-default .navbar-nav > .open > a:hover,
.navbar-default .navbar-nav > .open > a:focus {
  background-color: #e7e7e7;
  color: #555;
}
@media (max-width: 540px) {
  .navbar-default .navbar-nav .open .dropdown-menu > li > a {
    color: #777;
  }
  .navbar-default .navbar-nav .open .dropdown-menu > li > a:hover,
  .navbar-default .navbar-nav .open .dropdown-menu > li > a:focus {
    color: #333;
    background-color: transparent;
  }
  .navbar-default .navbar-nav .open .dropdown-menu > .active > a,
  .navbar-default .navbar-nav .open .dropdown-menu > .active > a:hover,
  .navbar-default .navbar-nav .open .dropdown-menu > .active > a:focus {
    color: #555;
    background-color: #e7e7e7;
  }
  .navbar-default .navbar-nav .open .dropdown-menu > .disabled > a,
  .navbar-default .navbar-nav .open .dropdown-menu > .disabled > a:hover,
  .navbar-default .navbar-nav .open .dropdown-menu > .disabled > a:focus {
    color: #ccc;
    background-color: transparent;
  }
}
.navbar-default .navbar-link {
  color: #777;
}
.navbar-default .navbar-link:hover {
  color: #333;
}
.navbar-default .btn-link {
  color: #777;
}
.navbar-default .btn-link:hover,
.navbar-default .btn-link:focus {
  color: #333;
}
.navbar-default .btn-link[disabled]:hover,
fieldset[disabled] .navbar-default .btn-link:hover,
.navbar-default .btn-link[disabled]:focus,
fieldset[disabled] .navbar-default .btn-link:focus {
  color: #ccc;
}
.navbar-inverse {
  background-color: #222;
  border-color: #080808;
}
.navbar-inverse .navbar-brand {
  color: #9d9d9d;
}
.navbar-inverse .navbar-brand:hover,
.navbar-inverse .navbar-brand:focus {
  color: #fff;
  background-color: transparent;
}
.navbar-inverse .navbar-text {
  color: #9d9d9d;
}
.navbar-inverse .navbar-nav > li > a {
  color: #9d9d9d;
}
.navbar-inverse .navbar-nav > li > a:hover,
.navbar-inverse .navbar-nav > li > a:focus {
  color: #fff;
  background-color: transparent;
}
.navbar-inverse .navbar-nav > .active > a,
.navbar-inverse .navbar-nav > .active > a:hover,
.navbar-inverse .navbar-nav > .active > a:focus {
  color: #fff;
  background-color: #080808;
}
.navbar-inverse .navbar-nav > .disabled > a,
.navbar-inverse .navbar-nav > .disabled > a:hover,
.navbar-inverse .navbar-nav > .disabled > a:focus {
  color: #444;
  background-color: transparent;
}
.navbar-inverse .navbar-toggle {
  border-color: #333;
}
.navbar-inverse .navbar-toggle:hover,
.navbar-inverse .navbar-toggle:focus {
  background-color: #333;
}
.navbar-inverse .navbar-toggle .icon-bar {
  background-color: #fff;
}
.navbar-inverse .navbar-collapse,
.navbar-inverse .navbar-form {
  border-color: #101010;
}
.navbar-inverse .navbar-nav > .open > a,
.navbar-inverse .navbar-nav > .open > a:hover,
.navbar-inverse .navbar-nav > .open > a:focus {
  background-color: #080808;
  color: #fff;
}
@media (max-width: 540px) {
  .navbar-inverse .navbar-nav .open .dropdown-menu > .dropdown-header {
    border-color: #080808;
  }
  .navbar-inverse .navbar-nav .open .dropdown-menu .divider {
    background-color: #080808;
  }
  .navbar-inverse .navbar-nav .open .dropdown-menu > li > a {
    color: #9d9d9d;
  }
  .navbar-inverse .navbar-nav .open .dropdown-menu > li > a:hover,
  .navbar-inverse .navbar-nav .open .dropdown-menu > li > a:focus {
    color: #fff;
    background-color: transparent;
  }
  .navbar-inverse .navbar-nav .open .dropdown-menu > .active > a,
  .navbar-inverse .navbar-nav .open .dropdown-menu > .active > a:hover,
  .navbar-inverse .navbar-nav .open .dropdown-menu > .active > a:focus {
    color: #fff;
    background-color: #080808;
  }
  .navbar-inverse .navbar-nav .open .dropdown-menu > .disabled > a,
  .navbar-inverse .navbar-nav .open .dropdown-menu > .disabled > a:hover,
  .navbar-inverse .navbar-nav .open .dropdown-menu > .disabled > a:focus {
    color: #444;
    background-color: transparent;
  }
}
.navbar-inverse .navbar-link {
  color: #9d9d9d;
}
.navbar-inverse .navbar-link:hover {
  color: #fff;
}
.navbar-inverse .btn-link {
  color: #9d9d9d;
}
.navbar-inverse .btn-link:hover,
.navbar-inverse .btn-link:focus {
  color: #fff;
}
.navbar-inverse .btn-link[disabled]:hover,
fieldset[disabled] .navbar-inverse .btn-link:hover,
.navbar-inverse .btn-link[disabled]:focus,
fieldset[disabled] .navbar-inverse .btn-link:focus {
  color: #444;
}
.breadcrumb {
  padding: 8px 15px;
  margin-bottom: 18px;
  list-style: none;
  background-color: #f5f5f5;
  border-radius: 2px;
}
.breadcrumb > li {
  display: inline-block;
}
.breadcrumb > li + li:before {
  content: "/\00a0";
  padding: 0 5px;
  color: #5e5e5e;
}
.breadcrumb > .active {
  color: #777777;
}
.pagination {
  display: inline-block;
  padding-left: 0;
  margin: 18px 0;
  border-radius: 2px;
}
.pagination > li {
  display: inline;
}
.pagination > li > a,
.pagination > li > span {
  position: relative;
  float: left;
  padding: 6px 12px;
  line-height: 1.42857143;
  text-decoration: none;
  color: #337ab7;
  background-color: #fff;
  border: 1px solid #ddd;
  margin-left: -1px;
}
.pagination > li:first-child > a,
.pagination > li:first-child > span {
  margin-left: 0;
  border-bottom-left-radius: 2px;
  border-top-left-radius: 2px;
}
.pagination > li:last-child > a,
.pagination > li:last-child > span {
  border-bottom-right-radius: 2px;
  border-top-right-radius: 2px;
}
.pagination > li > a:hover,
.pagination > li > span:hover,
.pagination > li > a:focus,
.pagination > li > span:focus {
  z-index: 2;
  color: #23527c;
  background-color: #eeeeee;
  border-color: #ddd;
}
.pagination > .active > a,
.pagination > .active > span,
.pagination > .active > a:hover,
.pagination > .active > span:hover,
.pagination > .active > a:focus,
.pagination > .active > span:focus {
  z-index: 3;
  color: #fff;
  background-color: #337ab7;
  border-color: #337ab7;
  cursor: default;
}
.pagination > .disabled > span,
.pagination > .disabled > span:hover,
.pagination > .disabled > span:focus,
.pagination > .disabled > a,
.pagination > .disabled > a:hover,
.pagination > .disabled > a:focus {
  color: #777777;
  background-color: #fff;
  border-color: #ddd;
  cursor: not-allowed;
}
.pagination-lg > li > a,
.pagination-lg > li > span {
  padding: 10px 16px;
  font-size: 17px;
  line-height: 1.3333333;
}
.pagination-lg > li:first-child > a,
.pagination-lg > li:first-child > span {
  border-bottom-left-radius: 3px;
  border-top-left-radius: 3px;
}
.pagination-lg > li:last-child > a,
.pagination-lg > li:last-child > span {
  border-bottom-right-radius: 3px;
  border-top-right-radius: 3px;
}
.pagination-sm > li > a,
.pagination-sm > li > span {
  padding: 5px 10px;
  font-size: 12px;
  line-height: 1.5;
}
.pagination-sm > li:first-child > a,
.pagination-sm > li:first-child > span {
  border-bottom-left-radius: 1px;
  border-top-left-radius: 1px;
}
.pagination-sm > li:last-child > a,
.pagination-sm > li:last-child > span {
  border-bottom-right-radius: 1px;
  border-top-right-radius: 1px;
}
.pager {
  padding-left: 0;
  margin: 18px 0;
  list-style: none;
  text-align: center;
}
.pager li {
  display: inline;
}
.pager li > a,
.pager li > span {
  display: inline-block;
  padding: 5px 14px;
  background-color: #fff;
  border: 1px solid #ddd;
  border-radius: 15px;
}
.pager li > a:hover,
.pager li > a:focus {
  text-decoration: none;
  background-color: #eeeeee;
}
.pager .next > a,
.pager .next > span {
  float: right;
}
.pager .previous > a,
.pager .previous > span {
  float: left;
}
.pager .disabled > a,
.pager .disabled > a:hover,
.pager .disabled > a:focus,
.pager .disabled > span {
  color: #777777;
  background-color: #fff;
  cursor: not-allowed;
}
.label {
  display: inline;
  padding: .2em .6em .3em;
  font-size: 75%;
  font-weight: bold;
  line-height: 1;
  color: #fff;
  text-align: center;
  white-space: nowrap;
  vertical-align: baseline;
  border-radius: .25em;
}
a.label:hover,
a.label:focus {
  color: #fff;
  text-decoration: none;
  cursor: pointer;
}
.label:empty {
  display: none;
}
.btn .label {
  position: relative;
  top: -1px;
}
.label-default {
  background-color: #777777;
}
.label-default[href]:hover,
.label-default[href]:focus {
  background-color: #5e5e5e;
}
.label-primary {
  background-color: #337ab7;
}
.label-primary[href]:hover,
.label-primary[href]:focus {
  background-color: #286090;
}
.label-success {
  background-color: #5cb85c;
}
.label-success[href]:hover,
.label-success[href]:focus {
  background-color: #449d44;
}
.label-info {
  background-color: #5bc0de;
}
.label-info[href]:hover,
.label-info[href]:focus {
  background-color: #31b0d5;
}
.label-warning {
  background-color: #f0ad4e;
}
.label-warning[href]:hover,
.label-warning[href]:focus {
  background-color: #ec971f;
}
.label-danger {
  background-color: #d9534f;
}
.label-danger[href]:hover,
.label-danger[href]:focus {
  background-color: #c9302c;
}
.badge {
  display: inline-block;
  min-width: 10px;
  padding: 3px 7px;
  font-size: 12px;
  font-weight: bold;
  color: #fff;
  line-height: 1;
  vertical-align: middle;
  white-space: nowrap;
  text-align: center;
  background-color: #777777;
  border-radius: 10px;
}
.badge:empty {
  display: none;
}
.btn .badge {
  position: relative;
  top: -1px;
}
.btn-xs .badge,
.btn-group-xs > .btn .badge {
  top: 0;
  padding: 1px 5px;
}
a.badge:hover,
a.badge:focus {
  color: #fff;
  text-decoration: none;
  cursor: pointer;
}
.list-group-item.active > .badge,
.nav-pills > .active > a > .badge {
  color: #337ab7;
  background-color: #fff;
}
.list-group-item > .badge {
  float: right;
}
.list-group-item > .badge + .badge {
  margin-right: 5px;
}
.nav-pills > li > a > .badge {
  margin-left: 3px;
}
.jumbotron {
  padding-top: 30px;
  padding-bottom: 30px;
  margin-bottom: 30px;
  color: inherit;
  background-color: #eeeeee;
}
.jumbotron h1,
.jumbotron .h1 {
  color: inherit;
}
.jumbotron p {
  margin-bottom: 15px;
  font-size: 20px;
  font-weight: 200;
}
.jumbotron > hr {
  border-top-color: #d5d5d5;
}
.container .jumbotron,
.container-fluid .jumbotron {
  border-radius: 3px;
  padding-left: 0px;
  padding-right: 0px;
}
.jumbotron .container {
  max-width: 100%;
}
@media screen and (min-width: 768px) {
  .jumbotron {
    padding-top: 48px;
    padding-bottom: 48px;
  }
  .container .jumbotron,
  .container-fluid .jumbotron {
    padding-left: 60px;
    padding-right: 60px;
  }
  .jumbotron h1,
  .jumbotron .h1 {
    font-size: 59px;
  }
}
.thumbnail {
  display: block;
  padding: 4px;
  margin-bottom: 18px;
  line-height: 1.42857143;
  background-color: #fff;
  border: 1px solid #ddd;
  border-radius: 2px;
  -webkit-transition: border 0.2s ease-in-out;
  -o-transition: border 0.2s ease-in-out;
  transition: border 0.2s ease-in-out;
}
.thumbnail > img,
.thumbnail a > img {
  margin-left: auto;
  margin-right: auto;
}
a.thumbnail:hover,
a.thumbnail:focus,
a.thumbnail.active {
  border-color: #337ab7;
}
.thumbnail .caption {
  padding: 9px;
  color: #000;
}
.alert {
  padding: 15px;
  margin-bottom: 18px;
  border: 1px solid transparent;
  border-radius: 2px;
}
.alert h4 {
  margin-top: 0;
  color: inherit;
}
.alert .alert-link {
  font-weight: bold;
}
.alert > p,
.alert > ul {
  margin-bottom: 0;
}
.alert > p + p {
  margin-top: 5px;
}
.alert-dismissable,
.alert-dismissible {
  padding-right: 35px;
}
.alert-dismissable .close,
.alert-dismissible .close {
  position: relative;
  top: -2px;
  right: -21px;
  color: inherit;
}
.alert-success {
  background-color: #dff0d8;
  border-color: #d6e9c6;
  color: #3c763d;
}
.alert-success hr {
  border-top-color: #c9e2b3;
}
.alert-success .alert-link {
  color: #2b542c;
}
.alert-info {
  background-color: #d9edf7;
  border-color: #bce8f1;
  color: #31708f;
}
.alert-info hr {
  border-top-color: #a6e1ec;
}
.alert-info .alert-link {
  color: #245269;
}
.alert-warning {
  background-color: #fcf8e3;
  border-color: #faebcc;
  color: #8a6d3b;
}
.alert-warning hr {
  border-top-color: #f7e1b5;
}
.alert-warning .alert-link {
  color: #66512c;
}
.alert-danger {
  background-color: #f2dede;
  border-color: #ebccd1;
  color: #a94442;
}
.alert-danger hr {
  border-top-color: #e4b9c0;
}
.alert-danger .alert-link {
  color: #843534;
}
@-webkit-keyframes progress-bar-stripes {
  from {
    background-position: 40px 0;
  }
  to {
    background-position: 0 0;
  }
}
@keyframes progress-bar-stripes {
  from {
    background-position: 40px 0;
  }
  to {
    background-position: 0 0;
  }
}
.progress {
  overflow: hidden;
  height: 18px;
  margin-bottom: 18px;
  background-color: #f5f5f5;
  border-radius: 2px;
  -webkit-box-shadow: inset 0 1px 2px rgba(0, 0, 0, 0.1);
  box-shadow: inset 0 1px 2px rgba(0, 0, 0, 0.1);
}
.progress-bar {
  float: left;
  width: 0%;
  height: 100%;
  font-size: 12px;
  line-height: 18px;
  color: #fff;
  text-align: center;
  background-color: #337ab7;
  -webkit-box-shadow: inset 0 -1px 0 rgba(0, 0, 0, 0.15);
  box-shadow: inset 0 -1px 0 rgba(0, 0, 0, 0.15);
  -webkit-transition: width 0.6s ease;
  -o-transition: width 0.6s ease;
  transition: width 0.6s ease;
}
.progress-striped .progress-bar,
.progress-bar-striped {
  background-image: -webkit-linear-gradient(45deg, rgba(255, 255, 255, 0.15) 25%, transparent 25%, transparent 50%, rgba(255, 255, 255, 0.15) 50%, rgba(255, 255, 255, 0.15) 75%, transparent 75%, transparent);
  background-image: -o-linear-gradient(45deg, rgba(255, 255, 255, 0.15) 25%, transparent 25%, transparent 50%, rgba(255, 255, 255, 0.15) 50%, rgba(255, 255, 255, 0.15) 75%, transparent 75%, transparent);
  background-image: linear-gradient(45deg, rgba(255, 255, 255, 0.15) 25%, transparent 25%, transparent 50%, rgba(255, 255, 255, 0.15) 50%, rgba(255, 255, 255, 0.15) 75%, transparent 75%, transparent);
  background-size: 40px 40px;
}
.progress.active .progress-bar,
.progress-bar.active {
  -webkit-animation: progress-bar-stripes 2s linear infinite;
  -o-animation: progress-bar-stripes 2s linear infinite;
  animation: progress-bar-stripes 2s linear infinite;
}
.progress-bar-success {
  background-color: #5cb85c;
}
.progress-striped .progress-bar-success {
  background-image: -webkit-linear-gradient(45deg, rgba(255, 255, 255, 0.15) 25%, transparent 25%, transparent 50%, rgba(255, 255, 255, 0.15) 50%, rgba(255, 255, 255, 0.15) 75%, transparent 75%, transparent);
  background-image: -o-linear-gradient(45deg, rgba(255, 255, 255, 0.15) 25%, transparent 25%, transparent 50%, rgba(255, 255, 255, 0.15) 50%, rgba(255, 255, 255, 0.15) 75%, transparent 75%, transparent);
  background-image: linear-gradient(45deg, rgba(255, 255, 255, 0.15) 25%, transparent 25%, transparent 50%, rgba(255, 255, 255, 0.15) 50%, rgba(255, 255, 255, 0.15) 75%, transparent 75%, transparent);
}
.progress-bar-info {
  background-color: #5bc0de;
}
.progress-striped .progress-bar-info {
  background-image: -webkit-linear-gradient(45deg, rgba(255, 255, 255, 0.15) 25%, transparent 25%, transparent 50%, rgba(255, 255, 255, 0.15) 50%, rgba(255, 255, 255, 0.15) 75%, transparent 75%, transparent);
  background-image: -o-linear-gradient(45deg, rgba(255, 255, 255, 0.15) 25%, transparent 25%, transparent 50%, rgba(255, 255, 255, 0.15) 50%, rgba(255, 255, 255, 0.15) 75%, transparent 75%, transparent);
  background-image: linear-gradient(45deg, rgba(255, 255, 255, 0.15) 25%, transparent 25%, transparent 50%, rgba(255, 255, 255, 0.15) 50%, rgba(255, 255, 255, 0.15) 75%, transparent 75%, transparent);
}
.progress-bar-warning {
  background-color: #f0ad4e;
}
.progress-striped .progress-bar-warning {
  background-image: -webkit-linear-gradient(45deg, rgba(255, 255, 255, 0.15) 25%, transparent 25%, transparent 50%, rgba(255, 255, 255, 0.15) 50%, rgba(255, 255, 255, 0.15) 75%, transparent 75%, transparent);
  background-image: -o-linear-gradient(45deg, rgba(255, 255, 255, 0.15) 25%, transparent 25%, transparent 50%, rgba(255, 255, 255, 0.15) 50%, rgba(255, 255, 255, 0.15) 75%, transparent 75%, transparent);
  background-image: linear-gradient(45deg, rgba(255, 255, 255, 0.15) 25%, transparent 25%, transparent 50%, rgba(255, 255, 255, 0.15) 50%, rgba(255, 255, 255, 0.15) 75%, transparent 75%, transparent);
}
.progress-bar-danger {
  background-color: #d9534f;
}
.progress-striped .progress-bar-danger {
  background-image: -webkit-linear-gradient(45deg, rgba(255, 255, 255, 0.15) 25%, transparent 25%, transparent 50%, rgba(255, 255, 255, 0.15) 50%, rgba(255, 255, 255, 0.15) 75%, transparent 75%, transparent);
  background-image: -o-linear-gradient(45deg, rgba(255, 255, 255, 0.15) 25%, transparent 25%, transparent 50%, rgba(255, 255, 255, 0.15) 50%, rgba(255, 255, 255, 0.15) 75%, transparent 75%, transparent);
  background-image: linear-gradient(45deg, rgba(255, 255, 255, 0.15) 25%, transparent 25%, transparent 50%, rgba(255, 255, 255, 0.15) 50%, rgba(255, 255, 255, 0.15) 75%, transparent 75%, transparent);
}
.media {
  margin-top: 15px;
}
.media:first-child {
  margin-top: 0;
}
.media,
.media-body {
  zoom: 1;
  overflow: hidden;
}
.media-body {
  width: 10000px;
}
.media-object {
  display: block;
}
.media-object.img-thumbnail {
  max-width: none;
}
.media-right,
.media > .pull-right {
  padding-left: 10px;
}
.media-left,
.media > .pull-left {
  padding-right: 10px;
}
.media-left,
.media-right,
.media-body {
  display: table-cell;
  vertical-align: top;
}
.media-middle {
  vertical-align: middle;
}
.media-bottom {
  vertical-align: bottom;
}
.media-heading {
  margin-top: 0;
  margin-bottom: 5px;
}
.media-list {
  padding-left: 0;
  list-style: none;
}
.list-group {
  margin-bottom: 20px;
  padding-left: 0;
}
.list-group-item {
  position: relative;
  display: block;
  padding: 10px 15px;
  margin-bottom: -1px;
  background-color: #fff;
  border: 1px solid #ddd;
}
.list-group-item:first-child {
  border-top-right-radius: 2px;
  border-top-left-radius: 2px;
}
.list-group-item:last-child {
  margin-bottom: 0;
  border-bottom-right-radius: 2px;
  border-bottom-left-radius: 2px;
}
a.list-group-item,
button.list-group-item {
  color: #555;
}
a.list-group-item .list-group-item-heading,
button.list-group-item .list-group-item-heading {
  color: #333;
}
a.list-group-item:hover,
button.list-group-item:hover,
a.list-group-item:focus,
button.list-group-item:focus {
  text-decoration: none;
  color: #555;
  background-color: #f5f5f5;
}
button.list-group-item {
  width: 100%;
  text-align: left;
}
.list-group-item.disabled,
.list-group-item.disabled:hover,
.list-group-item.disabled:focus {
  background-color: #eeeeee;
  color: #777777;
  cursor: not-allowed;
}
.list-group-item.disabled .list-group-item-heading,
.list-group-item.disabled:hover .list-group-item-heading,
.list-group-item.disabled:focus .list-group-item-heading {
  color: inherit;
}
.list-group-item.disabled .list-group-item-text,
.list-group-item.disabled:hover .list-group-item-text,
.list-group-item.disabled:focus .list-group-item-text {
  color: #777777;
}
.list-group-item.active,
.list-group-item.active:hover,
.list-group-item.active:focus {
  z-index: 2;
  color: #fff;
  background-color: #337ab7;
  border-color: #337ab7;
}
.list-group-item.active .list-group-item-heading,
.list-group-item.active:hover .list-group-item-heading,
.list-group-item.active:focus .list-group-item-heading,
.list-group-item.active .list-group-item-heading > small,
.list-group-item.active:hover .list-group-item-heading > small,
.list-group-item.active:focus .list-group-item-heading > small,
.list-group-item.active .list-group-item-heading > .small,
.list-group-item.active:hover .list-group-item-heading > .small,
.list-group-item.active:focus .list-group-item-heading > .small {
  color: inherit;
}
.list-group-item.active .list-group-item-text,
.list-group-item.active:hover .list-group-item-text,
.list-group-item.active:focus .list-group-item-text {
  color: #c7ddef;
}
.list-group-item-success {
  color: #3c763d;
  background-color: #dff0d8;
}
a.list-group-item-success,
button.list-group-item-success {
  color: #3c763d;
}
a.list-group-item-success .list-group-item-heading,
button.list-group-item-success .list-group-item-heading {
  color: inherit;
}
a.list-group-item-success:hover,
button.list-group-item-success:hover,
a.list-group-item-success:focus,
button.list-group-item-success:focus {
  color: #3c763d;
  background-color: #d0e9c6;
}
a.list-group-item-success.active,
button.list-group-item-success.active,
a.list-group-item-success.active:hover,
button.list-group-item-success.active:hover,
a.list-group-item-success.active:focus,
button.list-group-item-success.active:focus {
  color: #fff;
  background-color: #3c763d;
  border-color: #3c763d;
}
.list-group-item-info {
  color: #31708f;
  background-color: #d9edf7;
}
a.list-group-item-info,
button.list-group-item-info {
  color: #31708f;
}
a.list-group-item-info .list-group-item-heading,
button.list-group-item-info .list-group-item-heading {
  color: inherit;
}
a.list-group-item-info:hover,
button.list-group-item-info:hover,
a.list-group-item-info:focus,
button.list-group-item-info:focus {
  color: #31708f;
  background-color: #c4e3f3;
}
a.list-group-item-info.active,
button.list-group-item-info.active,
a.list-group-item-info.active:hover,
button.list-group-item-info.active:hover,
a.list-group-item-info.active:focus,
button.list-group-item-info.active:focus {
  color: #fff;
  background-color: #31708f;
  border-color: #31708f;
}
.list-group-item-warning {
  color: #8a6d3b;
  background-color: #fcf8e3;
}
a.list-group-item-warning,
button.list-group-item-warning {
  color: #8a6d3b;
}
a.list-group-item-warning .list-group-item-heading,
button.list-group-item-warning .list-group-item-heading {
  color: inherit;
}
a.list-group-item-warning:hover,
button.list-group-item-warning:hover,
a.list-group-item-warning:focus,
button.list-group-item-warning:focus {
  color: #8a6d3b;
  background-color: #faf2cc;
}
a.list-group-item-warning.active,
button.list-group-item-warning.active,
a.list-group-item-warning.active:hover,
button.list-group-item-warning.active:hover,
a.list-group-item-warning.active:focus,
button.list-group-item-warning.active:focus {
  color: #fff;
  background-color: #8a6d3b;
  border-color: #8a6d3b;
}
.list-group-item-danger {
  color: #a94442;
  background-color: #f2dede;
}
a.list-group-item-danger,
button.list-group-item-danger {
  color: #a94442;
}
a.list-group-item-danger .list-group-item-heading,
button.list-group-item-danger .list-group-item-heading {
  color: inherit;
}
a.list-group-item-danger:hover,
button.list-group-item-danger:hover,
a.list-group-item-danger:focus,
button.list-group-item-danger:focus {
  color: #a94442;
  background-color: #ebcccc;
}
a.list-group-item-danger.active,
button.list-group-item-danger.active,
a.list-group-item-danger.active:hover,
button.list-group-item-danger.active:hover,
a.list-group-item-danger.active:focus,
button.list-group-item-danger.active:focus {
  color: #fff;
  background-color: #a94442;
  border-color: #a94442;
}
.list-group-item-heading {
  margin-top: 0;
  margin-bottom: 5px;
}
.list-group-item-text {
  margin-bottom: 0;
  line-height: 1.3;
}
.panel {
  margin-bottom: 18px;
  background-color: #fff;
  border: 1px solid transparent;
  border-radius: 2px;
  -webkit-box-shadow: 0 1px 1px rgba(0, 0, 0, 0.05);
  box-shadow: 0 1px 1px rgba(0, 0, 0, 0.05);
}
.panel-body {
  padding: 15px;
}
.panel-heading {
  padding: 10px 15px;
  border-bottom: 1px solid transparent;
  border-top-right-radius: 1px;
  border-top-left-radius: 1px;
}
.panel-heading > .dropdown .dropdown-toggle {
  color: inherit;
}
.panel-title {
  margin-top: 0;
  margin-bottom: 0;
  font-size: 15px;
  color: inherit;
}
.panel-title > a,
.panel-title > small,
.panel-title > .small,
.panel-title > small > a,
.panel-title > .small > a {
  color: inherit;
}
.panel-footer {
  padding: 10px 15px;
  background-color: #f5f5f5;
  border-top: 1px solid #ddd;
  border-bottom-right-radius: 1px;
  border-bottom-left-radius: 1px;
}
.panel > .list-group,
.panel > .panel-collapse > .list-group {
  margin-bottom: 0;
}
.panel > .list-group .list-group-item,
.panel > .panel-collapse > .list-group .list-group-item {
  border-width: 1px 0;
  border-radius: 0;
}
.panel > .list-group:first-child .list-group-item:first-child,
.panel > .panel-collapse > .list-group:first-child .list-group-item:first-child {
  border-top: 0;
  border-top-right-radius: 1px;
  border-top-left-radius: 1px;
}
.panel > .list-group:last-child .list-group-item:last-child,
.panel > .panel-collapse > .list-group:last-child .list-group-item:last-child {
  border-bottom: 0;
  border-bottom-right-radius: 1px;
  border-bottom-left-radius: 1px;
}
.panel > .panel-heading + .panel-collapse > .list-group .list-group-item:first-child {
  border-top-right-radius: 0;
  border-top-left-radius: 0;
}
.panel-heading + .list-group .list-group-item:first-child {
  border-top-width: 0;
}
.list-group + .panel-footer {
  border-top-width: 0;
}
.panel > .table,
.panel > .table-responsive > .table,
.panel > .panel-collapse > .table {
  margin-bottom: 0;
}
.panel > .table caption,
.panel > .table-responsive > .table caption,
.panel > .panel-collapse > .table caption {
  padding-left: 15px;
  padding-right: 15px;
}
.panel > .table:first-child,
.panel > .table-responsive:first-child > .table:first-child {
  border-top-right-radius: 1px;
  border-top-left-radius: 1px;
}
.panel > .table:first-child > thead:first-child > tr:first-child,
.panel > .table-responsive:first-child > .table:first-child > thead:first-child > tr:first-child,
.panel > .table:first-child > tbody:first-child > tr:first-child,
.panel > .table-responsive:first-child > .table:first-child > tbody:first-child > tr:first-child {
  border-top-left-radius: 1px;
  border-top-right-radius: 1px;
}
.panel > .table:first-child > thead:first-child > tr:first-child td:first-child,
.panel > .table-responsive:first-child > .table:first-child > thead:first-child > tr:first-child td:first-child,
.panel > .table:first-child > tbody:first-child > tr:first-child td:first-child,
.panel > .table-responsive:first-child > .table:first-child > tbody:first-child > tr:first-child td:first-child,
.panel > .table:first-child > thead:first-child > tr:first-child th:first-child,
.panel > .table-responsive:first-child > .table:first-child > thead:first-child > tr:first-child th:first-child,
.panel > .table:first-child > tbody:first-child > tr:first-child th:first-child,
.panel > .table-responsive:first-child > .table:first-child > tbody:first-child > tr:first-child th:first-child {
  border-top-left-radius: 1px;
}
.panel > .table:first-child > thead:first-child > tr:first-child td:last-child,
.panel > .table-responsive:first-child > .table:first-child > thead:first-child > tr:first-child td:last-child,
.panel > .table:first-child > tbody:first-child > tr:first-child td:last-child,
.panel > .table-responsive:first-child > .table:first-child > tbody:first-child > tr:first-child td:last-child,
.panel > .table:first-child > thead:first-child > tr:first-child th:last-child,
.panel > .table-responsive:first-child > .table:first-child > thead:first-child > tr:first-child th:last-child,
.panel > .table:first-child > tbody:first-child > tr:first-child th:last-child,
.panel > .table-responsive:first-child > .table:first-child > tbody:first-child > tr:first-child th:last-child {
  border-top-right-radius: 1px;
}
.panel > .table:last-child,
.panel > .table-responsive:last-child > .table:last-child {
  border-bottom-right-radius: 1px;
  border-bottom-left-radius: 1px;
}
.panel > .table:last-child > tbody:last-child > tr:last-child,
.panel > .table-responsive:last-child > .table:last-child > tbody:last-child > tr:last-child,
.panel > .table:last-child > tfoot:last-child > tr:last-child,
.panel > .table-responsive:last-child > .table:last-child > tfoot:last-child > tr:last-child {
  border-bottom-left-radius: 1px;
  border-bottom-right-radius: 1px;
}
.panel > .table:last-child > tbody:last-child > tr:last-child td:first-child,
.panel > .table-responsive:last-child > .table:last-child > tbody:last-child > tr:last-child td:first-child,
.panel > .table:last-child > tfoot:last-child > tr:last-child td:first-child,
.panel > .table-responsive:last-child > .table:last-child > tfoot:last-child > tr:last-child td:first-child,
.panel > .table:last-child > tbody:last-child > tr:last-child th:first-child,
.panel > .table-responsive:last-child > .table:last-child > tbody:last-child > tr:last-child th:first-child,
.panel > .table:last-child > tfoot:last-child > tr:last-child th:first-child,
.panel > .table-responsive:last-child > .table:last-child > tfoot:last-child > tr:last-child th:first-child {
  border-bottom-left-radius: 1px;
}
.panel > .table:last-child > tbody:last-child > tr:last-child td:last-child,
.panel > .table-responsive:last-child > .table:last-child > tbody:last-child > tr:last-child td:last-child,
.panel > .table:last-child > tfoot:last-child > tr:last-child td:last-child,
.panel > .table-responsive:last-child > .table:last-child > tfoot:last-child > tr:last-child td:last-child,
.panel > .table:last-child > tbody:last-child > tr:last-child th:last-child,
.panel > .table-responsive:last-child > .table:last-child > tbody:last-child > tr:last-child th:last-child,
.panel > .table:last-child > tfoot:last-child > tr:last-child th:last-child,
.panel > .table-responsive:last-child > .table:last-child > tfoot:last-child > tr:last-child th:last-child {
  border-bottom-right-radius: 1px;
}
.panel > .panel-body + .table,
.panel > .panel-body + .table-responsive,
.panel > .table + .panel-body,
.panel > .table-responsive + .panel-body {
  border-top: 1px solid #ddd;
}
.panel > .table > tbody:first-child > tr:first-child th,
.panel > .table > tbody:first-child > tr:first-child td {
  border-top: 0;
}
.panel > .table-bordered,
.panel > .table-responsive > .table-bordered {
  border: 0;
}
.panel > .table-bordered > thead > tr > th:first-child,
.panel > .table-responsive > .table-bordered > thead > tr > th:first-child,
.panel > .table-bordered > tbody > tr > th:first-child,
.panel > .table-responsive > .table-bordered > tbody > tr > th:first-child,
.panel > .table-bordered > tfoot > tr > th:first-child,
.panel > .table-responsive > .table-bordered > tfoot > tr > th:first-child,
.panel > .table-bordered > thead > tr > td:first-child,
.panel > .table-responsive > .table-bordered > thead > tr > td:first-child,
.panel > .table-bordered > tbody > tr > td:first-child,
.panel > .table-responsive > .table-bordered > tbody > tr > td:first-child,
.panel > .table-bordered > tfoot > tr > td:first-child,
.panel > .table-responsive > .table-bordered > tfoot > tr > td:first-child {
  border-left: 0;
}
.panel > .table-bordered > thead > tr > th:last-child,
.panel > .table-responsive > .table-bordered > thead > tr > th:last-child,
.panel > .table-bordered > tbody > tr > th:last-child,
.panel > .table-responsive > .table-bordered > tbody > tr > th:last-child,
.panel > .table-bordered > tfoot > tr > th:last-child,
.panel > .table-responsive > .table-bordered > tfoot > tr > th:last-child,
.panel > .table-bordered > thead > tr > td:last-child,
.panel > .table-responsive > .table-bordered > thead > tr > td:last-child,
.panel > .table-bordered > tbody > tr > td:last-child,
.panel > .table-responsive > .table-bordered > tbody > tr > td:last-child,
.panel > .table-bordered > tfoot > tr > td:last-child,
.panel > .table-responsive > .table-bordered > tfoot > tr > td:last-child {
  border-right: 0;
}
.panel > .table-bordered > thead > tr:first-child > td,
.panel > .table-responsive > .table-bordered > thead > tr:first-child > td,
.panel > .table-bordered > tbody > tr:first-child > td,
.panel > .table-responsive > .table-bordered > tbody > tr:first-child > td,
.panel > .table-bordered > thead > tr:first-child > th,
.panel > .table-responsive > .table-bordered > thead > tr:first-child > th,
.panel > .table-bordered > tbody > tr:first-child > th,
.panel > .table-responsive > .table-bordered > tbody > tr:first-child > th {
  border-bottom: 0;
}
.panel > .table-bordered > tbody > tr:last-child > td,
.panel > .table-responsive > .table-bordered > tbody > tr:last-child > td,
.panel > .table-bordered > tfoot > tr:last-child > td,
.panel > .table-responsive > .table-bordered > tfoot > tr:last-child > td,
.panel > .table-bordered > tbody > tr:last-child > th,
.panel > .table-responsive > .table-bordered > tbody > tr:last-child > th,
.panel > .table-bordered > tfoot > tr:last-child > th,
.panel > .table-responsive > .table-bordered > tfoot > tr:last-child > th {
  border-bottom: 0;
}
.panel > .table-responsive {
  border: 0;
  margin-bottom: 0;
}
.panel-group {
  margin-bottom: 18px;
}
.panel-group .panel {
  margin-bottom: 0;
  border-radius: 2px;
}
.panel-group .panel + .panel {
  margin-top: 5px;
}
.panel-group .panel-heading {
  border-bottom: 0;
}
.panel-group .panel-heading + .panel-collapse > .panel-body,
.panel-group .panel-heading + .panel-collapse > .list-group {
  border-top: 1px solid #ddd;
}
.panel-group .panel-footer {
  border-top: 0;
}
.panel-group .panel-footer + .panel-collapse .panel-body {
  border-bottom: 1px solid #ddd;
}
.panel-default {
  border-color: #ddd;
}
.panel-default > .panel-heading {
  color: #333333;
  background-color: #f5f5f5;
  border-color: #ddd;
}
.panel-default > .panel-heading + .panel-collapse > .panel-body {
  border-top-color: #ddd;
}
.panel-default > .panel-heading .badge {
  color: #f5f5f5;
  background-color: #333333;
}
.panel-default > .panel-footer + .panel-collapse > .panel-body {
  border-bottom-color: #ddd;
}
.panel-primary {
  border-color: #337ab7;
}
.panel-primary > .panel-heading {
  color: #fff;
  background-color: #337ab7;
  border-color: #337ab7;
}
.panel-primary > .panel-heading + .panel-collapse > .panel-body {
  border-top-color: #337ab7;
}
.panel-primary > .panel-heading .badge {
  color: #337ab7;
  background-color: #fff;
}
.panel-primary > .panel-footer + .panel-collapse > .panel-body {
  border-bottom-color: #337ab7;
}
.panel-success {
  border-color: #d6e9c6;
}
.panel-success > .panel-heading {
  color: #3c763d;
  background-color: #dff0d8;
  border-color: #d6e9c6;
}
.panel-success > .panel-heading + .panel-collapse > .panel-body {
  border-top-color: #d6e9c6;
}
.panel-success > .panel-heading .badge {
  color: #dff0d8;
  background-color: #3c763d;
}
.panel-success > .panel-footer + .panel-collapse > .panel-body {
  border-bottom-color: #d6e9c6;
}
.panel-info {
  border-color: #bce8f1;
}
.panel-info > .panel-heading {
  color: #31708f;
  background-color: #d9edf7;
  border-color: #bce8f1;
}
.panel-info > .panel-heading + .panel-collapse > .panel-body {
  border-top-color: #bce8f1;
}
.panel-info > .panel-heading .badge {
  color: #d9edf7;
  background-color: #31708f;
}
.panel-info > .panel-footer + .panel-collapse > .panel-body {
  border-bottom-color: #bce8f1;
}
.panel-warning {
  border-color: #faebcc;
}
.panel-warning > .panel-heading {
  color: #8a6d3b;
  background-color: #fcf8e3;
  border-color: #faebcc;
}
.panel-warning > .panel-heading + .panel-collapse > .panel-body {
  border-top-color: #faebcc;
}
.panel-warning > .panel-heading .badge {
  color: #fcf8e3;
  background-color: #8a6d3b;
}
.panel-warning > .panel-footer + .panel-collapse > .panel-body {
  border-bottom-color: #faebcc;
}
.panel-danger {
  border-color: #ebccd1;
}
.panel-danger > .panel-heading {
  color: #a94442;
  background-color: #f2dede;
  border-color: #ebccd1;
}
.panel-danger > .panel-heading + .panel-collapse > .panel-body {
  border-top-color: #ebccd1;
}
.panel-danger > .panel-heading .badge {
  color: #f2dede;
  background-color: #a94442;
}
.panel-danger > .panel-footer + .panel-collapse > .panel-body {
  border-bottom-color: #ebccd1;
}
.embed-responsive {
  position: relative;
  display: block;
  height: 0;
  padding: 0;
  overflow: hidden;
}
.embed-responsive .embed-responsive-item,
.embed-responsive iframe,
.embed-responsive embed,
.embed-responsive object,
.embed-responsive video {
  position: absolute;
  top: 0;
  left: 0;
  bottom: 0;
  height: 100%;
  width: 100%;
  border: 0;
}
.embed-responsive-16by9 {
  padding-bottom: 56.25%;
}
.embed-responsive-4by3 {
  padding-bottom: 75%;
}
.well {
  min-height: 20px;
  padding: 19px;
  margin-bottom: 20px;
  background-color: #f5f5f5;
  border: 1px solid #e3e3e3;
  border-radius: 2px;
  -webkit-box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.05);
  box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.05);
}
.well blockquote {
  border-color: #ddd;
  border-color: rgba(0, 0, 0, 0.15);
}
.well-lg {
  padding: 24px;
  border-radius: 3px;
}
.well-sm {
  padding: 9px;
  border-radius: 1px;
}
.close {
  float: right;
  font-size: 19.5px;
  font-weight: bold;
  line-height: 1;
  color: #000;
  text-shadow: 0 1px 0 #fff;
  opacity: 0.2;
  filter: alpha(opacity=20);
}
.close:hover,
.close:focus {
  color: #000;
  text-decoration: none;
  cursor: pointer;
  opacity: 0.5;
  filter: alpha(opacity=50);
}
button.close {
  padding: 0;
  cursor: pointer;
  background: transparent;
  border: 0;
  -webkit-appearance: none;
}
.modal-open {
  overflow: hidden;
}
.modal {
  display: none;
  overflow: hidden;
  position: fixed;
  top: 0;
  right: 0;
  bottom: 0;
  left: 0;
  z-index: 1050;
  -webkit-overflow-scrolling: touch;
  outline: 0;
}
.modal.fade .modal-dialog {
  -webkit-transform: translate(0, -25%);
  -ms-transform: translate(0, -25%);
  -o-transform: translate(0, -25%);
  transform: translate(0, -25%);
  -webkit-transition: -webkit-transform 0.3s ease-out;
  -moz-transition: -moz-transform 0.3s ease-out;
  -o-transition: -o-transform 0.3s ease-out;
  transition: transform 0.3s ease-out;
}
.modal.in .modal-dialog {
  -webkit-transform: translate(0, 0);
  -ms-transform: translate(0, 0);
  -o-transform: translate(0, 0);
  transform: translate(0, 0);
}
.modal-open .modal {
  overflow-x: hidden;
  overflow-y: auto;
}
.modal-dialog {
  position: relative;
  width: auto;
  margin: 10px;
}
.modal-content {
  position: relative;
  background-color: #fff;
  border: 1px solid #999;
  border: 1px solid rgba(0, 0, 0, 0.2);
  border-radius: 3px;
  -webkit-box-shadow: 0 3px 9px rgba(0, 0, 0, 0.5);
  box-shadow: 0 3px 9px rgba(0, 0, 0, 0.5);
  background-clip: padding-box;
  outline: 0;
}
.modal-backdrop {
  position: fixed;
  top: 0;
  right: 0;
  bottom: 0;
  left: 0;
  z-index: 1040;
  background-color: #000;
}
.modal-backdrop.fade {
  opacity: 0;
  filter: alpha(opacity=0);
}
.modal-backdrop.in {
  opacity: 0.5;
  filter: alpha(opacity=50);
}
.modal-header {
  padding: 15px;
  border-bottom: 1px solid #e5e5e5;
}
.modal-header .close {
  margin-top: -2px;
}
.modal-title {
  margin: 0;
  line-height: 1.42857143;
}
.modal-body {
  position: relative;
  padding: 15px;
}
.modal-footer {
  padding: 15px;
  text-align: right;
  border-top: 1px solid #e5e5e5;
}
.modal-footer .btn + .btn {
  margin-left: 5px;
  margin-bottom: 0;
}
.modal-footer .btn-group .btn + .btn {
  margin-left: -1px;
}
.modal-footer .btn-block + .btn-block {
  margin-left: 0;
}
.modal-scrollbar-measure {
  position: absolute;
  top: -9999px;
  width: 50px;
  height: 50px;
  overflow: scroll;
}
@media (min-width: 768px) {
  .modal-dialog {
    width: 600px;
    margin: 30px auto;
  }
  .modal-content {
    -webkit-box-shadow: 0 5px 15px rgba(0, 0, 0, 0.5);
    box-shadow: 0 5px 15px rgba(0, 0, 0, 0.5);
  }
  .modal-sm {
    width: 300px;
  }
}
@media (min-width: 992px) {
  .modal-lg {
    width: 900px;
  }
}
.tooltip {
  position: absolute;
  z-index: 1070;
  display: block;
  font-family: "Helvetica Neue", Helvetica, Arial, sans-serif;
  font-style: normal;
  font-weight: normal;
  letter-spacing: normal;
  line-break: auto;
  line-height: 1.42857143;
  text-align: left;
  text-align: start;
  text-decoration: none;
  text-shadow: none;
  text-transform: none;
  white-space: normal;
  word-break: normal;
  word-spacing: normal;
  word-wrap: normal;
  font-size: 12px;
  opacity: 0;
  filter: alpha(opacity=0);
}
.tooltip.in {
  opacity: 0.9;
  filter: alpha(opacity=90);
}
.tooltip.top {
  margin-top: -3px;
  padding: 5px 0;
}
.tooltip.right {
  margin-left: 3px;
  padding: 0 5px;
}
.tooltip.bottom {
  margin-top: 3px;
  padding: 5px 0;
}
.tooltip.left {
  margin-left: -3px;
  padding: 0 5px;
}
.tooltip-inner {
  max-width: 200px;
  padding: 3px 8px;
  color: #fff;
  text-align: center;
  background-color: #000;
  border-radius: 2px;
}
.tooltip-arrow {
  position: absolute;
  width: 0;
  height: 0;
  border-color: transparent;
  border-style: solid;
}
.tooltip.top .tooltip-arrow {
  bottom: 0;
  left: 50%;
  margin-left: -5px;
  border-width: 5px 5px 0;
  border-top-color: #000;
}
.tooltip.top-left .tooltip-arrow {
  bottom: 0;
  right: 5px;
  margin-bottom: -5px;
  border-width: 5px 5px 0;
  border-top-color: #000;
}
.tooltip.top-right .tooltip-arrow {
  bottom: 0;
  left: 5px;
  margin-bottom: -5px;
  border-width: 5px 5px 0;
  border-top-color: #000;
}
.tooltip.right .tooltip-arrow {
  top: 50%;
  left: 0;
  margin-top: -5px;
  border-width: 5px 5px 5px 0;
  border-right-color: #000;
}
.tooltip.left .tooltip-arrow {
  top: 50%;
  right: 0;
  margin-top: -5px;
  border-width: 5px 0 5px 5px;
  border-left-color: #000;
}
.tooltip.bottom .tooltip-arrow {
  top: 0;
  left: 50%;
  margin-left: -5px;
  border-width: 0 5px 5px;
  border-bottom-color: #000;
}
.tooltip.bottom-left .tooltip-arrow {
  top: 0;
  right: 5px;
  margin-top: -5px;
  border-width: 0 5px 5px;
  border-bottom-color: #000;
}
.tooltip.bottom-right .tooltip-arrow {
  top: 0;
  left: 5px;
  margin-top: -5px;
  border-width: 0 5px 5px;
  border-bottom-color: #000;
}
.popover {
  position: absolute;
  top: 0;
  left: 0;
  z-index: 1060;
  display: none;
  max-width: 276px;
  padding: 1px;
  font-family: "Helvetica Neue", Helvetica, Arial, sans-serif;
  font-style: normal;
  font-weight: normal;
  letter-spacing: normal;
  line-break: auto;
  line-height: 1.42857143;
  text-align: left;
  text-align: start;
  text-decoration: none;
  text-shadow: none;
  text-transform: none;
  white-space: normal;
  word-break: normal;
  word-spacing: normal;
  word-wrap: normal;
  font-size: 13px;
  background-color: #fff;
  background-clip: padding-box;
  border: 1px solid #ccc;
  border: 1px solid rgba(0, 0, 0, 0.2);
  border-radius: 3px;
  -webkit-box-shadow: 0 5px 10px rgba(0, 0, 0, 0.2);
  box-shadow: 0 5px 10px rgba(0, 0, 0, 0.2);
}
.popover.top {
  margin-top: -10px;
}
.popover.right {
  margin-left: 10px;
}
.popover.bottom {
  margin-top: 10px;
}
.popover.left {
  margin-left: -10px;
}
.popover-title {
  margin: 0;
  padding: 8px 14px;
  font-size: 13px;
  background-color: #f7f7f7;
  border-bottom: 1px solid #ebebeb;
  border-radius: 2px 2px 0 0;
}
.popover-content {
  padding: 9px 14px;
}
.popover > .arrow,
.popover > .arrow:after {
  position: absolute;
  display: block;
  width: 0;
  height: 0;
  border-color: transparent;
  border-style: solid;
}
.popover > .arrow {
  border-width: 11px;
}
.popover > .arrow:after {
  border-width: 10px;
  content: "";
}
.popover.top > .arrow {
  left: 50%;
  margin-left: -11px;
  border-bottom-width: 0;
  border-top-color: #999999;
  border-top-color: rgba(0, 0, 0, 0.25);
  bottom: -11px;
}
.popover.top > .arrow:after {
  content: " ";
  bottom: 1px;
  margin-left: -10px;
  border-bottom-width: 0;
  border-top-color: #fff;
}
.popover.right > .arrow {
  top: 50%;
  left: -11px;
  margin-top: -11px;
  border-left-width: 0;
  border-right-color: #999999;
  border-right-color: rgba(0, 0, 0, 0.25);
}
.popover.right > .arrow:after {
  content: " ";
  left: 1px;
  bottom: -10px;
  border-left-width: 0;
  border-right-color: #fff;
}
.popover.bottom > .arrow {
  left: 50%;
  margin-left: -11px;
  border-top-width: 0;
  border-bottom-color: #999999;
  border-bottom-color: rgba(0, 0, 0, 0.25);
  top: -11px;
}
.popover.bottom > .arrow:after {
  content: " ";
  top: 1px;
  margin-left: -10px;
  border-top-width: 0;
  border-bottom-color: #fff;
}
.popover.left > .arrow {
  top: 50%;
  right: -11px;
  margin-top: -11px;
  border-right-width: 0;
  border-left-color: #999999;
  border-left-color: rgba(0, 0, 0, 0.25);
}
.popover.left > .arrow:after {
  content: " ";
  right: 1px;
  border-right-width: 0;
  border-left-color: #fff;
  bottom: -10px;
}
.carousel {
  position: relative;
}
.carousel-inner {
  position: relative;
  overflow: hidden;
  width: 100%;
}
.carousel-inner > .item {
  display: none;
  position: relative;
  -webkit-transition: 0.6s ease-in-out left;
  -o-transition: 0.6s ease-in-out left;
  transition: 0.6s ease-in-out left;
}
.carousel-inner > .item > img,
.carousel-inner > .item > a > img {
  line-height: 1;
}
@media all and (transform-3d), (-webkit-transform-3d) {
  .carousel-inner > .item {
    -webkit-transition: -webkit-transform 0.6s ease-in-out;
    -moz-transition: -moz-transform 0.6s ease-in-out;
    -o-transition: -o-transform 0.6s ease-in-out;
    transition: transform 0.6s ease-in-out;
    -webkit-backface-visibility: hidden;
    -moz-backface-visibility: hidden;
    backface-visibility: hidden;
    -webkit-perspective: 1000px;
    -moz-perspective: 1000px;
    perspective: 1000px;
  }
  .carousel-inner > .item.next,
  .carousel-inner > .item.active.right {
    -webkit-transform: translate3d(100%, 0, 0);
    transform: translate3d(100%, 0, 0);
    left: 0;
  }
  .carousel-inner > .item.prev,
  .carousel-inner > .item.active.left {
    -webkit-transform: translate3d(-100%, 0, 0);
    transform: translate3d(-100%, 0, 0);
    left: 0;
  }
  .carousel-inner > .item.next.left,
  .carousel-inner > .item.prev.right,
  .carousel-inner > .item.active {
    -webkit-transform: translate3d(0, 0, 0);
    transform: translate3d(0, 0, 0);
    left: 0;
  }
}
.carousel-inner > .active,
.carousel-inner > .next,
.carousel-inner > .prev {
  display: block;
}
.carousel-inner > .active {
  left: 0;
}
.carousel-inner > .next,
.carousel-inner > .prev {
  position: absolute;
  top: 0;
  width: 100%;
}
.carousel-inner > .next {
  left: 100%;
}
.carousel-inner > .prev {
  left: -100%;
}
.carousel-inner > .next.left,
.carousel-inner > .prev.right {
  left: 0;
}
.carousel-inner > .active.left {
  left: -100%;
}
.carousel-inner > .active.right {
  left: 100%;
}
.carousel-control {
  position: absolute;
  top: 0;
  left: 0;
  bottom: 0;
  width: 15%;
  opacity: 0.5;
  filter: alpha(opacity=50);
  font-size: 20px;
  color: #fff;
  text-align: center;
  text-shadow: 0 1px 2px rgba(0, 0, 0, 0.6);
  background-color: rgba(0, 0, 0, 0);
}
.carousel-control.left {
  background-image: -webkit-linear-gradient(left, rgba(0, 0, 0, 0.5) 0%, rgba(0, 0, 0, 0.0001) 100%);
  background-image: -o-linear-gradient(left, rgba(0, 0, 0, 0.5) 0%, rgba(0, 0, 0, 0.0001) 100%);
  background-image: linear-gradient(to right, rgba(0, 0, 0, 0.5) 0%, rgba(0, 0, 0, 0.0001) 100%);
  background-repeat: repeat-x;
  filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='#80000000', endColorstr='#00000000', GradientType=1);
}
.carousel-control.right {
  left: auto;
  right: 0;
  background-image: -webkit-linear-gradient(left, rgba(0, 0, 0, 0.0001) 0%, rgba(0, 0, 0, 0.5) 100%);
  background-image: -o-linear-gradient(left, rgba(0, 0, 0, 0.0001) 0%, rgba(0, 0, 0, 0.5) 100%);
  background-image: linear-gradient(to right, rgba(0, 0, 0, 0.0001) 0%, rgba(0, 0, 0, 0.5) 100%);
  background-repeat: repeat-x;
  filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='#00000000', endColorstr='#80000000', GradientType=1);
}
.carousel-control:hover,
.carousel-control:focus {
  outline: 0;
  color: #fff;
  text-decoration: none;
  opacity: 0.9;
  filter: alpha(opacity=90);
}
.carousel-control .icon-prev,
.carousel-control .icon-next,
.carousel-control .glyphicon-chevron-left,
.carousel-control .glyphicon-chevron-right {
  position: absolute;
  top: 50%;
  margin-top: -10px;
  z-index: 5;
  display: inline-block;
}
.carousel-control .icon-prev,
.carousel-control .glyphicon-chevron-left {
  left: 50%;
  margin-left: -10px;
}
.carousel-control .icon-next,
.carousel-control .glyphicon-chevron-right {
  right: 50%;
  margin-right: -10px;
}
.carousel-control .icon-prev,
.carousel-control .icon-next {
  width: 20px;
  height: 20px;
  line-height: 1;
  font-family: serif;
}
.carousel-control .icon-prev:before {
  content: '\2039';
}
.carousel-control .icon-next:before {
  content: '\203a';
}
.carousel-indicators {
  position: absolute;
  bottom: 10px;
  left: 50%;
  z-index: 15;
  width: 60%;
  margin-left: -30%;
  padding-left: 0;
  list-style: none;
  text-align: center;
}
.carousel-indicators li {
  display: inline-block;
  width: 10px;
  height: 10px;
  margin: 1px;
  text-indent: -999px;
  border: 1px solid #fff;
  border-radius: 10px;
  cursor: pointer;
  background-color: #000 \9;
  background-color: rgba(0, 0, 0, 0);
}
.carousel-indicators .active {
  margin: 0;
  width: 12px;
  height: 12px;
  background-color: #fff;
}
.carousel-caption {
  position: absolute;
  left: 15%;
  right: 15%;
  bottom: 20px;
  z-index: 10;
  padding-top: 20px;
  padding-bottom: 20px;
  color: #fff;
  text-align: center;
  text-shadow: 0 1px 2px rgba(0, 0, 0, 0.6);
}
.carousel-caption .btn {
  text-shadow: none;
}
@media screen and (min-width: 768px) {
  .carousel-control .glyphicon-chevron-left,
  .carousel-control .glyphicon-chevron-right,
  .carousel-control .icon-prev,
  .carousel-control .icon-next {
    width: 30px;
    height: 30px;
    margin-top: -10px;
    font-size: 30px;
  }
  .carousel-control .glyphicon-chevron-left,
  .carousel-control .icon-prev {
    margin-left: -10px;
  }
  .carousel-control .glyphicon-chevron-right,
  .carousel-control .icon-next {
    margin-right: -10px;
  }
  .carousel-caption {
    left: 20%;
    right: 20%;
    padding-bottom: 30px;
  }
  .carousel-indicators {
    bottom: 20px;
  }
}
.clearfix:before,
.clearfix:after,
.dl-horizontal dd:before,
.dl-horizontal dd:after,
.container:before,
.container:after,
.container-fluid:before,
.container-fluid:after,
.row:before,
.row:after,
.form-horizontal .form-group:before,
.form-horizontal .form-group:after,
.btn-toolbar:before,
.btn-toolbar:after,
.btn-group-vertical > .btn-group:before,
.btn-group-vertical > .btn-group:after,
.nav:before,
.nav:after,
.navbar:before,
.navbar:after,
.navbar-header:before,
.navbar-header:after,
.navbar-collapse:before,
.navbar-collapse:after,
.pager:before,
.pager:after,
.panel-body:before,
.panel-body:after,
.modal-header:before,
.modal-header:after,
.modal-footer:before,
.modal-footer:after,
.item_buttons:before,
.item_buttons:after {
  content: " ";
  display: table;
}
.clearfix:after,
.dl-horizontal dd:after,
.container:after,
.container-fluid:after,
.row:after,
.form-horizontal .form-group:after,
.btn-toolbar:after,
.btn-group-vertical > .btn-group:after,
.nav:after,
.navbar:after,
.navbar-header:after,
.navbar-collapse:after,
.pager:after,
.panel-body:after,
.modal-header:after,
.modal-footer:after,
.item_buttons:after {
  clear: both;
}
.center-block {
  display: block;
  margin-left: auto;
  margin-right: auto;
}
.pull-right {
  float: right !important;
}
.pull-left {
  float: left !important;
}
.hide {
  display: none !important;
}
.show {
  display: block !important;
}
.invisible {
  visibility: hidden;
}
.text-hide {
  font: 0/0 a;
  color: transparent;
  text-shadow: none;
  background-color: transparent;
  border: 0;
}
.hidden {
  display: none !important;
}
.affix {
  position: fixed;
}
@-ms-viewport {
  width: device-width;
}
.visible-xs,
.visible-sm,
.visible-md,
.visible-lg {
  display: none !important;
}
.visible-xs-block,
.visible-xs-inline,
.visible-xs-inline-block,
.visible-sm-block,
.visible-sm-inline,
.visible-sm-inline-block,
.visible-md-block,
.visible-md-inline,
.visible-md-inline-block,
.visible-lg-block,
.visible-lg-inline,
.visible-lg-inline-block {
  display: none !important;
}
@media (max-width: 767px) {
  .visible-xs {
    display: block !important;
  }
  table.visible-xs {
    display: table !important;
  }
  tr.visible-xs {
    display: table-row !important;
  }
  th.visible-xs,
  td.visible-xs {
    display: table-cell !important;
  }
}
@media (max-width: 767px) {
  .visible-xs-block {
    display: block !important;
  }
}
@media (max-width: 767px) {
  .visible-xs-inline {
    display: inline !important;
  }
}
@media (max-width: 767px) {
  .visible-xs-inline-block {
    display: inline-block !important;
  }
}
@media (min-width: 768px) and (max-width: 991px) {
  .visible-sm {
    display: block !important;
  }
  table.visible-sm {
    display: table !important;
  }
  tr.visible-sm {
    display: table-row !important;
  }
  th.visible-sm,
  td.visible-sm {
    display: table-cell !important;
  }
}
@media (min-width: 768px) and (max-width: 991px) {
  .visible-sm-block {
    display: block !important;
  }
}
@media (min-width: 768px) and (max-width: 991px) {
  .visible-sm-inline {
    display: inline !important;
  }
}
@media (min-width: 768px) and (max-width: 991px) {
  .visible-sm-inline-block {
    display: inline-block !important;
  }
}
@media (min-width: 992px) and (max-width: 1199px) {
  .visible-md {
    display: block !important;
  }
  table.visible-md {
    display: table !important;
  }
  tr.visible-md {
    display: table-row !important;
  }
  th.visible-md,
  td.visible-md {
    display: table-cell !important;
  }
}
@media (min-width: 992px) and (max-width: 1199px) {
  .visible-md-block {
    display: block !important;
  }
}
@media (min-width: 992px) and (max-width: 1199px) {
  .visible-md-inline {
    display: inline !important;
  }
}
@media (min-width: 992px) and (max-width: 1199px) {
  .visible-md-inline-block {
    display: inline-block !important;
  }
}
@media (min-width: 1200px) {
  .visible-lg {
    display: block !important;
  }
  table.visible-lg {
    display: table !important;
  }
  tr.visible-lg {
    display: table-row !important;
  }
  th.visible-lg,
  td.visible-lg {
    display: table-cell !important;
  }
}
@media (min-width: 1200px) {
  .visible-lg-block {
    display: block !important;
  }
}
@media (min-width: 1200px) {
  .visible-lg-inline {
    display: inline !important;
  }
}
@media (min-width: 1200px) {
  .visible-lg-inline-block {
    display: inline-block !important;
  }
}
@media (max-width: 767px) {
  .hidden-xs {
    display: none !important;
  }
}
@media (min-width: 768px) and (max-width: 991px) {
  .hidden-sm {
    display: none !important;
  }
}
@media (min-width: 992px) and (max-width: 1199px) {
  .hidden-md {
    display: none !important;
  }
}
@media (min-width: 1200px) {
  .hidden-lg {
    display: none !important;
  }
}
.visible-print {
  display: none !important;
}
@media print {
  .visible-print {
    display: block !important;
  }
  table.visible-print {
    display: table !important;
  }
  tr.visible-print {
    display: table-row !important;
  }
  th.visible-print,
  td.visible-print {
    display: table-cell !important;
  }
}
.visible-print-block {
  display: none !important;
}
@media print {
  .visible-print-block {
    display: block !important;
  }
}
.visible-print-inline {
  display: none !important;
}
@media print {
  .visible-print-inline {
    display: inline !important;
  }
}
.visible-print-inline-block {
  display: none !important;
}
@media print {
  .visible-print-inline-block {
    display: inline-block !important;
  }
}
@media print {
  .hidden-print {
    display: none !important;
  }
}
/*!
*
* Font Awesome
*
*/
/*!
 *  Font Awesome 4.7.0 by @davegandy - http://fontawesome.io - @fontawesome
 *  License - http://fontawesome.io/license (Font: SIL OFL 1.1, CSS: MIT License)
 */
/* FONT PATH
 * -------------------------- */
@font-face {
  font-family: 'FontAwesome';
  src: url('../components/font-awesome/fonts/fontawesome-webfont.eot?v=4.7.0');
  src: url('../components/font-awesome/fonts/fontawesome-webfont.eot?#iefix&v=4.7.0') format('embedded-opentype'), url('../components/font-awesome/fonts/fontawesome-webfont.woff2?v=4.7.0') format('woff2'), url('../components/font-awesome/fonts/fontawesome-webfont.woff?v=4.7.0') format('woff'), url('../components/font-awesome/fonts/fontawesome-webfont.ttf?v=4.7.0') format('truetype'), url('../components/font-awesome/fonts/fontawesome-webfont.svg?v=4.7.0#fontawesomeregular') format('svg');
  font-weight: normal;
  font-style: normal;
}
.fa {
  display: inline-block;
  font: normal normal normal 14px/1 FontAwesome;
  font-size: inherit;
  text-rendering: auto;
  -webkit-font-smoothing: antialiased;
  -moz-osx-font-smoothing: grayscale;
}
/* makes the font 33% larger relative to the icon container */
.fa-lg {
  font-size: 1.33333333em;
  line-height: 0.75em;
  vertical-align: -15%;
}
.fa-2x {
  font-size: 2em;
}
.fa-3x {
  font-size: 3em;
}
.fa-4x {
  font-size: 4em;
}
.fa-5x {
  font-size: 5em;
}
.fa-fw {
  width: 1.28571429em;
  text-align: center;
}
.fa-ul {
  padding-left: 0;
  margin-left: 2.14285714em;
  list-style-type: none;
}
.fa-ul > li {
  position: relative;
}
.fa-li {
  position: absolute;
  left: -2.14285714em;
  width: 2.14285714em;
  top: 0.14285714em;
  text-align: center;
}
.fa-li.fa-lg {
  left: -1.85714286em;
}
.fa-border {
  padding: .2em .25em .15em;
  border: solid 0.08em #eee;
  border-radius: .1em;
}
.fa-pull-left {
  float: left;
}
.fa-pull-right {
  float: right;
}
.fa.fa-pull-left {
  margin-right: .3em;
}
.fa.fa-pull-right {
  margin-left: .3em;
}
/* Deprecated as of 4.4.0 */
.pull-right {
  float: right;
}
.pull-left {
  float: left;
}
.fa.pull-left {
  margin-right: .3em;
}
.fa.pull-right {
  margin-left: .3em;
}
.fa-spin {
  -webkit-animation: fa-spin 2s infinite linear;
  animation: fa-spin 2s infinite linear;
}
.fa-pulse {
  -webkit-animation: fa-spin 1s infinite steps(8);
  animation: fa-spin 1s infinite steps(8);
}
@-webkit-keyframes fa-spin {
  0% {
    -webkit-transform: rotate(0deg);
    transform: rotate(0deg);
  }
  100% {
    -webkit-transform: rotate(359deg);
    transform: rotate(359deg);
  }
}
@keyframes fa-spin {
  0% {
    -webkit-transform: rotate(0deg);
    transform: rotate(0deg);
  }
  100% {
    -webkit-transform: rotate(359deg);
    transform: rotate(359deg);
  }
}
.fa-rotate-90 {
  -ms-filter: "progid:DXImageTransform.Microsoft.BasicImage(rotation=1)";
  -webkit-transform: rotate(90deg);
  -ms-transform: rotate(90deg);
  transform: rotate(90deg);
}
.fa-rotate-180 {
  -ms-filter: "progid:DXImageTransform.Microsoft.BasicImage(rotation=2)";
  -webkit-transform: rotate(180deg);
  -ms-transform: rotate(180deg);
  transform: rotate(180deg);
}
.fa-rotate-270 {
  -ms-filter: "progid:DXImageTransform.Microsoft.BasicImage(rotation=3)";
  -webkit-transform: rotate(270deg);
  -ms-transform: rotate(270deg);
  transform: rotate(270deg);
}
.fa-flip-horizontal {
  -ms-filter: "progid:DXImageTransform.Microsoft.BasicImage(rotation=0, mirror=1)";
  -webkit-transform: scale(-1, 1);
  -ms-transform: scale(-1, 1);
  transform: scale(-1, 1);
}
.fa-flip-vertical {
  -ms-filter: "progid:DXImageTransform.Microsoft.BasicImage(rotation=2, mirror=1)";
  -webkit-transform: scale(1, -1);
  -ms-transform: scale(1, -1);
  transform: scale(1, -1);
}
:root .fa-rotate-90,
:root .fa-rotate-180,
:root .fa-rotate-270,
:root .fa-flip-horizontal,
:root .fa-flip-vertical {
  filter: none;
}
.fa-stack {
  position: relative;
  display: inline-block;
  width: 2em;
  height: 2em;
  line-height: 2em;
  vertical-align: middle;
}
.fa-stack-1x,
.fa-stack-2x {
  position: absolute;
  left: 0;
  width: 100%;
  text-align: center;
}
.fa-stack-1x {
  line-height: inherit;
}
.fa-stack-2x {
  font-size: 2em;
}
.fa-inverse {
  color: #fff;
}
/* Font Awesome uses the Unicode Private Use Area (PUA) to ensure screen
   readers do not read off random characters that represent icons */
.fa-glass:before {
  content: "\f000";
}
.fa-music:before {
  content: "\f001";
}
.fa-search:before {
  content: "\f002";
}
.fa-envelope-o:before {
  content: "\f003";
}
.fa-heart:before {
  content: "\f004";
}
.fa-star:before {
  content: "\f005";
}
.fa-star-o:before {
  content: "\f006";
}
.fa-user:before {
  content: "\f007";
}
.fa-film:before {
  content: "\f008";
}
.fa-th-large:before {
  content: "\f009";
}
.fa-th:before {
  content: "\f00a";
}
.fa-th-list:before {
  content: "\f00b";
}
.fa-check:before {
  content: "\f00c";
}
.fa-remove:before,
.fa-close:before,
.fa-times:before {
  content: "\f00d";
}
.fa-search-plus:before {
  content: "\f00e";
}
.fa-search-minus:before {
  content: "\f010";
}
.fa-power-off:before {
  content: "\f011";
}
.fa-signal:before {
  content: "\f012";
}
.fa-gear:before,
.fa-cog:before {
  content: "\f013";
}
.fa-trash-o:before {
  content: "\f014";
}
.fa-home:before {
  content: "\f015";
}
.fa-file-o:before {
  content: "\f016";
}
.fa-clock-o:before {
  content: "\f017";
}
.fa-road:before {
  content: "\f018";
}
.fa-download:before {
  content: "\f019";
}
.fa-arrow-circle-o-down:before {
  content: "\f01a";
}
.fa-arrow-circle-o-up:before {
  content: "\f01b";
}
.fa-inbox:before {
  content: "\f01c";
}
.fa-play-circle-o:before {
  content: "\f01d";
}
.fa-rotate-right:before,
.fa-repeat:before {
  content: "\f01e";
}
.fa-refresh:before {
  content: "\f021";
}
.fa-list-alt:before {
  content: "\f022";
}
.fa-lock:before {
  content: "\f023";
}
.fa-flag:before {
  content: "\f024";
}
.fa-headphones:before {
  content: "\f025";
}
.fa-volume-off:before {
  content: "\f026";
}
.fa-volume-down:before {
  content: "\f027";
}
.fa-volume-up:before {
  content: "\f028";
}
.fa-qrcode:before {
  content: "\f029";
}
.fa-barcode:before {
  content: "\f02a";
}
.fa-tag:before {
  content: "\f02b";
}
.fa-tags:before {
  content: "\f02c";
}
.fa-book:before {
  content: "\f02d";
}
.fa-bookmark:before {
  content: "\f02e";
}
.fa-print:before {
  content: "\f02f";
}
.fa-camera:before {
  content: "\f030";
}
.fa-font:before {
  content: "\f031";
}
.fa-bold:before {
  content: "\f032";
}
.fa-italic:before {
  content: "\f033";
}
.fa-text-height:before {
  content: "\f034";
}
.fa-text-width:before {
  content: "\f035";
}
.fa-align-left:before {
  content: "\f036";
}
.fa-align-center:before {
  content: "\f037";
}
.fa-align-right:before {
  content: "\f038";
}
.fa-align-justify:before {
  content: "\f039";
}
.fa-list:before {
  content: "\f03a";
}
.fa-dedent:before,
.fa-outdent:before {
  content: "\f03b";
}
.fa-indent:before {
  content: "\f03c";
}
.fa-video-camera:before {
  content: "\f03d";
}
.fa-photo:before,
.fa-image:before,
.fa-picture-o:before {
  content: "\f03e";
}
.fa-pencil:before {
  content: "\f040";
}
.fa-map-marker:before {
  content: "\f041";
}
.fa-adjust:before {
  content: "\f042";
}
.fa-tint:before {
  content: "\f043";
}
.fa-edit:before,
.fa-pencil-square-o:before {
  content: "\f044";
}
.fa-share-square-o:before {
  content: "\f045";
}
.fa-check-square-o:before {
  content: "\f046";
}
.fa-arrows:before {
  content: "\f047";
}
.fa-step-backward:before {
  content: "\f048";
}
.fa-fast-backward:before {
  content: "\f049";
}
.fa-backward:before {
  content: "\f04a";
}
.fa-play:before {
  content: "\f04b";
}
.fa-pause:before {
  content: "\f04c";
}
.fa-stop:before {
  content: "\f04d";
}
.fa-forward:before {
  content: "\f04e";
}
.fa-fast-forward:before {
  content: "\f050";
}
.fa-step-forward:before {
  content: "\f051";
}
.fa-eject:before {
  content: "\f052";
}
.fa-chevron-left:before {
  content: "\f053";
}
.fa-chevron-right:before {
  content: "\f054";
}
.fa-plus-circle:before {
  content: "\f055";
}
.fa-minus-circle:before {
  content: "\f056";
}
.fa-times-circle:before {
  content: "\f057";
}
.fa-check-circle:before {
  content: "\f058";
}
.fa-question-circle:before {
  content: "\f059";
}
.fa-info-circle:before {
  content: "\f05a";
}
.fa-crosshairs:before {
  content: "\f05b";
}
.fa-times-circle-o:before {
  content: "\f05c";
}
.fa-check-circle-o:before {
  content: "\f05d";
}
.fa-ban:before {
  content: "\f05e";
}
.fa-arrow-left:before {
  content: "\f060";
}
.fa-arrow-right:before {
  content: "\f061";
}
.fa-arrow-up:before {
  content: "\f062";
}
.fa-arrow-down:before {
  content: "\f063";
}
.fa-mail-forward:before,
.fa-share:before {
  content: "\f064";
}
.fa-expand:before {
  content: "\f065";
}
.fa-compress:before {
  content: "\f066";
}
.fa-plus:before {
  content: "\f067";
}
.fa-minus:before {
  content: "\f068";
}
.fa-asterisk:before {
  content: "\f069";
}
.fa-exclamation-circle:before {
  content: "\f06a";
}
.fa-gift:before {
  content: "\f06b";
}
.fa-leaf:before {
  content: "\f06c";
}
.fa-fire:before {
  content: "\f06d";
}
.fa-eye:before {
  content: "\f06e";
}
.fa-eye-slash:before {
  content: "\f070";
}
.fa-warning:before,
.fa-exclamation-triangle:before {
  content: "\f071";
}
.fa-plane:before {
  content: "\f072";
}
.fa-calendar:before {
  content: "\f073";
}
.fa-random:before {
  content: "\f074";
}
.fa-comment:before {
  content: "\f075";
}
.fa-magnet:before {
  content: "\f076";
}
.fa-chevron-up:before {
  content: "\f077";
}
.fa-chevron-down:before {
  content: "\f078";
}
.fa-retweet:before {
  content: "\f079";
}
.fa-shopping-cart:before {
  content: "\f07a";
}
.fa-folder:before {
  content: "\f07b";
}
.fa-folder-open:before {
  content: "\f07c";
}
.fa-arrows-v:before {
  content: "\f07d";
}
.fa-arrows-h:before {
  content: "\f07e";
}
.fa-bar-chart-o:before,
.fa-bar-chart:before {
  content: "\f080";
}
.fa-twitter-square:before {
  content: "\f081";
}
.fa-facebook-square:before {
  content: "\f082";
}
.fa-camera-retro:before {
  content: "\f083";
}
.fa-key:before {
  content: "\f084";
}
.fa-gears:before,
.fa-cogs:before {
  content: "\f085";
}
.fa-comments:before {
  content: "\f086";
}
.fa-thumbs-o-up:before {
  content: "\f087";
}
.fa-thumbs-o-down:before {
  content: "\f088";
}
.fa-star-half:before {
  content: "\f089";
}
.fa-heart-o:before {
  content: "\f08a";
}
.fa-sign-out:before {
  content: "\f08b";
}
.fa-linkedin-square:before {
  content: "\f08c";
}
.fa-thumb-tack:before {
  content: "\f08d";
}
.fa-external-link:before {
  content: "\f08e";
}
.fa-sign-in:before {
  content: "\f090";
}
.fa-trophy:before {
  content: "\f091";
}
.fa-github-square:before {
  content: "\f092";
}
.fa-upload:before {
  content: "\f093";
}
.fa-lemon-o:before {
  content: "\f094";
}
.fa-phone:before {
  content: "\f095";
}
.fa-square-o:before {
  content: "\f096";
}
.fa-bookmark-o:before {
  content: "\f097";
}
.fa-phone-square:before {
  content: "\f098";
}
.fa-twitter:before {
  content: "\f099";
}
.fa-facebook-f:before,
.fa-facebook:before {
  content: "\f09a";
}
.fa-github:before {
  content: "\f09b";
}
.fa-unlock:before {
  content: "\f09c";
}
.fa-credit-card:before {
  content: "\f09d";
}
.fa-feed:before,
.fa-rss:before {
  content: "\f09e";
}
.fa-hdd-o:before {
  content: "\f0a0";
}
.fa-bullhorn:before {
  content: "\f0a1";
}
.fa-bell:before {
  content: "\f0f3";
}
.fa-certificate:before {
  content: "\f0a3";
}
.fa-hand-o-right:before {
  content: "\f0a4";
}
.fa-hand-o-left:before {
  content: "\f0a5";
}
.fa-hand-o-up:before {
  content: "\f0a6";
}
.fa-hand-o-down:before {
  content: "\f0a7";
}
.fa-arrow-circle-left:before {
  content: "\f0a8";
}
.fa-arrow-circle-right:before {
  content: "\f0a9";
}
.fa-arrow-circle-up:before {
  content: "\f0aa";
}
.fa-arrow-circle-down:before {
  content: "\f0ab";
}
.fa-globe:before {
  content: "\f0ac";
}
.fa-wrench:before {
  content: "\f0ad";
}
.fa-tasks:before {
  content: "\f0ae";
}
.fa-filter:before {
  content: "\f0b0";
}
.fa-briefcase:before {
  content: "\f0b1";
}
.fa-arrows-alt:before {
  content: "\f0b2";
}
.fa-group:before,
.fa-users:before {
  content: "\f0c0";
}
.fa-chain:before,
.fa-link:before {
  content: "\f0c1";
}
.fa-cloud:before {
  content: "\f0c2";
}
.fa-flask:before {
  content: "\f0c3";
}
.fa-cut:before,
.fa-scissors:before {
  content: "\f0c4";
}
.fa-copy:before,
.fa-files-o:before {
  content: "\f0c5";
}
.fa-paperclip:before {
  content: "\f0c6";
}
.fa-save:before,
.fa-floppy-o:before {
  content: "\f0c7";
}
.fa-square:before {
  content: "\f0c8";
}
.fa-navicon:before,
.fa-reorder:before,
.fa-bars:before {
  content: "\f0c9";
}
.fa-list-ul:before {
  content: "\f0ca";
}
.fa-list-ol:before {
  content: "\f0cb";
}
.fa-strikethrough:before {
  content: "\f0cc";
}
.fa-underline:before {
  content: "\f0cd";
}
.fa-table:before {
  content: "\f0ce";
}
.fa-magic:before {
  content: "\f0d0";
}
.fa-truck:before {
  content: "\f0d1";
}
.fa-pinterest:before {
  content: "\f0d2";
}
.fa-pinterest-square:before {
  content: "\f0d3";
}
.fa-google-plus-square:before {
  content: "\f0d4";
}
.fa-google-plus:before {
  content: "\f0d5";
}
.fa-money:before {
  content: "\f0d6";
}
.fa-caret-down:before {
  content: "\f0d7";
}
.fa-caret-up:before {
  content: "\f0d8";
}
.fa-caret-left:before {
  content: "\f0d9";
}
.fa-caret-right:before {
  content: "\f0da";
}
.fa-columns:before {
  content: "\f0db";
}
.fa-unsorted:before,
.fa-sort:before {
  content: "\f0dc";
}
.fa-sort-down:before,
.fa-sort-desc:before {
  content: "\f0dd";
}
.fa-sort-up:before,
.fa-sort-asc:before {
  content: "\f0de";
}
.fa-envelope:before {
  content: "\f0e0";
}
.fa-linkedin:before {
  content: "\f0e1";
}
.fa-rotate-left:before,
.fa-undo:before {
  content: "\f0e2";
}
.fa-legal:before,
.fa-gavel:before {
  content: "\f0e3";
}
.fa-dashboard:before,
.fa-tachometer:before {
  content: "\f0e4";
}
.fa-comment-o:before {
  content: "\f0e5";
}
.fa-comments-o:before {
  content: "\f0e6";
}
.fa-flash:before,
.fa-bolt:before {
  content: "\f0e7";
}
.fa-sitemap:before {
  content: "\f0e8";
}
.fa-umbrella:before {
  content: "\f0e9";
}
.fa-paste:before,
.fa-clipboard:before {
  content: "\f0ea";
}
.fa-lightbulb-o:before {
  content: "\f0eb";
}
.fa-exchange:before {
  content: "\f0ec";
}
.fa-cloud-download:before {
  content: "\f0ed";
}
.fa-cloud-upload:before {
  content: "\f0ee";
}
.fa-user-md:before {
  content: "\f0f0";
}
.fa-stethoscope:before {
  content: "\f0f1";
}
.fa-suitcase:before {
  content: "\f0f2";
}
.fa-bell-o:before {
  content: "\f0a2";
}
.fa-coffee:before {
  content: "\f0f4";
}
.fa-cutlery:before {
  content: "\f0f5";
}
.fa-file-text-o:before {
  content: "\f0f6";
}
.fa-building-o:before {
  content: "\f0f7";
}
.fa-hospital-o:before {
  content: "\f0f8";
}
.fa-ambulance:before {
  content: "\f0f9";
}
.fa-medkit:before {
  content: "\f0fa";
}
.fa-fighter-jet:before {
  content: "\f0fb";
}
.fa-beer:before {
  content: "\f0fc";
}
.fa-h-square:before {
  content: "\f0fd";
}
.fa-plus-square:before {
  content: "\f0fe";
}
.fa-angle-double-left:before {
  content: "\f100";
}
.fa-angle-double-right:before {
  content: "\f101";
}
.fa-angle-double-up:before {
  content: "\f102";
}
.fa-angle-double-down:before {
  content: "\f103";
}
.fa-angle-left:before {
  content: "\f104";
}
.fa-angle-right:before {
  content: "\f105";
}
.fa-angle-up:before {
  content: "\f106";
}
.fa-angle-down:before {
  content: "\f107";
}
.fa-desktop:before {
  content: "\f108";
}
.fa-laptop:before {
  content: "\f109";
}
.fa-tablet:before {
  content: "\f10a";
}
.fa-mobile-phone:before,
.fa-mobile:before {
  content: "\f10b";
}
.fa-circle-o:before {
  content: "\f10c";
}
.fa-quote-left:before {
  content: "\f10d";
}
.fa-quote-right:before {
  content: "\f10e";
}
.fa-spinner:before {
  content: "\f110";
}
.fa-circle:before {
  content: "\f111";
}
.fa-mail-reply:before,
.fa-reply:before {
  content: "\f112";
}
.fa-github-alt:before {
  content: "\f113";
}
.fa-folder-o:before {
  content: "\f114";
}
.fa-folder-open-o:before {
  content: "\f115";
}
.fa-smile-o:before {
  content: "\f118";
}
.fa-frown-o:before {
  content: "\f119";
}
.fa-meh-o:before {
  content: "\f11a";
}
.fa-gamepad:before {
  content: "\f11b";
}
.fa-keyboard-o:before {
  content: "\f11c";
}
.fa-flag-o:before {
  content: "\f11d";
}
.fa-flag-checkered:before {
  content: "\f11e";
}
.fa-terminal:before {
  content: "\f120";
}
.fa-code:before {
  content: "\f121";
}
.fa-mail-reply-all:before,
.fa-reply-all:before {
  content: "\f122";
}
.fa-star-half-empty:before,
.fa-star-half-full:before,
.fa-star-half-o:before {
  content: "\f123";
}
.fa-location-arrow:before {
  content: "\f124";
}
.fa-crop:before {
  content: "\f125";
}
.fa-code-fork:before {
  content: "\f126";
}
.fa-unlink:before,
.fa-chain-broken:before {
  content: "\f127";
}
.fa-question:before {
  content: "\f128";
}
.fa-info:before {
  content: "\f129";
}
.fa-exclamation:before {
  content: "\f12a";
}
.fa-superscript:before {
  content: "\f12b";
}
.fa-subscript:before {
  content: "\f12c";
}
.fa-eraser:before {
  content: "\f12d";
}
.fa-puzzle-piece:before {
  content: "\f12e";
}
.fa-microphone:before {
  content: "\f130";
}
.fa-microphone-slash:before {
  content: "\f131";
}
.fa-shield:before {
  content: "\f132";
}
.fa-calendar-o:before {
  content: "\f133";
}
.fa-fire-extinguisher:before {
  content: "\f134";
}
.fa-rocket:before {
  content: "\f135";
}
.fa-maxcdn:before {
  content: "\f136";
}
.fa-chevron-circle-left:before {
  content: "\f137";
}
.fa-chevron-circle-right:before {
  content: "\f138";
}
.fa-chevron-circle-up:before {
  content: "\f139";
}
.fa-chevron-circle-down:before {
  content: "\f13a";
}
.fa-html5:before {
  content: "\f13b";
}
.fa-css3:before {
  content: "\f13c";
}
.fa-anchor:before {
  content: "\f13d";
}
.fa-unlock-alt:before {
  content: "\f13e";
}
.fa-bullseye:before {
  content: "\f140";
}
.fa-ellipsis-h:before {
  content: "\f141";
}
.fa-ellipsis-v:before {
  content: "\f142";
}
.fa-rss-square:before {
  content: "\f143";
}
.fa-play-circle:before {
  content: "\f144";
}
.fa-ticket:before {
  content: "\f145";
}
.fa-minus-square:before {
  content: "\f146";
}
.fa-minus-square-o:before {
  content: "\f147";
}
.fa-level-up:before {
  content: "\f148";
}
.fa-level-down:before {
  content: "\f149";
}
.fa-check-square:before {
  content: "\f14a";
}
.fa-pencil-square:before {
  content: "\f14b";
}
.fa-external-link-square:before {
  content: "\f14c";
}
.fa-share-square:before {
  content: "\f14d";
}
.fa-compass:before {
  content: "\f14e";
}
.fa-toggle-down:before,
.fa-caret-square-o-down:before {
  content: "\f150";
}
.fa-toggle-up:before,
.fa-caret-square-o-up:before {
  content: "\f151";
}
.fa-toggle-right:before,
.fa-caret-square-o-right:before {
  content: "\f152";
}
.fa-euro:before,
.fa-eur:before {
  content: "\f153";
}
.fa-gbp:before {
  content: "\f154";
}
.fa-dollar:before,
.fa-usd:before {
  content: "\f155";
}
.fa-rupee:before,
.fa-inr:before {
  content: "\f156";
}
.fa-cny:before,
.fa-rmb:before,
.fa-yen:before,
.fa-jpy:before {
  content: "\f157";
}
.fa-ruble:before,
.fa-rouble:before,
.fa-rub:before {
  content: "\f158";
}
.fa-won:before,
.fa-krw:before {
  content: "\f159";
}
.fa-bitcoin:before,
.fa-btc:before {
  content: "\f15a";
}
.fa-file:before {
  content: "\f15b";
}
.fa-file-text:before {
  content: "\f15c";
}
.fa-sort-alpha-asc:before {
  content: "\f15d";
}
.fa-sort-alpha-desc:before {
  content: "\f15e";
}
.fa-sort-amount-asc:before {
  content: "\f160";
}
.fa-sort-amount-desc:before {
  content: "\f161";
}
.fa-sort-numeric-asc:before {
  content: "\f162";
}
.fa-sort-numeric-desc:before {
  content: "\f163";
}
.fa-thumbs-up:before {
  content: "\f164";
}
.fa-thumbs-down:before {
  content: "\f165";
}
.fa-youtube-square:before {
  content: "\f166";
}
.fa-youtube:before {
  content: "\f167";
}
.fa-xing:before {
  content: "\f168";
}
.fa-xing-square:before {
  content: "\f169";
}
.fa-youtube-play:before {
  content: "\f16a";
}
.fa-dropbox:before {
  content: "\f16b";
}
.fa-stack-overflow:before {
  content: "\f16c";
}
.fa-instagram:before {
  content: "\f16d";
}
.fa-flickr:before {
  content: "\f16e";
}
.fa-adn:before {
  content: "\f170";
}
.fa-bitbucket:before {
  content: "\f171";
}
.fa-bitbucket-square:before {
  content: "\f172";
}
.fa-tumblr:before {
  content: "\f173";
}
.fa-tumblr-square:before {
  content: "\f174";
}
.fa-long-arrow-down:before {
  content: "\f175";
}
.fa-long-arrow-up:before {
  content: "\f176";
}
.fa-long-arrow-left:before {
  content: "\f177";
}
.fa-long-arrow-right:before {
  content: "\f178";
}
.fa-apple:before {
  content: "\f179";
}
.fa-windows:before {
  content: "\f17a";
}
.fa-android:before {
  content: "\f17b";
}
.fa-linux:before {
  content: "\f17c";
}
.fa-dribbble:before {
  content: "\f17d";
}
.fa-skype:before {
  content: "\f17e";
}
.fa-foursquare:before {
  content: "\f180";
}
.fa-trello:before {
  content: "\f181";
}
.fa-female:before {
  content: "\f182";
}
.fa-male:before {
  content: "\f183";
}
.fa-gittip:before,
.fa-gratipay:before {
  content: "\f184";
}
.fa-sun-o:before {
  content: "\f185";
}
.fa-moon-o:before {
  content: "\f186";
}
.fa-archive:before {
  content: "\f187";
}
.fa-bug:before {
  content: "\f188";
}
.fa-vk:before {
  content: "\f189";
}
.fa-weibo:before {
  content: "\f18a";
}
.fa-renren:before {
  content: "\f18b";
}
.fa-pagelines:before {
  content: "\f18c";
}
.fa-stack-exchange:before {
  content: "\f18d";
}
.fa-arrow-circle-o-right:before {
  content: "\f18e";
}
.fa-arrow-circle-o-left:before {
  content: "\f190";
}
.fa-toggle-left:before,
.fa-caret-square-o-left:before {
  content: "\f191";
}
.fa-dot-circle-o:before {
  content: "\f192";
}
.fa-wheelchair:before {
  content: "\f193";
}
.fa-vimeo-square:before {
  content: "\f194";
}
.fa-turkish-lira:before,
.fa-try:before {
  content: "\f195";
}
.fa-plus-square-o:before {
  content: "\f196";
}
.fa-space-shuttle:before {
  content: "\f197";
}
.fa-slack:before {
  content: "\f198";
}
.fa-envelope-square:before {
  content: "\f199";
}
.fa-wordpress:before {
  content: "\f19a";
}
.fa-openid:before {
  content: "\f19b";
}
.fa-institution:before,
.fa-bank:before,
.fa-university:before {
  content: "\f19c";
}
.fa-mortar-board:before,
.fa-graduation-cap:before {
  content: "\f19d";
}
.fa-yahoo:before {
  content: "\f19e";
}
.fa-google:before {
  content: "\f1a0";
}
.fa-reddit:before {
  content: "\f1a1";
}
.fa-reddit-square:before {
  content: "\f1a2";
}
.fa-stumbleupon-circle:before {
  content: "\f1a3";
}
.fa-stumbleupon:before {
  content: "\f1a4";
}
.fa-delicious:before {
  content: "\f1a5";
}
.fa-digg:before {
  content: "\f1a6";
}
.fa-pied-piper-pp:before {
  content: "\f1a7";
}
.fa-pied-piper-alt:before {
  content: "\f1a8";
}
.fa-drupal:before {
  content: "\f1a9";
}
.fa-joomla:before {
  content: "\f1aa";
}
.fa-language:before {
  content: "\f1ab";
}
.fa-fax:before {
  content: "\f1ac";
}
.fa-building:before {
  content: "\f1ad";
}
.fa-child:before {
  content: "\f1ae";
}
.fa-paw:before {
  content: "\f1b0";
}
.fa-spoon:before {
  content: "\f1b1";
}
.fa-cube:before {
  content: "\f1b2";
}
.fa-cubes:before {
  content: "\f1b3";
}
.fa-behance:before {
  content: "\f1b4";
}
.fa-behance-square:before {
  content: "\f1b5";
}
.fa-steam:before {
  content: "\f1b6";
}
.fa-steam-square:before {
  content: "\f1b7";
}
.fa-recycle:before {
  content: "\f1b8";
}
.fa-automobile:before,
.fa-car:before {
  content: "\f1b9";
}
.fa-cab:before,
.fa-taxi:before {
  content: "\f1ba";
}
.fa-tree:before {
  content: "\f1bb";
}
.fa-spotify:before {
  content: "\f1bc";
}
.fa-deviantart:before {
  content: "\f1bd";
}
.fa-soundcloud:before {
  content: "\f1be";
}
.fa-database:before {
  content: "\f1c0";
}
.fa-file-pdf-o:before {
  content: "\f1c1";
}
.fa-file-word-o:before {
  content: "\f1c2";
}
.fa-file-excel-o:before {
  content: "\f1c3";
}
.fa-file-powerpoint-o:before {
  content: "\f1c4";
}
.fa-file-photo-o:before,
.fa-file-picture-o:before,
.fa-file-image-o:before {
  content: "\f1c5";
}
.fa-file-zip-o:before,
.fa-file-archive-o:before {
  content: "\f1c6";
}
.fa-file-sound-o:before,
.fa-file-audio-o:before {
  content: "\f1c7";
}
.fa-file-movie-o:before,
.fa-file-video-o:before {
  content: "\f1c8";
}
.fa-file-code-o:before {
  content: "\f1c9";
}
.fa-vine:before {
  content: "\f1ca";
}
.fa-codepen:before {
  content: "\f1cb";
}
.fa-jsfiddle:before {
  content: "\f1cc";
}
.fa-life-bouy:before,
.fa-life-buoy:before,
.fa-life-saver:before,
.fa-support:before,
.fa-life-ring:before {
  content: "\f1cd";
}
.fa-circle-o-notch:before {
  content: "\f1ce";
}
.fa-ra:before,
.fa-resistance:before,
.fa-rebel:before {
  content: "\f1d0";
}
.fa-ge:before,
.fa-empire:before {
  content: "\f1d1";
}
.fa-git-square:before {
  content: "\f1d2";
}
.fa-git:before {
  content: "\f1d3";
}
.fa-y-combinator-square:before,
.fa-yc-square:before,
.fa-hacker-news:before {
  content: "\f1d4";
}
.fa-tencent-weibo:before {
  content: "\f1d5";
}
.fa-qq:before {
  content: "\f1d6";
}
.fa-wechat:before,
.fa-weixin:before {
  content: "\f1d7";
}
.fa-send:before,
.fa-paper-plane:before {
  content: "\f1d8";
}
.fa-send-o:before,
.fa-paper-plane-o:before {
  content: "\f1d9";
}
.fa-history:before {
  content: "\f1da";
}
.fa-circle-thin:before {
  content: "\f1db";
}
.fa-header:before {
  content: "\f1dc";
}
.fa-paragraph:before {
  content: "\f1dd";
}
.fa-sliders:before {
  content: "\f1de";
}
.fa-share-alt:before {
  content: "\f1e0";
}
.fa-share-alt-square:before {
  content: "\f1e1";
}
.fa-bomb:before {
  content: "\f1e2";
}
.fa-soccer-ball-o:before,
.fa-futbol-o:before {
  content: "\f1e3";
}
.fa-tty:before {
  content: "\f1e4";
}
.fa-binoculars:before {
  content: "\f1e5";
}
.fa-plug:before {
  content: "\f1e6";
}
.fa-slideshare:before {
  content: "\f1e7";
}
.fa-twitch:before {
  content: "\f1e8";
}
.fa-yelp:before {
  content: "\f1e9";
}
.fa-newspaper-o:before {
  content: "\f1ea";
}
.fa-wifi:before {
  content: "\f1eb";
}
.fa-calculator:before {
  content: "\f1ec";
}
.fa-paypal:before {
  content: "\f1ed";
}
.fa-google-wallet:before {
  content: "\f1ee";
}
.fa-cc-visa:before {
  content: "\f1f0";
}
.fa-cc-mastercard:before {
  content: "\f1f1";
}
.fa-cc-discover:before {
  content: "\f1f2";
}
.fa-cc-amex:before {
  content: "\f1f3";
}
.fa-cc-paypal:before {
  content: "\f1f4";
}
.fa-cc-stripe:before {
  content: "\f1f5";
}
.fa-bell-slash:before {
  content: "\f1f6";
}
.fa-bell-slash-o:before {
  content: "\f1f7";
}
.fa-trash:before {
  content: "\f1f8";
}
.fa-copyright:before {
  content: "\f1f9";
}
.fa-at:before {
  content: "\f1fa";
}
.fa-eyedropper:before {
  content: "\f1fb";
}
.fa-paint-brush:before {
  content: "\f1fc";
}
.fa-birthday-cake:before {
  content: "\f1fd";
}
.fa-area-chart:before {
  content: "\f1fe";
}
.fa-pie-chart:before {
  content: "\f200";
}
.fa-line-chart:before {
  content: "\f201";
}
.fa-lastfm:before {
  content: "\f202";
}
.fa-lastfm-square:before {
  content: "\f203";
}
.fa-toggle-off:before {
  content: "\f204";
}
.fa-toggle-on:before {
  content: "\f205";
}
.fa-bicycle:before {
  content: "\f206";
}
.fa-bus:before {
  content: "\f207";
}
.fa-ioxhost:before {
  content: "\f208";
}
.fa-angellist:before {
  content: "\f209";
}
.fa-cc:before {
  content: "\f20a";
}
.fa-shekel:before,
.fa-sheqel:before,
.fa-ils:before {
  content: "\f20b";
}
.fa-meanpath:before {
  content: "\f20c";
}
.fa-buysellads:before {
  content: "\f20d";
}
.fa-connectdevelop:before {
  content: "\f20e";
}
.fa-dashcube:before {
  content: "\f210";
}
.fa-forumbee:before {
  content: "\f211";
}
.fa-leanpub:before {
  content: "\f212";
}
.fa-sellsy:before {
  content: "\f213";
}
.fa-shirtsinbulk:before {
  content: "\f214";
}
.fa-simplybuilt:before {
  content: "\f215";
}
.fa-skyatlas:before {
  content: "\f216";
}
.fa-cart-plus:before {
  content: "\f217";
}
.fa-cart-arrow-down:before {
  content: "\f218";
}
.fa-diamond:before {
  content: "\f219";
}
.fa-ship:before {
  content: "\f21a";
}
.fa-user-secret:before {
  content: "\f21b";
}
.fa-motorcycle:before {
  content: "\f21c";
}
.fa-street-view:before {
  content: "\f21d";
}
.fa-heartbeat:before {
  content: "\f21e";
}
.fa-venus:before {
  content: "\f221";
}
.fa-mars:before {
  content: "\f222";
}
.fa-mercury:before {
  content: "\f223";
}
.fa-intersex:before,
.fa-transgender:before {
  content: "\f224";
}
.fa-transgender-alt:before {
  content: "\f225";
}
.fa-venus-double:before {
  content: "\f226";
}
.fa-mars-double:before {
  content: "\f227";
}
.fa-venus-mars:before {
  content: "\f228";
}
.fa-mars-stroke:before {
  content: "\f229";
}
.fa-mars-stroke-v:before {
  content: "\f22a";
}
.fa-mars-stroke-h:before {
  content: "\f22b";
}
.fa-neuter:before {
  content: "\f22c";
}
.fa-genderless:before {
  content: "\f22d";
}
.fa-facebook-official:before {
  content: "\f230";
}
.fa-pinterest-p:before {
  content: "\f231";
}
.fa-whatsapp:before {
  content: "\f232";
}
.fa-server:before {
  content: "\f233";
}
.fa-user-plus:before {
  content: "\f234";
}
.fa-user-times:before {
  content: "\f235";
}
.fa-hotel:before,
.fa-bed:before {
  content: "\f236";
}
.fa-viacoin:before {
  content: "\f237";
}
.fa-train:before {
  content: "\f238";
}
.fa-subway:before {
  content: "\f239";
}
.fa-medium:before {
  content: "\f23a";
}
.fa-yc:before,
.fa-y-combinator:before {
  content: "\f23b";
}
.fa-optin-monster:before {
  content: "\f23c";
}
.fa-opencart:before {
  content: "\f23d";
}
.fa-expeditedssl:before {
  content: "\f23e";
}
.fa-battery-4:before,
.fa-battery:before,
.fa-battery-full:before {
  content: "\f240";
}
.fa-battery-3:before,
.fa-battery-three-quarters:before {
  content: "\f241";
}
.fa-battery-2:before,
.fa-battery-half:before {
  content: "\f242";
}
.fa-battery-1:before,
.fa-battery-quarter:before {
  content: "\f243";
}
.fa-battery-0:before,
.fa-battery-empty:before {
  content: "\f244";
}
.fa-mouse-pointer:before {
  content: "\f245";
}
.fa-i-cursor:before {
  content: "\f246";
}
.fa-object-group:before {
  content: "\f247";
}
.fa-object-ungroup:before {
  content: "\f248";
}
.fa-sticky-note:before {
  content: "\f249";
}
.fa-sticky-note-o:before {
  content: "\f24a";
}
.fa-cc-jcb:before {
  content: "\f24b";
}
.fa-cc-diners-club:before {
  content: "\f24c";
}
.fa-clone:before {
  content: "\f24d";
}
.fa-balance-scale:before {
  content: "\f24e";
}
.fa-hourglass-o:before {
  content: "\f250";
}
.fa-hourglass-1:before,
.fa-hourglass-start:before {
  content: "\f251";
}
.fa-hourglass-2:before,
.fa-hourglass-half:before {
  content: "\f252";
}
.fa-hourglass-3:before,
.fa-hourglass-end:before {
  content: "\f253";
}
.fa-hourglass:before {
  content: "\f254";
}
.fa-hand-grab-o:before,
.fa-hand-rock-o:before {
  content: "\f255";
}
.fa-hand-stop-o:before,
.fa-hand-paper-o:before {
  content: "\f256";
}
.fa-hand-scissors-o:before {
  content: "\f257";
}
.fa-hand-lizard-o:before {
  content: "\f258";
}
.fa-hand-spock-o:before {
  content: "\f259";
}
.fa-hand-pointer-o:before {
  content: "\f25a";
}
.fa-hand-peace-o:before {
  content: "\f25b";
}
.fa-trademark:before {
  content: "\f25c";
}
.fa-registered:before {
  content: "\f25d";
}
.fa-creative-commons:before {
  content: "\f25e";
}
.fa-gg:before {
  content: "\f260";
}
.fa-gg-circle:before {
  content: "\f261";
}
.fa-tripadvisor:before {
  content: "\f262";
}
.fa-odnoklassniki:before {
  content: "\f263";
}
.fa-odnoklassniki-square:before {
  content: "\f264";
}
.fa-get-pocket:before {
  content: "\f265";
}
.fa-wikipedia-w:before {
  content: "\f266";
}
.fa-safari:before {
  content: "\f267";
}
.fa-chrome:before {
  content: "\f268";
}
.fa-firefox:before {
  content: "\f269";
}
.fa-opera:before {
  content: "\f26a";
}
.fa-internet-explorer:before {
  content: "\f26b";
}
.fa-tv:before,
.fa-television:before {
  content: "\f26c";
}
.fa-contao:before {
  content: "\f26d";
}
.fa-500px:before {
  content: "\f26e";
}
.fa-amazon:before {
  content: "\f270";
}
.fa-calendar-plus-o:before {
  content: "\f271";
}
.fa-calendar-minus-o:before {
  content: "\f272";
}
.fa-calendar-times-o:before {
  content: "\f273";
}
.fa-calendar-check-o:before {
  content: "\f274";
}
.fa-industry:before {
  content: "\f275";
}
.fa-map-pin:before {
  content: "\f276";
}
.fa-map-signs:before {
  content: "\f277";
}
.fa-map-o:before {
  content: "\f278";
}
.fa-map:before {
  content: "\f279";
}
.fa-commenting:before {
  content: "\f27a";
}
.fa-commenting-o:before {
  content: "\f27b";
}
.fa-houzz:before {
  content: "\f27c";
}
.fa-vimeo:before {
  content: "\f27d";
}
.fa-black-tie:before {
  content: "\f27e";
}
.fa-fonticons:before {
  content: "\f280";
}
.fa-reddit-alien:before {
  content: "\f281";
}
.fa-edge:before {
  content: "\f282";
}
.fa-credit-card-alt:before {
  content: "\f283";
}
.fa-codiepie:before {
  content: "\f284";
}
.fa-modx:before {
  content: "\f285";
}
.fa-fort-awesome:before {
  content: "\f286";
}
.fa-usb:before {
  content: "\f287";
}
.fa-product-hunt:before {
  content: "\f288";
}
.fa-mixcloud:before {
  content: "\f289";
}
.fa-scribd:before {
  content: "\f28a";
}
.fa-pause-circle:before {
  content: "\f28b";
}
.fa-pause-circle-o:before {
  content: "\f28c";
}
.fa-stop-circle:before {
  content: "\f28d";
}
.fa-stop-circle-o:before {
  content: "\f28e";
}
.fa-shopping-bag:before {
  content: "\f290";
}
.fa-shopping-basket:before {
  content: "\f291";
}
.fa-hashtag:before {
  content: "\f292";
}
.fa-bluetooth:before {
  content: "\f293";
}
.fa-bluetooth-b:before {
  content: "\f294";
}
.fa-percent:before {
  content: "\f295";
}
.fa-gitlab:before {
  content: "\f296";
}
.fa-wpbeginner:before {
  content: "\f297";
}
.fa-wpforms:before {
  content: "\f298";
}
.fa-envira:before {
  content: "\f299";
}
.fa-universal-access:before {
  content: "\f29a";
}
.fa-wheelchair-alt:before {
  content: "\f29b";
}
.fa-question-circle-o:before {
  content: "\f29c";
}
.fa-blind:before {
  content: "\f29d";
}
.fa-audio-description:before {
  content: "\f29e";
}
.fa-volume-control-phone:before {
  content: "\f2a0";
}
.fa-braille:before {
  content: "\f2a1";
}
.fa-assistive-listening-systems:before {
  content: "\f2a2";
}
.fa-asl-interpreting:before,
.fa-american-sign-language-interpreting:before {
  content: "\f2a3";
}
.fa-deafness:before,
.fa-hard-of-hearing:before,
.fa-deaf:before {
  content: "\f2a4";
}
.fa-glide:before {
  content: "\f2a5";
}
.fa-glide-g:before {
  content: "\f2a6";
}
.fa-signing:before,
.fa-sign-language:before {
  content: "\f2a7";
}
.fa-low-vision:before {
  content: "\f2a8";
}
.fa-viadeo:before {
  content: "\f2a9";
}
.fa-viadeo-square:before {
  content: "\f2aa";
}
.fa-snapchat:before {
  content: "\f2ab";
}
.fa-snapchat-ghost:before {
  content: "\f2ac";
}
.fa-snapchat-square:before {
  content: "\f2ad";
}
.fa-pied-piper:before {
  content: "\f2ae";
}
.fa-first-order:before {
  content: "\f2b0";
}
.fa-yoast:before {
  content: "\f2b1";
}
.fa-themeisle:before {
  content: "\f2b2";
}
.fa-google-plus-circle:before,
.fa-google-plus-official:before {
  content: "\f2b3";
}
.fa-fa:before,
.fa-font-awesome:before {
  content: "\f2b4";
}
.fa-handshake-o:before {
  content: "\f2b5";
}
.fa-envelope-open:before {
  content: "\f2b6";
}
.fa-envelope-open-o:before {
  content: "\f2b7";
}
.fa-linode:before {
  content: "\f2b8";
}
.fa-address-book:before {
  content: "\f2b9";
}
.fa-address-book-o:before {
  content: "\f2ba";
}
.fa-vcard:before,
.fa-address-card:before {
  content: "\f2bb";
}
.fa-vcard-o:before,
.fa-address-card-o:before {
  content: "\f2bc";
}
.fa-user-circle:before {
  content: "\f2bd";
}
.fa-user-circle-o:before {
  content: "\f2be";
}
.fa-user-o:before {
  content: "\f2c0";
}
.fa-id-badge:before {
  content: "\f2c1";
}
.fa-drivers-license:before,
.fa-id-card:before {
  content: "\f2c2";
}
.fa-drivers-license-o:before,
.fa-id-card-o:before {
  content: "\f2c3";
}
.fa-quora:before {
  content: "\f2c4";
}
.fa-free-code-camp:before {
  content: "\f2c5";
}
.fa-telegram:before {
  content: "\f2c6";
}
.fa-thermometer-4:before,
.fa-thermometer:before,
.fa-thermometer-full:before {
  content: "\f2c7";
}
.fa-thermometer-3:before,
.fa-thermometer-three-quarters:before {
  content: "\f2c8";
}
.fa-thermometer-2:before,
.fa-thermometer-half:before {
  content: "\f2c9";
}
.fa-thermometer-1:before,
.fa-thermometer-quarter:before {
  content: "\f2ca";
}
.fa-thermometer-0:before,
.fa-thermometer-empty:before {
  content: "\f2cb";
}
.fa-shower:before {
  content: "\f2cc";
}
.fa-bathtub:before,
.fa-s15:before,
.fa-bath:before {
  content: "\f2cd";
}
.fa-podcast:before {
  content: "\f2ce";
}
.fa-window-maximize:before {
  content: "\f2d0";
}
.fa-window-minimize:before {
  content: "\f2d1";
}
.fa-window-restore:before {
  content: "\f2d2";
}
.fa-times-rectangle:before,
.fa-window-close:before {
  content: "\f2d3";
}
.fa-times-rectangle-o:before,
.fa-window-close-o:before {
  content: "\f2d4";
}
.fa-bandcamp:before {
  content: "\f2d5";
}
.fa-grav:before {
  content: "\f2d6";
}
.fa-etsy:before {
  content: "\f2d7";
}
.fa-imdb:before {
  content: "\f2d8";
}
.fa-ravelry:before {
  content: "\f2d9";
}
.fa-eercast:before {
  content: "\f2da";
}
.fa-microchip:before {
  content: "\f2db";
}
.fa-snowflake-o:before {
  content: "\f2dc";
}
.fa-superpowers:before {
  content: "\f2dd";
}
.fa-wpexplorer:before {
  content: "\f2de";
}
.fa-meetup:before {
  content: "\f2e0";
}
.sr-only {
  position: absolute;
  width: 1px;
  height: 1px;
  padding: 0;
  margin: -1px;
  overflow: hidden;
  clip: rect(0, 0, 0, 0);
  border: 0;
}
.sr-only-focusable:active,
.sr-only-focusable:focus {
  position: static;
  width: auto;
  height: auto;
  margin: 0;
  overflow: visible;
  clip: auto;
}
.sr-only-focusable:active,
.sr-only-focusable:focus {
  position: static;
  width: auto;
  height: auto;
  margin: 0;
  overflow: visible;
  clip: auto;
}
/*!
*
* IPython base
*
*/
.modal.fade .modal-dialog {
  -webkit-transform: translate(0, 0);
  -ms-transform: translate(0, 0);
  -o-transform: translate(0, 0);
  transform: translate(0, 0);
}
code {
  color: #000;
}
pre {
  font-size: inherit;
  line-height: inherit;
}
label {
  font-weight: normal;
}
/* Make the page background atleast 100% the height of the view port */
/* Make the page itself atleast 70% the height of the view port */
.border-box-sizing {
  box-sizing: border-box;
  -moz-box-sizing: border-box;
  -webkit-box-sizing: border-box;
}
.corner-all {
  border-radius: 2px;
}
.no-padding {
  padding: 0px;
}
/* Flexible box model classes */
/* Taken from Alex Russell http://infrequently.org/2009/08/css-3-progress/ */
/* This file is a compatability layer.  It allows the usage of flexible box 
model layouts accross multiple browsers, including older browsers.  The newest,
universal implementation of the flexible box model is used when available (see
`Modern browsers` comments below).  Browsers that are known to implement this 
new spec completely include:

    Firefox 28.0+
    Chrome 29.0+
    Internet Explorer 11+ 
    Opera 17.0+

Browsers not listed, including Safari, are supported via the styling under the
`Old browsers` comments below.
*/
.hbox {
  /* Old browsers */
  display: -webkit-box;
  -webkit-box-orient: horizontal;
  -webkit-box-align: stretch;
  display: -moz-box;
  -moz-box-orient: horizontal;
  -moz-box-align: stretch;
  display: box;
  box-orient: horizontal;
  box-align: stretch;
  /* Modern browsers */
  display: flex;
  flex-direction: row;
  align-items: stretch;
}
.hbox > * {
  /* Old browsers */
  -webkit-box-flex: 0;
  -moz-box-flex: 0;
  box-flex: 0;
  /* Modern browsers */
  flex: none;
}
.vbox {
  /* Old browsers */
  display: -webkit-box;
  -webkit-box-orient: vertical;
  -webkit-box-align: stretch;
  display: -moz-box;
  -moz-box-orient: vertical;
  -moz-box-align: stretch;
  display: box;
  box-orient: vertical;
  box-align: stretch;
  /* Modern browsers */
  display: flex;
  flex-direction: column;
  align-items: stretch;
}
.vbox > * {
  /* Old browsers */
  -webkit-box-flex: 0;
  -moz-box-flex: 0;
  box-flex: 0;
  /* Modern browsers */
  flex: none;
}
.hbox.reverse,
.vbox.reverse,
.reverse {
  /* Old browsers */
  -webkit-box-direction: reverse;
  -moz-box-direction: reverse;
  box-direction: reverse;
  /* Modern browsers */
  flex-direction: row-reverse;
}
.hbox.box-flex0,
.vbox.box-flex0,
.box-flex0 {
  /* Old browsers */
  -webkit-box-flex: 0;
  -moz-box-flex: 0;
  box-flex: 0;
  /* Modern browsers */
  flex: none;
  width: auto;
}
.hbox.box-flex1,
.vbox.box-flex1,
.box-flex1 {
  /* Old browsers */
  -webkit-box-flex: 1;
  -moz-box-flex: 1;
  box-flex: 1;
  /* Modern browsers */
  flex: 1;
}
.hbox.box-flex,
.vbox.box-flex,
.box-flex {
  /* Old browsers */
  /* Old browsers */
  -webkit-box-flex: 1;
  -moz-box-flex: 1;
  box-flex: 1;
  /* Modern browsers */
  flex: 1;
}
.hbox.box-flex2,
.vbox.box-flex2,
.box-flex2 {
  /* Old browsers */
  -webkit-box-flex: 2;
  -moz-box-flex: 2;
  box-flex: 2;
  /* Modern browsers */
  flex: 2;
}
.box-group1 {
  /*  Deprecated */
  -webkit-box-flex-group: 1;
  -moz-box-flex-group: 1;
  box-flex-group: 1;
}
.box-group2 {
  /* Deprecated */
  -webkit-box-flex-group: 2;
  -moz-box-flex-group: 2;
  box-flex-group: 2;
}
.hbox.start,
.vbox.start,
.start {
  /* Old browsers */
  -webkit-box-pack: start;
  -moz-box-pack: start;
  box-pack: start;
  /* Modern browsers */
  justify-content: flex-start;
}
.hbox.end,
.vbox.end,
.end {
  /* Old browsers */
  -webkit-box-pack: end;
  -moz-box-pack: end;
  box-pack: end;
  /* Modern browsers */
  justify-content: flex-end;
}
.hbox.center,
.vbox.center,
.center {
  /* Old browsers */
  -webkit-box-pack: center;
  -moz-box-pack: center;
  box-pack: center;
  /* Modern browsers */
  justify-content: center;
}
.hbox.baseline,
.vbox.baseline,
.baseline {
  /* Old browsers */
  -webkit-box-pack: baseline;
  -moz-box-pack: baseline;
  box-pack: baseline;
  /* Modern browsers */
  justify-content: baseline;
}
.hbox.stretch,
.vbox.stretch,
.stretch {
  /* Old browsers */
  -webkit-box-pack: stretch;
  -moz-box-pack: stretch;
  box-pack: stretch;
  /* Modern browsers */
  justify-content: stretch;
}
.hbox.align-start,
.vbox.align-start,
.align-start {
  /* Old browsers */
  -webkit-box-align: start;
  -moz-box-align: start;
  box-align: start;
  /* Modern browsers */
  align-items: flex-start;
}
.hbox.align-end,
.vbox.align-end,
.align-end {
  /* Old browsers */
  -webkit-box-align: end;
  -moz-box-align: end;
  box-align: end;
  /* Modern browsers */
  align-items: flex-end;
}
.hbox.align-center,
.vbox.align-center,
.align-center {
  /* Old browsers */
  -webkit-box-align: center;
  -moz-box-align: center;
  box-align: center;
  /* Modern browsers */
  align-items: center;
}
.hbox.align-baseline,
.vbox.align-baseline,
.align-baseline {
  /* Old browsers */
  -webkit-box-align: baseline;
  -moz-box-align: baseline;
  box-align: baseline;
  /* Modern browsers */
  align-items: baseline;
}
.hbox.align-stretch,
.vbox.align-stretch,
.align-stretch {
  /* Old browsers */
  -webkit-box-align: stretch;
  -moz-box-align: stretch;
  box-align: stretch;
  /* Modern browsers */
  align-items: stretch;
}
div.error {
  margin: 2em;
  text-align: center;
}
div.error > h1 {
  font-size: 500%;
  line-height: normal;
}
div.error > p {
  font-size: 200%;
  line-height: normal;
}
div.traceback-wrapper {
  text-align: left;
  max-width: 800px;
  margin: auto;
}
div.traceback-wrapper pre.traceback {
  max-height: 600px;
  overflow: auto;
}
/**
 * Primary styles
 *
 * Author: Jupyter Development Team
 */
body {
  background-color: #fff;
  /* This makes sure that the body covers the entire window and needs to
       be in a different element than the display: box in wrapper below */
  position: absolute;
  left: 0px;
  right: 0px;
  top: 0px;
  bottom: 0px;
  overflow: visible;
}
body > #header {
  /* Initially hidden to prevent FLOUC */
  display: none;
  background-color: #fff;
  /* Display over codemirror */
  position: relative;
  z-index: 100;
}
body > #header #header-container {
  display: flex;
  flex-direction: row;
  justify-content: space-between;
  padding: 5px;
  padding-bottom: 5px;
  padding-top: 5px;
  box-sizing: border-box;
  -moz-box-sizing: border-box;
  -webkit-box-sizing: border-box;
}
body > #header .header-bar {
  width: 100%;
  height: 1px;
  background: #e7e7e7;
  margin-bottom: -1px;
}
@media print {
  body > #header {
    display: none !important;
  }
}
#header-spacer {
  width: 100%;
  visibility: hidden;
}
@media print {
  #header-spacer {
    display: none;
  }
}
#ipython_notebook {
  padding-left: 0px;
  padding-top: 1px;
  padding-bottom: 1px;
}
[dir="rtl"] #ipython_notebook {
  margin-right: 10px;
  margin-left: 0;
}
[dir="rtl"] #ipython_notebook.pull-left {
  float: right !important;
  float: right;
}
.flex-spacer {
  flex: 1;
}
#noscript {
  width: auto;
  padding-top: 16px;
  padding-bottom: 16px;
  text-align: center;
  font-size: 22px;
  color: red;
  font-weight: bold;
}
#ipython_notebook img {
  height: 28px;
}
#site {
  width: 100%;
  display: none;
  box-sizing: border-box;
  -moz-box-sizing: border-box;
  -webkit-box-sizing: border-box;
  overflow: auto;
}
@media print {
  #site {
    height: auto !important;
  }
}
/* Smaller buttons */
.ui-button .ui-button-text {
  padding: 0.2em 0.8em;
  font-size: 77%;
}
input.ui-button {
  padding: 0.3em 0.9em;
}
span#kernel_logo_widget {
  margin: 0 10px;
}
span#login_widget {
  float: right;
}
[dir="rtl"] span#login_widget {
  float: left;
}
span#login_widget > .button,
#logout {
  color: #333;
  background-color: #fff;
  border-color: #ccc;
}
span#login_widget > .button:focus,
#logout:focus,
span#login_widget > .button.focus,
#logout.focus {
  color: #333;
  background-color: #e6e6e6;
  border-color: #8c8c8c;
}
span#login_widget > .button:hover,
#logout:hover {
  color: #333;
  background-color: #e6e6e6;
  border-color: #adadad;
}
span#login_widget > .button:active,
#logout:active,
span#login_widget > .button.active,
#logout.active,
.open > .dropdown-togglespan#login_widget > .button,
.open > .dropdown-toggle#logout {
  color: #333;
  background-color: #e6e6e6;
  border-color: #adadad;
}
span#login_widget > .button:active:hover,
#logout:active:hover,
span#login_widget > .button.active:hover,
#logout.active:hover,
.open > .dropdown-togglespan#login_widget > .button:hover,
.open > .dropdown-toggle#logout:hover,
span#login_widget > .button:active:focus,
#logout:active:focus,
span#login_widget > .button.active:focus,
#logout.active:focus,
.open > .dropdown-togglespan#login_widget > .button:focus,
.open > .dropdown-toggle#logout:focus,
span#login_widget > .button:active.focus,
#logout:active.focus,
span#login_widget > .button.active.focus,
#logout.active.focus,
.open > .dropdown-togglespan#login_widget > .button.focus,
.open > .dropdown-toggle#logout.focus {
  color: #333;
  background-color: #d4d4d4;
  border-color: #8c8c8c;
}
span#login_widget > .button:active,
#logout:active,
span#login_widget > .button.active,
#logout.active,
.open > .dropdown-togglespan#login_widget > .button,
.open > .dropdown-toggle#logout {
  background-image: none;
}
span#login_widget > .button.disabled:hover,
#logout.disabled:hover,
span#login_widget > .button[disabled]:hover,
#logout[disabled]:hover,
fieldset[disabled] span#login_widget > .button:hover,
fieldset[disabled] #logout:hover,
span#login_widget > .button.disabled:focus,
#logout.disabled:focus,
span#login_widget > .button[disabled]:focus,
#logout[disabled]:focus,
fieldset[disabled] span#login_widget > .button:focus,
fieldset[disabled] #logout:focus,
span#login_widget > .button.disabled.focus,
#logout.disabled.focus,
span#login_widget > .button[disabled].focus,
#logout[disabled].focus,
fieldset[disabled] span#login_widget > .button.focus,
fieldset[disabled] #logout.focus {
  background-color: #fff;
  border-color: #ccc;
}
span#login_widget > .button .badge,
#logout .badge {
  color: #fff;
  background-color: #333;
}
.nav-header {
  text-transform: none;
}
#header > span {
  margin-top: 10px;
}
.modal_stretch .modal-dialog {
  /* Old browsers */
  display: -webkit-box;
  -webkit-box-orient: vertical;
  -webkit-box-align: stretch;
  display: -moz-box;
  -moz-box-orient: vertical;
  -moz-box-align: stretch;
  display: box;
  box-orient: vertical;
  box-align: stretch;
  /* Modern browsers */
  display: flex;
  flex-direction: column;
  align-items: stretch;
  min-height: 80vh;
}
.modal_stretch .modal-dialog .modal-body {
  max-height: calc(100vh - 200px);
  overflow: auto;
  flex: 1;
}
.modal-header {
  cursor: move;
}
@media (min-width: 768px) {
  .modal .modal-dialog {
    width: 700px;
  }
}
@media (min-width: 768px) {
  select.form-control {
    margin-left: 12px;
    margin-right: 12px;
  }
}
/*!
*
* IPython auth
*
*/
.center-nav {
  display: inline-block;
  margin-bottom: -4px;
}
[dir="rtl"] .center-nav form.pull-left {
  float: right !important;
  float: right;
}
[dir="rtl"] .center-nav .navbar-text {
  float: right;
}
[dir="rtl"] .navbar-inner {
  text-align: right;
}
[dir="rtl"] div.text-left {
  text-align: right;
}
/*!
*
* IPython tree view
*
*/
/* We need an invisible input field on top of the sentense*/
/* "Drag file onto the list ..." */
.alternate_upload {
  background-color: none;
  display: inline;
}
.alternate_upload.form {
  padding: 0;
  margin: 0;
}
.alternate_upload input.fileinput {
  position: absolute;
  display: block;
  width: 100%;
  height: 100%;
  overflow: hidden;
  cursor: pointer;
  opacity: 0;
  z-index: 2;
}
.alternate_upload .btn-xs > input.fileinput {
  margin: -1px -5px;
}
.alternate_upload .btn-upload {
  position: relative;
  height: 22px;
}
::-webkit-file-upload-button {
  cursor: pointer;
}
/**
 * Primary styles
 *
 * Author: Jupyter Development Team
 */
ul#tabs {
  margin-bottom: 4px;
}
ul#tabs a {
  padding-top: 6px;
  padding-bottom: 4px;
}
[dir="rtl"] ul#tabs.nav-tabs > li {
  float: right;
}
[dir="rtl"] ul#tabs.nav.nav-tabs {
  padding-right: 0;
}
ul.breadcrumb a:focus,
ul.breadcrumb a:hover {
  text-decoration: none;
}
ul.breadcrumb i.icon-home {
  font-size: 16px;
  margin-right: 4px;
}
ul.breadcrumb span {
  color: #5e5e5e;
}
.list_toolbar {
  padding: 4px 0 4px 0;
  vertical-align: middle;
}
.list_toolbar .tree-buttons {
  padding-top: 1px;
}
[dir="rtl"] .list_toolbar .tree-buttons .pull-right {
  float: left !important;
  float: left;
}
[dir="rtl"] .list_toolbar .col-sm-4,
[dir="rtl"] .list_toolbar .col-sm-8 {
  float: right;
}
.dynamic-buttons {
  padding-top: 3px;
  display: inline-block;
}
.list_toolbar [class*="span"] {
  min-height: 24px;
}
.list_header {
  font-weight: bold;
  background-color: #EEE;
}
.list_placeholder {
  font-weight: bold;
  padding-top: 4px;
  padding-bottom: 4px;
  padding-left: 7px;
  padding-right: 7px;
}
.list_container {
  margin-top: 4px;
  margin-bottom: 20px;
  border: 1px solid #ddd;
  border-radius: 2px;
}
.list_container > div {
  border-bottom: 1px solid #ddd;
}
.list_container > div:hover .list-item {
  background-color: red;
}
.list_container > div:last-child {
  border: none;
}
.list_item:hover .list_item {
  background-color: #ddd;
}
.list_item a {
  text-decoration: none;
}
.list_item:hover {
  background-color: #fafafa;
}
.list_header > div,
.list_item > div {
  padding-top: 4px;
  padding-bottom: 4px;
  padding-left: 7px;
  padding-right: 7px;
  line-height: 22px;
}
.list_header > div input,
.list_item > div input {
  margin-right: 7px;
  margin-left: 14px;
  vertical-align: text-bottom;
  line-height: 22px;
  position: relative;
  top: -1px;
}
.list_header > div .item_link,
.list_item > div .item_link {
  margin-left: -1px;
  vertical-align: baseline;
  line-height: 22px;
}
[dir="rtl"] .list_item > div input {
  margin-right: 0;
}
.new-file input[type=checkbox] {
  visibility: hidden;
}
.item_name {
  line-height: 22px;
  height: 24px;
}
.item_icon {
  font-size: 14px;
  color: #5e5e5e;
  margin-right: 7px;
  margin-left: 7px;
  line-height: 22px;
  vertical-align: baseline;
}
.item_modified {
  margin-right: 7px;
  margin-left: 7px;
}
[dir="rtl"] .item_modified.pull-right {
  float: left !important;
  float: left;
}
.item_buttons {
  line-height: 1em;
  margin-left: -5px;
}
.item_buttons .btn,
.item_buttons .btn-group,
.item_buttons .input-group {
  float: left;
}
.item_buttons > .btn,
.item_buttons > .btn-group,
.item_buttons > .input-group {
  margin-left: 5px;
}
.item_buttons .btn {
  min-width: 13ex;
}
.item_buttons .running-indicator {
  padding-top: 4px;
  color: #5cb85c;
}
.item_buttons .kernel-name {
  padding-top: 4px;
  color: #5bc0de;
  margin-right: 7px;
  float: left;
}
[dir="rtl"] .item_buttons.pull-right {
  float: left !important;
  float: left;
}
[dir="rtl"] .item_buttons .kernel-name {
  margin-left: 7px;
  float: right;
}
.toolbar_info {
  height: 24px;
  line-height: 24px;
}
.list_item input:not([type=checkbox]) {
  padding-top: 3px;
  padding-bottom: 3px;
  height: 22px;
  line-height: 14px;
  margin: 0px;
}
.highlight_text {
  color: blue;
}
#project_name {
  display: inline-block;
  padding-left: 7px;
  margin-left: -2px;
}
#project_name > .breadcrumb {
  padding: 0px;
  margin-bottom: 0px;
  background-color: transparent;
  font-weight: bold;
}
.sort_button {
  display: inline-block;
  padding-left: 7px;
}
[dir="rtl"] .sort_button.pull-right {
  float: left !important;
  float: left;
}
#tree-selector {
  padding-right: 0px;
}
#button-select-all {
  min-width: 50px;
}
[dir="rtl"] #button-select-all.btn {
  float: right ;
}
#select-all {
  margin-left: 7px;
  margin-right: 2px;
  margin-top: 2px;
  height: 16px;
}
[dir="rtl"] #select-all.pull-left {
  float: right !important;
  float: right;
}
.menu_icon {
  margin-right: 2px;
}
.tab-content .row {
  margin-left: 0px;
  margin-right: 0px;
}
.folder_icon:before {
  display: inline-block;
  font: normal normal normal 14px/1 FontAwesome;
  font-size: inherit;
  text-rendering: auto;
  -webkit-font-smoothing: antialiased;
  -moz-osx-font-smoothing: grayscale;
  content: "\f114";
}
.folder_icon:before.fa-pull-left {
  margin-right: .3em;
}
.folder_icon:before.fa-pull-right {
  margin-left: .3em;
}
.folder_icon:before.pull-left {
  margin-right: .3em;
}
.folder_icon:before.pull-right {
  margin-left: .3em;
}
.notebook_icon:before {
  display: inline-block;
  font: normal normal normal 14px/1 FontAwesome;
  font-size: inherit;
  text-rendering: auto;
  -webkit-font-smoothing: antialiased;
  -moz-osx-font-smoothing: grayscale;
  content: "\f02d";
  position: relative;
  top: -1px;
}
.notebook_icon:before.fa-pull-left {
  margin-right: .3em;
}
.notebook_icon:before.fa-pull-right {
  margin-left: .3em;
}
.notebook_icon:before.pull-left {
  margin-right: .3em;
}
.notebook_icon:before.pull-right {
  margin-left: .3em;
}
.running_notebook_icon:before {
  display: inline-block;
  font: normal normal normal 14px/1 FontAwesome;
  font-size: inherit;
  text-rendering: auto;
  -webkit-font-smoothing: antialiased;
  -moz-osx-font-smoothing: grayscale;
  content: "\f02d";
  position: relative;
  top: -1px;
  color: #5cb85c;
}
.running_notebook_icon:before.fa-pull-left {
  margin-right: .3em;
}
.running_notebook_icon:before.fa-pull-right {
  margin-left: .3em;
}
.running_notebook_icon:before.pull-left {
  margin-right: .3em;
}
.running_notebook_icon:before.pull-right {
  margin-left: .3em;
}
.file_icon:before {
  display: inline-block;
  font: normal normal normal 14px/1 FontAwesome;
  font-size: inherit;
  text-rendering: auto;
  -webkit-font-smoothing: antialiased;
  -moz-osx-font-smoothing: grayscale;
  content: "\f016";
  position: relative;
  top: -2px;
}
.file_icon:before.fa-pull-left {
  margin-right: .3em;
}
.file_icon:before.fa-pull-right {
  margin-left: .3em;
}
.file_icon:before.pull-left {
  margin-right: .3em;
}
.file_icon:before.pull-right {
  margin-left: .3em;
}
#notebook_toolbar .pull-right {
  padding-top: 0px;
  margin-right: -1px;
}
ul#new-menu {
  left: auto;
  right: 0;
}
#new-menu .dropdown-header {
  font-size: 10px;
  border-bottom: 1px solid #e5e5e5;
  padding: 0 0 3px;
  margin: -3px 20px 0;
}
.kernel-menu-icon {
  padding-right: 12px;
  width: 24px;
  content: "\f096";
}
.kernel-menu-icon:before {
  content: "\f096";
}
.kernel-menu-icon-current:before {
  content: "\f00c";
}
#tab_content {
  padding-top: 20px;
}
#running .panel-group .panel {
  margin-top: 3px;
  margin-bottom: 1em;
}
#running .panel-group .panel .panel-heading {
  background-color: #EEE;
  padding-top: 4px;
  padding-bottom: 4px;
  padding-left: 7px;
  padding-right: 7px;
  line-height: 22px;
}
#running .panel-group .panel .panel-heading a:focus,
#running .panel-group .panel .panel-heading a:hover {
  text-decoration: none;
}
#running .panel-group .panel .panel-body {
  padding: 0px;
}
#running .panel-group .panel .panel-body .list_container {
  margin-top: 0px;
  margin-bottom: 0px;
  border: 0px;
  border-radius: 0px;
}
#running .panel-group .panel .panel-body .list_container .list_item {
  border-bottom: 1px solid #ddd;
}
#running .panel-group .panel .panel-body .list_container .list_item:last-child {
  border-bottom: 0px;
}
.delete-button {
  display: none;
}
.duplicate-button {
  display: none;
}
.rename-button {
  display: none;
}
.move-button {
  display: none;
}
.download-button {
  display: none;
}
.shutdown-button {
  display: none;
}
.dynamic-instructions {
  display: inline-block;
  padding-top: 4px;
}
/*!
*
* IPython text editor webapp
*
*/
.selected-keymap i.fa {
  padding: 0px 5px;
}
.selected-keymap i.fa:before {
  content: "\f00c";
}
#mode-menu {
  overflow: auto;
  max-height: 20em;
}
.edit_app #header {
  -webkit-box-shadow: 0px 0px 12px 1px rgba(87, 87, 87, 0.2);
  box-shadow: 0px 0px 12px 1px rgba(87, 87, 87, 0.2);
}
.edit_app #menubar .navbar {
  /* Use a negative 1 bottom margin, so the border overlaps the border of the
    header */
  margin-bottom: -1px;
}
.dirty-indicator {
  display: inline-block;
  font: normal normal normal 14px/1 FontAwesome;
  font-size: inherit;
  text-rendering: auto;
  -webkit-font-smoothing: antialiased;
  -moz-osx-font-smoothing: grayscale;
  width: 20px;
}
.dirty-indicator.fa-pull-left {
  margin-right: .3em;
}
.dirty-indicator.fa-pull-right {
  margin-left: .3em;
}
.dirty-indicator.pull-left {
  margin-right: .3em;
}
.dirty-indicator.pull-right {
  margin-left: .3em;
}
.dirty-indicator-dirty {
  display: inline-block;
  font: normal normal normal 14px/1 FontAwesome;
  font-size: inherit;
  text-rendering: auto;
  -webkit-font-smoothing: antialiased;
  -moz-osx-font-smoothing: grayscale;
  width: 20px;
}
.dirty-indicator-dirty.fa-pull-left {
  margin-right: .3em;
}
.dirty-indicator-dirty.fa-pull-right {
  margin-left: .3em;
}
.dirty-indicator-dirty.pull-left {
  margin-right: .3em;
}
.dirty-indicator-dirty.pull-right {
  margin-left: .3em;
}
.dirty-indicator-clean {
  display: inline-block;
  font: normal normal normal 14px/1 FontAwesome;
  font-size: inherit;
  text-rendering: auto;
  -webkit-font-smoothing: antialiased;
  -moz-osx-font-smoothing: grayscale;
  width: 20px;
}
.dirty-indicator-clean.fa-pull-left {
  margin-right: .3em;
}
.dirty-indicator-clean.fa-pull-right {
  margin-left: .3em;
}
.dirty-indicator-clean.pull-left {
  margin-right: .3em;
}
.dirty-indicator-clean.pull-right {
  margin-left: .3em;
}
.dirty-indicator-clean:before {
  display: inline-block;
  font: normal normal normal 14px/1 FontAwesome;
  font-size: inherit;
  text-rendering: auto;
  -webkit-font-smoothing: antialiased;
  -moz-osx-font-smoothing: grayscale;
  content: "\f00c";
}
.dirty-indicator-clean:before.fa-pull-left {
  margin-right: .3em;
}
.dirty-indicator-clean:before.fa-pull-right {
  margin-left: .3em;
}
.dirty-indicator-clean:before.pull-left {
  margin-right: .3em;
}
.dirty-indicator-clean:before.pull-right {
  margin-left: .3em;
}
#filename {
  font-size: 16pt;
  display: table;
  padding: 0px 5px;
}
#current-mode {
  padding-left: 5px;
  padding-right: 5px;
}
#texteditor-backdrop {
  padding-top: 20px;
  padding-bottom: 20px;
}
@media not print {
  #texteditor-backdrop {
    background-color: #EEE;
  }
}
@media print {
  #texteditor-backdrop #texteditor-container .CodeMirror-gutter,
  #texteditor-backdrop #texteditor-container .CodeMirror-gutters {
    background-color: #fff;
  }
}
@media not print {
  #texteditor-backdrop #texteditor-container .CodeMirror-gutter,
  #texteditor-backdrop #texteditor-container .CodeMirror-gutters {
    background-color: #fff;
  }
}
@media not print {
  #texteditor-backdrop #texteditor-container {
    padding: 0px;
    background-color: #fff;
    -webkit-box-shadow: 0px 0px 12px 1px rgba(87, 87, 87, 0.2);
    box-shadow: 0px 0px 12px 1px rgba(87, 87, 87, 0.2);
  }
}
.CodeMirror-dialog {
  background-color: #fff;
}
/*!
*
* IPython notebook
*
*/
/* CSS font colors for translated ANSI escape sequences */
/* The color values are a mix of
   http://www.xcolors.net/dl/baskerville-ivorylight and
   http://www.xcolors.net/dl/euphrasia */
.ansi-black-fg {
  color: #3E424D;
}
.ansi-black-bg {
  background-color: #3E424D;
}
.ansi-black-intense-fg {
  color: #282C36;
}
.ansi-black-intense-bg {
  background-color: #282C36;
}
.ansi-red-fg {
  color: #E75C58;
}
.ansi-red-bg {
  background-color: #E75C58;
}
.ansi-red-intense-fg {
  color: #B22B31;
}
.ansi-red-intense-bg {
  background-color: #B22B31;
}
.ansi-green-fg {
  color: #00A250;
}
.ansi-green-bg {
  background-color: #00A250;
}
.ansi-green-intense-fg {
  color: #007427;
}
.ansi-green-intense-bg {
  background-color: #007427;
}
.ansi-yellow-fg {
  color: #DDB62B;
}
.ansi-yellow-bg {
  background-color: #DDB62B;
}
.ansi-yellow-intense-fg {
  color: #B27D12;
}
.ansi-yellow-intense-bg {
  background-color: #B27D12;
}
.ansi-blue-fg {
  color: #208FFB;
}
.ansi-blue-bg {
  background-color: #208FFB;
}
.ansi-blue-intense-fg {
  color: #0065CA;
}
.ansi-blue-intense-bg {
  background-color: #0065CA;
}
.ansi-magenta-fg {
  color: #D160C4;
}
.ansi-magenta-bg {
  background-color: #D160C4;
}
.ansi-magenta-intense-fg {
  color: #A03196;
}
.ansi-magenta-intense-bg {
  background-color: #A03196;
}
.ansi-cyan-fg {
  color: #60C6C8;
}
.ansi-cyan-bg {
  background-color: #60C6C8;
}
.ansi-cyan-intense-fg {
  color: #258F8F;
}
.ansi-cyan-intense-bg {
  background-color: #258F8F;
}
.ansi-white-fg {
  color: #C5C1B4;
}
.ansi-white-bg {
  background-color: #C5C1B4;
}
.ansi-white-intense-fg {
  color: #A1A6B2;
}
.ansi-white-intense-bg {
  background-color: #A1A6B2;
}
.ansi-default-inverse-fg {
  color: #FFFFFF;
}
.ansi-default-inverse-bg {
  background-color: #000000;
}
.ansi-bold {
  font-weight: bold;
}
.ansi-underline {
  text-decoration: underline;
}
/* The following styles are deprecated an will be removed in a future version */
.ansibold {
  font-weight: bold;
}
.ansi-inverse {
  outline: 0.5px dotted;
}
/* use dark versions for foreground, to improve visibility */
.ansiblack {
  color: black;
}
.ansired {
  color: darkred;
}
.ansigreen {
  color: darkgreen;
}
.ansiyellow {
  color: #c4a000;
}
.ansiblue {
  color: darkblue;
}
.ansipurple {
  color: darkviolet;
}
.ansicyan {
  color: steelblue;
}
.ansigray {
  color: gray;
}
/* and light for background, for the same reason */
.ansibgblack {
  background-color: black;
}
.ansibgred {
  background-color: red;
}
.ansibggreen {
  background-color: green;
}
.ansibgyellow {
  background-color: yellow;
}
.ansibgblue {
  background-color: blue;
}
.ansibgpurple {
  background-color: magenta;
}
.ansibgcyan {
  background-color: cyan;
}
.ansibggray {
  background-color: gray;
}
div.cell {
  /* Old browsers */
  display: -webkit-box;
  -webkit-box-orient: vertical;
  -webkit-box-align: stretch;
  display: -moz-box;
  -moz-box-orient: vertical;
  -moz-box-align: stretch;
  display: box;
  box-orient: vertical;
  box-align: stretch;
  /* Modern browsers */
  display: flex;
  flex-direction: column;
  align-items: stretch;
  border-radius: 2px;
  box-sizing: border-box;
  -moz-box-sizing: border-box;
  -webkit-box-sizing: border-box;
  border-width: 1px;
  border-style: solid;
  border-color: transparent;
  width: 100%;
  padding: 5px;
  /* This acts as a spacer between cells, that is outside the border */
  margin: 0px;
  outline: none;
  position: relative;
  overflow: visible;
}
div.cell:before {
  position: absolute;
  display: block;
  top: -1px;
  left: -1px;
  width: 5px;
  height: calc(100% +  2px);
  content: '';
  background: transparent;
}
div.cell.jupyter-soft-selected {
  border-left-color: #E3F2FD;
  border-left-width: 1px;
  padding-left: 5px;
  border-right-color: #E3F2FD;
  border-right-width: 1px;
  background: #E3F2FD;
}
@media print {
  div.cell.jupyter-soft-selected {
    border-color: transparent;
  }
}
div.cell.selected,
div.cell.selected.jupyter-soft-selected {
  border-color: #ababab;
}
div.cell.selected:before,
div.cell.selected.jupyter-soft-selected:before {
  position: absolute;
  display: block;
  top: -1px;
  left: -1px;
  width: 5px;
  height: calc(100% +  2px);
  content: '';
  background: #42A5F5;
}
@media print {
  div.cell.selected,
  div.cell.selected.jupyter-soft-selected {
    border-color: transparent;
  }
}
.edit_mode div.cell.selected {
  border-color: #66BB6A;
}
.edit_mode div.cell.selected:before {
  position: absolute;
  display: block;
  top: -1px;
  left: -1px;
  width: 5px;
  height: calc(100% +  2px);
  content: '';
  background: #66BB6A;
}
@media print {
  .edit_mode div.cell.selected {
    border-color: transparent;
  }
}
.prompt {
  /* This needs to be wide enough for 3 digit prompt numbers: In[100]: */
  min-width: 14ex;
  /* This padding is tuned to match the padding on the CodeMirror editor. */
  padding: 0.4em;
  margin: 0px;
  font-family: monospace;
  text-align: right;
  /* This has to match that of the the CodeMirror class line-height below */
  line-height: 1.21429em;
  /* Don't highlight prompt number selection */
  -webkit-touch-callout: none;
  -webkit-user-select: none;
  -khtml-user-select: none;
  -moz-user-select: none;
  -ms-user-select: none;
  user-select: none;
  /* Use default cursor */
  cursor: default;
}
@media (max-width: 540px) {
  .prompt {
    text-align: left;
  }
}
div.inner_cell {
  min-width: 0;
  /* Old browsers */
  display: -webkit-box;
  -webkit-box-orient: vertical;
  -webkit-box-align: stretch;
  display: -moz-box;
  -moz-box-orient: vertical;
  -moz-box-align: stretch;
  display: box;
  box-orient: vertical;
  box-align: stretch;
  /* Modern browsers */
  display: flex;
  flex-direction: column;
  align-items: stretch;
  /* Old browsers */
  -webkit-box-flex: 1;
  -moz-box-flex: 1;
  box-flex: 1;
  /* Modern browsers */
  flex: 1;
}
/* input_area and input_prompt must match in top border and margin for alignment */
div.input_area {
  border: 1px solid #cfcfcf;
  border-radius: 2px;
  background: #f7f7f7;
  line-height: 1.21429em;
}
/* This is needed so that empty prompt areas can collapse to zero height when there
   is no content in the output_subarea and the prompt. The main purpose of this is
   to make sure that empty JavaScript output_subareas have no height. */
div.prompt:empty {
  padding-top: 0;
  padding-bottom: 0;
}
div.unrecognized_cell {
  padding: 5px 5px 5px 0px;
  /* Old browsers */
  display: -webkit-box;
  -webkit-box-orient: horizontal;
  -webkit-box-align: stretch;
  display: -moz-box;
  -moz-box-orient: horizontal;
  -moz-box-align: stretch;
  display: box;
  box-orient: horizontal;
  box-align: stretch;
  /* Modern browsers */
  display: flex;
  flex-direction: row;
  align-items: stretch;
}
div.unrecognized_cell .inner_cell {
  border-radius: 2px;
  padding: 5px;
  font-weight: bold;
  color: red;
  border: 1px solid #cfcfcf;
  background: #eaeaea;
}
div.unrecognized_cell .inner_cell a {
  color: inherit;
  text-decoration: none;
}
div.unrecognized_cell .inner_cell a:hover {
  color: inherit;
  text-decoration: none;
}
@media (max-width: 540px) {
  div.unrecognized_cell > div.prompt {
    display: none;
  }
}
div.code_cell {
  /* avoid page breaking on code cells when printing */
}
@media print {
  div.code_cell {
    page-break-inside: avoid;
  }
}
/* any special styling for code cells that are currently running goes here */
div.input {
  page-break-inside: avoid;
  /* Old browsers */
  display: -webkit-box;
  -webkit-box-orient: horizontal;
  -webkit-box-align: stretch;
  display: -moz-box;
  -moz-box-orient: horizontal;
  -moz-box-align: stretch;
  display: box;
  box-orient: horizontal;
  box-align: stretch;
  /* Modern browsers */
  display: flex;
  flex-direction: row;
  align-items: stretch;
}
@media (max-width: 540px) {
  div.input {
    /* Old browsers */
    display: -webkit-box;
    -webkit-box-orient: vertical;
    -webkit-box-align: stretch;
    display: -moz-box;
    -moz-box-orient: vertical;
    -moz-box-align: stretch;
    display: box;
    box-orient: vertical;
    box-align: stretch;
    /* Modern browsers */
    display: flex;
    flex-direction: column;
    align-items: stretch;
  }
}
/* input_area and input_prompt must match in top border and margin for alignment */
div.input_prompt {
  color: #303F9F;
  border-top: 1px solid transparent;
}
div.input_area > div.highlight {
  margin: 0.4em;
  border: none;
  padding: 0px;
  background-color: transparent;
}
div.input_area > div.highlight > pre {
  margin: 0px;
  border: none;
  padding: 0px;
  background-color: transparent;
}
/* The following gets added to the <head> if it is detected that the user has a
 * monospace font with inconsistent normal/bold/italic height.  See
 * notebookmain.js.  Such fonts will have keywords vertically offset with
 * respect to the rest of the text.  The user should select a better font.
 * See: https://github.com/ipython/ipython/issues/1503
 *
 * .CodeMirror span {
 *      vertical-align: bottom;
 * }
 */
.CodeMirror {
  line-height: 1.21429em;
  /* Changed from 1em to our global default */
  font-size: 14px;
  height: auto;
  /* Changed to auto to autogrow */
  background: none;
  /* Changed from white to allow our bg to show through */
}
.CodeMirror-scroll {
  /*  The CodeMirror docs are a bit fuzzy on if overflow-y should be hidden or visible.*/
  /*  We have found that if it is visible, vertical scrollbars appear with font size changes.*/
  overflow-y: hidden;
  overflow-x: auto;
}
.CodeMirror-lines {
  /* In CM2, this used to be 0.4em, but in CM3 it went to 4px. We need the em value because */
  /* we have set a different line-height and want this to scale with that. */
  /* Note that this should set vertical padding only, since CodeMirror assumes
       that horizontal padding will be set on CodeMirror pre */
  padding: 0.4em 0;
}
.CodeMirror-linenumber {
  padding: 0 8px 0 4px;
}
.CodeMirror-gutters {
  border-bottom-left-radius: 2px;
  border-top-left-radius: 2px;
}
.CodeMirror pre {
  /* In CM3 this went to 4px from 0 in CM2. This sets horizontal padding only,
    use .CodeMirror-lines for vertical */
  padding: 0 0.4em;
  border: 0;
  border-radius: 0;
}
.CodeMirror-cursor {
  border-left: 1.4px solid black;
}
@media screen and (min-width: 2138px) and (max-width: 4319px) {
  .CodeMirror-cursor {
    border-left: 2px solid black;
  }
}
@media screen and (min-width: 4320px) {
  .CodeMirror-cursor {
    border-left: 4px solid black;
  }
}
/*

Original style from softwaremaniacs.org (c) Ivan Sagalaev <Maniac@SoftwareManiacs.Org>
Adapted from GitHub theme

*/
.highlight-base {
  color: #000;
}
.highlight-variable {
  color: #000;
}
.highlight-variable-2 {
  color: #1a1a1a;
}
.highlight-variable-3 {
  color: #333333;
}
.highlight-string {
  color: #BA2121;
}
.highlight-comment {
  color: #408080;
  font-style: italic;
}
.highlight-number {
  color: #080;
}
.highlight-atom {
  color: #88F;
}
.highlight-keyword {
  color: #008000;
  font-weight: bold;
}
.highlight-builtin {
  color: #008000;
}
.highlight-error {
  color: #f00;
}
.highlight-operator {
  color: #AA22FF;
  font-weight: bold;
}
.highlight-meta {
  color: #AA22FF;
}
/* previously not defined, copying from default codemirror */
.highlight-def {
  color: #00f;
}
.highlight-string-2 {
  color: #f50;
}
.highlight-qualifier {
  color: #555;
}
.highlight-bracket {
  color: #997;
}
.highlight-tag {
  color: #170;
}
.highlight-attribute {
  color: #00c;
}
.highlight-header {
  color: blue;
}
.highlight-quote {
  color: #090;
}
.highlight-link {
  color: #00c;
}
/* apply the same style to codemirror */
.cm-s-ipython span.cm-keyword {
  color: #008000;
  font-weight: bold;
}
.cm-s-ipython span.cm-atom {
  color: #88F;
}
.cm-s-ipython span.cm-number {
  color: #080;
}
.cm-s-ipython span.cm-def {
  color: #00f;
}
.cm-s-ipython span.cm-variable {
  color: #000;
}
.cm-s-ipython span.cm-operator {
  color: #AA22FF;
  font-weight: bold;
}
.cm-s-ipython span.cm-variable-2 {
  color: #1a1a1a;
}
.cm-s-ipython span.cm-variable-3 {
  color: #333333;
}
.cm-s-ipython span.cm-comment {
  color: #408080;
  font-style: italic;
}
.cm-s-ipython span.cm-string {
  color: #BA2121;
}
.cm-s-ipython span.cm-string-2 {
  color: #f50;
}
.cm-s-ipython span.cm-meta {
  color: #AA22FF;
}
.cm-s-ipython span.cm-qualifier {
  color: #555;
}
.cm-s-ipython span.cm-builtin {
  color: #008000;
}
.cm-s-ipython span.cm-bracket {
  color: #997;
}
.cm-s-ipython span.cm-tag {
  color: #170;
}
.cm-s-ipython span.cm-attribute {
  color: #00c;
}
.cm-s-ipython span.cm-header {
  color: blue;
}
.cm-s-ipython span.cm-quote {
  color: #090;
}
.cm-s-ipython span.cm-link {
  color: #00c;
}
.cm-s-ipython span.cm-error {
  color: #f00;
}
.cm-s-ipython span.cm-tab {
  background: url(data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAADAAAAAMCAYAAAAkuj5RAAAAAXNSR0IArs4c6QAAAGFJREFUSMft1LsRQFAQheHPowAKoACx3IgEKtaEHujDjORSgWTH/ZOdnZOcM/sgk/kFFWY0qV8foQwS4MKBCS3qR6ixBJvElOobYAtivseIE120FaowJPN75GMu8j/LfMwNjh4HUpwg4LUAAAAASUVORK5CYII=);
  background-position: right;
  background-repeat: no-repeat;
}
div.output_wrapper {
  /* this position must be relative to enable descendents to be absolute within it */
  position: relative;
  /* Old browsers */
  display: -webkit-box;
  -webkit-box-orient: vertical;
  -webkit-box-align: stretch;
  display: -moz-box;
  -moz-box-orient: vertical;
  -moz-box-align: stretch;
  display: box;
  box-orient: vertical;
  box-align: stretch;
  /* Modern browsers */
  display: flex;
  flex-direction: column;
  align-items: stretch;
  z-index: 1;
}
/* class for the output area when it should be height-limited */
div.output_scroll {
  /* ideally, this would be max-height, but FF barfs all over that */
  height: 24em;
  /* FF needs this *and the wrapper* to specify full width, or it will shrinkwrap */
  width: 100%;
  overflow: auto;
  border-radius: 2px;
  -webkit-box-shadow: inset 0 2px 8px rgba(0, 0, 0, 0.8);
  box-shadow: inset 0 2px 8px rgba(0, 0, 0, 0.8);
  display: block;
}
/* output div while it is collapsed */
div.output_collapsed {
  margin: 0px;
  padding: 0px;
  /* Old browsers */
  display: -webkit-box;
  -webkit-box-orient: vertical;
  -webkit-box-align: stretch;
  display: -moz-box;
  -moz-box-orient: vertical;
  -moz-box-align: stretch;
  display: box;
  box-orient: vertical;
  box-align: stretch;
  /* Modern browsers */
  display: flex;
  flex-direction: column;
  align-items: stretch;
}
div.out_prompt_overlay {
  height: 100%;
  padding: 0px 0.4em;
  position: absolute;
  border-radius: 2px;
}
div.out_prompt_overlay:hover {
  /* use inner shadow to get border that is computed the same on WebKit/FF */
  -webkit-box-shadow: inset 0 0 1px #000;
  box-shadow: inset 0 0 1px #000;
  background: rgba(240, 240, 240, 0.5);
}
div.output_prompt {
  color: #D84315;
}
/* This class is the outer container of all output sections. */
div.output_area {
  padding: 0px;
  page-break-inside: avoid;
  /* Old browsers */
  display: -webkit-box;
  -webkit-box-orient: horizontal;
  -webkit-box-align: stretch;
  display: -moz-box;
  -moz-box-orient: horizontal;
  -moz-box-align: stretch;
  display: box;
  box-orient: horizontal;
  box-align: stretch;
  /* Modern browsers */
  display: flex;
  flex-direction: row;
  align-items: stretch;
}
div.output_area .MathJax_Display {
  text-align: left !important;
}
div.output_area .rendered_html table {
  margin-left: 0;
  margin-right: 0;
}
div.output_area .rendered_html img {
  margin-left: 0;
  margin-right: 0;
}
div.output_area img,
div.output_area svg {
  max-width: 100%;
  height: auto;
}
div.output_area img.unconfined,
div.output_area svg.unconfined {
  max-width: none;
}
div.output_area .mglyph > img {
  max-width: none;
}
/* This is needed to protect the pre formating from global settings such
   as that of bootstrap */
.output {
  /* Old browsers */
  display: -webkit-box;
  -webkit-box-orient: vertical;
  -webkit-box-align: stretch;
  display: -moz-box;
  -moz-box-orient: vertical;
  -moz-box-align: stretch;
  display: box;
  box-orient: vertical;
  box-align: stretch;
  /* Modern browsers */
  display: flex;
  flex-direction: column;
  align-items: stretch;
}
@media (max-width: 540px) {
  div.output_area {
    /* Old browsers */
    display: -webkit-box;
    -webkit-box-orient: vertical;
    -webkit-box-align: stretch;
    display: -moz-box;
    -moz-box-orient: vertical;
    -moz-box-align: stretch;
    display: box;
    box-orient: vertical;
    box-align: stretch;
    /* Modern browsers */
    display: flex;
    flex-direction: column;
    align-items: stretch;
  }
}
div.output_area pre {
  margin: 0;
  padding: 1px 0 1px 0;
  border: 0;
  vertical-align: baseline;
  color: black;
  background-color: transparent;
  border-radius: 0;
}
/* This class is for the output subarea inside the output_area and after
   the prompt div. */
div.output_subarea {
  overflow-x: auto;
  padding: 0.4em;
  /* Old browsers */
  -webkit-box-flex: 1;
  -moz-box-flex: 1;
  box-flex: 1;
  /* Modern browsers */
  flex: 1;
  max-width: calc(100% - 14ex);
}
div.output_scroll div.output_subarea {
  overflow-x: visible;
}
/* The rest of the output_* classes are for special styling of the different
   output types */
/* all text output has this class: */
div.output_text {
  text-align: left;
  color: #000;
  /* This has to match that of the the CodeMirror class line-height below */
  line-height: 1.21429em;
}
/* stdout/stderr are 'text' as well as 'stream', but execute_result/error are *not* streams */
div.output_stderr {
  background: #fdd;
  /* very light red background for stderr */
}
div.output_latex {
  text-align: left;
}
/* Empty output_javascript divs should have no height */
div.output_javascript:empty {
  padding: 0;
}
.js-error {
  color: darkred;
}
/* raw_input styles */
div.raw_input_container {
  line-height: 1.21429em;
  padding-top: 5px;
}
pre.raw_input_prompt {
  /* nothing needed here. */
}
input.raw_input {
  font-family: monospace;
  font-size: inherit;
  color: inherit;
  width: auto;
  /* make sure input baseline aligns with prompt */
  vertical-align: baseline;
  /* padding + margin = 0.5em between prompt and cursor */
  padding: 0em 0.25em;
  margin: 0em 0.25em;
}
input.raw_input:focus {
  box-shadow: none;
}
p.p-space {
  margin-bottom: 10px;
}
div.output_unrecognized {
  padding: 5px;
  font-weight: bold;
  color: red;
}
div.output_unrecognized a {
  color: inherit;
  text-decoration: none;
}
div.output_unrecognized a:hover {
  color: inherit;
  text-decoration: none;
}
.rendered_html {
  color: #000;
  /* any extras will just be numbers: */
}
.rendered_html em {
  font-style: italic;
}
.rendered_html strong {
  font-weight: bold;
}
.rendered_html u {
  text-decoration: underline;
}
.rendered_html :link {
  text-decoration: underline;
}
.rendered_html :visited {
  text-decoration: underline;
}
.rendered_html h1 {
  font-size: 185.7%;
  margin: 1.08em 0 0 0;
  font-weight: bold;
  line-height: 1.0;
}
.rendered_html h2 {
  font-size: 157.1%;
  margin: 1.27em 0 0 0;
  font-weight: bold;
  line-height: 1.0;
}
.rendered_html h3 {
  font-size: 128.6%;
  margin: 1.55em 0 0 0;
  font-weight: bold;
  line-height: 1.0;
}
.rendered_html h4 {
  font-size: 100%;
  margin: 2em 0 0 0;
  font-weight: bold;
  line-height: 1.0;
}
.rendered_html h5 {
  font-size: 100%;
  margin: 2em 0 0 0;
  font-weight: bold;
  line-height: 1.0;
  font-style: italic;
}
.rendered_html h6 {
  font-size: 100%;
  margin: 2em 0 0 0;
  font-weight: bold;
  line-height: 1.0;
  font-style: italic;
}
.rendered_html h1:first-child {
  margin-top: 0.538em;
}
.rendered_html h2:first-child {
  margin-top: 0.636em;
}
.rendered_html h3:first-child {
  margin-top: 0.777em;
}
.rendered_html h4:first-child {
  margin-top: 1em;
}
.rendered_html h5:first-child {
  margin-top: 1em;
}
.rendered_html h6:first-child {
  margin-top: 1em;
}
.rendered_html ul:not(.list-inline),
.rendered_html ol:not(.list-inline) {
  padding-left: 2em;
}
.rendered_html ul {
  list-style: disc;
}
.rendered_html ul ul {
  list-style: square;
  margin-top: 0;
}
.rendered_html ul ul ul {
  list-style: circle;
}
.rendered_html ol {
  list-style: decimal;
}
.rendered_html ol ol {
  list-style: upper-alpha;
  margin-top: 0;
}
.rendered_html ol ol ol {
  list-style: lower-alpha;
}
.rendered_html ol ol ol ol {
  list-style: lower-roman;
}
.rendered_html ol ol ol ol ol {
  list-style: decimal;
}
.rendered_html * + ul {
  margin-top: 1em;
}
.rendered_html * + ol {
  margin-top: 1em;
}
.rendered_html hr {
  color: black;
  background-color: black;
}
.rendered_html pre {
  margin: 1em 2em;
  padding: 0px;
  background-color: #fff;
}
.rendered_html code {
  background-color: #eff0f1;
}
.rendered_html p code {
  padding: 1px 5px;
}
.rendered_html pre code {
  background-color: #fff;
}
.rendered_html pre,
.rendered_html code {
  border: 0;
  color: #000;
  font-size: 100%;
}
.rendered_html blockquote {
  margin: 1em 2em;
}
.rendered_html table {
  margin-left: auto;
  margin-right: auto;
  border: none;
  border-collapse: collapse;
  border-spacing: 0;
  color: black;
  font-size: 12px;
  table-layout: fixed;
}
.rendered_html thead {
  border-bottom: 1px solid black;
  vertical-align: bottom;
}
.rendered_html tr,
.rendered_html th,
.rendered_html td {
  text-align: right;
  vertical-align: middle;
  padding: 0.5em 0.5em;
  line-height: normal;
  white-space: normal;
  max-width: none;
  border: none;
}
.rendered_html th {
  font-weight: bold;
}
.rendered_html tbody tr:nth-child(odd) {
  background: #f5f5f5;
}
.rendered_html tbody tr:hover {
  background: rgba(66, 165, 245, 0.2);
}
.rendered_html * + table {
  margin-top: 1em;
}
.rendered_html p {
  text-align: left;
}
.rendered_html * + p {
  margin-top: 1em;
}
.rendered_html img {
  display: block;
  margin-left: auto;
  margin-right: auto;
}
.rendered_html * + img {
  margin-top: 1em;
}
.rendered_html img,
.rendered_html svg {
  max-width: 100%;
  height: auto;
}
.rendered_html img.unconfined,
.rendered_html svg.unconfined {
  max-width: none;
}
.rendered_html .alert {
  margin-bottom: initial;
}
.rendered_html * + .alert {
  margin-top: 1em;
}
[dir="rtl"] .rendered_html p {
  text-align: right;
}
div.text_cell {
  /* Old browsers */
  display: -webkit-box;
  -webkit-box-orient: horizontal;
  -webkit-box-align: stretch;
  display: -moz-box;
  -moz-box-orient: horizontal;
  -moz-box-align: stretch;
  display: box;
  box-orient: horizontal;
  box-align: stretch;
  /* Modern browsers */
  display: flex;
  flex-direction: row;
  align-items: stretch;
}
@media (max-width: 540px) {
  div.text_cell > div.prompt {
    display: none;
  }
}
div.text_cell_render {
  /*font-family: "Helvetica Neue", Arial, Helvetica, Geneva, sans-serif;*/
  outline: none;
  resize: none;
  width: inherit;
  border-style: none;
  padding: 0.5em 0.5em 0.5em 0.4em;
  color: #000;
  box-sizing: border-box;
  -moz-box-sizing: border-box;
  -webkit-box-sizing: border-box;
}
a.anchor-link:link {
  text-decoration: none;
  padding: 0px 20px;
  visibility: hidden;
}
h1:hover .anchor-link,
h2:hover .anchor-link,
h3:hover .anchor-link,
h4:hover .anchor-link,
h5:hover .anchor-link,
h6:hover .anchor-link {
  visibility: visible;
}
.text_cell.rendered .input_area {
  display: none;
}
.text_cell.rendered .rendered_html {
  overflow-x: auto;
  overflow-y: hidden;
}
.text_cell.rendered .rendered_html tr,
.text_cell.rendered .rendered_html th,
.text_cell.rendered .rendered_html td {
  max-width: none;
}
.text_cell.unrendered .text_cell_render {
  display: none;
}
.text_cell .dropzone .input_area {
  border: 2px dashed #bababa;
  margin: -1px;
}
.cm-header-1,
.cm-header-2,
.cm-header-3,
.cm-header-4,
.cm-header-5,
.cm-header-6 {
  font-weight: bold;
  font-family: "Helvetica Neue", Helvetica, Arial, sans-serif;
}
.cm-header-1 {
  font-size: 185.7%;
}
.cm-header-2 {
  font-size: 157.1%;
}
.cm-header-3 {
  font-size: 128.6%;
}
.cm-header-4 {
  font-size: 110%;
}
.cm-header-5 {
  font-size: 100%;
  font-style: italic;
}
.cm-header-6 {
  font-size: 100%;
  font-style: italic;
}
/*!
*
* IPython notebook webapp
*
*/
@media (max-width: 767px) {
  .notebook_app {
    padding-left: 0px;
    padding-right: 0px;
  }
}
#ipython-main-app {
  box-sizing: border-box;
  -moz-box-sizing: border-box;
  -webkit-box-sizing: border-box;
  height: 100%;
}
div#notebook_panel {
  margin: 0px;
  padding: 0px;
  box-sizing: border-box;
  -moz-box-sizing: border-box;
  -webkit-box-sizing: border-box;
  height: 100%;
}
div#notebook {
  font-size: 14px;
  line-height: 20px;
  overflow-y: hidden;
  overflow-x: auto;
  width: 100%;
  /* This spaces the page away from the edge of the notebook area */
  padding-top: 20px;
  margin: 0px;
  outline: none;
  box-sizing: border-box;
  -moz-box-sizing: border-box;
  -webkit-box-sizing: border-box;
  min-height: 100%;
}
@media not print {
  #notebook-container {
    padding: 15px;
    background-color: #fff;
    min-height: 0;
    -webkit-box-shadow: 0px 0px 12px 1px rgba(87, 87, 87, 0.2);
    box-shadow: 0px 0px 12px 1px rgba(87, 87, 87, 0.2);
  }
}
@media print {
  #notebook-container {
    width: 100%;
  }
}
div.ui-widget-content {
  border: 1px solid #ababab;
  outline: none;
}
pre.dialog {
  background-color: #f7f7f7;
  border: 1px solid #ddd;
  border-radius: 2px;
  padding: 0.4em;
  padding-left: 2em;
}
p.dialog {
  padding: 0.2em;
}
/* Word-wrap output correctly.  This is the CSS3 spelling, though Firefox seems
   to not honor it correctly.  Webkit browsers (Chrome, rekonq, Safari) do.
 */
pre,
code,
kbd,
samp {
  white-space: pre-wrap;
}
#fonttest {
  font-family: monospace;
}
p {
  margin-bottom: 0;
}
.end_space {
  min-height: 100px;
  transition: height .2s ease;
}
.notebook_app > #header {
  -webkit-box-shadow: 0px 0px 12px 1px rgba(87, 87, 87, 0.2);
  box-shadow: 0px 0px 12px 1px rgba(87, 87, 87, 0.2);
}
@media not print {
  .notebook_app {
    background-color: #EEE;
  }
}
kbd {
  border-style: solid;
  border-width: 1px;
  box-shadow: none;
  margin: 2px;
  padding-left: 2px;
  padding-right: 2px;
  padding-top: 1px;
  padding-bottom: 1px;
}
.jupyter-keybindings {
  padding: 1px;
  line-height: 24px;
  border-bottom: 1px solid gray;
}
.jupyter-keybindings input {
  margin: 0;
  padding: 0;
  border: none;
}
.jupyter-keybindings i {
  padding: 6px;
}
.well code {
  background-color: #ffffff;
  border-color: #ababab;
  border-width: 1px;
  border-style: solid;
  padding: 2px;
  padding-top: 1px;
  padding-bottom: 1px;
}
/* CSS for the cell toolbar */
.celltoolbar {
  border: thin solid #CFCFCF;
  border-bottom: none;
  background: #EEE;
  border-radius: 2px 2px 0px 0px;
  width: 100%;
  height: 29px;
  padding-right: 4px;
  /* Old browsers */
  display: -webkit-box;
  -webkit-box-orient: horizontal;
  -webkit-box-align: stretch;
  display: -moz-box;
  -moz-box-orient: horizontal;
  -moz-box-align: stretch;
  display: box;
  box-orient: horizontal;
  box-align: stretch;
  /* Modern browsers */
  display: flex;
  flex-direction: row;
  align-items: stretch;
  /* Old browsers */
  -webkit-box-pack: end;
  -moz-box-pack: end;
  box-pack: end;
  /* Modern browsers */
  justify-content: flex-end;
  display: -webkit-flex;
}
@media print {
  .celltoolbar {
    display: none;
  }
}
.ctb_hideshow {
  display: none;
  vertical-align: bottom;
}
/* ctb_show is added to the ctb_hideshow div to show the cell toolbar.
   Cell toolbars are only shown when the ctb_global_show class is also set.
*/
.ctb_global_show .ctb_show.ctb_hideshow {
  display: block;
}
.ctb_global_show .ctb_show + .input_area,
.ctb_global_show .ctb_show + div.text_cell_input,
.ctb_global_show .ctb_show ~ div.text_cell_render {
  border-top-right-radius: 0px;
  border-top-left-radius: 0px;
}
.ctb_global_show .ctb_show ~ div.text_cell_render {
  border: 1px solid #cfcfcf;
}
.celltoolbar {
  font-size: 87%;
  padding-top: 3px;
}
.celltoolbar select {
  display: block;
  width: 100%;
  height: 32px;
  padding: 6px 12px;
  font-size: 13px;
  line-height: 1.42857143;
  color: #555555;
  background-color: #fff;
  background-image: none;
  border: 1px solid #ccc;
  border-radius: 2px;
  -webkit-box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075);
  box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075);
  -webkit-transition: border-color ease-in-out .15s, box-shadow ease-in-out .15s;
  -o-transition: border-color ease-in-out .15s, box-shadow ease-in-out .15s;
  transition: border-color ease-in-out .15s, box-shadow ease-in-out .15s;
  height: 30px;
  padding: 5px 10px;
  font-size: 12px;
  line-height: 1.5;
  border-radius: 1px;
  width: inherit;
  font-size: inherit;
  height: 22px;
  padding: 0px;
  display: inline-block;
}
.celltoolbar select:focus {
  border-color: #66afe9;
  outline: 0;
  -webkit-box-shadow: inset 0 1px 1px rgba(0,0,0,.075), 0 0 8px rgba(102, 175, 233, 0.6);
  box-shadow: inset 0 1px 1px rgba(0,0,0,.075), 0 0 8px rgba(102, 175, 233, 0.6);
}
.celltoolbar select::-moz-placeholder {
  color: #999;
  opacity: 1;
}
.celltoolbar select:-ms-input-placeholder {
  color: #999;
}
.celltoolbar select::-webkit-input-placeholder {
  color: #999;
}
.celltoolbar select::-ms-expand {
  border: 0;
  background-color: transparent;
}
.celltoolbar select[disabled],
.celltoolbar select[readonly],
fieldset[disabled] .celltoolbar select {
  background-color: #eeeeee;
  opacity: 1;
}
.celltoolbar select[disabled],
fieldset[disabled] .celltoolbar select {
  cursor: not-allowed;
}
textarea.celltoolbar select {
  height: auto;
}
select.celltoolbar select {
  height: 30px;
  line-height: 30px;
}
textarea.celltoolbar select,
select[multiple].celltoolbar select {
  height: auto;
}
.celltoolbar label {
  margin-left: 5px;
  margin-right: 5px;
}
.tags_button_container {
  width: 100%;
  display: flex;
}
.tag-container {
  display: flex;
  flex-direction: row;
  flex-grow: 1;
  overflow: hidden;
  position: relative;
}
.tag-container > * {
  margin: 0 4px;
}
.remove-tag-btn {
  margin-left: 4px;
}
.tags-input {
  display: flex;
}
.cell-tag:last-child:after {
  content: "";
  position: absolute;
  right: 0;
  width: 40px;
  height: 100%;
  /* Fade to background color of cell toolbar */
  background: linear-gradient(to right, rgba(0, 0, 0, 0), #EEE);
}
.tags-input > * {
  margin-left: 4px;
}
.cell-tag,
.tags-input input,
.tags-input button {
  display: block;
  width: 100%;
  height: 32px;
  padding: 6px 12px;
  font-size: 13px;
  line-height: 1.42857143;
  color: #555555;
  background-color: #fff;
  background-image: none;
  border: 1px solid #ccc;
  border-radius: 2px;
  -webkit-box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075);
  box-shadow: inset 0 1px 1px rgba(0, 0, 0, 0.075);
  -webkit-transition: border-color ease-in-out .15s, box-shadow ease-in-out .15s;
  -o-transition: border-color ease-in-out .15s, box-shadow ease-in-out .15s;
  transition: border-color ease-in-out .15s, box-shadow ease-in-out .15s;
  height: 30px;
  padding: 5px 10px;
  font-size: 12px;
  line-height: 1.5;
  border-radius: 1px;
  box-shadow: none;
  width: inherit;
  font-size: inherit;
  height: 22px;
  line-height: 22px;
  padding: 0px 4px;
  display: inline-block;
}
.cell-tag:focus,
.tags-input input:focus,
.tags-input button:focus {
  border-color: #66afe9;
  outline: 0;
  -webkit-box-shadow: inset 0 1px 1px rgba(0,0,0,.075), 0 0 8px rgba(102, 175, 233, 0.6);
  box-shadow: inset 0 1px 1px rgba(0,0,0,.075), 0 0 8px rgba(102, 175, 233, 0.6);
}
.cell-tag::-moz-placeholder,
.tags-input input::-moz-placeholder,
.tags-input button::-moz-placeholder {
  color: #999;
  opacity: 1;
}
.cell-tag:-ms-input-placeholder,
.tags-input input:-ms-input-placeholder,
.tags-input button:-ms-input-placeholder {
  color: #999;
}
.cell-tag::-webkit-input-placeholder,
.tags-input input::-webkit-input-placeholder,
.tags-input button::-webkit-input-placeholder {
  color: #999;
}
.cell-tag::-ms-expand,
.tags-input input::-ms-expand,
.tags-input button::-ms-expand {
  border: 0;
  background-color: transparent;
}
.cell-tag[disabled],
.tags-input input[disabled],
.tags-input button[disabled],
.cell-tag[readonly],
.tags-input input[readonly],
.tags-input button[readonly],
fieldset[disabled] .cell-tag,
fieldset[disabled] .tags-input input,
fieldset[disabled] .tags-input button {
  background-color: #eeeeee;
  opacity: 1;
}
.cell-tag[disabled],
.tags-input input[disabled],
.tags-input button[disabled],
fieldset[disabled] .cell-tag,
fieldset[disabled] .tags-input input,
fieldset[disabled] .tags-input button {
  cursor: not-allowed;
}
textarea.cell-tag,
textarea.tags-input input,
textarea.tags-input button {
  height: auto;
}
select.cell-tag,
select.tags-input input,
select.tags-input button {
  height: 30px;
  line-height: 30px;
}
textarea.cell-tag,
textarea.tags-input input,
textarea.tags-input button,
select[multiple].cell-tag,
select[multiple].tags-input input,
select[multiple].tags-input button {
  height: auto;
}
.cell-tag,
.tags-input button {
  padding: 0px 4px;
}
.cell-tag {
  background-color: #fff;
  white-space: nowrap;
}
.tags-input input[type=text]:focus {
  outline: none;
  box-shadow: none;
  border-color: #ccc;
}
.completions {
  position: absolute;
  z-index: 110;
  overflow: hidden;
  border: 1px solid #ababab;
  border-radius: 2px;
  -webkit-box-shadow: 0px 6px 10px -1px #adadad;
  box-shadow: 0px 6px 10px -1px #adadad;
  line-height: 1;
}
.completions select {
  background: white;
  outline: none;
  border: none;
  padding: 0px;
  margin: 0px;
  overflow: auto;
  font-family: monospace;
  font-size: 110%;
  color: #000;
  width: auto;
}
.completions select option.context {
  color: #286090;
}
#kernel_logo_widget .current_kernel_logo {
  display: none;
  margin-top: -1px;
  margin-bottom: -1px;
  width: 32px;
  height: 32px;
}
[dir="rtl"] #kernel_logo_widget {
  float: left !important;
  float: left;
}
.modal .modal-body .move-path {
  display: flex;
  flex-direction: row;
  justify-content: space;
  align-items: center;
}
.modal .modal-body .move-path .server-root {
  padding-right: 20px;
}
.modal .modal-body .move-path .path-input {
  flex: 1;
}
#menubar {
  box-sizing: border-box;
  -moz-box-sizing: border-box;
  -webkit-box-sizing: border-box;
  margin-top: 1px;
}
#menubar .navbar {
  border-top: 1px;
  border-radius: 0px 0px 2px 2px;
  margin-bottom: 0px;
}
#menubar .navbar-toggle {
  float: left;
  padding-top: 7px;
  padding-bottom: 7px;
  border: none;
}
#menubar .navbar-collapse {
  clear: left;
}
[dir="rtl"] #menubar .navbar-toggle {
  float: right;
}
[dir="rtl"] #menubar .navbar-collapse {
  clear: right;
}
[dir="rtl"] #menubar .navbar-nav {
  float: right;
}
[dir="rtl"] #menubar .nav {
  padding-right: 0px;
}
[dir="rtl"] #menubar .navbar-nav > li {
  float: right;
}
[dir="rtl"] #menubar .navbar-right {
  float: left !important;
}
[dir="rtl"] ul.dropdown-menu {
  text-align: right;
  left: auto;
}
[dir="rtl"] ul#new-menu.dropdown-menu {
  right: auto;
  left: 0;
}
.nav-wrapper {
  border-bottom: 1px solid #e7e7e7;
}
i.menu-icon {
  padding-top: 4px;
}
[dir="rtl"] i.menu-icon.pull-right {
  float: left !important;
  float: left;
}
ul#help_menu li a {
  overflow: hidden;
  padding-right: 2.2em;
}
ul#help_menu li a i {
  margin-right: -1.2em;
}
[dir="rtl"] ul#help_menu li a {
  padding-left: 2.2em;
}
[dir="rtl"] ul#help_menu li a i {
  margin-right: 0;
  margin-left: -1.2em;
}
[dir="rtl"] ul#help_menu li a i.pull-right {
  float: left !important;
  float: left;
}
.dropdown-submenu {
  position: relative;
}
.dropdown-submenu > .dropdown-menu {
  top: 0;
  left: 100%;
  margin-top: -6px;
  margin-left: -1px;
}
[dir="rtl"] .dropdown-submenu > .dropdown-menu {
  right: 100%;
  margin-right: -1px;
}
.dropdown-submenu:hover > .dropdown-menu {
  display: block;
}
.dropdown-submenu > a:after {
  display: inline-block;
  font: normal normal normal 14px/1 FontAwesome;
  font-size: inherit;
  text-rendering: auto;
  -webkit-font-smoothing: antialiased;
  -moz-osx-font-smoothing: grayscale;
  display: block;
  content: "\f0da";
  float: right;
  color: #333333;
  margin-top: 2px;
  margin-right: -10px;
}
.dropdown-submenu > a:after.fa-pull-left {
  margin-right: .3em;
}
.dropdown-submenu > a:after.fa-pull-right {
  margin-left: .3em;
}
.dropdown-submenu > a:after.pull-left {
  margin-right: .3em;
}
.dropdown-submenu > a:after.pull-right {
  margin-left: .3em;
}
[dir="rtl"] .dropdown-submenu > a:after {
  float: left;
  content: "\f0d9";
  margin-right: 0;
  margin-left: -10px;
}
.dropdown-submenu:hover > a:after {
  color: #262626;
}
.dropdown-submenu.pull-left {
  float: none;
}
.dropdown-submenu.pull-left > .dropdown-menu {
  left: -100%;
  margin-left: 10px;
}
#notification_area {
  float: right !important;
  float: right;
  z-index: 10;
}
[dir="rtl"] #notification_area {
  float: left !important;
  float: left;
}
.indicator_area {
  float: right !important;
  float: right;
  color: #777;
  margin-left: 5px;
  margin-right: 5px;
  width: 11px;
  z-index: 10;
  text-align: center;
  width: auto;
}
[dir="rtl"] .indicator_area {
  float: left !important;
  float: left;
}
#kernel_indicator {
  float: right !important;
  float: right;
  color: #777;
  margin-left: 5px;
  margin-right: 5px;
  width: 11px;
  z-index: 10;
  text-align: center;
  width: auto;
  border-left: 1px solid;
}
#kernel_indicator .kernel_indicator_name {
  padding-left: 5px;
  padding-right: 5px;
}
[dir="rtl"] #kernel_indicator {
  float: left !important;
  float: left;
  border-left: 0;
  border-right: 1px solid;
}
#modal_indicator {
  float: right !important;
  float: right;
  color: #777;
  margin-left: 5px;
  margin-right: 5px;
  width: 11px;
  z-index: 10;
  text-align: center;
  width: auto;
}
[dir="rtl"] #modal_indicator {
  float: left !important;
  float: left;
}
#readonly-indicator {
  float: right !important;
  float: right;
  color: #777;
  margin-left: 5px;
  margin-right: 5px;
  width: 11px;
  z-index: 10;
  text-align: center;
  width: auto;
  margin-top: 2px;
  margin-bottom: 0px;
  margin-left: 0px;
  margin-right: 0px;
  display: none;
}
.modal_indicator:before {
  width: 1.28571429em;
  text-align: center;
}
.edit_mode .modal_indicator:before {
  display: inline-block;
  font: normal normal normal 14px/1 FontAwesome;
  font-size: inherit;
  text-rendering: auto;
  -webkit-font-smoothing: antialiased;
  -moz-osx-font-smoothing: grayscale;
  content: "\f040";
}
.edit_mode .modal_indicator:before.fa-pull-left {
  margin-right: .3em;
}
.edit_mode .modal_indicator:before.fa-pull-right {
  margin-left: .3em;
}
.edit_mode .modal_indicator:before.pull-left {
  margin-right: .3em;
}
.edit_mode .modal_indicator:before.pull-right {
  margin-left: .3em;
}
.command_mode .modal_indicator:before {
  display: inline-block;
  font: normal normal normal 14px/1 FontAwesome;
  font-size: inherit;
  text-rendering: auto;
  -webkit-font-smoothing: antialiased;
  -moz-osx-font-smoothing: grayscale;
  content: ' ';
}
.command_mode .modal_indicator:before.fa-pull-left {
  margin-right: .3em;
}
.command_mode .modal_indicator:before.fa-pull-right {
  margin-left: .3em;
}
.command_mode .modal_indicator:before.pull-left {
  margin-right: .3em;
}
.command_mode .modal_indicator:before.pull-right {
  margin-left: .3em;
}
.kernel_idle_icon:before {
  display: inline-block;
  font: normal normal normal 14px/1 FontAwesome;
  font-size: inherit;
  text-rendering: auto;
  -webkit-font-smoothing: antialiased;
  -moz-osx-font-smoothing: grayscale;
  content: "\f10c";
}
.kernel_idle_icon:before.fa-pull-left {
  margin-right: .3em;
}
.kernel_idle_icon:before.fa-pull-right {
  margin-left: .3em;
}
.kernel_idle_icon:before.pull-left {
  margin-right: .3em;
}
.kernel_idle_icon:before.pull-right {
  margin-left: .3em;
}
.kernel_busy_icon:before {
  display: inline-block;
  font: normal normal normal 14px/1 FontAwesome;
  font-size: inherit;
  text-rendering: auto;
  -webkit-font-smoothing: antialiased;
  -moz-osx-font-smoothing: grayscale;
  content: "\f111";
}
.kernel_busy_icon:before.fa-pull-left {
  margin-right: .3em;
}
.kernel_busy_icon:before.fa-pull-right {
  margin-left: .3em;
}
.kernel_busy_icon:before.pull-left {
  margin-right: .3em;
}
.kernel_busy_icon:before.pull-right {
  margin-left: .3em;
}
.kernel_dead_icon:before {
  display: inline-block;
  font: normal normal normal 14px/1 FontAwesome;
  font-size: inherit;
  text-rendering: auto;
  -webkit-font-smoothing: antialiased;
  -moz-osx-font-smoothing: grayscale;
  content: "\f1e2";
}
.kernel_dead_icon:before.fa-pull-left {
  margin-right: .3em;
}
.kernel_dead_icon:before.fa-pull-right {
  margin-left: .3em;
}
.kernel_dead_icon:before.pull-left {
  margin-right: .3em;
}
.kernel_dead_icon:before.pull-right {
  margin-left: .3em;
}
.kernel_disconnected_icon:before {
  display: inline-block;
  font: normal normal normal 14px/1 FontAwesome;
  font-size: inherit;
  text-rendering: auto;
  -webkit-font-smoothing: antialiased;
  -moz-osx-font-smoothing: grayscale;
  content: "\f127";
}
.kernel_disconnected_icon:before.fa-pull-left {
  margin-right: .3em;
}
.kernel_disconnected_icon:before.fa-pull-right {
  margin-left: .3em;
}
.kernel_disconnected_icon:before.pull-left {
  margin-right: .3em;
}
.kernel_disconnected_icon:before.pull-right {
  margin-left: .3em;
}
.notification_widget {
  color: #777;
  z-index: 10;
  background: rgba(240, 240, 240, 0.5);
  margin-right: 4px;
  color: #333;
  background-color: #fff;
  border-color: #ccc;
}
.notification_widget:focus,
.notification_widget.focus {
  color: #333;
  background-color: #e6e6e6;
  border-color: #8c8c8c;
}
.notification_widget:hover {
  color: #333;
  background-color: #e6e6e6;
  border-color: #adadad;
}
.notification_widget:active,
.notification_widget.active,
.open > .dropdown-toggle.notification_widget {
  color: #333;
  background-color: #e6e6e6;
  border-color: #adadad;
}
.notification_widget:active:hover,
.notification_widget.active:hover,
.open > .dropdown-toggle.notification_widget:hover,
.notification_widget:active:focus,
.notification_widget.active:focus,
.open > .dropdown-toggle.notification_widget:focus,
.notification_widget:active.focus,
.notification_widget.active.focus,
.open > .dropdown-toggle.notification_widget.focus {
  color: #333;
  background-color: #d4d4d4;
  border-color: #8c8c8c;
}
.notification_widget:active,
.notification_widget.active,
.open > .dropdown-toggle.notification_widget {
  background-image: none;
}
.notification_widget.disabled:hover,
.notification_widget[disabled]:hover,
fieldset[disabled] .notification_widget:hover,
.notification_widget.disabled:focus,
.notification_widget[disabled]:focus,
fieldset[disabled] .notification_widget:focus,
.notification_widget.disabled.focus,
.notification_widget[disabled].focus,
fieldset[disabled] .notification_widget.focus {
  background-color: #fff;
  border-color: #ccc;
}
.notification_widget .badge {
  color: #fff;
  background-color: #333;
}
.notification_widget.warning {
  color: #fff;
  background-color: #f0ad4e;
  border-color: #eea236;
}
.notification_widget.warning:focus,
.notification_widget.warning.focus {
  color: #fff;
  background-color: #ec971f;
  border-color: #985f0d;
}
.notification_widget.warning:hover {
  color: #fff;
  background-color: #ec971f;
  border-color: #d58512;
}
.notification_widget.warning:active,
.notification_widget.warning.active,
.open > .dropdown-toggle.notification_widget.warning {
  color: #fff;
  background-color: #ec971f;
  border-color: #d58512;
}
.notification_widget.warning:active:hover,
.notification_widget.warning.active:hover,
.open > .dropdown-toggle.notification_widget.warning:hover,
.notification_widget.warning:active:focus,
.notification_widget.warning.active:focus,
.open > .dropdown-toggle.notification_widget.warning:focus,
.notification_widget.warning:active.focus,
.notification_widget.warning.active.focus,
.open > .dropdown-toggle.notification_widget.warning.focus {
  color: #fff;
  background-color: #d58512;
  border-color: #985f0d;
}
.notification_widget.warning:active,
.notification_widget.warning.active,
.open > .dropdown-toggle.notification_widget.warning {
  background-image: none;
}
.notification_widget.warning.disabled:hover,
.notification_widget.warning[disabled]:hover,
fieldset[disabled] .notification_widget.warning:hover,
.notification_widget.warning.disabled:focus,
.notification_widget.warning[disabled]:focus,
fieldset[disabled] .notification_widget.warning:focus,
.notification_widget.warning.disabled.focus,
.notification_widget.warning[disabled].focus,
fieldset[disabled] .notification_widget.warning.focus {
  background-color: #f0ad4e;
  border-color: #eea236;
}
.notification_widget.warning .badge {
  color: #f0ad4e;
  background-color: #fff;
}
.notification_widget.success {
  color: #fff;
  background-color: #5cb85c;
  border-color: #4cae4c;
}
.notification_widget.success:focus,
.notification_widget.success.focus {
  color: #fff;
  background-color: #449d44;
  border-color: #255625;
}
.notification_widget.success:hover {
  color: #fff;
  background-color: #449d44;
  border-color: #398439;
}
.notification_widget.success:active,
.notification_widget.success.active,
.open > .dropdown-toggle.notification_widget.success {
  color: #fff;
  background-color: #449d44;
  border-color: #398439;
}
.notification_widget.success:active:hover,
.notification_widget.success.active:hover,
.open > .dropdown-toggle.notification_widget.success:hover,
.notification_widget.success:active:focus,
.notification_widget.success.active:focus,
.open > .dropdown-toggle.notification_widget.success:focus,
.notification_widget.success:active.focus,
.notification_widget.success.active.focus,
.open > .dropdown-toggle.notification_widget.success.focus {
  color: #fff;
  background-color: #398439;
  border-color: #255625;
}
.notification_widget.success:active,
.notification_widget.success.active,
.open > .dropdown-toggle.notification_widget.success {
  background-image: none;
}
.notification_widget.success.disabled:hover,
.notification_widget.success[disabled]:hover,
fieldset[disabled] .notification_widget.success:hover,
.notification_widget.success.disabled:focus,
.notification_widget.success[disabled]:focus,
fieldset[disabled] .notification_widget.success:focus,
.notification_widget.success.disabled.focus,
.notification_widget.success[disabled].focus,
fieldset[disabled] .notification_widget.success.focus {
  background-color: #5cb85c;
  border-color: #4cae4c;
}
.notification_widget.success .badge {
  color: #5cb85c;
  background-color: #fff;
}
.notification_widget.info {
  color: #fff;
  background-color: #5bc0de;
  border-color: #46b8da;
}
.notification_widget.info:focus,
.notification_widget.info.focus {
  color: #fff;
  background-color: #31b0d5;
  border-color: #1b6d85;
}
.notification_widget.info:hover {
  color: #fff;
  background-color: #31b0d5;
  border-color: #269abc;
}
.notification_widget.info:active,
.notification_widget.info.active,
.open > .dropdown-toggle.notification_widget.info {
  color: #fff;
  background-color: #31b0d5;
  border-color: #269abc;
}
.notification_widget.info:active:hover,
.notification_widget.info.active:hover,
.open > .dropdown-toggle.notification_widget.info:hover,
.notification_widget.info:active:focus,
.notification_widget.info.active:focus,
.open > .dropdown-toggle.notification_widget.info:focus,
.notification_widget.info:active.focus,
.notification_widget.info.active.focus,
.open > .dropdown-toggle.notification_widget.info.focus {
  color: #fff;
  background-color: #269abc;
  border-color: #1b6d85;
}
.notification_widget.info:active,
.notification_widget.info.active,
.open > .dropdown-toggle.notification_widget.info {
  background-image: none;
}
.notification_widget.info.disabled:hover,
.notification_widget.info[disabled]:hover,
fieldset[disabled] .notification_widget.info:hover,
.notification_widget.info.disabled:focus,
.notification_widget.info[disabled]:focus,
fieldset[disabled] .notification_widget.info:focus,
.notification_widget.info.disabled.focus,
.notification_widget.info[disabled].focus,
fieldset[disabled] .notification_widget.info.focus {
  background-color: #5bc0de;
  border-color: #46b8da;
}
.notification_widget.info .badge {
  color: #5bc0de;
  background-color: #fff;
}
.notification_widget.danger {
  color: #fff;
  background-color: #d9534f;
  border-color: #d43f3a;
}
.notification_widget.danger:focus,
.notification_widget.danger.focus {
  color: #fff;
  background-color: #c9302c;
  border-color: #761c19;
}
.notification_widget.danger:hover {
  color: #fff;
  background-color: #c9302c;
  border-color: #ac2925;
}
.notification_widget.danger:active,
.notification_widget.danger.active,
.open > .dropdown-toggle.notification_widget.danger {
  color: #fff;
  background-color: #c9302c;
  border-color: #ac2925;
}
.notification_widget.danger:active:hover,
.notification_widget.danger.active:hover,
.open > .dropdown-toggle.notification_widget.danger:hover,
.notification_widget.danger:active:focus,
.notification_widget.danger.active:focus,
.open > .dropdown-toggle.notification_widget.danger:focus,
.notification_widget.danger:active.focus,
.notification_widget.danger.active.focus,
.open > .dropdown-toggle.notification_widget.danger.focus {
  color: #fff;
  background-color: #ac2925;
  border-color: #761c19;
}
.notification_widget.danger:active,
.notification_widget.danger.active,
.open > .dropdown-toggle.notification_widget.danger {
  background-image: none;
}
.notification_widget.danger.disabled:hover,
.notification_widget.danger[disabled]:hover,
fieldset[disabled] .notification_widget.danger:hover,
.notification_widget.danger.disabled:focus,
.notification_widget.danger[disabled]:focus,
fieldset[disabled] .notification_widget.danger:focus,
.notification_widget.danger.disabled.focus,
.notification_widget.danger[disabled].focus,
fieldset[disabled] .notification_widget.danger.focus {
  background-color: #d9534f;
  border-color: #d43f3a;
}
.notification_widget.danger .badge {
  color: #d9534f;
  background-color: #fff;
}
div#pager {
  background-color: #fff;
  font-size: 14px;
  line-height: 20px;
  overflow: hidden;
  display: none;
  position: fixed;
  bottom: 0px;
  width: 100%;
  max-height: 50%;
  padding-top: 8px;
  -webkit-box-shadow: 0px 0px 12px 1px rgba(87, 87, 87, 0.2);
  box-shadow: 0px 0px 12px 1px rgba(87, 87, 87, 0.2);
  /* Display over codemirror */
  z-index: 100;
  /* Hack which prevents jquery ui resizable from changing top. */
  top: auto !important;
}
div#pager pre {
  line-height: 1.21429em;
  color: #000;
  background-color: #f7f7f7;
  padding: 0.4em;
}
div#pager #pager-button-area {
  position: absolute;
  top: 8px;
  right: 20px;
}
div#pager #pager-contents {
  position: relative;
  overflow: auto;
  width: 100%;
  height: 100%;
}
div#pager #pager-contents #pager-container {
  position: relative;
  padding: 15px 0px;
  box-sizing: border-box;
  -moz-box-sizing: border-box;
  -webkit-box-sizing: border-box;
}
div#pager .ui-resizable-handle {
  top: 0px;
  height: 8px;
  background: #f7f7f7;
  border-top: 1px solid #cfcfcf;
  border-bottom: 1px solid #cfcfcf;
  /* This injects handle bars (a short, wide = symbol) for 
        the resize handle. */
}
div#pager .ui-resizable-handle::after {
  content: '';
  top: 2px;
  left: 50%;
  height: 3px;
  width: 30px;
  margin-left: -15px;
  position: absolute;
  border-top: 1px solid #cfcfcf;
}
.quickhelp {
  /* Old browsers */
  display: -webkit-box;
  -webkit-box-orient: horizontal;
  -webkit-box-align: stretch;
  display: -moz-box;
  -moz-box-orient: horizontal;
  -moz-box-align: stretch;
  display: box;
  box-orient: horizontal;
  box-align: stretch;
  /* Modern browsers */
  display: flex;
  flex-direction: row;
  align-items: stretch;
  line-height: 1.8em;
}
.shortcut_key {
  display: inline-block;
  width: 21ex;
  text-align: right;
  font-family: monospace;
}
.shortcut_descr {
  display: inline-block;
  /* Old browsers */
  -webkit-box-flex: 1;
  -moz-box-flex: 1;
  box-flex: 1;
  /* Modern browsers */
  flex: 1;
}
span.save_widget {
  height: 30px;
  margin-top: 4px;
  display: flex;
  justify-content: flex-start;
  align-items: baseline;
  width: 50%;
  flex: 1;
}
span.save_widget span.filename {
  height: 100%;
  line-height: 1em;
  margin-left: 16px;
  border: none;
  font-size: 146.5%;
  text-overflow: ellipsis;
  overflow: hidden;
  white-space: nowrap;
  border-radius: 2px;
}
span.save_widget span.filename:hover {
  background-color: #e6e6e6;
}
[dir="rtl"] span.save_widget.pull-left {
  float: right !important;
  float: right;
}
[dir="rtl"] span.save_widget span.filename {
  margin-left: 0;
  margin-right: 16px;
}
span.checkpoint_status,
span.autosave_status {
  font-size: small;
  white-space: nowrap;
  padding: 0 5px;
}
@media (max-width: 767px) {
  span.save_widget {
    font-size: small;
    padding: 0 0 0 5px;
  }
  span.checkpoint_status,
  span.autosave_status {
    display: none;
  }
}
@media (min-width: 768px) and (max-width: 991px) {
  span.checkpoint_status {
    display: none;
  }
  span.autosave_status {
    font-size: x-small;
  }
}
.toolbar {
  padding: 0px;
  margin-left: -5px;
  margin-top: 2px;
  margin-bottom: 5px;
  box-sizing: border-box;
  -moz-box-sizing: border-box;
  -webkit-box-sizing: border-box;
}
.toolbar select,
.toolbar label {
  width: auto;
  vertical-align: middle;
  margin-right: 2px;
  margin-bottom: 0px;
  display: inline;
  font-size: 92%;
  margin-left: 0.3em;
  margin-right: 0.3em;
  padding: 0px;
  padding-top: 3px;
}
.toolbar .btn {
  padding: 2px 8px;
}
.toolbar .btn-group {
  margin-top: 0px;
  margin-left: 5px;
}
.toolbar-btn-label {
  margin-left: 6px;
}
#maintoolbar {
  margin-bottom: -3px;
  margin-top: -8px;
  border: 0px;
  min-height: 27px;
  margin-left: 0px;
  padding-top: 11px;
  padding-bottom: 3px;
}
#maintoolbar .navbar-text {
  float: none;
  vertical-align: middle;
  text-align: right;
  margin-left: 5px;
  margin-right: 0px;
  margin-top: 0px;
}
.select-xs {
  height: 24px;
}
[dir="rtl"] .btn-group > .btn,
.btn-group-vertical > .btn {
  float: right;
}
.pulse,
.dropdown-menu > li > a.pulse,
li.pulse > a.dropdown-toggle,
li.pulse.open > a.dropdown-toggle {
  background-color: #F37626;
  color: white;
}
/**
 * Primary styles
 *
 * Author: Jupyter Development Team
 */
/** WARNING IF YOU ARE EDITTING THIS FILE, if this is a .css file, It has a lot
 * of chance of beeing generated from the ../less/[samename].less file, you can
 * try to get back the less file by reverting somme commit in history
 **/
/*
 * We'll try to get something pretty, so we
 * have some strange css to have the scroll bar on
 * the left with fix button on the top right of the tooltip
 */
@-moz-keyframes fadeOut {
  from {
    opacity: 1;
  }
  to {
    opacity: 0;
  }
}
@-webkit-keyframes fadeOut {
  from {
    opacity: 1;
  }
  to {
    opacity: 0;
  }
}
@-moz-keyframes fadeIn {
  from {
    opacity: 0;
  }
  to {
    opacity: 1;
  }
}
@-webkit-keyframes fadeIn {
  from {
    opacity: 0;
  }
  to {
    opacity: 1;
  }
}
/*properties of tooltip after "expand"*/
.bigtooltip {
  overflow: auto;
  height: 200px;
  -webkit-transition-property: height;
  -webkit-transition-duration: 500ms;
  -moz-transition-property: height;
  -moz-transition-duration: 500ms;
  transition-property: height;
  transition-duration: 500ms;
}
/*properties of tooltip before "expand"*/
.smalltooltip {
  -webkit-transition-property: height;
  -webkit-transition-duration: 500ms;
  -moz-transition-property: height;
  -moz-transition-duration: 500ms;
  transition-property: height;
  transition-duration: 500ms;
  text-overflow: ellipsis;
  overflow: hidden;
  height: 80px;
}
.tooltipbuttons {
  position: absolute;
  padding-right: 15px;
  top: 0px;
  right: 0px;
}
.tooltiptext {
  /*avoid the button to overlap on some docstring*/
  padding-right: 30px;
}
.ipython_tooltip {
  max-width: 700px;
  /*fade-in animation when inserted*/
  -webkit-animation: fadeOut 400ms;
  -moz-animation: fadeOut 400ms;
  animation: fadeOut 400ms;
  -webkit-animation: fadeIn 400ms;
  -moz-animation: fadeIn 400ms;
  animation: fadeIn 400ms;
  vertical-align: middle;
  background-color: #f7f7f7;
  overflow: visible;
  border: #ababab 1px solid;
  outline: none;
  padding: 3px;
  margin: 0px;
  padding-left: 7px;
  font-family: monospace;
  min-height: 50px;
  -moz-box-shadow: 0px 6px 10px -1px #adadad;
  -webkit-box-shadow: 0px 6px 10px -1px #adadad;
  box-shadow: 0px 6px 10px -1px #adadad;
  border-radius: 2px;
  position: absolute;
  z-index: 1000;
}
.ipython_tooltip a {
  float: right;
}
.ipython_tooltip .tooltiptext pre {
  border: 0;
  border-radius: 0;
  font-size: 100%;
  background-color: #f7f7f7;
}
.pretooltiparrow {
  left: 0px;
  margin: 0px;
  top: -16px;
  width: 40px;
  height: 16px;
  overflow: hidden;
  position: absolute;
}
.pretooltiparrow:before {
  background-color: #f7f7f7;
  border: 1px #ababab solid;
  z-index: 11;
  content: "";
  position: absolute;
  left: 15px;
  top: 10px;
  width: 25px;
  height: 25px;
  -webkit-transform: rotate(45deg);
  -moz-transform: rotate(45deg);
  -ms-transform: rotate(45deg);
  -o-transform: rotate(45deg);
}
ul.typeahead-list i {
  margin-left: -10px;
  width: 18px;
}
[dir="rtl"] ul.typeahead-list i {
  margin-left: 0;
  margin-right: -10px;
}
ul.typeahead-list {
  max-height: 80vh;
  overflow: auto;
}
ul.typeahead-list > li > a {
  /** Firefox bug **/
  /* see https://github.com/jupyter/notebook/issues/559 */
  white-space: normal;
}
ul.typeahead-list  > li > a.pull-right {
  float: left !important;
  float: left;
}
[dir="rtl"] .typeahead-list {
  text-align: right;
}
.cmd-palette .modal-body {
  padding: 7px;
}
.cmd-palette form {
  background: white;
}
.cmd-palette input {
  outline: none;
}
.no-shortcut {
  min-width: 20px;
  color: transparent;
}
[dir="rtl"] .no-shortcut.pull-right {
  float: left !important;
  float: left;
}
[dir="rtl"] .command-shortcut.pull-right {
  float: left !important;
  float: left;
}
.command-shortcut:before {
  content: "(command mode)";
  padding-right: 3px;
  color: #777777;
}
.edit-shortcut:before {
  content: "(edit)";
  padding-right: 3px;
  color: #777777;
}
[dir="rtl"] .edit-shortcut.pull-right {
  float: left !important;
  float: left;
}
#find-and-replace #replace-preview .match,
#find-and-replace #replace-preview .insert {
  background-color: #BBDEFB;
  border-color: #90CAF9;
  border-style: solid;
  border-width: 1px;
  border-radius: 0px;
}
[dir="ltr"] #find-and-replace .input-group-btn + .form-control {
  border-left: none;
}
[dir="rtl"] #find-and-replace .input-group-btn + .form-control {
  border-right: none;
}
#find-and-replace #replace-preview .replace .match {
  background-color: #FFCDD2;
  border-color: #EF9A9A;
  border-radius: 0px;
}
#find-and-replace #replace-preview .replace .insert {
  background-color: #C8E6C9;
  border-color: #A5D6A7;
  border-radius: 0px;
}
#find-and-replace #replace-preview {
  max-height: 60vh;
  overflow: auto;
}
#find-and-replace #replace-preview pre {
  padding: 5px 10px;
}
.terminal-app {
  background: #EEE;
}
.terminal-app #header {
  background: #fff;
  -webkit-box-shadow: 0px 0px 12px 1px rgba(87, 87, 87, 0.2);
  box-shadow: 0px 0px 12px 1px rgba(87, 87, 87, 0.2);
}
.terminal-app .terminal {
  width: 100%;
  float: left;
  font-family: monospace;
  color: white;
  background: black;
  padding: 0.4em;
  border-radius: 2px;
  -webkit-box-shadow: 0px 0px 12px 1px rgba(87, 87, 87, 0.4);
  box-shadow: 0px 0px 12px 1px rgba(87, 87, 87, 0.4);
}
.terminal-app .terminal,
.terminal-app .terminal dummy-screen {
  line-height: 1em;
  font-size: 14px;
}
.terminal-app .terminal .xterm-rows {
  padding: 10px;
}
.terminal-app .terminal-cursor {
  color: black;
  background: white;
}
.terminal-app #terminado-container {
  margin-top: 20px;
}
/*# sourceMappingURL=style.min.css.map */
    </style>
<style type="text/css">
    .highlight .hll { background-color: #ffffcc }
.highlight  { background: #f8f8f8; }
.highlight .c { color: #408080; font-style: italic } /* Comment */
.highlight .err { border: 1px solid #FF0000 } /* Error */
.highlight .k { color: #008000; font-weight: bold } /* Keyword */
.highlight .o { color: #666666 } /* Operator */
.highlight .ch { color: #408080; font-style: italic } /* Comment.Hashbang */
.highlight .cm { color: #408080; font-style: italic } /* Comment.Multiline */
.highlight .cp { color: #BC7A00 } /* Comment.Preproc */
.highlight .cpf { color: #408080; font-style: italic } /* Comment.PreprocFile */
.highlight .c1 { color: #408080; font-style: italic } /* Comment.Single */
.highlight .cs { color: #408080; font-style: italic } /* Comment.Special */
.highlight .gd { color: #A00000 } /* Generic.Deleted */
.highlight .ge { font-style: italic } /* Generic.Emph */
.highlight .gr { color: #FF0000 } /* Generic.Error */
.highlight .gh { color: #000080; font-weight: bold } /* Generic.Heading */
.highlight .gi { color: #00A000 } /* Generic.Inserted */
.highlight .go { color: #888888 } /* Generic.Output */
.highlight .gp { color: #000080; font-weight: bold } /* Generic.Prompt */
.highlight .gs { font-weight: bold } /* Generic.Strong */
.highlight .gu { color: #800080; font-weight: bold } /* Generic.Subheading */
.highlight .gt { color: #0044DD } /* Generic.Traceback */
.highlight .kc { color: #008000; font-weight: bold } /* Keyword.Constant */
.highlight .kd { color: #008000; font-weight: bold } /* Keyword.Declaration */
.highlight .kn { color: #008000; font-weight: bold } /* Keyword.Namespace */
.highlight .kp { color: #008000 } /* Keyword.Pseudo */
.highlight .kr { color: #008000; font-weight: bold } /* Keyword.Reserved */
.highlight .kt { color: #B00040 } /* Keyword.Type */
.highlight .m { color: #666666 } /* Literal.Number */
.highlight .s { color: #BA2121 } /* Literal.String */
.highlight .na { color: #7D9029 } /* Name.Attribute */
.highlight .nb { color: #008000 } /* Name.Builtin */
.highlight .nc { color: #0000FF; font-weight: bold } /* Name.Class */
.highlight .no { color: #880000 } /* Name.Constant */
.highlight .nd { color: #AA22FF } /* Name.Decorator */
.highlight .ni { color: #999999; font-weight: bold } /* Name.Entity */
.highlight .ne { color: #D2413A; font-weight: bold } /* Name.Exception */
.highlight .nf { color: #0000FF } /* Name.Function */
.highlight .nl { color: #A0A000 } /* Name.Label */
.highlight .nn { color: #0000FF; font-weight: bold } /* Name.Namespace */
.highlight .nt { color: #008000; font-weight: bold } /* Name.Tag */
.highlight .nv { color: #19177C } /* Name.Variable */
.highlight .ow { color: #AA22FF; font-weight: bold } /* Operator.Word */
.highlight .w { color: #bbbbbb } /* Text.Whitespace */
.highlight .mb { color: #666666 } /* Literal.Number.Bin */
.highlight .mf { color: #666666 } /* Literal.Number.Float */
.highlight .mh { color: #666666 } /* Literal.Number.Hex */
.highlight .mi { color: #666666 } /* Literal.Number.Integer */
.highlight .mo { color: #666666 } /* Literal.Number.Oct */
.highlight .sa { color: #BA2121 } /* Literal.String.Affix */
.highlight .sb { color: #BA2121 } /* Literal.String.Backtick */
.highlight .sc { color: #BA2121 } /* Literal.String.Char */
.highlight .dl { color: #BA2121 } /* Literal.String.Delimiter */
.highlight .sd { color: #BA2121; font-style: italic } /* Literal.String.Doc */
.highlight .s2 { color: #BA2121 } /* Literal.String.Double */
.highlight .se { color: #BB6622; font-weight: bold } /* Literal.String.Escape */
.highlight .sh { color: #BA2121 } /* Literal.String.Heredoc */
.highlight .si { color: #BB6688; font-weight: bold } /* Literal.String.Interpol */
.highlight .sx { color: #008000 } /* Literal.String.Other */
.highlight .sr { color: #BB6688 } /* Literal.String.Regex */
.highlight .s1 { color: #BA2121 } /* Literal.String.Single */
.highlight .ss { color: #19177C } /* Literal.String.Symbol */
.highlight .bp { color: #008000 } /* Name.Builtin.Pseudo */
.highlight .fm { color: #0000FF } /* Name.Function.Magic */
.highlight .vc { color: #19177C } /* Name.Variable.Class */
.highlight .vg { color: #19177C } /* Name.Variable.Global */
.highlight .vi { color: #19177C } /* Name.Variable.Instance */
.highlight .vm { color: #19177C } /* Name.Variable.Magic */
.highlight .il { color: #666666 } /* Literal.Number.Integer.Long */
    </style>
<style type="text/css">
    
/* Temporary definitions which will become obsolete with Notebook release 5.0 */
.ansi-black-fg { color: #3E424D; }
.ansi-black-bg { background-color: #3E424D; }
.ansi-black-intense-fg { color: #282C36; }
.ansi-black-intense-bg { background-color: #282C36; }
.ansi-red-fg { color: #E75C58; }
.ansi-red-bg { background-color: #E75C58; }
.ansi-red-intense-fg { color: #B22B31; }
.ansi-red-intense-bg { background-color: #B22B31; }
.ansi-green-fg { color: #00A250; }
.ansi-green-bg { background-color: #00A250; }
.ansi-green-intense-fg { color: #007427; }
.ansi-green-intense-bg { background-color: #007427; }
.ansi-yellow-fg { color: #DDB62B; }
.ansi-yellow-bg { background-color: #DDB62B; }
.ansi-yellow-intense-fg { color: #B27D12; }
.ansi-yellow-intense-bg { background-color: #B27D12; }
.ansi-blue-fg { color: #208FFB; }
.ansi-blue-bg { background-color: #208FFB; }
.ansi-blue-intense-fg { color: #0065CA; }
.ansi-blue-intense-bg { background-color: #0065CA; }
.ansi-magenta-fg { color: #D160C4; }
.ansi-magenta-bg { background-color: #D160C4; }
.ansi-magenta-intense-fg { color: #A03196; }
.ansi-magenta-intense-bg { background-color: #A03196; }
.ansi-cyan-fg { color: #60C6C8; }
.ansi-cyan-bg { background-color: #60C6C8; }
.ansi-cyan-intense-fg { color: #258F8F; }
.ansi-cyan-intense-bg { background-color: #258F8F; }
.ansi-white-fg { color: #C5C1B4; }
.ansi-white-bg { background-color: #C5C1B4; }
.ansi-white-intense-fg { color: #A1A6B2; }
.ansi-white-intense-bg { background-color: #A1A6B2; }

.ansi-bold { font-weight: bold; }

    </style>


<style type="text/css">
/* Overrides of notebook CSS for static HTML export */
body {
  overflow: visible;
  padding: 8px;
}

div#notebook {
  overflow: visible;
  border-top: none;
}@media print {
  div.cell {
    display: block;
    page-break-inside: avoid;
  } 
  div.output_wrapper { 
    display: block;
    page-break-inside: avoid; 
  }
  div.output { 
    display: block;
    page-break-inside: avoid; 
  }
}
</style>

<!-- Custom stylesheet, it must be in the same directory as the html file -->
<link rel="stylesheet" href="custom.css">

<!-- Loading mathjax macro -->
<!-- Load mathjax -->
    <script src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-AMS_HTML"></script>
    <!-- MathJax configuration -->
    <script type="text/x-mathjax-config">
    MathJax.Hub.Config({
        tex2jax: {
            inlineMath: [ ['$','$'], ["\\(","\\)"] ],
            displayMath: [ ['$$','$$'], ["\\[","\\]"] ],
            processEscapes: true,
            processEnvironments: true
        },
        // Center justify equations in code and markdown cells. Elsewhere
        // we use CSS to left justify single line equations in code cells.
        displayAlign: 'center',
        "HTML-CSS": {
            styles: {'.MathJax_Display': {"margin": 0}},
            linebreaks: { automatic: true }
        }
    });
    </script>
    <!-- End of mathjax configuration --></head>
<body>
  <div tabindex="-1" id="notebook" class="border-box-sizing">
    <div class="container" id="notebook-container">

<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p><h1 align='center' style="margin-bottom: 0px"> An end to end implementation of a Machine Learning pipeline </h1></p>
<h4 align='center' style="margin-top: 0px"> SPANDAN MADAN</h4>
<h4 align='center' style="margin-top: 0px"> Visual Computing Group, Harvard University</h4>
<h4 align='center' style="margin-top: 0px"> Computer Science and Artificial Intelligence Laboratory, MIT</h4>
</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h2 align='center' style="margin-top: 0px"><a href='https://github.com/Spandan-Madan/DeepLearningProject'>Link to Github Repo</a></h2>
</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h1 id="Section-1.-Introduction">Section 1. Introduction<a class="anchor-link" href="#Section-1.-Introduction">&#182;</a></h1><h3 id="Background">Background<a class="anchor-link" href="#Background">&#182;</a></h3><p>In the fall of 2016, I was a Teaching Fellow (Harvard's version of TA) for the graduate class on "Advanced Topics in Data Science (CS209/109)" at Harvard University. I was in-charge of designing the class project given to the students, and this tutorial has been built on top of the project I designed for the class.</p>
<h3 id="Why-write-yet-another-Tutorial-on-Machine-Learning-and-Deep-Learning?">Why write yet another Tutorial on Machine Learning and Deep Learning?<a class="anchor-link" href="#Why-write-yet-another-Tutorial-on-Machine-Learning-and-Deep-Learning?">&#182;</a></h3><p>As a researcher on Computer Vision, I come across new blogs and tutorials on ML (Machine Learning) every day. However, most of them are just focussing on introducing the syntax and the terminology relevant to the field. For example - a 15 minute tutorial on Tensorflow using MNIST dataset, or a 10 minute intro to Deep Learning in Keras on Imagenet.</p>
<p>While people are able to copy paste and run the code in these tutorials and feel that working in ML is really not that hard, it doesn't help them at all in using ML for their own purposes. For example, they never introduce you to how you can run the same algorithm on your own dataset. Or, how do you get the dataset if you want to solve a problem. Or, which algorithms do you use - Conventional ML, or Deep Learning? How do you evaluate your models performance? How do you write your own model, as opposed to choosing a ready made architecture? All these form fundamental steps in any Machine Learning pipeline, and it is these steps that take most of our time as ML practitioners.</p>
<p>This tutorial breaks down the whole pipeline, and leads the reader through it step by step in an hope to empower you to actually use ML, and not just feel that it was not too hard. Needless to say, this will take much longer than 15-30 minutes. I believe a weekend would be a good enough estimate.</p>
<h3 id="About-the-Author">About the Author<a class="anchor-link" href="#About-the-Author">&#182;</a></h3><p>I am <a href="http://spandanmadan.com/">Spandan Madan</a>, a graduate student at Harvard University working on Computer Vision. My research work is supervised collaboratively by Professor Hanspeter Pfister at Harvard, and Professor Aude Oliva at MIT. My current research focusses on using Computer Vision and Natural Language Techniques in tandem to build systems capable of reasoning using text and visual elements simultaneusly.</p>

</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h1 id="Section-2.-Project-Outline-:-Multi-Modal-Genre-Classification-for-Movies">Section 2. Project Outline : Multi-Modal Genre Classification for Movies<a class="anchor-link" href="#Section-2.-Project-Outline-:-Multi-Modal-Genre-Classification-for-Movies">&#182;</a></h1>
</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h2 id="Wow,-that-title-sounds-like-a-handful,-right?-Let's-break-it-down-step-by-step.">Wow, that title sounds like a handful, right? Let's break it down step by step.<a class="anchor-link" href="#Wow,-that-title-sounds-like-a-handful,-right?-Let's-break-it-down-step-by-step.">&#182;</a></h2><h3 id="Q.1.-what-do-we-mean-by-Classification?">Q.1. what do we mean by Classification?<a class="anchor-link" href="#Q.1.-what-do-we-mean-by-Classification?">&#182;</a></h3><p>In machine learning, the task of classification means to use the available data to learn a <i>function</i> which can assign a category to a data point. For example, assign a genre to a movie, like "Romantic Comedy", "Action", "Thriller". Another example could be automatically assigning a category to news articles, like "Sports" and "Politics".</p>
<h3 id="More-Formally">More Formally<a class="anchor-link" href="#More-Formally">&#182;</a></h3><h4 id="Given:">Given:<a class="anchor-link" href="#Given:">&#182;</a></h4><ul>
<li>A data point $x_i$ </li>
<li>A set of categories $y_1,y_2...y_n$ that $x_i$ can belong to. <br></li>
</ul>
<h4 id="Task-:">Task :<a class="anchor-link" href="#Task-:">&#182;</a></h4><p>Predict the correct category $y_k$ for a new data point $x_k$ not present in the given dataset.</p>
<h4 id="Problem-:">Problem :<a class="anchor-link" href="#Problem-:">&#182;</a></h4><p>We don't know how the $x$ and $y$ are related mathematically.</p>
<h4 id="Assumption-:">Assumption :<a class="anchor-link" href="#Assumption-:">&#182;</a></h4><p>We assume there exists a function $f$ relating $x$ and $y$ i.e. $f(x_i)=y_i$</p>
<h4 id="Approach-:">Approach :<a class="anchor-link" href="#Approach-:">&#182;</a></h4><p>Since $f$ is not known, we learn a function $g$, which approximates $f$.</p>
<h4 id="Important-consideration-:">Important consideration :<a class="anchor-link" href="#Important-consideration-:">&#182;</a></h4><ul>
<li><p>If $f(x_i)=g(x_i)=y_i$ for all $x_i$, then the two functions $f$ and $g$ are exactly equal. Needless to say, this won't realistically ever happen, and we'll only be able to approximate the true function $f$ using $g$. This means, sometimes the prediction $g(x_i)$ will not be correct. And essentially, our whole goal is to find a $g$ which makes a really low number of such errors. That's basically all that we're trying to do.</p>
</li>
<li><p>For the sake of completeness, I should mention that this is a specific kind of learning problem which we call "Supervised Learning". Also, the idea that $g$ approximates $f$ well for data not present in our dataset is called "Generalization". It is absolutely paramount that our model generalizes, or else all our claims will only be true about data we already have and our predictions will not be correct.</p>
</li>
<li><p>We will look into generalization a little bit more a little ahead in the tutorial.</p>
</li>
<li><p>Finally, There are several other kinds, but supervised learning is the most popular and well studied kind.</p>
</li>
</ul>

</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h3 id="Q.2.-What's-Multi-Modal-Classification-then?">Q.2. What's Multi-Modal Classification then?<a class="anchor-link" href="#Q.2.-What's-Multi-Modal-Classification-then?">&#182;</a></h3><p>In the machine learning community, the term Multi-Modal is used to refer to multiple <i>kinds</i> of data. For example, consider a YouTube video. It can be thought to contain 3 different modalities -</p>
<ul>
<li>The video frames (visual modality)</li>
<li>The audio clip of what's being spoken (audio modality)</li>
<li>Some videos also come with the transcription of the words spoken in the form of subtitles (textual modality)</li>
</ul>
<p>Consider, that I'm interested in classifying a song on YouTube as pop or rock. You can use any of the above 3 modalities to predict the genre - The video, the song itself, or the lyrics. But, needless to say, you can predict it much better if you could use all three simultaneously. This is what we mean by multi-modal classification.</p>

</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h1 id="For-this-project,-we-will-be-using-visual-and-textual-data-to-classify-movie-genres.">For this project, we will be using visual and textual data to classify movie genres.<a class="anchor-link" href="#For-this-project,-we-will-be-using-visual-and-textual-data-to-classify-movie-genres.">&#182;</a></h1>
</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h1 id="Project-Outline">Project Outline<a class="anchor-link" href="#Project-Outline">&#182;</a></h1><ul>
<li><strong>Scraping a dataset</strong> : The first step is to build a rich data set. We will collect textual and visual data for each movie.</li>
<li><strong>Data pre-processing</strong></li>
<li><strong>Non-deep Machine Learning models : Probabilistic and Max-Margin Classifiers.</strong></li>
<li><strong>Intuitive theory behind Deep Learning</strong></li>
<li><strong>Deep Models for Visual Data</strong></li>
<li><strong>Deep Models for Text</strong></li>
<li><strong>Potential Extensions</strong></li>
<li><strong>Food for Thought</strong></li>
</ul>

</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h1 id="Section-3.-Building-your-very-own-DataSet.">Section 3. Building your very own DataSet.<a class="anchor-link" href="#Section-3.-Building-your-very-own-DataSet.">&#182;</a></h1>
</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>For any machine learning algorithm to work, it is imperative that we collect data which is "representative". Now, let's take a moment to discuss what the word representative mean.</p>
<h3 id="What-data-is-good-data?-OR-What-do-you-mean-by-data-being-&quot;representative&quot;?">What data is good data? OR What do you mean by data being "representative"?<a class="anchor-link" href="#What-data-is-good-data?-OR-What-do-you-mean-by-data-being-&quot;representative&quot;?">&#182;</a></h3><p>Let's look at this from first principles. Mathematically, the premise of machine learning (to be precise, the strand of machine learning we'll be working with here) is that given input variable X, and an output variable y, <strong>IF</strong> there is a function such that g(X)=y, then if g is unknown, we can "learn" a function f which approximates g. At the very heart, its not at all different from what you may have earlier studied as "curve fitting". For example, if you're trying to predict someone's movie preferences then X can be information about the person's gender, age, nationality and so on, while y can be the genre they most like to listen to!</p>
<p>Let's do a thought experiment. Consider the same example - I'm trying to predict people's movie preferences. I walk into a classroom today, and collect information about some students and their movie preferences. Now, I use that data to build a model. How well do you think I can predict my father's movie preferences? The answer is - probably not very well. Why? Intuitively, there was probably no one in the classroom who was my father's age. My model can tell me that as people go from age 18 to 30, they have a higher preference for documentaries over superhero movies. But does this trend continue at 55? Probably, they may start liking family dramas more. Probably they don't. In a nutshell, we cannot say with certainty, as our data tells us nothing about it. So, if the task was to make predictions about ANYONE's movie preferences, then the data collected from just undergraduates is NOT representative.</p>
<p>Now, let's see why this makes sense Mathematically. Look at the graph below.</p>

</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p><img src="contour.png"></p>
<center>Fig.1: Plot of a function we are trying to approximate(<a href="http://www.jzy3d.org/js/slider/images/ContourPlotsDemo.png">source</a>)</center>
</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>If we consider that the variable plotted on the vertical axis is $y$, and the values of the 2 variables on the horizontal axes make the input vector $X$, then, our hope is that we are able to find a function $f$ which can approximate the function plotted here. If all the data I collect is such that $x_1$ belongs to (80,100) and $x_2$ belongs to (80,100), the learned function will only be able to learn the "yellow-green dipping bellow" part of the function.  Our function will never be able to predict the behavior in the "red" regions of the true function. So, in order to be able to learn a good function, we need data sampled from a diverse set of values of $x_1$ and x2. That would be representative data to learn this contour.</p>

</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>Therefore, we want to collect data which is representative of all possible movies that we want to make predictions about. Or else (which is often the case), we need to be aware of the limitations of the model we have trained, and the predictions we can make with confidence. The easiest way to do this is to only make predictions about the domain of data we collected the training data from. For example, in our case, let us start by assuming that our model will predict genres for only English movies. Now, the task is to collect data about a diverse collection of movies.</p>
<p>So how do we get this data then? Neither google, nor any university has released such a dataset. We want to collect visual and textual data about these movies. The simple answer is to scrape it from the internet to build our own dataset. For the purpose of this project, we will use movie posters as our visual data, and movie plots as textual data. Using these, we will build a model that can predict movie genres!</p>

</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h1 id="We-will-be-scraping-data-from-2-different-movie-sources---IMDB-and-TMDB">We will be scraping data from 2 different movie sources - IMDB and TMDB<a class="anchor-link" href="#We-will-be-scraping-data-from-2-different-movie-sources---IMDB-and-TMDB">&#182;</a></h1>
</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h3>IMDB:http://www.imdb.com/</h3><p>For those unaware, IMDB is the primary source of information about movies on the internet. It is immensely rich with posters, reviews, synopsis, ratings and many other information on every movie. We will use this as our primary data source.</p>
<h3>TMDB:https://www.themoviedb.org/</h3><p>TMDB, or The Movie DataBase, is an open source version of IMDB, with a free to use API that can be used to collect information. You do need an API key, but it can be obtained for free by just making a request after making a free account.</p>

</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h4 id="Note--">Note -<a class="anchor-link" href="#Note--">&#182;</a></h4><p>IMDB gives some information for free through the API, but doesn't release other information about movies. Here, we will keep it legal and only use information given to us for free and legally. However, scraping does reside on the moral fence, so to say. People often scrape data which isn't exactly publicly available for use from websites.</p>

</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[4]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="kn">import</span> <span class="nn">warnings</span>
<span class="n">warnings</span><span class="o">.</span><span class="n">filterwarnings</span><span class="p">(</span><span class="s1">&#39;ignore&#39;</span><span class="p">)</span>
<span class="kn">import</span> <span class="nn">torchvision</span>
<span class="kn">import</span> <span class="nn">urllib2</span>
<span class="kn">import</span> <span class="nn">requests</span>
<span class="kn">import</span> <span class="nn">json</span>
<span class="kn">import</span> <span class="nn">imdb</span>
<span class="kn">import</span> <span class="nn">time</span>
<span class="kn">import</span> <span class="nn">itertools</span>
<span class="kn">import</span> <span class="nn">wget</span>
<span class="kn">import</span> <span class="nn">os</span>
<span class="kn">import</span> <span class="nn">tmdbsimple</span> <span class="kn">as</span> <span class="nn">tmdb</span>
<span class="kn">import</span> <span class="nn">numpy</span> <span class="kn">as</span> <span class="nn">np</span>
<span class="kn">import</span> <span class="nn">random</span>
<span class="kn">import</span> <span class="nn">matplotlib</span>
<span class="kn">import</span> <span class="nn">matplotlib.pyplot</span> <span class="kn">as</span> <span class="nn">plt</span>
<span class="o">%</span><span class="k">matplotlib</span> inline
<span class="kn">import</span> <span class="nn">seaborn</span> <span class="kn">as</span> <span class="nn">sns</span>
<span class="kn">import</span> <span class="nn">pickle</span>
<span class="kn">import</span> <span class="nn">sys</span>
</pre></div>

    </div>
</div>
</div>

</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h1 id="Here-is-a-broad-outline-of-technical-steps-to-be-done-for-data-collection">Here is a broad outline of technical steps to be done for data collection<a class="anchor-link" href="#Here-is-a-broad-outline-of-technical-steps-to-be-done-for-data-collection">&#182;</a></h1><ul>
<li>Sign up for TMDB (themoviedatabase.org), and set up API to scrape movie posters for above movies.</li>
<li>Set up and work with TMDb to get movie information from their database</li>
<li>Do the same for IMDb</li>
<li>Compare the entries of IMDb and TMDb for a movie</li>
<li>Get a listing and information of a few movies</li>
<li>Think and ponder over the potential challenges that may come our way, and think about interesting questions we can answer given the API's we have in our hands.</li>
<li>Get data from the TMDb</li>
</ul>
<p>Let's go over each one of these one by one.</p>

</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h2 id="Signing-up-for-TMDB-and-getting-set-up-for-getting-movie-metadata.">Signing up for TMDB and getting set up for getting movie metadata.<a class="anchor-link" href="#Signing-up-for-TMDB-and-getting-set-up-for-getting-movie-metadata.">&#182;</a></h2><ul>
<li>Step 1. Head over to [tmdb.org] (<a href="https://www.themoviedb.org/?language=en">https://www.themoviedb.org/?language=en</a>) and create a new account there by signing up.</li>
<li>Step 2. Click on your account icon on the top right, then from drop down menu select "Settings".</li>
<li>Step 3. On the settings page, you will see the option "API" on the left pane. Click on that.</li>
<li>Step 4. Apply for a new developer key. Fill out the form as required. The fields "Application Name" and "Application URL" are not important. Fill anything there.</li>
<li>Step 5. It should generate a new API key for you and you should also receive a mail.</li>
</ul>
<p>Now that you have the API key for TMDB, you can query using TMDB. Remember, it allows only 40 queries per 10 seconds.</p>
<p>An easy way to respect this is to just have a call to <i>time.sleep(1)</i> after each iteration. This is also being very nice to the server.</p>
<p>If you want to try and maximize your throughput you can embed every TMDB request in a nested try except block. If the first try fails, the second try first uses python's sleep function to give it a little rest, and then try again to make a request. Something like this -</p>

<pre><code>try:
    search.movie(query=movie) #An API request
except:
    try:
        time.sleep(10) #sleep for a bit, to give API requests a rest.
        search.movie(query=&lt;i&gt;movie_name&lt;/i&gt;) #Make second API request
    except:
        print "Failed second attempt too, check if there's any error in request"</code></pre>

</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h2 id="Using-TMDB-using-the-obtained-API-Key-to-get-movie-information">Using TMDB using the obtained API Key to get movie information<a class="anchor-link" href="#Using-TMDB-using-the-obtained-API-Key-to-get-movie-information">&#182;</a></h2>
</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>I have made these functions which make things easy. Basically, I'm making use of a library called tmdbsimple which makes TMDB using even easier. This library was installed at the time of setup.</p>
<p>However, if you  want to avoid the library, it is also easy enough to load the API output directly into a dictionary like this without using tmdbsimple:</p>

<pre><code>url = 'https://api.themoviedb.org/3/movie/1581?api_key=' + api_key
data = urllib2.urlopen(url).read()

# create dictionary from JSON 
dataDict = json.loads(data)</code></pre>

</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[5]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="kn">import</span> <span class="nn">os</span>
<span class="c1"># set here the path where you want the scraped folders to be saved!</span>
<span class="n">poster_folder</span><span class="o">=</span><span class="s1">&#39;posters_final/&#39;</span>
<span class="k">if</span> <span class="n">poster_folder</span><span class="o">.</span><span class="n">split</span><span class="p">(</span><span class="s1">&#39;/&#39;</span><span class="p">)[</span><span class="mi">0</span><span class="p">]</span> <span class="ow">in</span> <span class="n">os</span><span class="o">.</span><span class="n">listdir</span><span class="p">(</span><span class="s1">&#39;./&#39;</span><span class="p">):</span>
    <span class="k">print</span><span class="p">(</span><span class="s1">&#39;Folder already exists&#39;</span><span class="p">)</span>
<span class="k">else</span><span class="p">:</span>
    <span class="n">os</span><span class="o">.</span><span class="n">mkdir</span><span class="p">(</span><span class="s1">&#39;./&#39;</span><span class="o">+</span><span class="n">poster_folder</span><span class="p">)</span>
</pre></div>

    </div>
</div>
</div>

<div class="output_wrapper">
<div class="output">


<div class="output_area">

    <div class="prompt"></div>


<div class="output_subarea output_stream output_stdout output_text">
<pre>Folder already exists
</pre>
</div>
</div>

</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[6]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="n">poster_folder</span>
</pre></div>

    </div>
</div>
</div>

<div class="output_wrapper">
<div class="output">


<div class="output_area">

    <div class="prompt output_prompt">Out[6]:</div>


<div class="output_text output_subarea output_execute_result">
<pre>&#39;posters_final/&#39;</pre>
</div>

</div>

</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[128]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="c1"># For the purpose of this example, i will be working with the 1999 Sci-Fi movie - &quot;The Matrix&quot;!</span>
<span class="kn">import</span> <span class="nn">tmdbsimple</span> <span class="kn">as</span> <span class="nn">tmdb</span>
<span class="n">api_key</span> <span class="o">=</span> <span class="s1">&#39;&#39;</span> <span class="c1">#Enter your own API key here to run the code below. </span>
<span class="c1"># Generate your own API key as explained above :)</span>


<span class="n">tmdb</span><span class="o">.</span><span class="n">API_KEY</span> <span class="o">=</span> <span class="n">api_key</span> <span class="c1">#This sets the API key setting for the tmdb object</span>
<span class="n">search</span> <span class="o">=</span> <span class="n">tmdb</span><span class="o">.</span><span class="n">Search</span><span class="p">()</span> <span class="c1">#this instantiates a tmdb &quot;search&quot; object which allows your to search for the movie</span>
<span class="kn">import</span> <span class="nn">os.path</span>
<span class="c1"># These functions take in a string movie name i.e. like &quot;The Matrix&quot; or &quot;Interstellar&quot;</span>
<span class="c1"># What they return is pretty much clear in the name - Poster, ID , Info or genre of the Movie!</span>
<span class="k">def</span> <span class="nf">grab_poster_tmdb</span><span class="p">(</span><span class="n">movie</span><span class="p">):</span>
    <span class="n">response</span> <span class="o">=</span> <span class="n">search</span><span class="o">.</span><span class="n">movie</span><span class="p">(</span><span class="n">query</span><span class="o">=</span><span class="n">movie</span><span class="p">)</span>
    <span class="nb">id</span><span class="o">=</span><span class="n">response</span><span class="p">[</span><span class="s1">&#39;results&#39;</span><span class="p">][</span><span class="mi">0</span><span class="p">][</span><span class="s1">&#39;id&#39;</span><span class="p">]</span>
    <span class="n">movie</span> <span class="o">=</span> <span class="n">tmdb</span><span class="o">.</span><span class="n">Movies</span><span class="p">(</span><span class="nb">id</span><span class="p">)</span>
    <span class="n">posterp</span><span class="o">=</span><span class="n">movie</span><span class="o">.</span><span class="n">info</span><span class="p">()[</span><span class="s1">&#39;poster_path&#39;</span><span class="p">]</span>
    <span class="n">title</span><span class="o">=</span><span class="n">movie</span><span class="o">.</span><span class="n">info</span><span class="p">()[</span><span class="s1">&#39;original_title&#39;</span><span class="p">]</span>
    <span class="k">if</span> <span class="n">os</span><span class="o">.</span><span class="n">path</span><span class="o">.</span><span class="n">isfile</span><span class="p">(</span><span class="n">poster_folder</span><span class="o">+</span><span class="n">title</span><span class="o">+</span><span class="s1">&#39;.jpg &#39;</span><span class="p">):</span>
        <span class="k">return</span>
    <span class="n">url</span><span class="o">=</span><span class="s1">&#39;image.tmdb.org/t/p/original&#39;</span><span class="o">+</span><span class="n">posterp</span>
    <span class="n">title</span><span class="o">=</span><span class="s1">&#39;_&#39;</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">title</span><span class="o">.</span><span class="n">split</span><span class="p">(</span><span class="s1">&#39; &#39;</span><span class="p">))</span>
    <span class="n">strcmd</span><span class="o">=</span><span class="s1">&#39;wget -O &#39;</span><span class="o">+</span><span class="n">poster_folder</span><span class="o">+</span><span class="n">title</span><span class="o">+</span><span class="s1">&#39;.jpg &#39;</span><span class="o">+</span><span class="n">url</span>
    <span class="n">os</span><span class="o">.</span><span class="n">system</span><span class="p">(</span><span class="n">strcmd</span><span class="p">)</span>

<span class="k">def</span> <span class="nf">get_movie_id_tmdb</span><span class="p">(</span><span class="n">movie</span><span class="p">):</span>
    <span class="n">response</span> <span class="o">=</span> <span class="n">search</span><span class="o">.</span><span class="n">movie</span><span class="p">(</span><span class="n">query</span><span class="o">=</span><span class="n">movie</span><span class="p">)</span>
    <span class="n">movie_id</span><span class="o">=</span><span class="n">response</span><span class="p">[</span><span class="s1">&#39;results&#39;</span><span class="p">][</span><span class="mi">0</span><span class="p">][</span><span class="s1">&#39;id&#39;</span><span class="p">]</span>
    <span class="k">return</span> <span class="n">movie_id</span>

<span class="k">def</span> <span class="nf">get_movie_info_tmdb</span><span class="p">(</span><span class="n">movie</span><span class="p">):</span>
    <span class="n">response</span> <span class="o">=</span> <span class="n">search</span><span class="o">.</span><span class="n">movie</span><span class="p">(</span><span class="n">query</span><span class="o">=</span><span class="n">movie</span><span class="p">)</span>
    <span class="nb">id</span><span class="o">=</span><span class="n">response</span><span class="p">[</span><span class="s1">&#39;results&#39;</span><span class="p">][</span><span class="mi">0</span><span class="p">][</span><span class="s1">&#39;id&#39;</span><span class="p">]</span>
    <span class="n">movie</span> <span class="o">=</span> <span class="n">tmdb</span><span class="o">.</span><span class="n">Movies</span><span class="p">(</span><span class="nb">id</span><span class="p">)</span>
    <span class="n">info</span><span class="o">=</span><span class="n">movie</span><span class="o">.</span><span class="n">info</span><span class="p">()</span>
    <span class="k">return</span> <span class="n">info</span>

<span class="k">def</span> <span class="nf">get_movie_genres_tmdb</span><span class="p">(</span><span class="n">movie</span><span class="p">):</span>
    <span class="n">response</span> <span class="o">=</span> <span class="n">search</span><span class="o">.</span><span class="n">movie</span><span class="p">(</span><span class="n">query</span><span class="o">=</span><span class="n">movie</span><span class="p">)</span>
    <span class="nb">id</span><span class="o">=</span><span class="n">response</span><span class="p">[</span><span class="s1">&#39;results&#39;</span><span class="p">][</span><span class="mi">0</span><span class="p">][</span><span class="s1">&#39;id&#39;</span><span class="p">]</span>
    <span class="n">movie</span> <span class="o">=</span> <span class="n">tmdb</span><span class="o">.</span><span class="n">Movies</span><span class="p">(</span><span class="nb">id</span><span class="p">)</span>
    <span class="n">genres</span><span class="o">=</span><span class="n">movie</span><span class="o">.</span><span class="n">info</span><span class="p">()[</span><span class="s1">&#39;genres&#39;</span><span class="p">]</span>
    <span class="k">return</span> <span class="n">genres</span>
</pre></div>

    </div>
</div>
</div>

</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>While the above functions have been made to make it easy to get genres, posters and ID, all the information that can be accessed can be seen by calling the function get_movie_info() as shown below</p>

</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[8]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="k">print</span> <span class="n">get_movie_genres_tmdb</span><span class="p">(</span><span class="s2">&quot;The Matrix&quot;</span><span class="p">)</span>
</pre></div>

    </div>
</div>
</div>

<div class="output_wrapper">
<div class="output">


<div class="output_area">

    <div class="prompt"></div>


<div class="output_subarea output_stream output_stdout output_text">
<pre>[{u&#39;id&#39;: 28, u&#39;name&#39;: u&#39;Action&#39;}, {u&#39;id&#39;: 878, u&#39;name&#39;: u&#39;Science Fiction&#39;}]
</pre>
</div>
</div>

</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[9]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="n">info</span><span class="o">=</span><span class="n">get_movie_info_tmdb</span><span class="p">(</span><span class="s2">&quot;The Matrix&quot;</span><span class="p">)</span>
<span class="k">print</span> <span class="s2">&quot;All the Movie information from TMDB gets stored in a dictionary with the following keys for easy access -&quot;</span>
<span class="n">info</span><span class="o">.</span><span class="n">keys</span><span class="p">()</span>
</pre></div>

    </div>
</div>
</div>

<div class="output_wrapper">
<div class="output">


<div class="output_area">

    <div class="prompt"></div>


<div class="output_subarea output_stream output_stdout output_text">
<pre>All the Movie information from TMDB gets stored in a dictionary with the following keys for easy access -
</pre>
</div>
</div>

<div class="output_area">

    <div class="prompt output_prompt">Out[9]:</div>


<div class="output_text output_subarea output_execute_result">
<pre>[u&#39;poster_path&#39;,
 u&#39;production_countries&#39;,
 u&#39;revenue&#39;,
 u&#39;overview&#39;,
 u&#39;video&#39;,
 u&#39;id&#39;,
 u&#39;genres&#39;,
 u&#39;title&#39;,
 u&#39;tagline&#39;,
 u&#39;vote_count&#39;,
 u&#39;homepage&#39;,
 u&#39;belongs_to_collection&#39;,
 u&#39;original_language&#39;,
 u&#39;status&#39;,
 u&#39;spoken_languages&#39;,
 u&#39;imdb_id&#39;,
 u&#39;adult&#39;,
 u&#39;backdrop_path&#39;,
 u&#39;production_companies&#39;,
 u&#39;release_date&#39;,
 u&#39;popularity&#39;,
 u&#39;original_title&#39;,
 u&#39;budget&#39;,
 u&#39;vote_average&#39;,
 u&#39;runtime&#39;]</pre>
</div>

</div>

</div>
</div>

</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>So, to get the tagline of the movie we can use the above dictionary key -</p>

</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[10]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="n">info</span><span class="o">=</span><span class="n">get_movie_info_tmdb</span><span class="p">(</span><span class="s2">&quot;The Matrix&quot;</span><span class="p">)</span>
<span class="k">print</span> <span class="n">info</span><span class="p">[</span><span class="s1">&#39;tagline&#39;</span><span class="p">]</span>
</pre></div>

    </div>
</div>
</div>

<div class="output_wrapper">
<div class="output">


<div class="output_area">

    <div class="prompt"></div>


<div class="output_subarea output_stream output_stdout output_text">
<pre>Welcome to the Real World.
</pre>
</div>
</div>

</div>
</div>

</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h2 id="Getting-movie-information-from-IMDB">Getting movie information from IMDB<a class="anchor-link" href="#Getting-movie-information-from-IMDB">&#182;</a></h2>
</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>Now that we know how to get information from TMDB, here's how we can get information about the same movie from IMDB. This makes it possible for us to combine more information, and get a richer dataset. I urge you to try and see what dataset you can make, and go above and beyond the basic things I've done in this tutorial. Due to the differences between the two datasets, you will have to do some cleaning, however both of these datasets are extremely clean and it will be minimal.</p>

</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[11]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="kn">import</span> <span class="nn">imdb</span>
<span class="c1"># Create the IMDB object that will be used to access the IMDb&#39;s database.</span>
<span class="n">imbd_object</span> <span class="o">=</span> <span class="n">imdb</span><span class="o">.</span><span class="n">IMDb</span><span class="p">()</span> <span class="c1"># by default access the web.</span>

<span class="c1"># Search for a movie (get a list of Movie objects).</span>
<span class="n">results</span> <span class="o">=</span> <span class="n">imbd_object</span><span class="o">.</span><span class="n">search_movie</span><span class="p">(</span><span class="s1">&#39;The Matrix&#39;</span><span class="p">)</span>
<span class="c1">#print(len(results))</span>
<span class="c1"># As this returns a list of all movies containing the word &quot;The Matrix&quot;, we pick the first element</span>
<span class="n">movie</span> <span class="o">=</span> <span class="n">results</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span>

<span class="n">imbd_object</span><span class="o">.</span><span class="n">update</span><span class="p">(</span><span class="n">movie</span><span class="p">)</span>

<span class="k">print</span> <span class="s2">&quot;All the information we can get about this movie from IMDB-&quot;</span>
<span class="n">movie</span><span class="o">.</span><span class="n">keys</span><span class="p">()</span>
</pre></div>

    </div>
</div>
</div>

<div class="output_wrapper">
<div class="output">


<div class="output_area">

    <div class="prompt"></div>


<div class="output_subarea output_stream output_stdout output_text">
<pre>All the information we can get about this movie from IMDB-
</pre>
</div>
</div>

<div class="output_area">

    <div class="prompt output_prompt">Out[11]:</div>


<div class="output_text output_subarea output_execute_result">
<pre>[u&#39;production managers&#39;,
 u&#39;rating&#39;,
 u&#39;casting directors&#39;,
 u&#39;distributors&#39;,
 u&#39;music department&#39;,
 u&#39;runtimes&#39;,
 u&#39;special effects&#39;,
 u&#39;year&#39;,
 u&#39;color info&#39;,
 u&#39;composers&#39;,
 u&#39;costume designers&#39;,
 u&#39;votes&#39;,
 u&#39;top 250 rank&#39;,
 u&#39;title&#39;,
 u&#39;writer&#39;,
 u&#39;editors&#39;,
 u&#39;visual effects&#39;,
 u&#39;cinematographers&#39;,
 u&#39;writers&#39;,
 u&#39;camera department&#39;,
 u&#39;directors&#39;,
 u&#39;certificates&#39;,
 u&#39;countries&#39;,
 u&#39;country codes&#39;,
 u&#39;language codes&#39;,
 u&#39;production designers&#39;,
 u&#39;director&#39;,
 u&#39;casting department&#39;,
 u&#39;special effects companies&#39;,
 u&#39;assistant directors&#39;,
 u&#39;sound mix&#39;,
 u&#39;location management&#39;,
 u&#39;genres&#39;,
 u&#39;miscellaneous&#39;,
 u&#39;production companies&#39;,
 u&#39;producers&#39;,
 u&#39;set decorators&#39;,
 u&#39;original air date&#39;,
 u&#39;costume departmen&#39;,
 u&#39;akas&#39;,
 u&#39;aspect ratio&#39;,
 u&#39;sound department&#39;,
 u&#39;stunts&#39;,
 u&#39;kind&#39;,
 u&#39;make up department&#39;,
 u&#39;other companies&#39;,
 u&#39;art department&#39;,
 u&#39;languages&#39;,
 u&#39;transportation department&#39;,
 u&#39;plot outline&#39;,
 u&#39;plot&#39;,
 u&#39;cast&#39;,
 u&#39;synopsis&#39;,
 u&#39;art directors&#39;,
 u&#39;animation department&#39;,
 u&#39;cover url&#39;,
 u&#39;editorial department&#39;,
 u&#39;canonical title&#39;,
 u&#39;long imdb title&#39;,
 u&#39;long imdb canonical title&#39;,
 u&#39;smart canonical title&#39;,
 u&#39;smart long imdb canonical title&#39;,
 u&#39;full-size cover url&#39;]</pre>
</div>

</div>

</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[12]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="k">print</span> <span class="s2">&quot;The genres associated with the movie are - &quot;</span><span class="p">,</span><span class="n">movie</span><span class="p">[</span><span class="s1">&#39;genres&#39;</span><span class="p">]</span>
</pre></div>

    </div>
</div>
</div>

<div class="output_wrapper">
<div class="output">


<div class="output_area">

    <div class="prompt"></div>


<div class="output_subarea output_stream output_stdout output_text">
<pre>The genres associated with the movie are -  [u&#39;Action&#39;, u&#39;Sci-Fi&#39;]
</pre>
</div>
</div>

</div>
</div>

</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h2 id="A-small-comparison-of-IMDB-and-TMDB">A small comparison of IMDB and TMDB<a class="anchor-link" href="#A-small-comparison-of-IMDB-and-TMDB">&#182;</a></h2>
</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>Now that we have both systems running, let's do a very short comparison for the same movie?</p>

</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[13]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="k">print</span> <span class="s2">&quot;The genres for The Matrix pulled from IMDB are -&quot;</span><span class="p">,</span><span class="n">movie</span><span class="p">[</span><span class="s1">&#39;genres&#39;</span><span class="p">]</span>
<span class="k">print</span> <span class="s2">&quot;The genres for The Matrix pulled from TMDB are -&quot;</span><span class="p">,</span><span class="n">get_movie_genres_tmdb</span><span class="p">(</span><span class="s2">&quot;The Matrix&quot;</span><span class="p">)</span>
</pre></div>

    </div>
</div>
</div>

<div class="output_wrapper">
<div class="output">


<div class="output_area">

    <div class="prompt"></div>


<div class="output_subarea output_stream output_stdout output_text">
<pre>The genres for The Matrix pulled from IMDB are - [u&#39;Action&#39;, u&#39;Sci-Fi&#39;]
The genres for The Matrix pulled from TMDB are - [{u&#39;id&#39;: 28, u&#39;name&#39;: u&#39;Action&#39;}, {u&#39;id&#39;: 878, u&#39;name&#39;: u&#39;Science Fiction&#39;}]
</pre>
</div>
</div>

</div>
</div>

</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>As we can see, both the systems are correct, but the way they package information is different. TMDB calls it "Science Fiction" and has an ID for every genre. While IMDB calls it "Sci-Fi". Thus, it is important to keep track of these things when making use of both the datasets simultaneously.</p>

</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>Now that we know how to scrape information for one movie, let's take a bigger step towards scraping multiple movies?</p>

</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h2 id="Working-with-multiple-movies-:-Obtaining-Top-20-movies-from-TMDB">Working with multiple movies : Obtaining Top 20 movies from TMDB<a class="anchor-link" href="#Working-with-multiple-movies-:-Obtaining-Top-20-movies-from-TMDB">&#182;</a></h2>
</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>We first instantiate an object that inherits from class Movies from TMDB. Then We use the <strong>popular()</strong> class method (i.e. function) to get top movies. To get more than one page of results, the optional page argument lets us see movies from any specified page number.</p>

</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[14]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="n">all_movies</span><span class="o">=</span><span class="n">tmdb</span><span class="o">.</span><span class="n">Movies</span><span class="p">()</span>
<span class="n">top_movies</span><span class="o">=</span><span class="n">all_movies</span><span class="o">.</span><span class="n">popular</span><span class="p">()</span>

<span class="c1"># This is a dictionary, and to access results we use the key &#39;results&#39; which returns info on 20 movies</span>
<span class="k">print</span><span class="p">(</span><span class="nb">len</span><span class="p">(</span><span class="n">top_movies</span><span class="p">[</span><span class="s1">&#39;results&#39;</span><span class="p">]))</span>
<span class="n">top20_movs</span><span class="o">=</span><span class="n">top_movies</span><span class="p">[</span><span class="s1">&#39;results&#39;</span><span class="p">]</span>
</pre></div>

    </div>
</div>
</div>

<div class="output_wrapper">
<div class="output">


<div class="output_area">

    <div class="prompt"></div>


<div class="output_subarea output_stream output_stdout output_text">
<pre>20
</pre>
</div>
</div>

</div>
</div>

</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>Let's look at one of these movies. It's the same format as above, as we had information on the movie "The Matrix", as you can see below. It's a dictionary which can be queried  for specific information on that movie</p>

</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[15]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="n">first_movie</span><span class="o">=</span><span class="n">top20_movs</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span>
<span class="k">print</span> <span class="s2">&quot;Here is all the information you can get on this movie - &quot;</span>
<span class="k">print</span> <span class="n">first_movie</span>
<span class="k">print</span> <span class="s2">&quot;</span><span class="se">\n\n</span><span class="s2">The title of the first movie is - &quot;</span><span class="p">,</span> <span class="n">first_movie</span><span class="p">[</span><span class="s1">&#39;title&#39;</span><span class="p">]</span>
</pre></div>

    </div>
</div>
</div>

<div class="output_wrapper">
<div class="output">


<div class="output_area">

    <div class="prompt"></div>


<div class="output_subarea output_stream output_stdout output_text">
<pre>Here is all the information you can get on this movie - 
{u&#39;poster_path&#39;: u&#39;/2uNW4WbgBXL25BAbXGLnLqX71Sw.jpg&#39;, u&#39;title&#39;: u&#39;Venom&#39;, u&#39;overview&#39;: u&#39;When Eddie Brock acquires the powers of a symbiote, he will have to release his alter-ego &#34;Venom&#34; to save his life.&#39;, u&#39;release_date&#39;: u&#39;2018-10-03&#39;, u&#39;popularity&#39;: 371.566, u&#39;original_title&#39;: u&#39;Venom&#39;, u&#39;backdrop_path&#39;: u&#39;/VuukZLgaCrho2Ar8Scl9HtV3yD.jpg&#39;, u&#39;vote_count&#39;: 1576, u&#39;video&#39;: False, u&#39;adult&#39;: False, u&#39;vote_average&#39;: 6.6, u&#39;genre_ids&#39;: [878, 28, 80, 28, 27], u&#39;id&#39;: 335983, u&#39;original_language&#39;: u&#39;en&#39;}


The title of the first movie is -  Venom
</pre>
</div>
</div>

</div>
</div>

</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>Let's print out top 5 movie's titles!</p>

</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[16]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="nb">len</span><span class="p">(</span><span class="n">top20_movs</span><span class="p">)):</span>
    <span class="n">mov</span><span class="o">=</span><span class="n">top20_movs</span><span class="p">[</span><span class="n">i</span><span class="p">]</span>
    <span class="n">title</span><span class="o">=</span><span class="n">mov</span><span class="p">[</span><span class="s1">&#39;title&#39;</span><span class="p">]</span>
    <span class="k">print</span> <span class="n">title</span>
    <span class="k">if</span> <span class="n">i</span><span class="o">==</span><span class="mi">4</span><span class="p">:</span>
        <span class="k">break</span>
</pre></div>

    </div>
</div>
</div>

<div class="output_wrapper">
<div class="output">


<div class="output_area">

    <div class="prompt"></div>


<div class="output_subarea output_stream output_stdout output_text">
<pre>Venom
Halloween
A Star Is Born
The Seven Deadly Sins: Prisoners of the Sky
Incredibles 2
</pre>
</div>
</div>

</div>
</div>

</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h3 id="Yes,-I-know.-I'm-a-little-upset-too-seeing-The-Nun-above-Avengers:-Infinity-War!">Yes, I know. I'm a little upset too seeing The Nun above Avengers: Infinity War!<a class="anchor-link" href="#Yes,-I-know.-I'm-a-little-upset-too-seeing-The-Nun-above-Avengers:-Infinity-War!">&#182;</a></h3>
</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>Moving on, we can get their genres the same way.</p>

</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[110]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="nb">len</span><span class="p">(</span><span class="n">top20_movs</span><span class="p">)):</span>
    <span class="n">mov</span><span class="o">=</span><span class="n">top20_movs</span><span class="p">[</span><span class="n">i</span><span class="p">]</span>
    <span class="n">genres</span><span class="o">=</span><span class="n">mov</span><span class="p">[</span><span class="s1">&#39;genre_ids&#39;</span><span class="p">]</span>
    <span class="k">print</span> <span class="n">genres</span>
    <span class="k">if</span> <span class="n">i</span><span class="o">==</span><span class="mi">4</span><span class="p">:</span>
        <span class="k">break</span>
</pre></div>

    </div>
</div>
</div>

<div class="output_wrapper">
<div class="output">


<div class="output_area">

    <div class="prompt"></div>


<div class="output_subarea output_stream output_stdout output_text">
<pre>[878, 28, 80, 28, 27]
[27]
[18, 10402, 10749]
[28, 12, 14, 16]
[28, 12, 16, 10751]
</pre>
</div>
</div>

</div>
</div>

</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>So, TMDB doesn't want to make your job as easy as you thought. Why these random numbers? Want to see their genre names? Well, there's the Genre() class for it. Let's get this done!</p>

</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[18]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="c1"># Create a tmdb genre object!</span>
<span class="n">genres</span><span class="o">=</span><span class="n">tmdb</span><span class="o">.</span><span class="n">Genres</span><span class="p">()</span>
<span class="c1"># the list() method of the Genres() class returns a listing of all genres in the form of a dictionary.</span>
<span class="n">list_of_genres</span><span class="o">=</span><span class="n">genres</span><span class="o">.</span><span class="n">list</span><span class="p">()[</span><span class="s1">&#39;genres&#39;</span><span class="p">]</span>
</pre></div>

    </div>
</div>
</div>

</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>Let's convert this list into a nice dictionary to look up genre names from genre IDs!</p>

</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[19]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="n">Genre_ID_to_name</span><span class="o">=</span><span class="p">{}</span>
<span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="nb">len</span><span class="p">(</span><span class="n">list_of_genres</span><span class="p">)):</span>
    <span class="n">genre_id</span><span class="o">=</span><span class="n">list_of_genres</span><span class="p">[</span><span class="n">i</span><span class="p">][</span><span class="s1">&#39;id&#39;</span><span class="p">]</span>
    <span class="n">genre_name</span><span class="o">=</span><span class="n">list_of_genres</span><span class="p">[</span><span class="n">i</span><span class="p">][</span><span class="s1">&#39;name&#39;</span><span class="p">]</span>
    <span class="n">Genre_ID_to_name</span><span class="p">[</span><span class="n">genre_id</span><span class="p">]</span><span class="o">=</span><span class="n">genre_name</span>
</pre></div>

    </div>
</div>
</div>

</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>Now, let's re-print the genres of top 20 movies?</p>

</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[20]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="nb">len</span><span class="p">(</span><span class="n">top20_movs</span><span class="p">)):</span>
    <span class="n">mov</span><span class="o">=</span><span class="n">top20_movs</span><span class="p">[</span><span class="n">i</span><span class="p">]</span>
    <span class="n">title</span><span class="o">=</span><span class="n">mov</span><span class="p">[</span><span class="s1">&#39;title&#39;</span><span class="p">]</span>
    <span class="n">genre_ids</span><span class="o">=</span><span class="n">mov</span><span class="p">[</span><span class="s1">&#39;genre_ids&#39;</span><span class="p">]</span>
    <span class="n">genre_names</span><span class="o">=</span><span class="p">[]</span>
    <span class="k">for</span> <span class="nb">id</span> <span class="ow">in</span> <span class="n">genre_ids</span><span class="p">:</span>
        <span class="n">genre_name</span><span class="o">=</span><span class="n">Genre_ID_to_name</span><span class="p">[</span><span class="nb">id</span><span class="p">]</span>
        <span class="n">genre_names</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">genre_name</span><span class="p">)</span>
    <span class="k">print</span> <span class="n">title</span><span class="p">,</span><span class="n">genre_names</span>
    <span class="k">if</span> <span class="n">i</span><span class="o">==</span><span class="mi">4</span><span class="p">:</span>
        <span class="k">break</span>
</pre></div>

    </div>
</div>
</div>

<div class="output_wrapper">
<div class="output">


<div class="output_area">

    <div class="prompt"></div>


<div class="output_subarea output_stream output_stdout output_text">
<pre>Venom [u&#39;Science Fiction&#39;, u&#39;Action&#39;, u&#39;Crime&#39;, u&#39;Action&#39;, u&#39;Horror&#39;]
Halloween [u&#39;Horror&#39;]
A Star Is Born [u&#39;Drama&#39;, u&#39;Music&#39;, u&#39;Romance&#39;]
The Seven Deadly Sins: Prisoners of the Sky [u&#39;Action&#39;, u&#39;Adventure&#39;, u&#39;Fantasy&#39;, u&#39;Animation&#39;]
Incredibles 2 [u&#39;Action&#39;, u&#39;Adventure&#39;, u&#39;Animation&#39;, u&#39;Family&#39;]
</pre>
</div>
</div>

</div>
</div>

</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h1 id="Section-4---Building-a-dataset-to-work-with-:-Let's-take-a-look-at-the-top-1000-movies-from-the-database">Section 4 - Building a dataset to work with : Let's take a look at the top 1000 movies from the database<a class="anchor-link" href="#Section-4---Building-a-dataset-to-work-with-:-Let's-take-a-look-at-the-top-1000-movies-from-the-database">&#182;</a></h1>
</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>Making use of the same api as before, we will just pull results from the top 50 pages. As mentioned earlier, the "page" attribute of the command top_movies=all_movies.popular() can be used for this purpose.</p>

</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>Please note: Some of the code below will store the data into python "pickle" files so that it can be ready directly from memory, as opposed to being downloaded every time. Once done, you should comment out any code which generated an object that was pickled and is no longer needed.</p>

</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[21]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="n">all_movies</span><span class="o">=</span><span class="n">tmdb</span><span class="o">.</span><span class="n">Movies</span><span class="p">()</span>
<span class="n">top_movies</span><span class="o">=</span><span class="n">all_movies</span><span class="o">.</span><span class="n">popular</span><span class="p">()</span>

<span class="c1"># This is a dictionary, and to access results we use the key &#39;results&#39; which returns info on 20 movies</span>
<span class="nb">len</span><span class="p">(</span><span class="n">top_movies</span><span class="p">[</span><span class="s1">&#39;results&#39;</span><span class="p">])</span>
<span class="n">top20_movs</span><span class="o">=</span><span class="n">top_movies</span><span class="p">[</span><span class="s1">&#39;results&#39;</span><span class="p">]</span>
</pre></div>

    </div>
</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[23]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="c1"># Comment out this cell once the data is saved into pickle file.</span>
<span class="n">all_movies</span><span class="o">=</span><span class="n">tmdb</span><span class="o">.</span><span class="n">Movies</span><span class="p">()</span>
<span class="n">top1000_movies</span><span class="o">=</span><span class="p">[]</span>
<span class="k">print</span><span class="p">(</span><span class="s1">&#39;Pulling movie list, Please wait...&#39;</span><span class="p">)</span>
<span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span><span class="mi">51</span><span class="p">):</span>
    <span class="k">if</span> <span class="n">i</span><span class="o">%</span><span class="k">15</span>==0:
        <span class="n">time</span><span class="o">.</span><span class="n">sleep</span><span class="p">(</span><span class="mi">7</span><span class="p">)</span>
    <span class="n">movies_on_this_page</span><span class="o">=</span><span class="n">all_movies</span><span class="o">.</span><span class="n">popular</span><span class="p">(</span><span class="n">page</span><span class="o">=</span><span class="n">i</span><span class="p">)[</span><span class="s1">&#39;results&#39;</span><span class="p">]</span>
    <span class="n">top1000_movies</span><span class="o">.</span><span class="n">extend</span><span class="p">(</span><span class="n">movies_on_this_page</span><span class="p">)</span>
<span class="nb">len</span><span class="p">(</span><span class="n">top1000_movies</span><span class="p">)</span>
<span class="n">f3</span><span class="o">=</span><span class="nb">open</span><span class="p">(</span><span class="s1">&#39;movie_list.pckl&#39;</span><span class="p">,</span><span class="s1">&#39;wb&#39;</span><span class="p">)</span>
<span class="n">pickle</span><span class="o">.</span><span class="n">dump</span><span class="p">(</span><span class="n">top1000_movies</span><span class="p">,</span><span class="n">f3</span><span class="p">)</span>
<span class="n">f3</span><span class="o">.</span><span class="n">close</span><span class="p">()</span>
<span class="k">print</span><span class="p">(</span><span class="s1">&#39;Done!&#39;</span><span class="p">)</span>
</pre></div>

    </div>
</div>
</div>

<div class="output_wrapper">
<div class="output">


<div class="output_area">

    <div class="prompt"></div>


<div class="output_subarea output_stream output_stdout output_text">
<pre>Pulling movie list, Please wait...
Done!
</pre>
</div>
</div>

</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[24]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="n">f3</span><span class="o">=</span><span class="nb">open</span><span class="p">(</span><span class="s1">&#39;movie_list.pckl&#39;</span><span class="p">,</span><span class="s1">&#39;rb&#39;</span><span class="p">)</span>
<span class="n">top1000_movies</span><span class="o">=</span><span class="n">pickle</span><span class="o">.</span><span class="n">load</span><span class="p">(</span><span class="n">f3</span><span class="p">)</span>
<span class="n">f3</span><span class="o">.</span><span class="n">close</span><span class="p">()</span>
</pre></div>

    </div>
</div>
</div>

</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h1 id="Pairwise-analysis-of-Movie-Genres">Pairwise analysis of Movie Genres<a class="anchor-link" href="#Pairwise-analysis-of-Movie-Genres">&#182;</a></h1>
</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>As our dataset is multi label, simply looking at the distribution of genres is not sufficient. It might be beneficial to see which genres co-occur, as it might shed some light on inherent biases in our dataset. For example, it would make sense if romance and comedy occur together more often than documentary and comedy. Such inherent biases tell us that the underlying population we are sampling from itself is skewed and not balanced. We may then take steps to account for such problems. Even if we don't take such steps, it is important to be aware that we are making the assumption that an unbalanced dataset is not hurting our performance and if need be, we can come back to address this assumption. Good old scientific method, eh?</p>
<p>So for the top 1000 movies let's do some pairwise analysis for genre distributions. Our main purpose is to see which genres occur together in the same movie. So, we first define a function which takes a list and makes all possible pairs from it. Then, we pull the list of genres for a movie and run this function on the list of genres to get all pairs of genres which occur together</p>

</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[25]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="c1"># This function just generates all possible pairs of movies</span>
<span class="k">def</span> <span class="nf">list2pairs</span><span class="p">(</span><span class="n">l</span><span class="p">):</span>
    <span class="c1"># itertools.combinations(l,2) makes all pairs of length 2 from list l.</span>
    <span class="n">pairs</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="n">itertools</span><span class="o">.</span><span class="n">combinations</span><span class="p">(</span><span class="n">l</span><span class="p">,</span> <span class="mi">2</span><span class="p">))</span>
    <span class="c1"># then the one item pairs, as duplicate pairs aren&#39;t accounted for by itertools</span>
    <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="n">l</span><span class="p">:</span>
        <span class="n">pairs</span><span class="o">.</span><span class="n">append</span><span class="p">([</span><span class="n">i</span><span class="p">,</span><span class="n">i</span><span class="p">])</span>
    <span class="k">return</span> <span class="n">pairs</span>
</pre></div>

    </div>
</div>
</div>

</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>As mentioned, now we will pull genres for each movie, and use above function to count occurrences of when two genres occurred together</p>

</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[26]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="c1"># get all genre lists pairs from all movies</span>
<span class="n">allPairs</span> <span class="o">=</span> <span class="p">[]</span>
<span class="k">for</span> <span class="n">movie</span> <span class="ow">in</span> <span class="n">top1000_movies</span><span class="p">:</span>
    <span class="n">allPairs</span><span class="o">.</span><span class="n">extend</span><span class="p">(</span><span class="n">list2pairs</span><span class="p">(</span><span class="n">movie</span><span class="p">[</span><span class="s1">&#39;genre_ids&#39;</span><span class="p">]))</span>
    
<span class="n">nr_ids</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">unique</span><span class="p">(</span><span class="n">allPairs</span><span class="p">)</span>
<span class="n">visGrid</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">zeros</span><span class="p">((</span><span class="nb">len</span><span class="p">(</span><span class="n">nr_ids</span><span class="p">),</span> <span class="nb">len</span><span class="p">(</span><span class="n">nr_ids</span><span class="p">)))</span>
<span class="k">for</span> <span class="n">p</span> <span class="ow">in</span> <span class="n">allPairs</span><span class="p">:</span>
    <span class="n">visGrid</span><span class="p">[</span><span class="n">np</span><span class="o">.</span><span class="n">argwhere</span><span class="p">(</span><span class="n">nr_ids</span><span class="o">==</span><span class="n">p</span><span class="p">[</span><span class="mi">0</span><span class="p">]),</span> <span class="n">np</span><span class="o">.</span><span class="n">argwhere</span><span class="p">(</span><span class="n">nr_ids</span><span class="o">==</span><span class="n">p</span><span class="p">[</span><span class="mi">1</span><span class="p">])]</span><span class="o">+=</span><span class="mi">1</span>
    <span class="k">if</span> <span class="n">p</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span> <span class="o">!=</span> <span class="n">p</span><span class="p">[</span><span class="mi">0</span><span class="p">]:</span>
        <span class="n">visGrid</span><span class="p">[</span><span class="n">np</span><span class="o">.</span><span class="n">argwhere</span><span class="p">(</span><span class="n">nr_ids</span><span class="o">==</span><span class="n">p</span><span class="p">[</span><span class="mi">1</span><span class="p">]),</span> <span class="n">np</span><span class="o">.</span><span class="n">argwhere</span><span class="p">(</span><span class="n">nr_ids</span><span class="o">==</span><span class="n">p</span><span class="p">[</span><span class="mi">0</span><span class="p">])]</span><span class="o">+=</span><span class="mi">1</span>
</pre></div>

    </div>
</div>
</div>

</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>Let's take a look at the structure we just made. It is a 19X19 structure, as shown below. Also, see that we had 19 Genres. Needless to say, this structure counts the number of simultaneous occurrences of genres in same movie.</p>

</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[49]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="k">print</span> <span class="n">visGrid</span><span class="o">.</span><span class="n">shape</span>
<span class="k">print</span> <span class="nb">len</span><span class="p">(</span><span class="n">Genre_ID_to_name</span><span class="o">.</span><span class="n">keys</span><span class="p">())</span>
</pre></div>

    </div>
</div>
</div>

<div class="output_wrapper">
<div class="output">


<div class="output_area">

    <div class="prompt"></div>


<div class="output_subarea output_stream output_stdout output_text">
<pre>(19, 19)
19
</pre>
</div>
</div>

</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[24]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="n">annot_lookup</span> <span class="o">=</span> <span class="p">[]</span>
<span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">xrange</span><span class="p">(</span><span class="nb">len</span><span class="p">(</span><span class="n">nr_ids</span><span class="p">)):</span>
    <span class="n">annot_lookup</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">Genre_ID_to_name</span><span class="p">[</span><span class="n">nr_ids</span><span class="p">[</span><span class="n">i</span><span class="p">]])</span>

<span class="n">sns</span><span class="o">.</span><span class="n">heatmap</span><span class="p">(</span><span class="n">visGrid</span><span class="p">,</span> <span class="n">xticklabels</span><span class="o">=</span><span class="n">annot_lookup</span><span class="p">,</span> <span class="n">yticklabels</span><span class="o">=</span><span class="n">annot_lookup</span><span class="p">)</span>
</pre></div>

    </div>
</div>
</div>

<div class="output_wrapper">
<div class="output">


<div class="output_area">

    <div class="prompt output_prompt">Out[24]:</div>


<div class="output_text output_subarea output_execute_result">
<pre>&lt;matplotlib.axes._subplots.AxesSubplot at 0x11e9d1690&gt;</pre>
</div>

</div>

<div class="output_area">

    <div class="prompt"></div>


<div class="output_png output_subarea ">
<img src="data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAaEAAAE9CAYAAACiDN36AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz
AAALEgAACxIB0t1+/AAAIABJREFUeJzsnXecVcX5/9+fpShKUexiwa5YIIhEsQRrjH5ji0aNRjH5
fokpliQmMeWXYGISjSZG4zfxi8Ru1Gg0lhgVUWxgQRFQ7GLDgogIBBB29/n9MXPhcLnl3LN39967
PG9e57WnzDMz5+xynjMzz3xGZobjOI7j1IKmWlfAcRzHWXVxJ+Q4juPUDHdCjuM4Ts1wJ+Q4juPU
DHdCjuM4Ts1wJ+Q4juPUDHdCjuM4Ts1wJ+Q4juPUDHdCjuM4Ts3oWusKrAosuvfSTLIUv/3GxEzl
TWr9OJPdjMWzM9m9tWBWJruxfXbNZHdh96WZ7ADufO/pzLZZ2H297TPZTZk7I5PdwqWfZrI7deO9
MtndMndqJruPFs7LZLdWj56Z7I5ae5dMdle//0QmO4Dm1pZsdktmKnOhkaWzX0/9zum27pZtLq8t
eEvIcRzHqRk1d0KSjpBkkgp+Mkq6StLRVS5zuKRh1czTcRynbmhZmn6rMTV3QsDxwKPxZ0cxHKjI
CUnyrkvHcRqD1tb0W42pqROS1BPYC/g6cFw8J0mXSnpJ0v3A+vH8wZJuTtgOl3RX3D9I0kRJz0i6
OeaLpDcknRPPT5O0vaT+wKnAdyU9K2nv/NaWpAWJMh6RdAcwPZ47UdKT0fb/JHVp9wflOI5TAWat
qbdaU+uW0OHAPWb2MvCRpF2BI4HtgAHASSxvsdwPfFbSmvH4WOBGSesCPwMOMLPBwCTge4kyZsfz
fwHOMrM3gMuAi8xskJk9UqaOg4EzzGxbSTvEcvc0s0FAC3BCG+7fcRyn+nhLKDXHAzfG/Rvj8T7A
DWbWYmbvAg8AmFkzcA/wxdg1dihwO7A7wWE9JulZ4GRg80QZt8afTwP9M9TxSTPLhSrtD+wKPBXL
2h/YspCRpJGSJkma9Ne7H8tQrOM4TkasNf1WY2o2ziGpL7AfsLMkA7oABtxWwuxG4DvAHGCSmc2X
JGCsmRUbU8rFrLZQ/H6biQ5ZUhPQPXHtP8lqA1eb2Y9L1BEAMxsNjIbsIdqO4ziZyBgeXgtq2RI6
GrjWzDY3s/5mtikwA/gIOFZSF0kbAfsmbB4idI/9D8tbUI8De0raGkDSmpK2LVP2fKBX4vgNQgsH
4DCgWxG7ccDRknLjVH0lbV4kreM4Tm1oaU6/1ZhaOqHjWbnV8w9gI+AVQiDANcCyGZtm1gLcBXwh
/sTMPgRGADdImhrTl5sheCdwZC4wAbgc+JykKcAerNj6WYaZTSeMP90Xyxob6+s4jlM3NFJgQs26
48xs3wLnLklh9x1Cl1zy3APAbgXS9k/sTyKEZhMDIfKnUO+e2P9RTDceGJ+X503ATeXq6TiOUzPq
IOAgLT73pQPIKr9zWHPBBllZbl6STX6n2bL1I/fo2r18ogKc221RJrtZS+ZnsoMw6JiFrLomLy+Y
mclucfOSjCVm498LXslkt2DJ4irXpDTdm7K9sma0ZPubaa2DlkImGqje7oQcx3E6Gw0UmOBOyHEc
p7NRBwEHaan1PKGKkdQSAwpyW/+M+ZwpaY3q1s5xHKcO8HlC7cqiqFbQVs4ErgMWViEvx3Gc+qGB
AhMariVUCEn9o8bbM3EbFs8PlzRe0i2SXpR0fdSmOx3YGHhQ0oMx7V+iwsHzks5J5H2epOmSpkq6
UFIvSTMkdYvXeyePHcdxao1ZS+qt1jRiS6hHlMwBmGFmRwKzgAPNbLGkbYAbgCExzWeAHYF3gccI
um+XSPoesK+Z5ULJfmpmc6Ig6ThJuwAzCVp225uZSVorqjSMJ8gG/ZMgvHqrma2giS5pJDAS4NC+
Qxnca+v2eBaO4zgrUwfdbGlpxJbQoig8Oig6IAgKB5dLmgbcTNCSy/Gkmb1jYVbWsxTXj/uypGeA
yQSnNQD4BFgM/FXSUSzvuhsDnBL3TwGuzM/MzEab2RAzG+IOyHGcDqWBBEwbsSVUiO8CHwADCY41
OXkhud5xQf04SVsAZwG7mdnHkq4CVjezZklDCUKlRxMmye5nZo/FLsDhQBcze64d7slxHCcbdbBY
XVoasSVUiD7Ae7G181WCGGo5kvpxvQlSPZ9I2oAgC5Rb76iPmd1NcHQDE/bXAH+jQCvIcRynpnh0
XIfzZ+Afkk4iLPeQRmpgNHCPpHfNbF9Jk4EXgbcJY0cQnNTtklYnTJpPrlN0PXAuYfzJcRynfqiD
bra0NJwTMrOeBc69wopacAW136LuXG7/T8CfEscjihQ5tMj5vYBbzGxuupo7juN0EHXQwklLwzmh
ekDSnwhddoekST+p9eNM5WTVgDt19XIrWRRm1JzHM9nNX5JNA+7Jua9mshu6VscHemTVnPtoUXad
u47k7fmzMtk1KVuPflNTNrs5ixdkshvUZ+1Mdg9Ygy4F5i2hzo2ZnVbrOjiO4xSlgZxQXQQmSDpC
kkkqtw4QksZIGlAuXYp8+kv6SuJ4iKSyS0k4juPUO9ayNPVWa+rCCREWuHs0/iyJmf13XFyurfQH
ljkhM5tkZqdXIV/HcZza0kDRcTV3QjEMei/g6wT1gaJyO/HaeElD4v4CSRdEqZ37JQ2N11+XdFhM
U1DSBzgP2DuKoH43lnlXtOkr6Z9RqufxqJ6ApFGSrkiU4U7LcZz6o4Emq9bcCQGHA/fE1U4/krRr
PP8ZgsjoAGBLYM8CtmsCD5jZjoR5P+cCBxKkdn4Z0+QkfQYDxwK5LrezgUei8sJFefmeA0w2s12A
nxDmBOXYHvg8IWruF64Z5zhO3VHFlpCk1SU9KWlKUltT0haSnpD0qqSbJHWP51eLx6/G6/1L5V8P
Tuh44Ma4fyPLu+TSyO0sIcwLApgGPBQ13KYl0peS9CnGXsC1sGzp8HUk9Y7X/mVmn0bNuVnABoUy
kDQyCqJOemvBWymKdBzHqRLVbQl9SlCKGQgMAg6WtDtwPnCRmW0NfEzozSL+/DievyimK0pNnZCk
vsB+wBhJbwA/AL5MmBhaVm4HWGq2LIayNWcTHVcufVLSZwiQbS3q5aSp1wracZv13KyNRTqO41RA
S3P6rQwWyMXGd4ubEd7dt8TzVwNHxP3D4zHx+v654ZRC1LoldDRwrZltbmb9zWxTYAawdxXLKCbp
k5TtyecR4AQI41PAbDObV8U6OY7jtB8VtISSvTZxG5mfnaQucfWCWcBY4DVgrpnlvNg7QL+434+g
PEO8/gmwTrGq1nqe0PGs3FT7B/BNwk1Wg2KSPlOBFklTgKsI6tk5RgFXSJpKUM4+uUp1cRzHaX8q
iHozs9EEGbNSaVqAQZLWAm4jjI1XhZo6ITPbt8C5S1gePJA7l5TbGZ7Y75nYH5Vn0zP+LCbps5TQ
nEwyPl6bw/KmZTLP/DJ2WvmuHMdxakw7Rb2Z2dy4EOgewFqSusbWziaE9deIPzcF3pHUldAb9VGx
PGvdElolmLE4m/xOc8ZVD7PK78x6475Mdltte3g2uzU2zGT3cUs2mSCAjXv2zWQ3a+EnmezO3iBb
z/ITrXMy2Y39YGomuy37bJTJrkdTtiHWl+fNLJ+oAJ9bN9s89b/Nm5bJLuvfC8B7C7L9DqtCFef/
SFqPMP4+V1IPQgTy+cCDhCGVGwm9RbdHkzvi8cR4/YHE2P1KuBNyHMfpbFS3JbQRcHVcdboJ+LuZ
3SVpOnCjpHMJwxl/jen/Clwr6VVgDnH+ZzHcCTmO43Q2UkS9pcXMphLmbeaff50CqwyY2WLgmLT5
dxonJKmFMD+oG9BMmGB6UYyKcxzHWXWoAyWEtHQaJwQsMrNBAJLWJ6x62hv4RTJRYiDNcRync9JA
S1DUep5Qu2Bms4CRwHcUGCHpDkkPAOMk9ZQ0LmrJTZN0OCzTmXtR0lWSXo6adQdIekzSK5KGxnRD
JU2UNFnSBEnb1fB2HcdxVqSBtOM6U0toBczs9TiQtn48NRjYxczmxLDBI81snqR1gccl3RHTbU3o
z/wa8BRBaXsv4DCCjtwRhGXA9zazZkkHAL8BvpQsP074GgmwYc/NWavH+jiO43QIdeBc0tJpnVAB
xsb5PxBkgX4jaR+C3E8/lmvAzTCzaQCSngfGmZlF7bn+MU0fQrTINgT5ipVETJMTwHZYf2jjtI0d
x2l8GmgovNM6IUlbErTdcusW/ydx+QRgPWBXM1sadetWj9eS2nCtieOkHt2vgAfN7MioEDu+ytV3
HMfJTku2OYa1oFM6oTi56jLg0tiKyU/SB5gVHdC+wOYVFtGH5bODR7Slro7jOFXHu+NqQo8osJcL
0b4W+EORtNcDd8YutkmEMZ5K+B2hO+5nwL8y1tdxHKd9cCfU8ZhZlxLXriKIlOaOZxO0jwqxUyLd
iMT+G7lrZjYR2DZh87PKa+w4jtNO+JiQk+StBbPKJypAj67ZdLnmL8mmrZZVA+77PQdmsjtv3jOZ
7LJq6gF8vGhB+URVZLJl05wbP3t6lWtSmoXNi7PZkc3u0+almezue39KJrum4svZtBu1jEay1saJ
hXIn5DiO09moomxPe9OpJqtKWpB3PELSpbWqj+M4Tk1otfRbjfGWUIJ8SZ+0Ej8uBeQ4Tl3hgQn1
R5zPcwWwLvAhcIqZvSXpKmAxQSX2MUnzgK2ALYG3JJ0C/AUYQoi6+56ZPShpBHAU0JOwZPjnOvJ+
HMdxiuJOqGbkwrRz9CUssATwJ+BqM7ta0tcIq7fmVk/dBBhmZi2SRgEDgL3MbJGk7wNmZjtL2h64
T1IuMm6ZFFA735fjOE56GkjAtLM5oWVK2hDGhAgtGAgh2UfF/WsJc31y3BzXUM9xh5nlQsz2Ijgw
zOxFSW+yPDx7bDEHlNSO696tL1279sp8U47jOBXhLaGG4z9ljtPaLSOpHbfmGv0b57PEcZzGp4Fk
ezpVdFwZJrB8mdkTgEdS2j0S0xO74TYDXqp67RzHcaqFR8fVJacBV0r6ATEwIaXdn4G/RImfZmCE
mX1aQI/OcRynLjDvjqsNZtYz7/gqolyPmb0J7FfAZkTe8ai848UUcFj5UkCO4zh1Qx20cNLSqZxQ
vTK2z66Z7M7tlk1+58m5r2ay22qNDTPZZZXfeePxv2SyG77/qEx2AM98mu3ZWMZoo+Oa+2Sym9N3
60x2Ez+sVIs3cEZG6aU/zJ+cyS4rXZuKSkSWpNdqPTLZNbdhbCWrfFZVcO04x3Ecp2Y0N05ggjsh
x3GczkYDdcd1yug4SUdIsji5tFS6EZI2ThyPkTSg/WvoOI7Tjlhr+q3GdEonBBwPPBp/lmIEsMwJ
mdl/m1nHaug7juNUmwYK0e50TkhST4LKwddZPi8IST+SNE3SFEnnSTqaoKZwvaRnJfWQNF7SkJj+
+Jj+OUnnJ/JZIOnXMZ/HJW3QwbfoOI5TEmttTb3Vmk7nhIDDgXvM7GXgI0m7SvpCPP9ZMxsI/M7M
biEs7X2CmQ1KyPQQu+jOJ4R0DwJ2k5TTmVsTeDzm8zDwP4UqIWmkpEmSJt2+cEY73arjOE4BvCVU
U44Hboz7N8bjA4ArzWwhQArB0d2A8Wb2YVyi4Xpgn3htCXBX3H8a6F8oAzMbbWZDzGzI4WtskfVe
HMdxKqelJf1WYzqVE5LUl9B6GSPpDeAHwJerXMxSWz5ppAWPMHQcp96oYktI0qaSHpQ0XdLzks7I
u/79GAi2bjyWpEskvSppqqTBpfLvVE4IOBq41sw2N7P+ZrYpMAP4BDhF0hqwzFkBzAcKyVs/CXxO
0rqSuhBaUw+1f/Udx3HajrVa6i0FzcD3zWwAsDvw7VwUsaRNgYOAtxLpvwBsE7eRhPXYitLZnNDx
wG155/4BbERYV2hSXG/orHjtKuCyXGBCzsDM3gPOBh4EpgBPm9nt7Vx3x3Gc6lDFlpCZvWdmz8T9
+cALQL94+SLgh0Ayo8OBayzwOLCWpI2K5a+sciROeo7a/LBMD/mtJR9nKm/9btnWLvq4JZvMyKsL
3s1kt3XPjcsnKsD938hmB9Dn3PGZbbOwRrfVMtn16No9k91Hi+Znslt3jd6Z7D5evCCTXbembL3Y
2/bpVz5RAXZdreg7sCxXvjshk11WieOlS2a2WR15/ncOSf3O6XXp3anLiytUPwzsBOwL7GdmZ8Th
jyFmNlvSXcB5ZvZotBkH/MjMJhXK08czHMdxipDVAdWcCqLekgtwRkbH9dDy0/Uk9CydSeii+wmh
K65NuBNyHMfpZFhL+vk/yQU4iyGpG8EBXW9mt0raGdgCmBKXtdkEeEbSUGAmsGnCfJN4riANNSYk
aUNJN0p6TdLTku6OC811RNlv5KI/HMdx6prqRscJ+Cvwgpn9AcDMppnZ+jEArD/wDjDYzN4njL+f
FKPkdgc+iePsBWmYllB8ELcBV5vZcfHcQGAD4OVa1s1xHKeuqO4k1D2BrwLTYmAXwE/M7O4i6e8G
DgFeBRZSZgHRhnFChEGwpWZ2We6EmU2J3vYCQligAeea2U2ShgPnAHOBnYG/A9OAM4AewBFm9pqk
9YDLCMt2A5xpZo9JWge4gRAFMpE4zijpl8AcM/tjPP41MMvMLm7f23ccx0lHytDrdHmFAIOSwQux
NZTbN+DbafNvpO64nQgKBfkcRZDWGUhQRrggEQ44EDgV2IHgybc1s6HAGMJy3wAXAxeZ2W7Al+I1
gF8Aj5rZjoQWWM5JXQGcBCCpiaBPd12V7tFxHKftNJBsTyO1hIqxF3CDmbUAH0h6iCC7Mw94KtcX
Kek14L5oM43QsoLguAbEwTWA3jEKZB+Cg8PM/iXp47j/hqSPJH2G0BU42cw+yq9UMuJkUN9d2KLn
5lW+bcdxnMJYc+2dS1oayQk9T1BEqIRPE/utieNWlt97E7C7mS1OGiacUiHGEJaB2JDQMlqJZMRJ
1nlCjuM4maiDFk5aGqk77gFgtdjCAEDSLoQxn2MldYnjO/sQZHfSch/Lu+aQNCjuPgx8JZ77ArB2
wuY24GBCi+veym/FcRynHWmtYKsxDdMSMjOTdCTwR0k/AhYDbxAmTvUkyOsY8EMze7/cqqoJTgf+
V9JUwvN4mDCOdA5wg6TngQkktJHMbImkB4G5sRvQcRynbqhmYEJ70zBOCMDM3qWwKvYP4pZMOx4Y
nzgeXuiamc0Gji1Q1kcUmQ0cAxJ2B46ppP6O4zgdQh20cNLi2nEVEtVj7wJuM7Pvp7Hp1r1fpofc
0b+ZjXv2LZ+oAO8uKLc8U2G6NGXrDW5pw2qQ/Xqtk8lu5vyVYk9S0a1Ltu+8ltZsDexW//9cVbJq
/wEsXPpp+UQFaK6CdtycIz+X+g+h720Ptbm8ttBQLaF6wMymA1vWuh6O4zjFsOZa1yA97oQcx3E6
Gw3UHddI0XEFkbQg73iEpEvj/qmSTiphO1zSsPauo+M4Tkdirem3WtOpW0JJiZ8iDAcWEKLfUiGp
q1kjNXYdx1nlqAPnkpaGbwmVQtIoSWfF/dPjGulToxJ3f0Io9nfjyqp7S+ov6YGYZpykzaLtVZIu
k/QE8DtJr8Q5SUhqimupr1ej23Qcx1kBbwl1LD0Syq4AfQlS4vmcDWxhZp9KWsvM5kq6DFhgZhcC
SLqToNJ9taSvAZcAR0T7TYBhZtYi6RPgBOCPBNmfKWb2YbKwpGxPU5c+NDWtWbUbdhzHKUU9OJe0
dIaW0CIzG5TbgJ8XSTcVuF7SiYRVAQuxB/C3uH8tQZcux82JianLREyBrwFX5mdkZqPNbIiZDXEH
5DhOR2ItSr3Vms7ghNJyKPC/wGDgKUmVtgL/k9sxs7cJYqn7AUOBf1etlo7jOG2kkbrjVgknFBUO
NjWzB4EfAX0IUj/zgV6JpBMISzNA6G57pES2YwhLONzs0j2O49QT1qrUW61ZJZwQ0AW4TtI0YDJw
iZnNBe4EjswFJhCETE+JOnJfJSyAV4w7CI5spa44x3GcWtJILSGX7cmIpCGExfD2Lpc2q2xPVrIW
1rWpSya75owSM02ll8soSi2kabLKtyzKKN3i/ytXXaoh2zNzj/1S/wn1m/iAy/Y0GpLOBr5J6LJz
HMepK1qba9/NlpZVpTuuqpjZeWa2eVx73XEcp64wS7/VmoZwQpIuknRm4vheSWMSx7+X9L0K8zxT
0hrVrKfjOE494IEJ1ecxYBgsi3RbF9gxcX0YFUjvRM4EKnJCkrINmjiO43Qg7oSqzwTCRFIIzuc5
YL6ktSWtBuwAPCPpB5KeirI75wBIWlPSvyRNkfScpGMlnQ5sDDwYV0hF0kGSJkp6RtLNknrG829I
Ol/SM8AxksbH4yclvRyj6hzHceqGRuqOa4jABDN7V1Jz1HIbBkwE+hEc0yfANIIY6TaEyaMC7pC0
D7Ae8K6ZHQogqY+ZfRK77/Y1s9mS1gV+BhxgZv+Jy4d/D/hlrMJHZjY42p8KdDWzoZIOAX5BkO5x
HMepC+qhhZOWhnBCkQkEBzQM+APBCQ0jOKHHCEtxH0SYBwRhDs82hAmnv5d0PnCXmRWagLo7MAB4
TCFsuDvB0eW4KS/9rfHn00D/QpV17TjHcWpFax3I8aSlkZxQblxoZ0J33NvA94F5hAmjnwN+a2b/
l28oaTBwCHCupHFm9sv8JMBYMzu+SNn/yTvOTf5oocgzNLPRwGjo+HlCjuOs2rRa4zihRhkTgtAS
+i9gjpm1mNkcYC1Cl9wE4F7ga4mxnH6S1pe0MbDQzK4DLiBox8GKkj2PA3tK2jrarilp2466Mcdx
nGpiptRbrWmkltA0QlTc3/LO9TSz2cB9knYAJsYutQXAicDWwAWSWoGlhEmmEFop90h618z2lTQC
uCEGOkAYI3q5ne/JcRyn6jTSmJDL9nQALttTGJftKY7/r1x1qYZszwvbHJL6T2iHV+4uWZ6kKwi9
ULPMbKd4bhBwGbA6YWmcb5nZkwotgIsJwx8LgRFm9kyp/BupJdSw7L7e9pnsXl4wM5PdR4vmZ7I7
e4Ns0eaT7ZNMdsc198lk941PJpZPVISlGR1mVmcy75bvZrI76bRSAu7Fue29SZnsDt5wUCY7ke19
+e/3J5dPVIDVunbLZPdp89JMdt26ZH9FNrcUW7as/alyS+gq4FLgmsS53wHnmNm/Y5Tw7wgRyl8g
BIRtA3wW+Ev8WRR3Qo7jOJ2MltbqDfeb2cOS+uefBnrH/T7Au3H/cOAaC11sj0taS9JGZvZesfwb
KTBhJSStE5dheFbS+5Jmxv25kqanzONUSSfF/askHR33x0elbMdxnIaiAyarnkkYa38buBD4cTzf
jxC5nOOdeK4oDe2EzOyjxLLelxGWVhgEDALKrpQhqauZXWZm15RLmyIvl/RxHKcuaDWl3iSNlDQp
sY1MUcQ3ge+a2abAd4G/Zq1rZ+6O6yLpcsLcopnA4Wa2SNJ44FlgL0I0XC9ggZldWCwjSQcB5wCr
Aa8Bp5jZAklvECayHkjoE72xHe/HcRwnFZWEXifnNFbAySxf9PNmwkrTEN61mybSbRLPFaWhW0Jl
2Ab4XzPbEZgLfClxrbuZDTGz35fLJE/SZzAwiSDpk+MjMxtsZu6AHMepCzqgO+5dgkAAwH7AK3H/
DuAkBXYHPik1HgSduyU0w8yejfv58jr5MjylqFTSB1hRtmfrPtux4Zolu0Udx3GqRjUDEyTdQIh8
W1fSOwS9zP8BLpbUFVhMfNcBdxPCs18lhGifUi7/zuyEkjG1LUCPxHG+DE8pKpX0AVZs4u7db3+f
9uE4TodRTdmeEu++XQukNeDbleTfmbvjqoVL+jiO01BYBVut6cwtoapgZh+6pI/jOI1EIwmYdhon
ZGajEvtvADslji9M7A8vYTeiUDozewDYrUCZ/dtUacdxnHagHoRJ09JpnFA9M2XujEx2i5uXVLkm
pXmidU4mu/GzU80LXok5fbfOZNeja/dMdgCLFy/IZJe12yKr/M4+rb3KJyrAbZms4L4PpmSy26hn
34wlZiOr/E7WV/LSGkrvtIWykyTrCHdCjuM4nYwWbwk5juM4taI1c9uv4+m00XGSNpR0o6TXJD0t
6e5CUW2SJtSifo7jOO2FodRbremULaG4psVtwNVmdlw8NxDYgBjVFnXjms1sWO1q6jiOU30aaUyo
s7aE9gWWmtlluRNmNoWgJ/eIpDuA6QCSFsSfwyU9JOl2Sa9LOk/SCZKelDRN0lYx3XqS/iHpqbjt
WYP7cxzHKYq3hGrPTgSpnkIMBnYys0IhawOBHYA5wOvAGDMbKukM4DSCfPnFBLXuRyVtBtwbbVYg
KduzWvd16N61d34Sx3GcdqGRYvo6qxMqxZNFHBDAUzmxPUmvAffF89MIrSuAA4ABWr40dW9JPc1s
hdjfpGxP7zW3rIeJyY7jrCLUQwsnLZ3VCT0PHF3kWinduKTeXGviuJXlz6oJ2N3MFrepho7jOO1E
dVf3bl8665jQA8BqycWZJO0C7F2FvO8jdM3l8h1UhTwdx3GqRitKvdWaTumEopLrkcABMUT7eeC3
wPtVyP50YIikqXEJ8VOrkKfjOE7VcAHTOsDM3gW+XODS5Xnpesaf44HxifPDE/vLrpnZbODY6tbW
cRynejRSiLasDUvrOeno2r2fP2SnQ1i7R89Mdh8vyqap51Sf5iUz29xHdvNGJ6R+5xzz3vU17ZPr
tC0hx3GcVZVGagm12QlJaiGEMHcjhKdfQ5hHU/fPIQYVbGxmd9e6Lo7jONWikaLjqtESWmRmgwAk
rQ/8DehNWIe83hkEDCGsi56KnNxP+1XJcRynbdRD1FtaqhodZ2azCCoB31FgdUlXRtmbyZL2BZDU
RdKFkp6LUWanxfNvSFo37g+RND7uj5J0dZTceVPSUZJ+F/O9R1K3mG7XKL3ztKR7JW0Uz4+XdH6U
4HlZ0t6SugO/BI6V9KykYyUNlTQx1nWCpO2i/QhJd0h6ABgn6RpJR+TuW9L1kg6v5rN0HMfJyiod
HWdmr0vqAqwPnBhO2c6Stgfui0rWpwD9gUFm1iwpzcpYWxFUCwYAE4EvmdkPJd0GHCrpX8CfgMPj
ktzHAr9j0vEXAAAgAElEQVQGvhbtu0YJnkOAX5jZAZJ+Dgwxs+8ASOoN7B3rdADwG+BL0X4wsIuZ
zZH0OeC7wD8l9QGGASdnf2qO4zjVY1XrjivFXgTHgJm9KOlNYFuC9M1luW4tM0uzpOe/zWyppGlA
F+CeeH4awaFtR9CMGxsldboA7yXsb40/n47pC9EHuFrSNoSPhG6Ja2Nz9TSzhyT9WdJ6BCf1j/wu
uqR2nLr0oalpzRS36DiO03Zaal2BCqi6E5K0JeEZzMpg3szyLsLV8659CmBmrZKW2vLY8pykjoDn
zWyPInnnJHhaKH7fvwIeNLMjJfUnMW+IleV+riG09I4jtOxWIKkd5yHajuN0JI3UEqrqmFBsGVwG
XBqdxCPACfHatsBmwEvAWOAbkrrGa7nuuDeAXeP+l6iMl4D1JO0R8+wmaccyNvOBXonjPsDMuD+i
jO1VBFVtzGx6hXV1HMdpN1or2GpNNZxQjziw/zxwP0Fb7Zx47c9AU+xCuwkYYWafAmOAt4CpkqYA
X4npzwEuljSJCluUZraEIFp6fszzWcJYTSkeJChiPxvHkH4H/FbSZMq0Es3sA+AF4MpK6uk4jtPe
NJITcsWEjEhagzAeNdjMPimV1rvjnI7CFRMan2ooJly26Ymp3zmnvn2dKyY0GjFy7q+ESbklHRDA
qRvvlamcfy94JZPd2/OzDMfBln02ymS3sDnbqhZn9ByYye6CecXWKyzP7IXzMttm4eANs4ms3/fB
lEx2WZ3JtM2z/S72mfVaJrus9ezWJdsrq1tTl0x2i5uXZLIDaK3hB34jTWR0J5QBM7sf2LzW9XAc
xylEI3W9uBNyHMfpZHSq6DhJP5X0fFQ2eFbSZ0ukHSLpkupWsTyS+ktaFOuX27pLOkzS2SXs1pL0
rcTxxpJu6ZhaO47jtA/VDEyQdIWkWZKeS5y7QNKL0S/cJmmtxLUfS3pV0kuSPl8u/5JOKIY7/xdh
8H0XwiTTt4ulN7NJZnZ6ivtqD14zs0GJbYmZ3WFm55WwWQtY5oTM7F0zK7YsuOM4TkNQ5ei4q4CD
886NBXaKfuFl4McAkgYQ5k7uGG3+HBV0ilKuJbQRMDuGVWNms+NicUjaLeqrTYmabL0kDZd0V7y+
ZvSgT0YttsPj+RGSbo2ab69I+l2uMEkHS3om5jmuVD5piGVdGvc3iB57StyGAecBW8WW0wWxRfVc
TF9M965o/R3HceqBamrHmdnDwJy8c/clVGIeBzaJ+4cDN5rZp2Y2A3gVGFoq/3JjQvcBP5f0MmEO
0E1RsqY7Yd7PsWb2VNRcW5Rn+1PgATP7WmyqPSnp/nhtEPAZgorBS5L+BCwmrHq6j5nNSExgLZiP
meUrGGwl6dm4/5iZfTvv+iXAQ1ENoQvQEzib4M1zKuD9E+m/TWHdu4L1N7MVWohJ2Z7hfXdlx15b
FXq+juM4Vae5gjGh5LsqMjoqvqTlawR/ANCP4JRyvBPPFaXchMwFknYF9iaIh94Ux1ieBt4zs6di
unkA0gp3fhBwmKSz4vHqBMUEgHG50GZJ0wmRZmsDD0fvmdSTK5bPC3nVfS3nTIqwH3BSzLsF+ETS
2iXSF9O9K1b/FZxQUrbnO/2PbaRgFcdxGpxKXjjJd1WlSPopISL8+iz2kCI6Lr6wxwPjo/LByQQn
VLZ+BKXrl1Y4GQIbPk2cKqXlVjSfGlNJ/R3HcTqU1g4I0pY0ghAzsH9Cy3MmsGki2SYsl0IrSLnA
hO0UFKVzDALeJOi0bSRpt5iul6IOXIJ7gdMUm0eSPlPyjkITbh9JW8T0ue64SvMpxjjgmzGPLgpL
MORrxyUppnvnOI5T17S3bI+kg4EfAoeZ2cLEpTuA4yStFt/l2wBPlsqrXGBCT8LSBtMlTSWs5TMq
6rQdC/xJQadtLCurXv+KsBTCVAVduV+VKsjMPiT0S94a88z1MVaUTwnOAPaNrbmngQFm9hHwmMLi
ehfkpS+me+c4jlPXVDMwQdINhDXctpP0jqSvA5cSPuDHxsCuywDM7Hng78B0wnI73469acXzd+24
9ifrmJDL9hTGZXuKk1UqxmV7ClML2Z5qaMeN2vyE1IWPevN6147r7Nwyd2omuwVLsr3cm5RNHL1H
U/dMdgvJVs8/zJ+cye7jxY0jtimy/f/eqGeaxYZXZub8jzLZZXUmX1k7W+/4/y56JJNdS2u25dqy
2tVS/60tNKtx6u1OyHEcp5PROC6oyovadQSSTNJ1ieOukj7MTZKtMK9Bkg6pbg0dx3FqSyOtJ9Rw
ToiwzPZOknrE4wMpEwJYgkFARU6oQBSg4zhOXdGKpd5qTSM6IYC7gUPj/vHADQCSmqKUznqJ41cl
rSfpmBgFN0XSw1H14ZfAsTG649hiEkFRqucOSQ8A4yRdI+mIXGUkXV+JnJDjOE57Us3ouPamUZ3Q
jYRY9NWBXYAnAMysFbiOOL+HILg6JYZ//xz4vJkNJMS2L4nnboqCpzexXCJoKEEh4gJJa8a8BgNH
m9nnCAvajQCI842GAf9q53t2HMdJRTOWeqs1DemEzGwq0J/QCro77/IVRHkegqbRlXH/MeAqSf8D
FIvXPAg4O2rQjWdFqaGxOSkhM3sI2Ca2uI4H/pEQ8wOCHpOkSZImLVwyN9N9Oo7jZKGRWkKNPL5x
B3AhMBxYJ3fSzN6W9IGk/QjqrSfE86dGyaBDgaejJl4+paSG8gVTrwFOJMiWn5KfUVKPacO1dqiH
37XjOKsI9RBwkJaGbAlFrgDOMbNpBa6NIXTL3ZybrStpKzN7wsx+DnxI0DfKl+2pRCLoKuBMADOb
3sZ7cRzHqRpWwb9a07BOyMzeMbNiq7jeQZAcujJx7oK4NtBzwARgCvAgMCAXmEAFEkFm9gFByfvK
Ymkcx3FqQSOFaDdcd5yZ9SxwbjxhDCfHQEJAwouJNEcVyG4OsFveuW8UyP8qQstnGZLWIIjz3ZCq
4o7jOB1EPYRep6XhnFA54npH32R5hFx7lHEAIULuoty6QqX4qIP1ypqasjVwX56XbbrVp81LM9ll
ZfWu2eSFAFpas2uBZeHf72eTJuposmq5ZZXfWadHMfH60sxbkr92ZjqyasctXNqYmsUt7oRqh5md
R1i2uz3LuJ+wkJ3jOE7dUQ/dbGnpdE7IcRxnVaceAg7S0rCBCWmops5ctB8jaUD1aug4jlN9PDCh
flimM2dmi2ibzhxm9t9Vq5njOE474S2h+qKgzhyApFGSzkocPyepf9SQ+1fUmXsuhm8jabykIXH/
YEnPxDTjOvB+HMdxStJILaFVwQkV1Jkrw8HAu2Y20Mx2IixTu4wo13M5QV1hIHBMfgZJ2Z7W1nyx
BcdxnPajxSz1Vms6vRMqozNXjGnAgZLOl7R3gTDs3YGHzWxGLGNOgXJHm9kQMxvS1LRm/mXHcZx2
w5dyqD9yOnP5E0ubWfEZrA5gZi8TVLOnAedK+nlHVNJxHKcauGxP/VFMZ+4NgrNB0mBgi7i/MbDQ
zK4DLsilSfA4sI+kXPq+7Vd1x3GcymikMaHOHh0HBJ05oJDO3D+Ak6JO3BPAy/H8zgStuVZgKUGB
IZnfh5JGArdKagJmESLvHMdxak49dLOlRVYHA1OdnfX6bJfpIXdvyvaNMGdxNgmWfdfbMZPdfe9P
yWTXNaOUyoC1NyufqAhTP5qR2TYLq3Xtlsmuo6WQunXJ9rfW0tqSya5Lxt/9z9bfK5Pd7+c8mckO
YEFGqaDWjO/W5iUzlckwwVGbH5a68FvfvKPN5bWFVaIl5DiOk4WsDqjWNFLjwp2Q4zhOJ6ORuuPq
0glJaiFEpnUFZgBfNTNfI9txHCcF9RBwkJZ6jY5bZGaD4kTROcC3a10hx3GcRsFDtKvLRKAfgAIX
RCmdaQk5neGSHpJ0u6TXJZ0n6QRJT8Z0W8V0X5T0hKTJku6XtEE8P0rSFVGW53VJp+cKl3SSpKlR
nufaeG49Sf+Q9FTc9uzwp+I4jlOERpqsWpfdcTkkdQH2JywgB3AUMIiwcuq6wFOSHo7XBgI7EFpO
rwNjzGyopDOA04AzgUeB3c3MJP038EPg+9F+e2BfoBfwkqS/ANsCPwOGmdnsxHygiwkL2j0qaTPg
3li24zhOzam2HI+ktYAxwE6AAV8DXgJuIijSvAF82cw+rjTvenVCPSQ9S2gBvQCMjef3Am4wsxbg
A0kPEZbnngc8ZWbvAUh6Dbgv2kwjOBeATYCbJG0EdCeMN+X4l5l9CnwqaRawAbAfcLOZzYYV5HkO
AAZIyyIbe0vqaWbLYqPjPKKRAD1XX5/Vu6/V1mfiOI6TinboZrsYuMfMjpbUHVgD+AkwzszOiyta
nw38qNKM67U7bpGZDSKsXirSjQkl1+FtTRy3stzZ/gm41Mx2Br5BlOkpYN9CaQfdRGhRDYpbv6QD
ghW149wBOY7TkVSzO05SH2AfYo+UmS2JgWKHA1fHZFcDR2Spa706IQDMbCFwOvB9SV2BR4BjJXWJ
Stb7AJXMQuvD8vWETk6R/gHgGEnrwAryPPcRuviI5wdVUAfHcZx2xcxSb0nF/7iNzMtuC+BD4Mo4
nj5G0prABrneJ+B9Qu9RxdS1EwIws8nAVIIK9m1xfwrBQfzQzN6vILtRwM2SngZmpyj7eeDXwEOS
pgB/iJdOB4bEgIXpwKkV1MFxHKddqaQllOy1idvovOy6EvQz/2JmnyEsFnp2MoGF2bGZ+gDrckzI
zHrmHX8xcfiDuCWvjwfGJ46HF7pmZrcDtxcob1Te8U6J/atZ3uTMnZsNHJviVhzHcTqcFqvqTKF3
gHfMLLcW2y0EJ/SBpI3M7L04zj4rS+Z16YQ6G0etvUsmuxkt8zPZDeqzdia7v83LFxlPR5OySU/1
Wq1HJrtdV9sokx3AVDpWOy6rBlxWMa+sw9HdMmq5ZdWOy1peVg2429ccmMlu308nZrKD7L/DalDN
sAQze1/S25K2M7OXCBHL0+N2MnBe/LnSB34a3Ak5juN0Mtph/s9pwPUxMu514BTCcM7fJX0deBP4
cpaM3Qk5juN0MqrthMzsWWBIgUv7tzXvug9MSIukFknPJrb+VcjzVEknxf2rJB3d1jwdx3Ham0qi
42pNZ2oJ5eYWVQ0zu6ya+TmO43QE9SDHk5ZO0xIqhKT+kh6R9EzchsXzabXmRkk6Ky/P/ST9M3F8
oKTbOvbOHMdxitNqram3WtOZnFCPRFdczinMAg40s8GEkOrkEt8DCfN7dgC+CmxrZkMJ+kinUZwH
ge3jZFkIA3RX5CdKTgB7Yf7rbboxx3GcSmgkAdPO5IRyyz8MMrMj47luwOWSpgE3AwMS6Z8ys/ei
Xly+1lz/YoXESVnXAidGUb89gH8XSLdsAtgOvbZs6705juOkxseE6ofvAh8QWj1NwOLEtTRac8W4
Ergz5nezmTVXpbaO4zhVoB5aOGnp7E6oD2Gmb6ukk4FsM+TyMLN3Jb1LWObhgGrk6TiOUy3qYbG6
tHSm7rhC/Bk4Oeq+bU/QPKoW1wNvm9kLVczTcRynzbSapd5qjeqhT7ARkXQpMNnM/lou7eqrb5bp
IWeNXMn6h7Vxz77lExXg/f9UvI4VAGt2W718ogLMX7Iokx3AGt1Wy2S3cOmn5RMVoFuXbJ0NS1s6
toc3q/RSR7/EOrqem/fOJAwNwJvzPshk17xkZpsVf3ZYf2jqG35h1pO1VBjq9N1x7UJU4f4Py1dl
dRzHqRsaqTvOnVAGzGzXWtfBcRynGPXQzZaWzj4mVBJJF0k6M3F8r6QxiePfS/pebWrnOI6TDavg
X61ZpZ0Q8BiQU1FoAtYFdkxcHwZMKJeJAqv6s3Qcp05opMCEVf3FOYEw2RSC83kOmC9pbUmrEdQU
pksaF2V/pkk6HJZJAr0k6Zpot2ktbsBxHCefVmtJvdWaVXpMKM73aZa0GaHVMxHoR3BMnxDUExYC
R5rZPEnrAo9LuiNmsQ1wspk9XoPqO47jFMQnqzYWEwgOaBjwB4ITGkZwQo8RFkj8jaR9CGoK/YBc
3OabxRyQpJHASICuXdemS5eehZI5juNUnUaaeuNOaPm40M6EbrW3CaHX8wjyPCcA6wG7mtlSSW8A
uQkuRSe/mtloYDRknyfkOI6ThUZqCa3qY0IQWkL/BcwxsxYzmwPkhEknEKR/ZkUHtC+wee2q6jiO
Ux4XMG0sphGi4v6Wd66nmc2WdD1wZ1TingS8WIM6Oo7jpKYeot7Ssso7ITNrAXrnnRuR2J/N8gi6
fHZqv5o5juNkox4Wq0vLKu+EOoLm1tqHQabhvQVzMtll/ebKqgHXFqGrrBpwWWnuYA24rDTKl3PW
emb9m8mq/wbZdQOrQSONCbkTchzH6WTUw1hPWuoyMEHSOomlut+XNDPum6TP56U9U9JfCuRhkq5L
HHeV9KGkuzLW6VRJJ2WxdRzH6UgaSTGhLltCZvYRMAhA0ihggZldGOfeHAfcm0h+HPDDAtn8B9hJ
Ug8zWwQcCMxsQ50uy2rrOI7TkXhLqP24BThUUncI0jnAxsAjRdLfDRwa948HbshdkNRX0j8lTZX0
uKRdJDVJekPSWol0r0jaQNIoSWfFc1tJukfS05IekbR91e/UcRwnI61Y6q3WNJQTinN4ngS+EE8d
B/zdirv9G4HjJK0O7AI8kbh2DmFRul2AnwDXmFkrcDtwJICkzxJUEfJHJ0cDp8UlHc4irODqOI5T
F7S0tqbeak1DOaHIDQTnQ/x5Q7GEZjYV6E9oBd2dd3kv4NqY7gFgHUm9gZuAYxP535Q0ktSToLBw
s6Rngf8DNsovW9JISZMkTWptreaq4o7jOKVppKUc6nJMqAy3AxdJGgysYWZPl0l/B3AhMBxYJ0X+
E4GtJa0HHAGcm3e9CZhrZoNKZZKU7enavV/tf9OO46wy1EPAQVoariVkZguAB4ErKNEKSnAFcI6Z
Tcs7/whBFw5Jw4HZZjYvdu3dRhAzfSEGSSTLnwfMkHRMtJWkgW24JcdxnKpSTdkeSQfHZWtelXR2
tevacE4ocgMwkBROyMzeMbNLClwaBewqaSpwHnBy4tpNwInkdcUlOAH4uqQpwPPA4emr7jiO075U
qztOUhfgfwnj8AOA4yUNqGZd1UihfI1Ko3THZZ1V3tE31xbFhEapa0P8wTQQtfg9ZFVMWLTozbb8
iYeyK3jnLF0ys2h5kvYARpnZ5+PxjwHM7LdtrWOORm0JOY7jOEWwCrYy9CMsb5PjnXiuipWtoO/Q
t+pvwMiOtKtFmW7X2HaNVNfObtceG2HxzUmJbWTi2tHAmMTxV4FLq1m+t4Rqz8gOtqtFmW7X2Ha1
KNPtOggzG21mQxLb6MTlmcCmieNNaIPyTCHcCTmO4zjFeArYRtIWUanmOMK0l6rRiPOEHMdxnA7A
zJolfYeg19kFuMLMnq9mGe6Eas/o8kmqaleLMt2use1qUabb1QlmdjcrK85UDQ/RdhzHcWqGjwk5
juM4NcOdkOM4jlMzfEzIcToASU3A0Wb291rXJS2S+gGbk3hPmNnDtatRbZEkYBMze7tsYic13hLq
YKLg6YmSfh6PN5M0NKVtlzaU20XSxrG8zSRtljWvlGU92F75FyjvNElrZ7RNo6zeZiysVVVoBeCK
kLRGhem/GB1gpeWcDzwG/Az4QdzOas8y20qGZ3OrpEPT1tXCAHqmAXpJ20oaJ+m5eLyLpJ9lyauz
4U6o4/kzsAdhjSOA+QSBwDS8IumCSgUEJZ0GfACMBf4Vt7tS2m4j6RZJ0yW9nttK2ZhZC9AqqU+F
9VxP0k8kjZZ0RW5LYboB8JSkv0fF30q0tx6XdLOkQ9LaSdpT0lhJL8fnMaPcM4ncL+ksSZvGlX37
SuqbssxhkqYDL8bjgZLSLKZ4LOHv5ncVrgB8BLCdmR1iZl+M22EpbTOVGZ9pclXjtSXdm8Iu67P5
M/CVWNfzJG2XwuYZSbulSJfP5cCPgaWwbK2z40parCrUWjJiVduAZ+LPyYlzU1La9gL+B5gAPE6Y
dd07hd2rwDoZ6/sosD8wldA1Mwr4ZQq724G3gL8Cl+S2MjYTgPOBLwNfym0p6yng84TVdF8FfgNs
ldLuQIIie85u2zI2LxJUhdcnrFG1TprnC8wosL2e8v6eIMxcT/7dPJfStjfwjfg3MzH+3fQqY/Nv
oGcb/s6zlDk5zblqPpuYtg9wKkEjbQJwCtCtxO++GXgt/p+YBkxNUcZT+fcDPJv1+XamzceEOp6l
sVstvAHD4nmp1tg1s/mEL6rLJX0O+Bthgb9bgF+Z2atFTN8GPslY3x5mNk6SzOxNYJSkp4Gfl7G7
NW6VsIaZ/ShLJc3MJL0PvE94SawN3CJprJkV7Qaz8DYYC4yVtC9wHfCtuEzH2WY2sYDZJ2b27wx1
3KJSmzz7t/Maay0p7ebFv5EewJmE5et/IOkSM/tTEbOFwLOSxgGfJvI6vR3LbJW0mZm9BSBpc1IK
WWd9NrE79kSCJtpk4HrCqssnExbCzOfzafItwGxJW7H8//3RwHsZ8+pUuBPqeC4hLJq3vqRfEwQC
U/UNR+d1KOFLrT/we8J/mr0JfdXbFjF9HRgv6V+s+EL5Q4piP4195q/EmdMzgZ7ljMzs6ijzkavT
S2a2tIzZXZIOsTA5LjWSzgBOAmYDY4AfmNnSXL0pMRaT9xL6ADiNIEsyCLgZKOQ4HpR0AcHJJp/n
M2XquQbwPWAzMxspaRtCl1eartG3JQ0DTFI34AzghXJGkg4HRgBbA9cAQ81sVqzLdKCYQ7iDjPIs
bSjzp8Cjkh4itFD3Jp3GWtZncxuwHXAt8EUzyzmFmyRNKmQTP8SQtD6weoq65fg2YYLq9pJmElrB
J1Zg32nxyao1IPaT70/4jzbOzMr+h4l2rxNWlf2rmU3Iu3ZJsa9USb8odN7MzklR5m6E/9BrAb8i
dF38zsweL2M3HLgaeINwn5sCJ1uJ6CpJ84E1gSXEvvNQTetdpqxzCHIibxa4tkOp5yvpZcJL6Eoz
eyfv2o/M7PwCNoWCLszM9itTz5uAp4GTzGyn+FKeYGWWio+26wIXAwcQnud9wBmWt/JvAburCM9m
pecuaX8zG1fCttKPiGqUuS6wezx83Mxmpygv67PZ18wqCqCRdBjh429jYBahi/oFM9sxpf2aQFPs
1XBwJ9ShxJbM82ZWyQBx0r6nheXNs5bfE5Ytkd6uxC67r5jZS/F4W+AGM9u1imWUHNQ3szll7LsQ
HOr3Kygzc6i1pElmNkTSZDP7TDw3xczaZXn4eH/3m9m+GWyHU+FHRNYyJW1vZi9KGlzoerkWZqVI
OqrUdTMr2o0cu2n3I9zjZ2IX7olm9vUi6U80s+skfa9IWWl6Izo13h3XgZhZi8Ja7cv6vSvk55LO
BRYB9wC7AN81s+tKGUnaifC13zcezyZ8jZcVIpQ0hNBNkj9fZJcypt1yDiimfzl2lZQr7zBgn3g4
vkxX1dOEPnYBmwEfx/21CEERJcdg4u9jWLk65dm0SvohkGW+zxJJPVg+LrAVie68UkjagtBV2J8V
fw9FI9bi/bVK6mNmlY4J/h44KP8jAij5EZGxzO8Rut1+XyhLwku/KJKuJrR85sbjtYHfm9nXiph8
sUR2RumxzKVm9pGkJklNZvagpD+WSL9m/NmrRJpVGndCHc/awPOSngT+kztZ6mWS4CAz+6GkIwlf
qEcBDxMG00sxGvherushfuVeDqR5AV9PmCMyjZQBFJFJksYk6nYCYcGsokg6D9gtlglwhqQ9zezH
hdLnBvolXQ7clhtLkvQFQohxGp6VdAdh/Cf5+yj1Irpf0lnATXk2JVtewC8IHw+bSroe2JMwdpKG
fxIiDe+kst/DAmCapLF5dS0XYJDpIyJLmWY2Mv6suMUW2SXngGI+H0v6TLHEZnZKxnIA5sYehUeA
6yXNInGPBcr6v7j7ZzP7sA3ldlq8O66DUYhqWwkzeyiF7fNmtmN8ud9iZvek6c4plCZtN5CkR81s
r3LpCtitRhiMzdk+QviPWPTLX9JUYJCFiZ25rp3J5VpdkqaZ2c7lzhWxvbLAaSvxFY2kGUVstkxR
3jqEMQ+Rcswj2j1hZp9NkzbP7uRC583s6jJ2VxCcXfIjokup51KFMo8B7jGz+QoTOQcToj4nl7Gb
Agw3s4/jcV/goWK//yxdZJLOJIRvTydEDjYRnkkf4PoU408vEz4cbwJuzdXV8ZZQh5PG2ZTgTkkv
ErrjvqkQ3r04hd3rkv4foUsOQlROmsmVAL+ITi8/VLdUv3lu3ZETgEr7vNcCci2KtJNd340vreQL
8900hlm+iq3CUOsCYx25KKzNYtdsmjGPi2OAyX1UEJFX7sVfgm8SPiJyrZdHCJM7y2IhMrIHIQrw
pbIGy/l/ZnazpL0IQQYXAJcB5Zzv74GJkm4mOPejgV+XSJ+li2wT4I/A9oRegccITunOFC1gzGxb
BWWU44CfKkyuvbFcV/qqgLeEOpgYAZZ76N2BbsB/ykWAJez7EuaptMToqt5m9n4Zm7WBc1ixVTIq
zdeYpOsI//GeZ3k3UMmWQrR7FNjPzJaUKyNhczxwHiECUISxobPN7KYydn0JXV37EJ7tw4QJtWVf
DpI2IYQM7xlPPUIYX3inhE1FodZFoulymJWJqot5/JYQRv4aK/4eyo2XbAP8FhhAIqQ4TastK5K+
CFwIdDezLSQNIvw+SnY55wI24r1OM7O/JYM4ytjuCOS68x4ws+ltvY8i5XQHhhC6sveI21wzS61i
ohDN9wfgBDPLLMXVWXAnVEMkCTgc2N3Mzk5psxMrv1CuaZ8agqSXzCyNnEm+3TXADoS5JslxgZIt
I0kbEcaFAJ4s52DzbNc0s6L980VsxhIm/SZbiSeY2YElbDKHWmdF0qvAgEqcerR7lOCgLyIMyJ9C
CBEuONlY0t/N7MuSplFgomiKgJRcZOR+hMCSXBTgc2a2Uxm7uwjz0A4kdMUtIvwNpOk27kKQb0oG
bY89xa4AABXwSURBVJQM/skS7KEgRbUH4aNlD0LLfVq5FrWk3oQJu8cBWxHmCv7dzJ4uZbcq4E6o
Dqjga+8XhFncAwiTU78APGpmRxdJ/0czO1PSnRR+oZQNhohjJhdU+mWpCuYmqY0hujHCbQxBZmYz
SQOBb5jZt1LU89l851HoXN71ikKt1YaQ4EQe/wRGmtmscmnz7J42s12TY2S5c0XSb2Rm7ymoFRSq
60pzsQrk8biZ7Z73fKamGNtbAziY8FJ/JX6Q7Gxm95WxO43gaD8gKCUoVLVseVMIwR4rBN0U6jKX
NBrYkaD1+ARBjujxtGM7cRzxnwTHU0iFY5XFx4Q6mLwXUhOhaZ9mXAdCX/dAwmD9KZI2oHRkXO7r
/sKKK7qc3QkRZDMIYxFl/4PHr9JeZpZKdZk2hugSvvI/T5zhb2ZTJO1T2mQZH0k6kRB+DEFYtuQg
M5WHWrclJDjHWsCLkp5ixTGhch8SFSle2HLVgG9ZnoSSgrJ2Glml5yV9BegSuwNPJ4yflGNdYgSl
lqu8v5jC7gxCd2i531s+i83skpRpNwNWIyhwzATeAeaWtFiRLc3MJPVUG+f7dTa8JdTB5EVjNRMi
Zi5P84Ur6UkzGxq7O/YlfJW9YGUmv0o6w8wuLneuiG2mL2JJE81sj3L559msbmaLy50rYPeEmX02
bcskz3ZzwpjQHgSHMAE4zUqsGSPpIMLcqQGEQIE9gVOswtn3laCMUZVaWfGiN2GC7hNl7J4xs8F5
58q2ZmK6NQjP56B46l5ClFvJOVGJLkARupu3ICg1lFQjiGNuB5pZc7m65dl9BdiGlMEesft8R8J4
0DBgJ0IQzUQzK9jyT9gm5+oJ+JAw+fe5SurcGfGWUMczxsweS56QtCdBAqQckxSk7i8njEksICgU
l+NkgqxJkhEFzq2EZdfKyjL/ZgJhLKDcuXwyaYdFNslvTcTfR1EnZGb3xQ+BXKj1GVYi1FpVmDVf
ztmUoL+ZPUX4Wzkl1ucYQpdSobp+E/gWsJVCyHyOXqRrzQAcamY/JTiiXL7HEP4WimIrh9kPjnUp
R1ZtxJ0JwR77kQj2oEjL28IX+3OS5hIEgT8B/gsYSugOLEWhuXqjSTdXr1PjLaEOpsgX5krnUuTT
nxAZN7VEmuMJ66XsRYj6ytELaDWz/VOUk0krSxXMv5G0IdCP0LX4FcKLHcJX+2UpWnqZtMOibcW/
D0nj8p9doXOJa98ws/+rZJwsYfuome2lFaMqYXm3aDldvYruLw68r02IqEsGy8y3FNGGWcosk1fZ
+V5Znmu0Sx3sIel0lreAlhIccm6bZv+/vTMPlqyu7vjnyzAMWxGDA7JD0GSAqCBCwrBYEhIWgYih
2LRIUhjACkvYK4YYBlSiggEdAmFRCbIFMuDKjOxIQKrYxgIXCGGkikgFCKsMKbaTP86v593X033v
795+vbx+51P16r2+3b/+/fp19z33d5bvSbVtJeMb1+qNO7ETGhCS5uMf4PXarojXAbLSNIsnOjP7
VfuxDtyL16TMZXK85VW8F0oOn8ev+CdpZVUNqsoWamMvfGe2SVpnywi9AvxdxlzP47VB2TR5PySt
DqwJzJWnvReN5cYl67s4xcleMbPz6qzTUqGwmdWSfZGrRnwM2FhSMe6xDu4G7jbfy8DLkr4GvGBJ
aFPSOpL+sMyN13TOwvji+7AKvgOurPeqMjYlPIq7KXO8EFvgO7kTC3GzOvRSqzfWhBEaHKvhAeFV
mVwk9wqecNCVHk5+TwFP4fGOptTVymqtObv+xryg8l8lHWhmi+ouUA1SbWn2fhyN98bZCHeHFo3l
BWVrNK/rOgxPoqiFmgnf/hoP8v9pWmuLV4ETM8ZfxGQ36G86HJvqOYvvw1t4B+DKz4O8aPs0PF5T
LF2oSmjJTvYws46u1BocgdfqtdzRd6djM55wxw0YSZvnpLm2jfkbJk5+/83kk9+lZlZ6ApS0E24Q
tsZPvrPILJCVdCuuw/aP+I7qWWBHMyv1ZatZ/c3ZeNC8KER5spmV9ltSjVTbDmNXvB8pi2xtM3ul
Ysxx1r0xW9m48/Di5HbNuUrFBEnfxRMmagnfSpptmS0Y2sZ1Sl3PTUxYMWd6Dzctcxv3iqSb8f/p
KXiH1L8AnmvP7uswrrGEVjB1hBEaMHI14lNY+ao9p2q+6cnvAbxI7no8JfzP8RbWHYVB28auhRcN
1tXKalJ/s1K9VE4sQQ111dLYq/ET19vA/fju8mtmdk7JmE4aZ1+oMiZq2Icojf0x8CGglvBtSrJY
wIQKeiuWVKqYIOkG4E589wOeILC7mVUKw0q6E98NrYrviJ7Fi3k77oZSAktXMl5jqxZqhZGUdL+Z
7Vg2bhD0+tpmAuGOGzzX43pYl5HZgriFmS1MWWBbMNmAVSommNkTkmaZ2dvAtyQ9DJQaoeQG+oG5
uvE7eH+ZXJrU38ySNMdSKq+8FmdOxlyNdNUS25i3ov4UsBgPxj+I65Z1o5PG2UVUaJxZs74+78OV
AD7Xdtdu5LWH/gbuCnuQep+3z+BdgP8eT4i4jbwupwC/lf6nfwVcYWZntGXatTMfz0a8Bs/aU8lj
O9Ha6T0jaV/cLVjaawp68xDUoNfXNvaEERo8b5nZRdUPWxlJ38YlP5YycUIxvIVyGcvlmldLJX0F
P3mtUjWf9daP5gj8C34eE/U3VckKVwG3pcw64ckKOYavVqptG7Plad0HABeYtwWvcg+0/vf7ApeY
2Q/lfZ5KkSuLH8jKFxFnlQw7H/ismT3S9lwvAGfjRqaMl81scdXa2jGvWzu07rjEqnK1g4MppGmX
sAEu1dPK5vwh3gCxst9V4gspq+9k/DO3DnkxqAvo4CHInDOXXl/b2BPuuAEjaQHunriRyVftOWKb
v8Cv3Gu9afKCzGfxeMSJuEvtQjN7ImPsd3E3UN1+NI2QtDe+uzA85rWBmR1TMaaRrloaezyuAvBT
3KhsBlxpZruVjGmkcSZpCV5bMmlXYmadlCJaY7q6lTLTl7+EX+HfQF5B5mlm9hVJC+ks9VT5vid3
5edwSam/lrQlLv10YMbYOfgJ+xzgzKp4Zy9oQn6p6MbLktBqON/AXtt0InZCg6fVa+XUwjEDclSN
H8WvrGqliBYSIV7HM3TqcAN5sjIAdDt5FdZSdRL7nzT+IGAZGdlR1Eu1bV/P13G3U4unUhp6GQfj
GmfnmtlL6ar/1Iox4IWxe9dc4rtK7lsjY3zLRbhD4VjZLrFV5FvagLCC281sRWGqmT2J7wC7kk7Q
++In6S3w9+TGnMkaZkdCQw9BXXp5bTOBMEIDxmr2omljLvBzeVfWbP0wSfvh9T7twelK37d5b5j1
0t85nSGLJ68zqa4kbyVrHJZ+nscznVQjhlJbV00VKgaU9EEys+Xyjpq74lpib6XfVdwr6QPtrrUK
HpB0pJldWjyY4i2VCswN4lCL07imfYgA7pO0FPgWsLhq5y5XXH8/Lsp7ptWXsmnadfZw3Ogci3sI
NqXCWNZlCl7b2BPuuAGjmr1o2sY21Q97Am8F/kiuK0+ScANyLP5FFX6yXVgRwyg+R646+Dt43cSn
Wy5CSU9WZXAVxtf+v6g3FYMz8J3FPPNmZRsB15vZLl0e/yh+clwV1yp7knwx2PfgV81vMGF0dsAD
6Z+w6l5S78FjRxuZ2T6StgHmm1nHWFIxG1HSQjM7ruz5uzyHcJfqEXhbjuuAy83s8S6Pf4cJV28T
VYha2ZHyRoK1Ut2b0utrmwmEERowGk4vmjuAPaxCWqRtzEl4q4ijzGxZOrYlngW2xDIq/3PSq9Pj
DsADxLsAS4BrcY297F1jOtkW+xDVds3VmGspHid7yDJaFUh6Eej6/lpee4Td8Stq8MLV2zPXuhjf
kZxuZttKWhVXYe/W+rooAttIaqfDuq/Eu5n+FG9SOKWtDFRfiLRoaBflxKqC/hHuuMHzXjM7RF49
33LtZKVt9pBSehpwk6S7yBd4PBxXJn6+8PgnU9r1zTSo/O+GmX0H+I68JunjeGHu+pIuAm606n4y
B+PB3jvxK8yFkk41s38vGVMq4V8Ru3rDzKyVRZfWXcayHENTsZ478I6zdZlrZtdJ+mx6nrcklaVq
93xVKundeHHy4XiM7zi8zcZ2eCZaLy7pTtTNjix+3/rWYTbII4zQ4Knbi6ZI05TSL+KyK6vjxiuH
2dZBGdrMnkspzR3RZKHNNSW11Acq3Q/mXVGvBq6WV9ofhGeulRohPA14x9buJ8WwbgW6GiEmx1Oy
YlcFrpN0MfAuSUfibqdLSx6/fknsKUtFuwdeS0ah9XnbCc/Q68ZW8poeMVlJO6tRXOInuFLGATZZ
pukBSf9S+xVUcxDeryc3O9K6/B0MgTBCg2cB7nLaVNJVuAvqL3MHW4OiUzweUNpauQNlX+iu91lN
oc2S53kRl7q/JOPhq7S53/6XiiynYuBd0gk5gXhJJ+D1Tufj/ZxeAeYB/2Bmt5QMnYXr1A2jUPEk
fBfyXkn3AOtRrlW49RTMOa9b7NHMvjwFz99O3ezIbdPFkYA16lwoBVNPGKEBYzV70bTRNKX0Jkl7
Vrm12ti28OUs0mo4NkoskfQjJtQZDsGzkXLJvRreBDdAW+E6dffgRqkqS+2Z3GSOqcbMHkqJG/Pw
9+4xK9GS68VtqIJETScPc0bKdFNqZUeaWZZq/VQg6Z+Bq62th1gwQSQmDBhJ38ddTt9L7qc6YzfH
feyrUaPoNLnI1sK/oG8yJld8SpI2ZnaPvG36rumul3B9u//KfJ5aAfh0IbAD3gpifvp5ycy26fL4
vhVAViGXXtqXlWtoptwFKOk5SiRqqrI4e5h3ZIVI5eLDhwIb4lmC15jZw8Nd1WgRRmjApC/MIfiJ
4X48E+wHVtLCepAppdMJuXJBJ0mbDwBnm9n+JWMnxa6A5a27qDDQcomY+bgrdT5+Jf6IdemhJGld
y2wIN9VIugn4P1ZWGG/ag6dsrllMSNR8kAFK1AwyO7IJ6QLy0PSzBm6or+mWtj6TCCM0JNIX9o+A
I4G9K056jVJKJW1lZr+Ut0leiW4prNMF9Shp02C+S/CeNa/iV/r3Afel+NVIUpY6njF2Dbye7bEG
Ywcpv9OeHbkbUJodOUwkfQj4JvDBQboGR5WICQ2B9OXeH98RbU+1SGfTlNKTcOXjTtpkuQKfo0yv
kjZ12QxX9f5PXDvuadz1N8osbhAPRNL+wLm46/d3JG0HnFUV19FwJGqaZEcOlFSftQ++E9oDN5gL
hrikkSGM0ICRdB3wB3iG3AXAXRlFpI1SSs3sqPS7dguBaUJPkjZ1MbO9U03X7+PxoJOB98sVrX9i
ZnXSvAfFfcCN8oZ9deKBC/DP6Z34gKVyjbauaHgSNbWzIweFpJZ78mN4L6hr8QLwWvHgcSbccQNG
0l7ArSnFOnfM27j0h/Ar/Oz4ReE5GvUhGmXUo6RNj3NvgseEdgb2A95tZmU7s6EgaRleAJwt2ZTG
3WdmO7UpKJS69jQkiRpJ5+AxqGJ25CNmdlo/5qtD+u4eDSwaZbftMAkjNCBS9lZXzCxbqbrB3B37
EFWoAkwbmkraNJjneNzo7IzvKu4t/DySsaMdOPKOrB+tuzZJ38Ab2f0tLup5PF7A/JmpX2XvtGVH
3m1mI6FSPczMyOlCGKEBIW/UBrA+fhJrnSh3x7Xj9uvj3I36EAWTkfRPpNogM6vVTmNYSLocjyMu
Jl+yqSW0ezqwZzr0I7yFedcszlEhuR4PM7OrRmAtT1OuyN5PtYxpQcSEBkQrfVfSLbhBeCbd3hC4
vM/TN+pDFEzGzLpK74wwy9LPauRLNmFmy3EjlNMZdShIWgc4BtgYV4W4Jd0+BRdLHboRwtUypkRF
ZFyJndCAkfQLM9u6cHsV4OdmtlUf57wDF48s9iEyM/t4v+YMpjfpYukgM3sp3f5t4Foz22u4K5tA
3vX3RVyrbg/cy9BSIVk6zLW1qFsIPROJndDgua2DxEyt9NkGLCj83aqjOLTPcwYjQLoA6dSmuyo9
f27LAKXHvyhp/aleX49s2aoFk3QZvtPfbMRchsPQC5xWhBEaMGZ2rKRPAB9Jh+7FXWX9nPOuVCD3
SSbaZvdDzTgYPU4p/L06nmTwVsa4d4pKHanif9TcJis08MzsbUlPj5gBAt+hBSWEERoOv8KTE1oG
YVE/JlHvbbODaY6ZtddL3SNvD1/F6cB/yHtQtXbPR031+nqkKLJbVMQeGW3EYck1TSciJjQguhiE
U8xs8z7O2VPb7GD6I2ndws1VgA8DXzezeRlj5+Jq7+DyRLlq70GQTeyEBscvcYOwX8EgnNjnOf8M
j/3cIanVNjt81DOLB3E3mnA33DLg05lj5wAv4OeJbSRhZj/uyyqDGUvshAaEpANwg7ALLtlzLXCZ
mU11q+NOc7faZh+G68VdQUbb7GDmIunLeNLMzyi0zK7SjguCuoQRGjDDNgiaaJt9iJlF0HTMkXQM
3lupmGp9mJldWDHuMVzlObf1fBA0IozQEAmDEPQbSUvNbLu2Y5VSMpIW43VCv+nrAoMZT8SEhkgS
NLwk/QRBP5glSS3JptTHKkc5YTneRv42Jsv9jIXeYDA6hBEKgvFmCfBvki5Ot49Ox6r4XvoJgr4S
7rggGGOSLNTRTBRN3oInxFS2Eumls2oQ5BJGKAjGHEmrAfPwVO3HzOzNiiGTOquaWXZn1SCoy0h0
HwyCoD9I+ijejvwC4ELgcUkfKR3kLMA7q74E3lmVeq3lgyCLiAkFwXjzVWDPlkstKXdcgysnlPGm
mb3s3cxXMHJN+4LpT+yEgmC8mV2M6ZjZ48DsjHE/k/RJPLvudyUtxMV2g2BKiZhQEIwxkr6J72Cu
TIc+BcwysyMqxhU7qwrvrPr5EVSpDqY5YYSCYIyRNAfvNrprOnQ3cGEoIQSjQhihIBhzJK0HYGbP
ZTz2fDM7QdL36dwML7LjgiklEhOCYAyRZxScARxLiv1KehtYaGZnlQz9dvp9bn9XGARO7ISCYAyR
dBKwD3CUmS1Lx7YELgKWmNl5FePXAl43s3fS7VnAHDNb3t+VBzONMEJBMIZIehj4k/ZGdMk1d3OG
gOl9wB+3BEwlrZ3G7dyvNQczk0jRDoLxZHanTqgpLpSTor16UUE7/b3mFK4vCIAwQkEwrrzR8L4W
r0navnVD0oeB13teVRC0Ee64IBhDUhLCa53uwnc5pbshSTvi3X9/ncZsgPe9enCq1xrMbMIIBUHQ
EUmzceFTyBQ+DYK6hDsuCIIVSNpR0gYAyehsD3wR+KqkdYe6uGAsCSMUBEGRi0kxo6S2/SXgCuBl
ogNw0AeiWDUIgiKzzOyF9PchwCVmtghYJGnpENcVjCmxEwqCoMgsSa2L0z2A2wv3xUVrMOXEhyoI
giLXAHdJeh5Pyb4bQNL7cJdcEEwpkR0XBMEkJO0EbIgrJLyWjv0esLaZPTTUxQVjRxihIAiCYGhE
TCgIgiAYGmGEgiAIgqERRigIgiAYGmGEgiAIgqERRigIgiAYGv8PyUdpOiVAZz4AAAAASUVORK5C
YII=
"
>
</div>

</div>

</div>
</div>

</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>The above image shows how often the genres occur together, as a heatmap</p>

</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>Important thing to notice in the above plot is the diagonal. The diagonal corresponds to self-pairs, i.e. number of times a genre, say Drama occurred with Drama. Which is basically just a count of the total times that genre occurred!</p>
<p>As we can see there are a lot of dramas in the data set, it is also a very unspecific label. There are nearly no documentaries or TV Movies. Horror is a very distinct label, and romance is also not too widely spread.</p>
<p>To account for this unbalanced data, there are multiple things we can try to explore what interesting relationships can be found.</p>

</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h2 id="Delving-Deeper-into-co-occurrence-of-genres">Delving Deeper into co-occurrence of genres<a class="anchor-link" href="#Delving-Deeper-into-co-occurrence-of-genres">&#182;</a></h2>
</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>What we want to do now is to look for nice groups of genres that co-occur, and see if it makes sense to us logically? Intuitively speaking, wouldn't it be fun if we saw nice boxes on the above plot - boxes of high intensity i.e. genres that occur together and don't occur much with other genres. In some ways, that would isolate the co-occurrence of some genres, and heighten the co-occurrence of others.</p>
<p>While the data may not show that directly, we can play with the numbers to see if that's possible. The technique used for that is called biclustering.</p>

</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[25]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="kn">from</span> <span class="nn">sklearn.cluster</span> <span class="kn">import</span> <span class="n">SpectralCoclustering</span>
</pre></div>

    </div>
</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[26]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="n">model</span> <span class="o">=</span> <span class="n">SpectralCoclustering</span><span class="p">(</span><span class="n">n_clusters</span><span class="o">=</span><span class="mi">5</span><span class="p">)</span>
<span class="n">model</span><span class="o">.</span><span class="n">fit</span><span class="p">(</span><span class="n">visGrid</span><span class="p">)</span>

<span class="n">fit_data</span> <span class="o">=</span> <span class="n">visGrid</span><span class="p">[</span><span class="n">np</span><span class="o">.</span><span class="n">argsort</span><span class="p">(</span><span class="n">model</span><span class="o">.</span><span class="n">row_labels_</span><span class="p">)]</span>
<span class="n">fit_data</span> <span class="o">=</span> <span class="n">fit_data</span><span class="p">[:,</span> <span class="n">np</span><span class="o">.</span><span class="n">argsort</span><span class="p">(</span><span class="n">model</span><span class="o">.</span><span class="n">column_labels_</span><span class="p">)]</span>

<span class="n">annot_lookup_sorted</span> <span class="o">=</span> <span class="p">[]</span>
<span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="n">np</span><span class="o">.</span><span class="n">argsort</span><span class="p">(</span><span class="n">model</span><span class="o">.</span><span class="n">row_labels_</span><span class="p">):</span>
    <span class="n">annot_lookup_sorted</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">Genre_ID_to_name</span><span class="p">[</span><span class="n">nr_ids</span><span class="p">[</span><span class="n">i</span><span class="p">]])</span>
    
<span class="n">sns</span><span class="o">.</span><span class="n">heatmap</span><span class="p">(</span><span class="n">fit_data</span><span class="p">,</span> <span class="n">xticklabels</span><span class="o">=</span><span class="n">annot_lookup_sorted</span><span class="p">,</span> <span class="n">yticklabels</span><span class="o">=</span><span class="n">annot_lookup_sorted</span><span class="p">,</span> <span class="n">annot</span><span class="o">=</span><span class="bp">False</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">title</span><span class="p">(</span><span class="s2">&quot;After biclustering; rearranged to show biclusters&quot;</span><span class="p">)</span>

<span class="n">plt</span><span class="o">.</span><span class="n">show</span><span class="p">()</span>
</pre></div>

    </div>
</div>
</div>

<div class="output_wrapper">
<div class="output">


<div class="output_area">

    <div class="prompt"></div>


<div class="output_png output_subarea ">
<img src="data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAaEAAAFJCAYAAAAyv5ItAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz
AAALEgAACxIB0t1+/AAAIABJREFUeJzsnXm8XdP5/9+fezPLJMQUJIogRUIiNYuhqmiVUlQR1ear
39ZQ9e3op6i2VFst2ipqjlBzqqoIiZkkZJCYJUQMkZDIJMO9z++PtU6yc3LOPefue+49Q553XvuV
Paxn7bX32Xc/e631rM+SmeE4juM45aCu3AVwHMdx1l3cCTmO4zhlw52Q4ziOUzbcCTmO4zhlw52Q
4ziOUzbcCTmO4zhlw51QKyGps6R/SVog6Y5WOsdwSU82cfw/kk4uIp+Zkg4qbemaj6QTJD1U7nKs
C0i6QdJFbXzOfpJMUrs2POf5km5p4vg0ScOKyMckbVPSwjmAO6EWI2mspE8kdcw6dDSwMbCBmR1T
yGG0Bmb2ZTO7sS3OVeiPvRjMbKSZHVyqMjnpKMezWi7M7PNmNrYtzlUOx18NuBNqAZL6AfsABnw1
63Bf4DUzW1mic7XZ12M5KOf1lfrcufKr9d/PaX1q9hkyM19SLsB5wFPAH4H7E/svAJYDK4BFwPeB
z4CGuD0/pusI/B54B/gQuAroHI8NA94FfgJ8ANyc4/zD4/mvBBYArwAHJo6PBb6T2P4u8DKwEJgO
7Br3zwQOius3ABclbIYB7ya2fwLMjnm8ChwIHJJ1vZNj2h7AP4D3o81FQH1W2S8D5sVjw4EnE+cy
4DTgdWA+8BdA8Vg98AdgLjAD+EFM3y4e/2nyN8lx72bGa5kCLAPaAZsBdwEfxTzPSKQfCjwTy/F+
vOcdssr6/VjWGU3s+zMwC/gUmAjsk8jjfOCfwE3x/k4DhiSO7wq8GI/dAdye9VsdDkyKZXwa2Dlx
bBfghWh7O3Bb0jaRbgdyP6s9Yrk+At4GzgXq8tzbocCEeI0fAn+M+/vFe3Iy4ZmfC/wiYdcR+BPw
Xlz+BHSMx8YBX4/re8V8DovbBwKT8pTlfODOeM0L4z0YmPUcZJ79euDnwJsx7URgi8RvuU2ev6vh
xOcWEOGZnhOvfyqwIzCC8PexPN7Xf8X0TT1zmbLfEvP6Tr57W81L2QtQzQvwBvC/wOD4gG2cOHY+
cEtie9WDmth3GTAa6AV0A/4F/DYeGwasBC6Jf5ydc5x/eEzzQ6A9cCzBGfWKx1f9sQDHEBzBbvEP
ZRugbzyW/EO8gTxOCNiO8ALdLG73A7bOdb1x3z3A34H1gI2A54H/ySr76QQH0Dn7HhH+8O8HegJb
xj/UQ+Kx0wiOdHNgfeAREk6oiN9uJuGFvUU8dx3hpXMe0AH4HPAW8KWYfjCweyxrP4IzPyurrA/H
37JzE/u+BWwQ8/kR4QOjU+IefgYcSngh/hZ4Nh7rQHj5nxl/66MIL7SL4vFdCC++L0Tbk+M1dkzY
Zp6TownP61pOqIln9SbgPsJz2g94DTg1j/0zwIlxvSuwe+J5MeCaeM8HEj4AdojHLwSejc9Kb4Ij
/VXi2BVxPeMoLkkc+3Oespwfr/XoeO3nEF727XM8+/9HcBrbEf5GBhKa0zO/ZTFO6EuE56hnzGMH
YNM8f1uFnrlM2b8W03bOd2+refHmuJRI2pvQ5PZPM5tI+KP4ZjPsRfg6+qGZfWxmC4HfAMclkjUC
vzSzZWa2NE9Wc4A/mdkKM7udUDs5LEe67wC/M7PxFnjDzN4utryRBsJLbYCk9mY208zezHN9GxNe
pmeZ2WIzm0Nwusnre8/MrjCzlU1c38VmNt/M3gEeAwbF/d8gvHjeNbNPgIubeS0Al5vZrHju3YDe
ZnahmS03s7cIL8vjAMxsopk9G8s6k+Bc98vK77fxt1yab5+Z3WJm82I+fyDcz+0S6Z80swfMrAG4
mfAihNUO8PL4W99NcOoZRgB/N7PnzKzBQl/gsmi3O+EFnHlO7gTGF3uTJNXH+/AzM1sYr/8PwIl5
TFYA20ja0MwWmdmzWccvMLOlZjYZmJy4xhOAC81sjpl9RGhRyJxjHKvv974EB53Z3i8ez8dEM7vT
zFYQWi06Ee5JNt8BzjWzV+PfyGQzm9dEvrlYQXDU2xNq7S+b2ft50jb5zEWeMbN7zawxPkOF7m3V
4U4oPScDD5nZ3Lh9a9xXLL2BLsBESfMlzQcejPszfGRmnxXIZ7aZJVVo3yZU8bPZguAoU2NmbwBn
Eb7Q5ki6TVKuc0Fw0O2B9xPX93fCV26GWUWc9oPE+hLC1x+Ea0zaF5NXNkmbvsBmmbLG8v6cEFyC
pP6S7pf0gaRPCR8MGzaRX859ks6R9HKMmpxPaOZK5pN9vZ1iX8BmrP1bZ5f/R1nl3yLa5bJtzgfI
hoTfMmnzNtAnT/pTgf7AK5LGSzo863hTv2n2OTLP1zNA//hxM4hQM9tC0oaEJqrHmyj/qvtkZo2E
Zu7W+ht5lNBU+xfC38jVkrrnSd7kM5dd9kihe1t1uBNKgaTOhC/x/eJL6QNCU8dASQPzmGXLlc8F
lgKfN7OecelhZl2bsMlFn1iryrAloT09m1nA1kXkt5jgHDNskjxoZreaWaYWaITmwlxlnUX4Et8w
cX3dzezzyeyKKE8+3ic0xWXYIkUe2S/0GYmy9jSzbmZ2aDz+N0Kf27Zm1p3wslAT+a21T9I+wI8J
z876ZtaT0HyanU8u3mft3zp5zbOAX2eVv4uZjcpju2UT58r1rK4g/OZJ+9k5jc1eN7PjCR8clwB3
SlqvqYuLvJfjHO/FPJcQmq7OBF4ys+WE5rqzgTcTH4O5WHWfJNURnpvW/Bu53MwGAwMIDuP/Mody
nK+pZ24tmxbc24rFnVA6vkZomhpA+CobRGj7fQI4KY/Nh8DmkjrAqi+ya4DLJG0EIKmPpC81sywb
AWdIai/pmFiOB3KkuxY4R9JgBbaR1DdHuknAoZJ6SdqEUPMhlm87SQfEcPTPCE60MXF9/eIfObEJ
4iHgD5K6S6qTtLWk7CastPwTODPes56EIINVxJDxsc3I73lgoaSfxDFe9ZJ2lLRbPN6N0Bm8SNL2
wPdSlLkboR/sI6CdpPOAfF/J2TxDeOZ+IKmdpCMINYAM1wCnSfpC/H3Xk3SYpG7RdiWrn5Ojsmyz
yX5WGwj3+9eSusXn5mxCh/laSPqWpN7xGZ8fdzfmSpvFKOBcSb1jDee8rHOMIwSgZJrexmZt52Ow
pKNijfIswsdRrmasa4FfSdo23sOdJW2QI90k4ChJXRTGDp2aOSBpt/gbtCc4q89Y82/kc4l8Cj1z
a9GCe1uxuBNKx8nA9Wb2jpl9kFkI1fATlDuU8lFCtNMHkjJfbT8hBDc8G5t4HmHN/oFieA7YlvC1
+mvg6Fzt2GZ2Rzx+KyHy515Ch3k2NxPa6WcSnMjtiWMdCX0vcwlNKhsBP4vHMgNy50l6Ia6fROhw
nQ58Qoj02bSZ15ePa2L5phAixh4gvGgb4vEtCNF3RRFftIcTPihmEK7xWkJzGYQO7W8S7t01rHlf
iuW/hCbX1whNTZ9RZDNi/PI/ivDCm08IcLif8ELFzCYQoh+vJNzrNwgd5knb4cDHhACWu5s4Xa5n
9XTCS/Ut4EnCc3RdHvtDgGmSFhGiAY9ros8vyUWEyK8phACBF+K+DOMIjvzxPNv5uI9wzZ8Q+piO
iv1D2fyR4GwfInxw/IMQDJDNZYSgkA+BG4GRiWPdCc/HJ4TfeB5waTz2D0J/6nxJ9xbxzOUi7b2t
WDLhro5T1Uj6MnCVmfWN25MI4erN7ViuGiQ9R7jm68tdFsdJi9eEnKokNl8cGpum+gC/JISEA2Bm
g2rNAUnaT9Im8ZpPBnYm1Kwcp2pxJ+RUKyKE8H5CaI57mdCHUMtsR2gqnU8YY3R0E+G/jlMVeHOc
4ziOUza8JuQ4juOUjdoUxKswlo7+farq5jd+MDbV+dJ+WRy3sqmgnPyMbDe/cKIcdE6px3jP+xNS
2QF0bNc+ld2ylbmCqQpTzACgUvKLTYelsrtkTtuKZrevq09lt2TFslR2G3Tulspu3tKFqewA9ui9
fSq7J2aPafFjs2LuW0W/c9pv+Lm2fkzXwGtCjuM4TtnwmpDjOE6t0ZCu5l4OqromJGkDSZPi8oGk
2XF9vqTpReZxmqST4voNko6O62MlDWnN8juO47QKjY3FL2WmqmtCcRzIIAgyLcAiM/u9wmRz9xey
l9TOzK4qRVkk1ccR0I7jOGUlqPpUB1XthApQL+kaYE+C0OIRZrY06olNAvYGRkVtrUVm9vt8GUk6
mDAmpSNBZfcUM1skaSZBvuWLwO8IE4U5juOUlwqo4RRLVTfHFWBb4C9RtXk+8PXEsQ5mNiTO59Ik
UUjxXMLEV7sStK3OTiSZZ2a7mtltWXYjJE2QNOEf/636KT8cx6kmrLH4pczUck1ohplNiusTCbM6
ZmiO+OTuBLXsp6ISfgeCKnGTeZnZ1cDVkD5E23EcJxWN1dMzUMtOKDmgoIE11XAXNyMfAQ/HOTxy
0Zy8HMdxWp+GleUuQdHUcnNcqXgW2CvOG0Kcp6V/mcvkOI6TF7PGopdyU8s1oZJgZh9JGk4IYugY
d59LmBPGcRyn8qiiwAQXMG0DvrLl4alu8ufquhZOlIN7Pp2Wyq4h5VfR8sZ0Vf8u7ToWTpSD2QvT
z9BQp3QKJY1V8neyU69+qeymfjyzpOUoRH1dukYYpRRCapdSJmh5CwZ99kopFfTB/JdbLKOz7LUn
i35gO/bfu6yyPV4TchzHqTU8MMFxHMcpGx6Y0HpIMkm3JLbbSfpIUkGFhBx5DZJ0aGlL6DiOU2aq
aJxQ1TkhQkj0jpIyIddfJCgipGEQ0CwnJKWcf8BxHKetqCLtuGp0QgAPAIfF9eOBUQCS6iS9Lql3
YvsNSb0lHSPpJUmTJT0uqQNwIXBsFD09NoZfXyfpeUkvSjoi5jNc0mhJjwJjJN0k6WuZwkgamUnr
OI5Tbswail7KTbU6oduA4yR1AnYGngOwEPR+C3BCTHcQMNnMPgLOA75kZgOBr5rZ8rjvdjMbZGa3
A78AHjWzocD+wKWS1ot57QocbWb7Af8AhgNI6kHQp/t3soBJ2Z63F73TKjfBcRwnJ94c17qY2RSC
DM/xhFpRkuuAk+L6t4Hr4/pTwA2Svgvki9c8GPippEnAWKATsGU89rCZfRzPPw7YNta4jgfuMrM1
egLN7OqoTzekb9ctcRzHaTOqqDmumvs3RgO/B4YBG2R2mtksSR9KOgAYSqwVmdlpkr5AaMabKGlw
jjwFfN3MXl1jZ7DLlue5CfgWcBxwSkmuyHEcpxT4pHZtwnXABWY2NcexawnNcndk5viRtLWZPWdm
5wEfAVsAC4HkiLL/AqcrKpVK2qWJ898AnAVgZkVNoOc4jtMmeHNc62Nm75rZ5XkOjwa6sropDkL/
zlRJLwFPA5OBx4ABmcAE4FdAe2CKpGlxO9/5PwRezjqH4zhO+fHmuNbDzNbSsjGzsYQ+nAwDCQEJ
ryTSHJUju4+B3bL2/U+O/G8g1HxWIakLYc6iUUUV3HEcp62ogBpOsVSdEyqEpJ8C32N1hFxrnOMg
QoTcZWa2oFD6tNXNtBpwP+o6MJXdxZ++kMquc32HVHZzlhS8dSWnWjTg0jJv+adter60WnzdO3ZJ
ZffJ0kWp7Lp26JTKriXacQuXL01t22IqoIZTLDXnhMzsYuDiVj7HI0Df1jyH4zhOatwJOY7jOOXC
PDquPEhalLU9XNKV5SqP4zhOWaii6DivCSWQ1C456DR7u1g7x3GcsuLNcZWHpH6EsUUbEsYJnWJm
70i6AfgM2AV4StKnwNbA54B3JJ0C/A0YAqwEzjazx+Jsq0cRQsHrgf3a8nocx3HyUsIaTpRHexzo
SPAZd5rZLyVtRZBQ2wCYCJxoZsvjDNQ3AYOBecCxZjYzX/411RwHdI5jfiZF6Z0LE8euAG40s52B
kUByjNHmwJ5mdnbcHgAcZGbHA98HzMx2Ikj03Bh/FFhTT24NktpxM107znGctqS044SWAQdE3c1B
wCGSdgcuIUQIbwN8Apwa058KfBL3XxbT5aXWnNDSKEY6yMwGEQRKM+wB3BrXbwb2ThxbpawQGW1m
mfjKvQnqC8RxR28D/eOxVXpy2SS14/q5dpzjOG1Jw8rilwJYINPf3j4uBhwA3Bn33whkZhY4Im4T
jx+YUaHJRa05obRk68Jlbxdr5ziOU36aURNKttrEZUR2dpLqY+vSHOBh4E1gfqIv/F2gT1zvA8wC
iMcXkND3zGZdckJPE8RGIQxkfaJIuydieiT1J6hqv9qkheM4TjlpRnRcstUmLlevlZ1ZQ2xd2pwg
DL19qYq6Ljmh04FTJE0BTgTOLNLur0CdpKnA7cBwM1vWSmV0HMdpOa2kHWdm8wmam3sAPRMzTW/O
6hmuZxMEojMzUfcgBCjkpKai47J15ZKab2b2NqENM9tmeNb2+Vnbn5FjqoZcenL5OG5lj2KSrcWE
lBEuaeV3Zj77t1R2Vx58TSq7ezu/n8rumY9eKZwoD+3q8k0l1TQrG9t2Bsq0cjhbd9kkld2HS+an
suvSrmMqu226bpbK7nWbXThRDob23CaV3fgFb6ayA7ivy46pbVtMaaPjegMrzGy+pM7AFwnBBo8B
RxMi5E4G7osmo+P2M/H4o2b59bJqygk5juM4lHqc0KaEqOB6QuvZP83sfknTgdskXQS8SNDTJP5/
s6Q3CCLRx+XKNIM7IcdxnFqjiKi3YokzWa81t5qZvUXoH8re/xlwTLH512SfkKSvSTJJTXaeRVmf
zRLb10oa0PoldBzHaUWqaD6hmnRChEGlT8b/m2I4sMoJmdl3fJZUx3GqHrPilzJTc05IUlfCANNT
SbRFSvpJnFl1sqSLJR1NkOIZGRUWOksaK2lITH98ZiZWSZck8lkk6dcxn2clbdzGl+g4jtM0XhMq
K0cAD5rZa8A8SYMlfTnu/0KUnvidmd0JTABOiAoLq2agik10lxCi6QYBu0nKjAZeD3g25vM48N1c
hUgOABuz5I1WulTHcZwcuBMqK8cTQgaJ/x8PHARcb2ZLAPJJ7STYDRhrZh/FEb8jgX3jseXA/XF9
ItAvVwbJAWAHdkkXHuo4jpMKn8qhPEjqRai97CTJCOrWBtxRwtOsSMS8N1Bj99BxnBqgoW3HtbWE
WqsJHQ3cbGZ9zayfmW0BzCBoF50iqQusclYAC4FuOfJ5HthP0oYxNv54YFzrF99xHKcEeHNc2Tge
uCdr312EwVajgQlRhO+ceOwG4KpMYELGwMzeB35KGBE8GZhoZvfhOI5TDVSRE6qppiQz2z/HvuS8
QRdnHbuL4KQyDEscGwWMypFf18T6nayWMnccx6kMKqCvp1hqyglVKiPbpdPlWt6YbtRz5/oOqezS
asAd2rFQnEdufvZO24uRNzGtSUXRmHL8xvvL0z1rDSm/iBev+CyV3YzFH6Sy+3TZklR2ry5Jd75l
K1eksgO4tWO61+ueqc+4Gmss//ifYnEn5DiOU2uUULantXEn5DiOU2tUUU2oYGCCpF9ImiZpSuzA
/0ITaYdIujzf8dZCUj9JS2P5MksHSV+V9NMm7HpK+t/E9maSvI/HcZzqplYCEyTtARwO7GpmyyRt
COTtcDCzCQQVgnLwZpz5L8nouOSjJ/C/hInrMLP3CGHejuM41UsFOJdiKVQT2hSYm5lJ1Mzmxhc1
knaT9HTUUHteUjdJwyTdH4+vJ+m6eOxFSUfE/cMl3S3pQUmvS/pd5mSSDpH0QsxzTFP5FEM815Vx
fWNJ98S8J0vakxAtt3WsOV0aa1QvxfSdJF0f9eNelLR/ofI7juNUBFUkYFqoT+gh4DxJrwGPALeb
2ThJHQhTXR9rZuMldQeWZtn+gjCj3rcl9QSel/RIPDaIMD/FMuBVSVcAnwHXAPua2YzEgNKc+ZjZ
4qzzbR3HAAE8ZWbfzzp+OTDOzI6MA1C7EsYC7ZipQUnql0j/fcDMbKc4JcRDkvrnK7+ZzUqeTNII
YATATuvvyJZdt8x1fx3HcUpPFdWEmnRCZrZI0mBgH2B/4PbYxzIReN/Mxsd0n8Ja4a8HA1+VlBkY
2gnIvInHmNmCaDMd6AusDzxuZjNinh8XyOflrOLmao5LcgBwUsy7AVggaf0m0u8NXBHTvyLpbSDj
hHKVfw0nZGZXA1cDHL7lYeX/3HAcZ92himR7CkbHxRf2WGCspKmEucMnFpG3gK+b2RqDQWJgw7LE
rkL6aznzKTPNKb/jOE7bUivRcZK2k7RtYtcg4G3gVWBTSbvFdN0kZb+I/wucrlg9krTW9LBZPAvs
K2mrmD7THNfcfPIxBvhezKNeUg/ya8cBPAGcENP3J9S+KskROo7j5MQaG4teyk2hwISuwI2Spkua
AgwAzjez5cCxwBWSJgMPE5rJkvwKaA9MkTQtbufFzD4i9KHcHfO8PU0+TXAmsH+szU0EBpjZPOAp
hYnrLs1K/1egLqa/HRieCdBwHMepaBqt+KXMyCogOqLWOabvEalu8vOLZqQ635wlC1LZDemVbt6j
5+amqyD+YaNhqex++OFjqezWBb648c6p7MbMmZrKLq280Aad8zVANM28pQtT2dWllGtqyfuxc/uO
qew+XfxWi7WlFl/0raILvt65t5RVy8r7MhzHcWqNlTUUmOA4juNUGRXQzFYstTaf0CokbSLpNklv
Spoo6YHEOJ9kuqfLUT7HcZxWw6f3Li8xku4e4EYzOy7uGwhsDLwWt9uZ2UozK4VyuuM4TuXgNaGy
sz+wwsyuyuwws8lAvaQnJI0GpgNIWhT/HyZpnKT7JL0l6WJJJ0S5oKmSto7peku6S9L4uOxVhutz
HMfJSy2FaFcrO5J/QO2uwJlmtlbTHDAQOA3YATgR6G9mQ4FrgdNjmj8Dl5nZbsDX47G1kDRC0gRJ
E95aNDP1hTiO4zSbKgrRrsnmuAI8n5EGysF4M3sfQNKbBO08gKmE2hXAQcCAhERRd0ldzWxRMqOk
bE/aEG3HcZxU1JJsT5UyjfxTMmQLnyZJDkZtTGw3svpe1QG7m1m6eY0dx3Famwqo4RRLrTbHPQp0
jErWAEjamSDE2lIeYnXTHJKaEk11HMdpc6zRil4KIWkLSY9F5Zxpks7MOv4jSaYw3xwKXC7pDYXJ
UHdtKv+adEIWhjkfCRwUQ7SnAb8FPihB9mcAQ+LNnU7oQ3Icx6kcStsntBL4kZkNAHYHvi9pAAQH
RZjp4J1E+i8D28ZlBPC3pjJ32Z42oF2HPn6TS8j6nbumtv1k6aLCiRynjKxcPrvFMjoLf3Bo0e+c
blc+0KzzSboPuNLMHpZ0J0HP8z5giJnNlfR3YKyZjYrpXwWGZfrbs6nJmpDjOM46TTNqQslI3riM
yJdtnPhzF+A5hVmuZ8fhL0n6sOb8au/GfTmp1cAEx3GcdRZrKH78TzKStykkdQXuAs4iNNH9nNAU
1yLcCTmO49QaJY6Ok9Se4IBGmtndknYCtgImx+EqmwMvSBoKzAa2SJhvHvflpMXNcZIaJE2KUROT
Y6REVTTzSRok6dByl8NxHKeklDAwIcqg/QN42cz+CGBmU81sIzPrZ2b9CE1uu5rZB8Bo4KQYJbc7
sCBffxCUpia01MwGxcJuBNwKdAd+WYK8W5tBwBDggWINMppzrVckx3GcllFM6HUz2IugIDNV0qS4
7+dmlu+9+QBwKPAGsAQ4panMWxwdJ2mRmXVNbH8OGA9sCHQkhOcNIbQhnm1mj0mqBy4BDiEMBL3G
zK6QNJPVERZDgN+b2TBJ5xOqfp8jTLP9Q0Ko4JcJ1byvmNkKSYOBPxJmhJ1LmA31fUljgecIqgc9
gVPj9htA55jHb4EZBFmeTsBS4BQze1XScOComG89YYrzu83s3njNI4F/mtl9ue6RR8eVFo+Oc2qZ
UkTHLTj5wKLfOT1uHFNbk9qZ2VvRyWwEfCvssp0kbQ88FKdTOAXoBwwys5WSehWR9dYEJzIAeAb4
upn9WNI9wGGS/g1cARxhZh9JOhb4NfDtaN/OzIbG5rdfmtlBks4jOL0fAEjqDuwTy3QQ8BuCPhwE
zbmdzexjSfsRHOG9knoAewInJwsbI0xGAKi+B3V16zXrPjqO46TFVlbPd29rBybsTXAMmNkrkt4G
+hP0167KNGuZ2cdF5PWfWNuZSqiNPBj3TyU4tO0IwqUPx46yeiDZDnl3/H9iTJ+LHsCNkrYFDGif
OPZwppxmNk7SXyX1Jjipu7Kb6JIRJ14TchynTaki2Z6SO6HYHNcAzElhvpLVwRKdso4tAzCzRkkr
bHU7YkbXTcA0M9sjT94ZHbgG8l/3r4DHzOzIGA8/NnEsW3PuJkJN7zgKtHk6juO0KeWfoaFoShrF
FmsGVxFG0xrwBHBCPNaf0J/zKvAw8D+S2sVjmea4mcDguP51mserQG9Je8Q820v6fAGbhUC3xHYP
VocSDi9gewMhXh4zm97MsjqO47QapdSOa21K4YQ6Z0K0gUcIAp8XxGN/BepiE9rthECBZYQ5eN4B
pkiaDHwzpr8A+LOkCYQaS9GY2XKCcvYlMc9JhL6apniMMC3DpNiH9Dvgt5JepEAt0cw+BF4Grm9O
OR3HcVqdxmYsZca141IiqQuhP2pXM1vQVNr1uvRLdZNXNKSLBG9M+Zu2q6tPZZeYW6lZpL2+lnDB
psNS2V0058lUdm19jQdvMjCV3UMfZCuvFEddyt9+x179Utm9PH9W4UQ56NUpXUTl0pXLU9kB1Nel
+8b/aMGrLY5W+/jI/Yp+CfS6Z1xZo+OqYlBppREj514GrijkgBzHcdoaW1n8Um5cticFZvYI0Lfc
5XAcx8lJBTSzFUtFOiFJDYSmrnaEAaQnmtn88pbKcRynOrAqckKV2hy31MwGmdmOwMfA98tdIMdx
nKqhigITKtUJJXmGOBdFFMS7VNJLkqbGiDYkDZM0TtJ9kt6SdLGkEyQ9H9NtHdN9RdJzkl6U9Iik
jeP+8yVdJ2lstD8jc3JJJ8VZVCdLujnu6y3pLknj47JXm98Vx3GcPFhj8Uu5qcjmuAxR/udAgoIr
BP22QcAMZIb4AAAgAElEQVRAgjbdeEmPx2MDgR0INae3gGujTM+ZwOmEMT1PArubmUn6DvBj4EfR
fnuCLFA34FVJfyOoO5wL7Bn17DLjmf4MXGZmT0raEvhvPHey7Ktkezq070W7dsnhSI7jOK1HJTiX
YqlUJ9Q5qrX2IUShPRz37w2MMrMG4ENJ44DdgE+B8Rm5cElvEsYrQehb2j+ubw7cLmlToAOhvynD
v+MYpmWS5gAbAwcAd5jZXFhDXuggwviijG13SV3NbJU6ZlK2J22ItuM4ThqsoaxR182iUpvjMtND
9CXI8RTTJ7Qssd6Y2M7I+kDQsbvSzHYC/oc1pYGS9k1J+0C4b7vHfqtBZtYn6YAcx3HKSTU1x1Wq
EwLAzJYAZwA/ihI/TwDHSqqPEkH7As83I8ukLM/JTSWMPAocI2kDWENe6CFCEx9x/6BmlMFxHKdV
sUYVvZSbinZCAGb2IjAFOB64J65PJjiIH8eZ/IrlfOAOSRMJ8w0VOvc0wnQQ46IU0B/joTOAITFg
YTpwWjPK4DiO06pUU03IZXvaAJ/KoXJoX5+uG7R7h86p7OYtXZjKLi0d27UvnCgHy1auKHFJmqZT
uw6p7FY0ppSyakz3tu3cvmMqO4ClK5YVTpSDFSWY1G72HgcU/c7p88yjtTWpneM4jlNeGleWv5mt
WNwJOY7j1BjV1MBV8X1CLUGSSbolsd1O0keS7k+Z37WSBpSuhI7jOKWnmgITar0mtBjYUVJnM1sK
fJHV0XHNxsy+U7KSOY7jtBKV4FyKpaZrQpEHgMPi+vHAqMyBKNdzTmL7JUn9JK0n6d9RquelhDzQ
WElD4vohkl6Iaca04fU4juM0iVnxS7mp9ZoQwG3AebEJbmfgOmCfAjaHAO+Z2WEAknokD8YxStcA
+5rZjMT4IcdxnLLjNaEKwsymAP0ItaAHijSbCnxR0iWS9skxcd3uwONmNiOe4+PsDCSNkDRB0oTG
xsXpL8BxHKeZNDao6KXc1LwTiowGfk+iKS6ykjXvQScAM3sN2JXgjC6SdF5zT2hmV5vZEDMbUle3
XrpSO47jpKDRVPRSbtYVJ3QdcIGZTc3aP5PgbJC0K7BVXN8MWGJmtwCXZtIkeBbYV1ImvTfHOY5T
MZip6KXcrAt9QpjZu8DlOQ7dBZwkaRrwHPBa3L8TcKmkRmAF8L2s/D6KUzXcLakOmEOIvHMcxyk7
1dQnVNNOyMy65tg3Fhgb15cCB+cwnUmYIyjbdlhi/T/Af0pSUMdxnBJSCVFvxVLTTqhSSPtNUkXP
UdWwoiGd9lhaDbi0WnVpy7m8jTXg0rJs5fJUdm39N5FW/w3K+/dbypqQpOuAw4E5ZrZj3DcIuIrQ
j74S+F8ze15hkrU/A4cCS4DhZvZCU/mvK31CjuM46wwNjXVFL0VwA2HYSpLfEfrZBwHnxW2ALwPb
xmUE8LdCmbsTchzHqTFKOVjVzB4HsoehGNA9rvcA3ovrRwA3WeBZoGecyTovFemEJG0gaVJcPpA0
O66bpC9lpT1L0lrethV0406TdFIaW8dxnLakOSHayTGNcRlRxCnOIgRvzSIMf/lZ3N8HmJVI927c
l5eK7BMys3nAIAjSOsAiM/t9vDnHsWbQwHHAj3NkU2rduKvS2jqO47QlzQm9NrOrgaubeYrvAT80
s7skfQP4B3BQM/MAKrQm1AR3AodJ6gAgqR+wGWHa71w0pRvXS9K9cXbUZyXtLKlO0kxJPRPpXpe0
cVJnTtLWkh6UNFHSE5K2L/mVOo7jpKQNtONOBu6O63cAQ+P6bGCLRLrNKfDxX1VOKMrjPE/o/IJQ
C/qn5Z8e9jbgOEmdCLpxzyWOXQC8aGY7Az8ntGM2AvcBRwJI+gLwtpl9mJXv1cDpZjYYOAf4a/aJ
XbbHcZxyUeLAhFy8B+wX1w8AXo/rowljLyVpd2CBmb3fVEYV2RxXgFEE53Nf/P/UfAnNbEqsLeXS
jdsb+HpM92jsh+oO3E6I9rg+5n970khSV2BP4I4QjQjAWnMAJ6u47X16b8dx2pBSyvFIGgUMAzaU
9C7wS+C7wJ8ltQM+I0TCQXjPHgq8QQjRPqVQ/tXohO4DLosyO13MbGKB9BnduGHABkXk/wywTVTK
/hpwUdbxOmB+DE10HMepOEr51Wtmx+c5NDhHWgO+35z8q6o5DsDMFgGPEfTgsgVJc5FPN+4J4AQA
ScOAuWb2abyJ9wB/BF6OQRLJ838KzJB0TLSVpIEtuCTHcZyS4gKmrc8oYCBFOCEze9fMcunGnQ8M
ljQFuJjQ0ZbhduBbZDXFJTgBOFXSZGAaITbecRynIqgmAVPl79N3SkXaPqG2/mXqlO6BbPRnqOR0
ab9WN2NRLE8p97OysSGVXbVQDumstOdcsXx2iz3DE5scXXTR9/ngzrJ6omrsE3Icx3GaoKECajjF
4k7IcRynxmhMXQ9re6qiT0jSZZLOSmz/V9K1ie0/SDq7mXmeJalLKcvpOI5TCRgqeik3VeGEgKcI
Y3OIk8htCHw+cXxP4Olm5nkW0CwnJKm+medwHMdpcxqbsZSbanFCTwN7xPXPAy8BCyWtL6kjsAPw
gqT/kzQ+SvFcACBpPUn/ljRZ0kuSjpV0BkHu5zFJj8V0B0t6RtILku6Ig1KJMj6XSHoBOEbS2Lj9
vKTXJO3TxvfCcRynSaqpJlQVfUJm9p6klZK2JNR6niEos+4BLACmEgajbkvQMBIwWtK+QG/gPTM7
DEBSDzNbEJvv9jezuZI2BM4FDjKzxZJ+ApwNXBiLMM/Mdo32pwHtzGyopEMJo4fXEu6LYqsjAOrq
e1BXt17pb4zjOE4O0sVIloeqcEKRpwkOaE/CQNI+cX0Bobnu4Li8GNN3JTilJ4A/SLoEuN/Mcomd
7g4MAJ6KUjwdCI4uQ/Z4oYxw30SgX67CumyP4zjlohJqOMVSTU4o0y+0E6E5bhbwI+BTgs7bfsBv
zezv2YZR4udQ4CJJY8zswuwkwMNNyFNkK5Bm5vxtoLruoeM46wAlnN271amWPiEINaHDgY/NrCEq
avckNMk9TZhj6NuJvpw+kjaStBmwxMxuAS4Fdo35LQS6xfVngb0kbRNt15PUv60uzHEcp5Q0oqKX
clNNX/FTCVFxt2bt62pmc4GHJO0APBOb1BYRpHe2IcwA2AisIEzGBKGp7EFJ75nZ/pKGA6NioAOE
PqLXWvmaHMdxSk41tf+7bE8b4LI9TnNx2Z7Ssq7J9ty9yTeLLvpRH9zqsj21zi82HZbK7r5lM1PZ
zVv+aSq7rbtsksru/eXzU9lt1XHDVHYPfzgllR3AwZukEzwfN3d6KrvlK1eks0vpTBa+OzaV3db9
21aDt0/ndL/9KwtmpbLbsHOPVHZLG5YVTpSHbu3KNxa+IeUHZTlwJ+Q4jlNjVMIg1GJxJ+Q4jlNj
VFN0XM04IUkNhECF9oSxWjcBl5lZNX0UOI7jtJhKiHorlppxQsDSzJTbkjYiRNF1JygarEJSOzOr
pgHFjuM4zaKaQoWqaZxQ0ZjZHIJkzg/i9NvDJY2W9CgwRlJXSWOiTtxUSUcASOon6RVJN0RduJGS
DpL0lKTXJQ2N6YZGnbkXJT0tabsyXq7jOM4aNKr4pdzUpBMCMLO3gHpgo7hrV+BoM9sP+Aw4MurB
7U+Q9cn8HNsAfwC2j8s3gb2Bc4CfxzSvAPuY2S7AecBvss8vaYSkCZImTFz0RmtcouM4Tk4amrGU
m1pqjivEw1FlAUII/2+iwGkjQYdu43hshplNBZA0DRhjZiZpKqt14noAN0rallDzbZ99sqR23Pl9
T6im2rHjOFVOJdRwiqVma0KSPkdw9HPirqT+2wkEde3BsR/pQ6BTPJYcGNCY2G5ktdP+FfCYme0I
fCVh6ziOU3aqaT6hmqwJSeoNXAVcGWsx2Ul6AHPMbIWk/YG+zTxFD2B2XB/ekrI6juOUmkpwLsVS
S06os6RJrA7Rvpkw5UMuRgL/ik1sEwh9PM3hd4TmuHOBf6csr+M4TqtgVdQc59pxbUDnzn1T3eQV
KaVb0lJfl651tqEx3XeXa9WVns269kpll6O1oCg++WxRKrslK9LL4VQL5dSOu3KLbxX9R/KDWbe4
dpzjOI5TOqrpM82dkOM4To3h0XFtiKRFWdvDJV0Z10+TdFITtsMk7dnaZXQcx2lLShkdJ+k6SXMk
vZTYd2kc2D9F0j2SeiaO/UzSG5JelfSlQvlXvRNqCjO7ysxuaiLJMMKU4UUjyWuPjuNUNCUO0b4B
OCRr38PAjma2M2Hyz58BSBoAHAd8Ptr8VVJ9U5nXtBOSdL6kc+L6GZKmR899m6R+wGnADyVNkrRP
lO15NKYZI2nLaHuDpKskPQf8Lkr49I7H6qLX712my3Qcx1kDa8ZSMC+zx4GPs/Y9lNDgfBbYPK4f
AdxmZsvMbAbwBjC0qfxr4as+E5qdoRcwOke6nwJbmdkyST3NbL6kq4BFZvZ7AEn/Am40sxslfRu4
HPhatN8c2NPMGiQtIAx4/RNwEDDZzD5KnkzSCIJ+He3a9aJdu64lu2DHcZymWNmMPqHkuypydVR8
KZZvA7fH9T4Ep5Th3bgvL7XghFapZ0PoEwKG5Eg3BRgp6V7g3jx57QEcFddvJowHynCHmWWklq4D
7iM4oW8D12dnlJTtSRui7TiOk4bmvHCS76rmIukXhHGZI9PYQ403x2VxGPAXgpDp+BR9O6tkf8xs
FvChpAMIVc3/lKyUjuM4LaQRK3pJS/zgPxw4wVYPOJ0NbJFItjmr1WVysk44IUl1wBZm9hjwE4Ls
TldgIdAtkfRpQqcahOa2J5rI9lrgFtasITmO45Sd1taOk3QI8GPgq2a2JHFoNHCcpI6StgK2BZ5v
Kq91wgkRpnS4Jcr0vAhcbmbzgX8BR2YCE4DTgVMkTQFOBM5sIs/RBEe2VlOc4zhOOSllYIKkUcAz
wHaS3pV0KnAl4QP+4fj+vArAzKYB/wSmAw8C3y/0ke6yPSmRNIQwffg+hdK6bE9uXLan9LhsT+VQ
Ttme5kwfc/7bI122p9qQ9FPge4Qmu4oj7cu9S7uOqewWr/gslV05nIk7vtykdSa9O/csnCgHb6/4
MJVdWsrxu9fXNTk8plVZqep5XteV5riSYmYXm1lfM3uy3GVxHMfJppTNca2N14Qcx3FqjGqaT2id
rglJukzSWYnt/0q6NrH9B0lnl6d0juM46WiLEO1SsU47IeAponZcDOPekKB5lGFPQth2kyiwrt9L
x3EqhGpqjlvXX5xPE1QSIDifl4CFktaX1BHYAZgedeRekDRV0hEAUWfuVUk3Rbstcp3AcRynrVmJ
Fb2Um3W6T8jM3pO0MgqV7kmIhe9DcEwLgKnAEuBIM/tU0obAs5Iy2nTbAieb2bPZebt2nOM45aL8
rqV41mknFHma4ID2BP5IcEJ7EpzQU4Rw/99I2pfQ39cH2Djavp3LAYFrxzmOUz6qKTDBndDqfqGd
CM1qs4AfAZ8S1BBOAHoDg81shaSZQKdou3it3BzHccqMVVFdaF3vE4JQEzoc+NjMGszsY6AnoUnu
aYLO3JzogPYH+pavqI7jOIVpbe24UuI1odDvsyFwa9a+rmY2V9JI4F9Rd24C8EoZyug4jlM0lRB6
XSzrvBOK4nrds/YNT6zPZXUEXTY7FnOO9inlOxot3XdK945dUtlt03WzVHYzFn+Qyi6tbuG8pQtT
2QHs2KtfKrvXFjSpRp+XZSuXp7JL+wrp03nDVHbvLfq4cKIcpJXfWXDusFR2x1z7SSq7OSvSPTO7
dNwklR3APCufPl6DOyHHcRynXFRCM1uxuBNyHMepMTwwoZWQtImk2yS9KWmipAck9W+jc8+M44Qc
x3EqGg9MaAUUJjy5B7jRzI6L+wYSxuy8Vs6yOY7jVBJeE2od9gdWmNlVmR1mNhl4UtKlkl6KsjrH
AkgaJmmcpPskvSXpYkknSHo+pts6pust6S5J4+OyV9y/gaSHJE2LoqaK+y/MEj39taSmZmB1HMdp
U6qpJlRNTmhHYGKO/UcBg4CBwEHApZI2jccGAqcRNOBOBPqb2VDgWsJU3gB/JsyQuhvw9XgM4JfA
k2b2eUINbMu4/zrgJFglenoccEt2oSSNkDRB0oTlKz9NfdGO4zjNpcGs6KXcVE1zXBPsDYyKodYf
ShoH7EZQPBhvZu8DSHoTeCjaTCXUrCA4rgGJ6Y27S+oK7EtwcJjZvyV9EtdnSponaRdCU+CLZjYv
u1BJ2Z7u632u/L+04zjrDD5OqHWYBhzdTJtkoH5jYruR1ddeB+xuZmvMUa2mpwO+FhgObEKoGTmO
41QM3ifUOjwKdIzq1ABI2hmYDxwrqV5Sb0IN5vlm5PsQq5vmkDQorj4OfDPu+zKwfsLmHuAQQo3r
v82/FMdxnNajmvqEqqYmZGYm6UjgT5J+AnwGzATOAroCkwkDzX9sZh9I2r7IrM8A/iJpCuF+PE7o
R7oAGCVpGkFD7p1EWZZLegyYH5sBHcdxKoZqao5TWumUdZkYkPACcIyZvV4ofbsOfVLd5HYp5X5W
Nqbziz07rZfK7tNlS1LZNZbh2Wtfn+67K62EUkNj235rduvQOZXdwuVLS1ySpjl4k4Gp7L5Er1R2
V32WfhTHV7psncrupcYFqeweeOeBJvsCiuGovl8t+o/r7rdHt/h8LaGamuMqAkkDgDeAMcU4IMdx
qpe0DqjcmFnRS7mpmua4SsHMpgOfK3c5HMdx8lFNzXHuhBzHcWqMSgg4KJaKaI6T9DVJVkwwgaRr
Y5NYS8/ZT9I3E9tDJF3e0nwdx3HKjTXjX7mpCCcEHA88Gf9vEjP7TmwSayn9iCHYMd8JZnZGCfJ1
HMcpK41Y0Uu5KbsTiuoEewOnEiRwMrpvYyXdKekVSSOjgClx/5C4vijqxk2T9IikofH4W5K+GtP0
k/SEpBfismc89cXAPpImSfphPOf90aaXpHslTZH0bByPhKTzJV2XOIc7LcdxKo5Sy/ZI6pl4H78s
aY/4nnxY0uvx//UL57Q2ZXdCwBHAg2b2GjBP0uC4fxfCGKABhECAvXLYrgc8GvXdFgIXAV8EjgQu
jGnmAF80s12BY4FMk9tPgSfMbJCZXZaV7wUEOZ6dgZ8DNyWObQ98CRgK/FJS+1wXldSOa2xcXMx9
cBzHKQmt0Bz3Z8J7enuCJufLhHfoGDPbFhgTt5tNJTih44Hb4vptrG6Se97M3jWzRmASofksm+XA
g3F9KjDOzFbE9Uz69sA1kqYCdxCcWiH2Bm4GMLNHgQ0kZaYA/7eZLYvTfs8h6MethZldbWZDzGxI
XV268TeO4zhpKGVznKQeBCWaf0AYrG9m8wkViBtjshuBr6Upa1mj4yT1Ag4AdpJkQD1B9eDfrKn7
1kDusq6w1YHuq7ThzKxRUib9D4EPCd67jqC00BKKKZfjOE7ZaM74nyiFNiKx6+oowJxhK+Aj4Po4
h9tE4Exg44xANPABeT7IC1HumtDRwM1m1tfM+pnZFsAMYJ8SnqMH8H6sUZ1IcHQQmu+65bF5AjgB
Qv8UMNfMfD4Gx3GqgubUhJKtNnG5Oiu7dsCuwN/MbBdgMVlNb7EykCrKodxO6HiCGGiSuygiSq4Z
/BU4WdJkQn9OpoNmCtAgabKkH2bZnA8MjnpyFwMnl7A8juM4rUqDNRa9FMG7wLtm9lzcvpPglD7M
zN0W/5+TpqyuHdcGbNxj+1Q3efGKZYUT5aBTu5yxEgUZ2nObVHavLvkgld2shame2RZpzm28Xs9U
dnMWz09ll7akacW8+vXYJJXdjAXpfsO6pqc8ycvOvbZKZbe4Id3fxM0dNktl93Olb73fvX6DVHYX
zhzZYi23ffocWPSj98TsMQXPJ+kJ4Dtm9qqk8wlBYQDzzOxiST8FepnZj5tbVu/PcBzHqTFaYfzP
6cBISR2At4BTCC1p/5R0KvA28I00GbsTchzHqTFK7YTMbBIwJMehA1uad7n7hJqNpIY4wDSz9EuZ
z1mSupS2dI7jOOXHVbRbl6VmNqhwsoKcBdwCpJsMx3Ecp0KpBDmeYqm6mlAu8knz5JP/iXI7mwGP
xRlSkfS3qHAwTdIFibwvljQ9Svj8XlI3STMySgmSuie3Hcdxyk2jNRa9lJtqrAl1ljQprs8wsyNZ
Lc3zmaRtgVGsbr/cBfg88B7wFLCXmV0u6Wxg/6h8APALM/tYUj0wJurFzSZIAG0fpxfvaWYLJY0F
DgPuJejd3R2VGlaRHADWrdPGdO6QLirLcRynuXhNqHVZGvXeBkUHBE1L8xQj/wPwDUkvAC8SnNYA
YAFBYeEfko5iddPdtYToEOL/12dnlhwA5g7IcZy2pJr6hKrRCeUiKc0zBOiQOFZQZkfSVsA5wIFR
tPTfQCczW0kQKr0TOJyoU2dmTwH9oppCvZm9VOoLchzHSYtP5dD25JPmaYqkbE93gpLCAkkbA1+G
VdNM9DCzBwiObmDC/ibgVnLUghzHccpJNU1qV419Qrn4K3CXpJMItZVi5k64GnhQ0ntmtr+kF4FX
gFmEviMITuo+SZ0Ig9jPTtiPJEwdMapE1+A4jlMSWqIq0ta4bE9KJB0NHGFmJxZK265Dn3TCfikl
UdLSo1O6KSeWrVxROFEOlqaUJWrJE9u9Y7qhYSsbG1LZtfU1btI11bxifLDok5RnTMcpm+1ZOFEO
1k/53TypId31/b8VXVPZAdzSKd3f799n3tHiP/wdNhpa9CP08pzn2/ZFk0Wt1ITaFElXEJrsDi13
WRzHcbKphGa2YnEnlAIzO73cZXAcx8lHNTXH1UpgQsnkfLLyPC32MyHphtgE5ziOU9F4YEJ5KJWc
zyrM7KpS5uc4jtMWeE2oQigg5zNO0n2S3orSPCdIel7SVElbx3TnSzonK88DJN2b2P6ipOyJ+RzH
ccpGozUUvZSbWnJCnRNNcRmnkJHz2RU4Frg8kX4gcBqwA2FsUX8zG0pQQ2iqz+cxYHtJveP2KcB1
JbwOx3GcFlFNg1VrvTmuPXClpEEEtYT+iWPjzex9AElvAg/F/VOB/fOdJGrI3Qx8S9L1wB7ASdnp
ktpxqu9BXV268GfHcZzmUk1Db2rJCeUiKedTR9CBy5AcwNGY2G6k8H25HvhXzO+OKO+zBmZ2NWFA
bOpxQo7jOGmohBpOsdS6E+oBvGtmjZJOpjg5n4KY2XuS3gPOBQ4qRZ6O4ziloppqQrXUJ5SLvwIn
S5oMbE9xcj7FMhKYZWYvlzBPx3GcFtNoVvRSbmqmJmRma+lrmNnrwM6JXT+J+8cCYxPphiXWVx0z
s/MT+4dnZb83cE2LCu04jtMKVMJkdcVSM06oLZE0kVCr+lEx6ffovX2q87y++L1UdguXL01ld1+X
HVPZ3dox3WN000fjU9ktSanHBlBfl67yv3BZulng035nphXz6tYunTbeh6TTVquvS9fCPc/S/Yaz
LV1jxu71G6Syu6V+eSo7gD/9fb/Uti3F+4RqHDMbXO4yOI7j5KOa+oTcCTmO49QYldDXUyxlD0yQ
9DVJJilnm1VraLZFxYR0WvKO4zgVjk/v3TyOB56M/7cVw4BmOSFJXmt0HKcqqCbFhLI6oTh99t7A
qcBxcZ8kXSnpVUmPABvF/YdIuiNhO0zS/XH9YEnPRH24O2K+SJop6YK4f6qk7aO69mnAD6PEzz7Z
tS1JixLneELSaGB63PetqDE3SdLfJZVk7JHjOE6paGhsLHopN+WuCR0BPGhmrwHzJA0GjgS2AwYQ
5HAyNZZHgC9IyujfHAvcJmlD4qDRqBE3gTWn4Z4b9/8NOMfMZgJXAZeZ2SAze6JAGXcFzjSz/pJ2
iOfdK0oENQAn5DKSNELSBEkTPlg8u+gb4jiO01J8KofiOR74c1y/LW63A0aZWQPwnqRHAcxspaQH
ga9IuhM4DPgxsB/BYT2lMB12B+CZxDnujv9PBI5KUcbnzWxGXD8QGAyMj+fqTBBJXYukbM8+fQ4s
/y/tOM46QzUFJpTNCUnqBRwA7CTJCJI6BjQ1LcJtwA+Aj4EJZrZQwRs8bGb5+pQyAxIayH+9K4m1
Qkl1BEeWITkwQcCNZvazJsroOI5TVioh4KBYytkcdzRws5n1NbN+ZrYFMAOYBxwrqV7SpqypaD2O
0Dz2XYJDAngW2EvSNgCS1pOUVMvOxUKgW2J7JqGGA/BVgvp2LsYAR0vK9FP1ktS38KU6juO0HaVs
jov98a9KekPST0td1nI6oeNZu9ZzF7Ap8DohEOAmEk1rsYnufuDL8X/M7CNgODBK0pSYvpBEwb+A
IzOBCQT5nf2ixtwe5NGYM7PphP6nh+K5Ho7ldRzHqRgaGxuLXpoiBl79hfDOHQAcL2lAKctatuY4
M1trzh4zuzxX2qw0PyA0ySX3PQrsliNtv8T6BEJoNjEQYues5Lsn1nNqzMV9twO3Fyqn4zhOuShh
Y9xQ4A0zewtA0m2EgLLpJTtDcwY1+VL6BRjRlnblOKfbVbddNZW11u1aYyFMvjkhsYxIHDsauDax
fSJwZSnPX+4QbSfOvtqGduU4p9tVt105zul2bYSZXW1mQxLL1W15fndCjuM4Tj5mA1sktjeP+0qG
OyHHcRwnH+OBbSVtJakDQdlmdClPUO7Bqk4c0NqGduU4p9tVt105zul2FYAFkYAfAP8ljOW8zsym
lfIcip1NjuM4jtPmeHOc4ziOUzbcCTmO4zhlw51QGyKpTtI3yl0Ox3GcSsGdUBtiZo0E5e9USPpK
FFiteCT1kbSnpH0zSzNsu6Q4X4vuTZpzpjhHf0ljJL0Ut3eWdG5rnzctkjZoZnpJ2qJwysrA5wKr
DKrihVZjPCLpHElbRAHUXlFRvBiOBV6X9Lt806HnIorBPtbcgkrqLennkq6WdF1mKcLuEuApgs7e
/8XlnCLs9pQ0HXglbg+U9Ncii5v23qQ6p6SHJfVMbK8v6b8FzK4BfgasADCzKcTJHIs4317xnK9J
elTnOhIAAB+ySURBVEvSDElvFWG3raQ7JU2Pdm8VYxd5VmGSyEOjWn2TWIhyeqDIvLPLebqk9VPa
1kvaTNKWmaVI09clXdpcLTRJd0s6rDkfPdFBf0vSeXF7S0lDm3PemqXckhHr2kJQCs9e3mqGfXfg
fwjq4c8QRl53K8JuDNCjmWV9GrgE+Abw9cxShN2rQMcU9+Y5wsC4FxP7XmrNe5P2nMn0Te3LOj4+
Ox0wqchre4UgIrkRsEFmKcLuScI8WFOAvsD5wIVFnlPAF4FRwBvAb4D+BWxuBHZL8dtfFM/xT+AQ
YuRuEXanA3OBacDUuEwp0rYbQZH/6fjMjAC6F2F3EDASeBO4GNiuCJu/EYRAX47b62eeh3V9KXsB
fEnxo4UX0FmEKSj+Q1AdP72AzX3AO8A/gMszSwGbol6QOez+A3RNYfdc/D/5kp7cmvcm7TkJkyRu
mdjuC7xQxH3ZOpOOoMv1n+bcmxT3dGL8f2r2vmbmsz9hpPx8wpQqe+RJ9wphfq43o+NrjlMQ8CXC
NC0Zp7d1AZs3KMIZF3Hu/eL1LY6OdJsibHoApwGzoiM7BWifJ23mN0/9bNfq4oNV25jY93A24QU2
QtK2hC+p+4uwPYIwbcU2hGkuhprZnJjndOCKJszvZvUss8Vyv6RDzay5TSxLgEmSxrB6UkHM7IwC
drMk7QmYpPbAmcDLxZywBfcm7Tl/ATwpaRzh5bnP/2/vTMMkq6p0/X5VzCCTgIjIqIIMMtsiqCCi
IqKgIg3YoHYrfUUopka9DkwqIiAiXpBBQWQSmgZBZZa5xJahEPBagOBtwKEYC0QUKL77Y+2ojIyM
OFNGRBZZ+32efDLjxFmxd5yMOGvvNVJeD2xvIklxbUmPELvgj1UYC+BaSUcT/8P2a3p7idw/ktno
vpR0+AiwRJUBk0/oY0TRyr8Qu45LgA2BC4DVu4i9p8prd8O2Jf0Z+DOhyJYB/lPSVbZ7+VIfAmY3
GS/5hLYnlMdqwLHEDudthFmxZ1+yjmtzR5LbEtiTVK2/gxfSeE7yywPFfRTmE3Ky6pCR9GNiFb2H
7fXSTXK67Q0ryJ5BZCzf0OW5bWxfUyK/ECNfrJm2Xyg5/xlgceB5kh+DuFcsWSK3Z7fjtn9YIrcc
0e79XcSN/Upgmu3Hi+SS7Bk0uDbjHHM5RlqA3GL7sTKZJLc4MMX2M1XOTzLdfHq2/c4Suc0Ipbo0
cASxev+m7VsqjHkv8CPgdNsPdzz3OdtHFciuACzSNtH/KRlrGrAHYVo7DbjY9gstBWp7zR5y3wfW
An7GaOX8rZK3R/KNXQt83/b0jue+02vRJOmiNOaPgDNs/6ntuVttb9pFZnfCb7kxsdP6CPAl2xeU
zXOyk5XQkGl9SCXdYXujdOxO2xuUyE0FrnaXPkwVx92K+PD/gbjZvhbYs9tNux/UVXjjHGtc16bm
WGvb/p2kjbs9321nIuljts+SdEAPmcIbZroRf8T2+Y0m3YB0Tb9p+8Cach8gdhQrAbMIM+X/tb1u
idxhxCLi/3V57o22u+5OJR3S7bjtwyrMdQnbfy07r4vc1rabBPqsTfjnBFzT6z3Nb2Rz3PB5XtKi
jGzL16RtBdcL23MkvSRpKdtNzA/HAu+2PTON+wbC4bxJkVC6qbTCq6+raDbcig6FJ6lU4UlanTD5
rEbbZ9P2B4rkxnNtJP2Q2Pk8lR4vAxxr+5M9RA4gzG7HdpsK0G1nsnj6/Youz5Vi+yVJBxNO+1pI
2pQwHa7K6Gva2dSxc8w5yUxZlyOI3eHVtjeStDUFJkeNRIYe3/G4NY8nim7WLWUjaYn0uI5S+Yqk
rwLPAZcTjS73t31Wj7l+qNvfbXPpau5OCv0e22uTojAzI2QlNHwOIT7wr5V0NrAF4cuowl+BuyRd
RVsL8gq+FgiH6cw2mXuTD6Qnkr5BdKw9Ox2aJmkL218oGauRwgMuJgInLqW+vbzptXlTSwGl85+U
tFGvk21/Ov2uvOuyfXL680RHO/omXC3pIKKrb/v7e6JE7mwiRP4u6l/TGZIuIfw/7WMW+RZfsP24
IjF7iu1rJX274PzbCOUtYBXgyfT30kQgTTe/01wkrUeYxZZNjx8jTN1Vimy+2/bBknYiFkwfAm4A
uiohYIeC1zI9fK5Joc+UtEqZWXJ+JCuhIWP7Kkm3E6tFEavwSr4EmgUXtLhV0mmMfMF2J7ooFvE+
YENHkm1r13AHketSRG2Fl/i7K7R470HTazNF0jK2n4S5K/HS74WknYHLbT+jSDjdGDjC9h0FYjdL
+gOhSP6rNWZFdkm/9247ZmCNErlHbTctvb8I8Dijd3c9b7aJp9Ku5EbgbEmzaFNgndheHUDSqcBF
rSAYSdsBO1aY4ynAAS3zWNqFnwpU2cW1PpPbAxfYnq2CdCjbn6jwmr1YBrhH0n8zWqEX7vLnB7JP
aEj08iG0qBDl1HqdRYnIupmlJ4+WW5i4gW2ZDt1IrMx7mgIl/QbYqrXaTjfo68pMOYqE1pcYrfCm
Fpi4WnK7Aa8nggPqRIA1RtIewP8mVvsiHMZfs/2jErnf2H6TpC2JHJejga/Y/qcSuTcTCao7ElF7
5/Uy//QDSdsAuxJ5Yu3XtOliptc4+xFhyr8loiOnEP/3pYCzywI9JN1le/2yY13kxvhTq/hY03nf
IP4PzwFvJnZfP+31PxyPb0/SO3rIXF82z8lOVkJDokd0U4vSKKf0GjsAxwAL2V5d0oZE4mHhairZ
pM+0vXvNOe9KJONdS9yg3w583vaPS+RqK7wkdyQR8vp7RkxHVa/N64EjgXUYHZVVtlNA0rpEHgzA
L2z/toLMHcnncSSRg3NOe7BJBfnlgG8Bu9suLR+jhqH9ks4C1iaSOduvaeGCIMmuTIS2b5EO3Ujs
3B/ucu4xxO5jbcL0dzOhlKZXMBmiqDZxI6MXLm+3XRjynSLVbidMchD+p01s71Q2ZpJfFpidTGaL
Ecmqf+5x7l62Tx5PMERmLFkJvYyQdBthGrnOI5F1d9ter4LsTcA7bT9fc8xXE34hgP/u9QXtB5Lu
B9apO8ckexPhbzuOsN1/ggiD/koF2anAqxjtuC8LKf4pkXOzLWGKe464Pj1X4JKWBHYidkJrAhcB
59u+rcIcG4X2S5ppe62y1+8hexVwDqNv8Lvb3rZAZiFgU0IhbZ5+nrJdWBonKYNDiIWOCd/M4WUK
LAWSHMboBc+hVU2dyafUuXA5s4psHRTpDq2b7UKEKfBZl6Q7zA9kn9CQ6BZN005F88gLXezWVZ3N
DxA+iUsYbZMeY0LQ2DDk1sp3JUkr9TKPSTrf9kcl3cXIF24uZWY84G7CJDKr/O2MYVHb10iSI8z3
0KS0C5WQpH2Im99fgDnEjs9EpFQRHyXKyxxj+6mkrP+jROZOIvjicNu/LH1Ho1nT9i5pd4rtv6nI
gTHCdEnrVNnddWF526e3PT4jmd2KWJQon7RU+vkjsTMqJCmbaZIWt93Th9RF7kmgSmDOGNKOZitC
Cf2cKIt0E5HsXCRXO4rT9tzIyPR/+yAjOWbzNVkJDY9GkTUd3JP8JlOTOWZfwuRRhd+nnymUhwo3
CUOGqDYA8P6Kc+pkaeB3kn7NaP9FFedt08oA0wizVmlyagfLkQI7NFIwsyz8dg3blrSE6ueoNArt
J250MyQ9mM4XYY4rU7IAj0v6GBHZCOFb6nqdJJ0CrAs8Q9Tjmw58q8aO5K1EkuoSwCqSNgD2sv2Z
Hud/2/Z+ki6l+4KnymfmI8AGRCmdT0h6Fb0j49oZTxQnDvPTxUkJfr6u/GQjK6EhMc7Imhb7EDkf
/yDMJFcQeRmFJHPTK2yXVrKGkTBkYDvbf+94rUW6iLTkWpnjn7H9uQ65o4DPjZUaRVdbe0WmAYsR
ivkIwsezRwW5pmVffsZIaPEiRCjxTOJG3It1JbXCiSXpUSJh+O4K4x3K2ND+Kp+p91Y4pxefJHxC
xxHvdTq90wlWARYmavU9Quyen+pxbjeOI0r+XAJg+04Vt/9omQiPqTFGJ885crBeTKbSWUQSdxm1
ozg7LCFTCJPl33ucPl+RldCQGE9kTRvb2/4ioYhar7szEdnVk+R03aLonB5MJ/wdZcc62ZaxCme7
LsdGMc5IodVs/5rIF/oEzL02vyqRewC4TlKtsi9dIrk2Brqu2tvoFk58ChXCiW1fmcyLtUL7k2kS
dZTRqcjKnTuK9Dl6qMs4701mpnWJ93MgsJ6kJ4Bf2i5dYNh+qMPCOKfg3JYfbUPbx3fMcRpRZLWM
WxXtOE4l/G1/Jaqvl3F82sXUieJst4S8SOQlfbDCWJOerISGx7iy5hNfYKzC6XasG5UTDyWtCLwG
WFSRuNm6MyxJ7Da6Iul/ETfiNRXh3S1eQYHZUNJNtrfscN7CiOmoivO26bX5n/SzUPpphO3bJRWG
ZwOLu63ci+3rFHXkSpF0je1tiB1Y57Eiua5ldCjesbU4gbELjm7HgLlmprslPUXsLmcTptk3U77L
bVpIdk9StYU2Pt7lWLf5thYN35N0OREZ95simcT6RBTnO2mLOKS3mRrgNNs3tx9ICr2J/3NSkZXQ
kEihnVOBp20fV0dWkbj3PuA1ktrNAEsSq6oq1Ek8fA/xRV6ZuIG1lNDTRE5NL84h2hUcyWhb9zNF
UU62t0y/ayvo8V6bpmG1HTvaKcSN+Y8lYg9I+jKjo80KG8wl8+diwHIpEqx9QfCaClOtVUYnjbk5
sZtZvuN9Lgl0DSeXtG+SeStR7HZ6+vkBFQITiJYIxxPv6RFil7F3r5NTgMZuwOppcdXiFUBpSHh6
jblK3PYfOo8VsDPh36sTxVlLoc9PZCU0RJJZbFfC/l2HPxJO8A8QZoMWzwD7Vxy7sk/KUe36h5I+
bPvCGnKzgdmSjgeecKoSLWlJSf9ku6dpTKPra9VhXNdGUVL/YGJn0B6mW5ab1K4wXyR2KGXX6pNE
OHFL8d+YjhWxF9EfaSXi/bUvCL5bIgv1y+hA7AiXIO4P7e/zacKZ343ViF3n/m2+wcok02KdPLbp
wJ+IAJH2AJpniD5GPemDYq8cxdlEoc9v5DyhISPpOCJHoLMGWGlVAEkLOlWjTl+e11Y0H9RKPGyT
+TpRSbm9uOeBtr9UMtYdwMbJPEOKWrvVduGqT9JPiAZ0tetrtV+bmnJXEv+Lg4jV+J5EqZuyIIqh
Imkf20X9onrJXU1UBTiSuGHPIjqflvqhJK3a5lOaQjQqfLruHCrOs1Hx2oZjTWNEsT/CaMV+qu1C
5S7pOiKEvzSKU1EpYSvis/W9tqeeAS61fV/T9zFZyEpoyKhhX5gkex2x4l+AWBXPIhIWq6z4myQe
jqkAIOn2CspkhjuSKJXK3JTI3QBsBNSur5Xs64cyUi265U8qrJgg6Tbbm7TPT9KvbW/W4/zCOmw9
bkS1Zbq8RrdadV8tW7wkn9Nz1Cyjk2TPIW6ec4gb7pLA8baPLpOti6Q7ibDnUYVWy4JVJL2FWFy9
kdjBTaViEug4FHvtEjztCj0zmmyOGzIeX8+bpWw/LenfiDI8h3QEABTRJPFwqqSFncrtKPJUFq4w
1gPJR3BSevwZCnwfkl5HVCz4csdTbyNMLlX4PmF+u42CqKoutHZPf5K0PWHeW7bg/M2J6LBzici7
KgmjTWQ6+bLtCxS16t5F1Ko7CegZDJFMnD9Nn7mXiPYadVgnfd52J3x9nyeub9+VEM2L136XqEBx
ARH2vAcFHVHbsX1CCoZYjdG7r8Jk1YZRnAsrcqk6xypdfE52shIaMoq6ah9m7Ifx8AriCygy8z9K
W5h2RSonHrZxNnCNpNOJG+fHqXYj+3fgO8CXiOCHayhuff1t4Au2RzmwU3jv1wkFU8Zs25dVOK+T
r0paiggpPoFY7RftLFckQtBbjvGfAee6uHVAE5lOWop1e+AU2z9T9MLpicffg2rBFKm2I/BdR6fT
QZlOmoQ9t865X9JU23OA05M5uKzSO4qcrTWBGYxcX1NeMaHJ7usCwhx3GvUWSZOerISGz0+I0NXb
qJbx3s7hRILqTbZ/LWkNIjmwCt0SDwuDFWwflcwk70oyVxDmrkJszyJWp1V5VacCSq9zl6TVKr7G
tZKOJpz+lW9iHikAOpuRIqZF588hkkYvTwuKXYk8o8N6+RKayHThEUknE8rsqPQ6UyrIjacH1clE
PsudwA2SViX8JoOgSdgzwN8U9epmSPomsXOucl0gdk7rtHyXNWiy+3rR9kkl58yXZJ/QkFHFgqM9
ZF9ZxZbfTxR5QrsRYakPAhf2unFKOtj2NyWdQPdSKl1vfJLus/36Hs/db/t1FebZyNfWxCGeFMD2
hDJZjcjy/4HtR/op0yG/GFH94C7b96Ud8fq2ryyR27Pb8RQBWRtJC9iumhZQ53UbFa9NinEWEeyz
P+HzOtH2/RVkLwD2rRvNJ+lW25t2+BELK6hLOjTN8yJGL5IqhZNPZvJOaPhMl7R+t5V/BW6RNAM4
Hbisygqul0Jo0U0xKDqh7pp+HiOix1TBn9VKLixrltfJrZI+ZfvUjnn8G6PDrnsyDl9brTpgks4E
1iMKXh7mCiV3msh04ihYOouoFn0fERZeugu2/cMUho4rdnVVSXUPogVFv2lUvLbN2f8cEf5eh+WA
3yoazdWpVdhk99VaDLQXua3SlHDSk3dCQ0LS3cRNbgGicdsD1CwoKUmEaeyTRHuF84EzbN9bINO+
Ej6Mjsz1bitiSS8RIdz/2lpRSnqgQqRZo1WyonDkRcDzjCidTQl7+06u0D4ivcbXgZVsbydpHWBz
24X+JEm/ckkjuo7zX2LErFWpukMTmS6vcQhxTday/QZJKxHdQLuWY0qflUOAzxI3SBGK64Qy/6Mm
oG9OnbDnDrn3Ewm5nVGRVa5po0Zzaff1F+LzWWv3lRlLVkJDQtKTQM/eL3XDNxWZ72cR5YDuJJrN
Fda9KjMZtJ23I2Hz3oLwZZxHlB1ZvURubvi2pBNs71PpzYzIb03sGCASV39RQ/YyYof4RdsbSFqA
qI5c1plz6N1cm5B2wBsBt3ukl1TPsPe0i9kO+LTtB9OxNYiIustds2rHoBmHQrgf+BBhphzozUzS
Km6Qw5ZkGzUlnB/I5rjh8WBdRdOJpFcS+T3/QqzE9iF8CxsSTtJCJUGBWW7USfbFRKn5xYkii/sB
K0g6CbiowA/RHnpcu2Cqo65aUQfaIpazfb6kL6TXelFSlSikpg7xYfO8bbei01Rec+5fgG3dVuTU
9gMpQvJKCqp2aHT5ozFUDGqohe3r0262vYFiFdPcQ8DdTRRQgyi3i0lldiRdaPvDNYY7ndjlt5KE
HyG+s1kJTfQE5iNWKLCxV62i/Usi2XRHj650cKuk7/WQaYyjudg5wDmKagk7E5WweymhidxWP5uU
dOsm/RaqtWhoUgdsIjg/RcctLelThEn21ILzF3SXKtu2H01h10W0++HGmHAHgaSPEvlH1xGLmRMk
/Yft/ywRPRj4uaTrqVEFPVE3yq19kVXXl9O0KeGkJyuh4TGVqMc1ng/eWr1WfLaP6nZcoytTLyap
FWJbp0J1q4PlKemnF2srkmfF6EradRqpNeUAYle4pqSbgeXpXeesnfF0cx04ioTi6UQu1dZEiPRa
wFdsX1UgWqRUCxVuu59Q0n5NI+lq8kWinNCsNO7ywNVAmRL6GhGGvggNqqC7Xo6Re/xdhaZNCSc9
WQkNjz+VOYR7obayL90WT0XOWzeoTD0O3jjEsUbhaKXwDuIGLWCmq9WSG08310pI+j/AOe4o5V+R
lQkFtDZR0uZmQimVRQ1u0LbgGDUd6vUVGtbudkqH+e1xquX7rOSGKQ/Uj3JrXVMRbU7qLOgOZWxT
wo83nPekIgcmDImqQQE9ZB+loOxLmfN2fkBRomZ7xub7FJplmjrEa85tGmH2eTUR0Xiu7TtqvsZC
hMnorUQZoM2Bp2yv06959hi3tFZgn8Y5moiOa1X02AX4jUsKySblcXWBn7JIdqhRbslc3GpKeEs3
c+n8SFZCQ0LSsm6YmJZusK2yL2+iWdmXSY2knxPtkjsLYJaGEzd0iDeZ46qEMvpnYFHihntuUYh9
m+xShOLZIv1emogI60fb+M6xRplwgb+1nqKGCbfiWK8jKmbcrGiBvWV66imi0OrvK8x1cWIX+0KV
OY4nyq0pki4l/KuXJF9rJpGV0MsMjZR9OZpIfKxa9mVSUxSuXCLX6RB/G1DFIT4uFJUofgC8yXbP
vjKKopfrEqX/fwXcQqyinxzk/IaFpJ/SvW7g+sDXbe/QXXJcY7anEtSNcms65juI3d32RC7UeURx
2b8Peux5newTepmgsWVfvkMkeM6TJCfsKrZnDmnIyyS9u4FZpqlDvDYpd2k7Yie0DaH4Di0RW4Wo
XH4fEdb7MLFLmCw0qhsoaW3bv5PUq9V4UZ7XeKLcGpHMu9cnq8Y7gU8Ri5C+7SpfrmQl9DJAfSj7
Mkwk7QAcQ9jaV5e0IXB4P539XbgFuEjRfK2SWSbR1CFeGUktU+r7iF5J5xFJpKVmGdvvTaG86xL+
oAOB9RQVxn9pe+Dh0wNm6YLnFi147gCiMvuxXZ4ry/MaT5RbY9LCbAdiR7Qx9VtrTEqyOe5lgPpQ
9mWYSLqNuAlc15bdf1dZ9YJxjvkgkVhbK3O+h0P8LtsH93Fuc4g23ReOx4ym6I67BaGM3g+80nbR
TXyeR9K5wC/cvW7gtrZ3GcCYc4jvkwhFNzCfV9uY5wNvJiLkfgxcb7u0VuH8QFZCmb4j6Rbbb2mP
CGzqs6kx5g3AVk2+2B0O8Rtt99XMOc7IyH0JpfNWYoc3ve3nrpf7jUz9qRtYuzHdsJH0HiKKL/cS
6iAroUzfkfR9opHd54kGfvsSGfz/PsAxzyDs+5dRP3O+/XWmALvaPruPc3uYgsrTRXOU9C1SbpBr
thx4OaGGdQPVozGdB1BaqAlpgdMT2/81rLnMq2SfUGYQ7EM4/P9BhKVeARR2Ae0DD6afhaiQOS9p
SWBv4DVEpYWr0uODiIKwfVNCRLWMRknDtnuWeppMuHndwKaN6YZFK7pvBWI321KuWxO72fleCeWd
UGa+RNJPgCeJenzbEDcJAdNsz+jzWENJ+JwfUcPGdMNG0dl2j9Y8FU0Jz7D9nomd2cSTd0KZvpO+
cDvbfio9XgY4b5BfOEVn1W7dXHtFSa3RCpSQdBpRsmWVAeVt5EKVg6NbYzrb/uAEzqkbK3coyr8Q
PZDme7ISygyC5VoKCKL4qaQVBjzmQW1/L0L4oooa7M2tK2d7jqSHB5g4uM2AXjczOs+qlWz8zxMz
lUKukXQFo6Mwa5camoxkJZQZBC+1l0ZJ5WoGave13VnQ8+a0Ou5Fe4HP9oKUfQ/TbVquKVOOow/R
RsBuRFuOB4G+tzUZL7Y/K2kn4O3p0HRgxQmc0jxDVkKZQfBF4CZFj5fW6vTTgxxQ0rJtD6cAmxAF
KbtSVConM+8j6Q1EAvCuwGNE7o1sbz2hEyvmD0RwQktZXjihs5lHyIEJmYEgaTmiYjAMoWJwSlY1
ofReJL7kh9u+aZDjZiaGlMB9I/CvrarXkh6wPZQyPFXpoSwPsp39QYm8E8oMioWBJ4jP2DqSsH3D
oAazXdbaPDO5+BDh+7lW0uVEKaR5MQDkd4SyfH+bstx/Yqc0b5F3Qpm+I+kowvF6DyNtFTzI2nGS
9iZK/7dH5O1q+8RBjZmZeCQtTpRr2pUoFXUmcFGT/kKDQNKOhLLcgijZcx5wWl40jZCVUKbvSJpJ
tCgYWvtiSTNsb9hxrHG5nMzLj7Tw2BnYxfY8FZE4ryvLiaSv1YIzmcQDwIJDHnNqqjYNzG0EWFo5
ITN5sP2k7VPmNQUEYPtZ2+ek/kgrA3cAhV1j5xfyTijTdyRdCGxA1I9rr+M2sHpeqRr2qsDJ6dBe
wEO2DxzUmJlMZvxkJZTpO5L27Hbc9sD6p6TCo3sxkhh6FWF7z1WLM5l5mKyEMgNhAjqrImkhYC0i
VHum7RdKRDKZzASTfUKZvpM6q84gooGQtKGkSwY85lZEC+zvAicC90p6e6FQJpOZcPJOKNN3enRW
vdv2esWS4x5zt9bOKyUJnmt7k0GNmclkxk/eCWUGwQu2Z3ccG3QH0AXbTX+272X4EXqZTKYmuWJC
ZhDcI2k3Imz69URn1ekDHvPW1JLhrPR4d+DWAY+ZyWTGSTbHZfqOpMWIIqbvJkqpXAEcMcBWCUha
mOiMumU6dCNw4jATZjOZTH2yEspMGiQtD2D70YmeSyaTqUZWQpm+IenbtveTdCndu5z2vXZcqpJw
CPBZRnycc4ATbB/e7/EymUx/yT6hTD/5Ufp9zBDH3J8oDrmZ7QcBJK0BnCRpf9vHDXEumUymJnkn
lOk7qVjjc7ZfSo+nAgvb/tsAxroD2LazX1EyzV2ZC5hmMvM2OUQ7MwiuARZre7wocPWAxlqwW8O8
5BfKIdqZzDxOVkKZQbCI7b+2HqS/Fys4fzw83/C5TCYzD5B9QplB8KykjW3fDiBpE+C5AY21gaSn
uxwXsMiAxsxkMn0i+4QyfUfSZkQHyT8SymBFotHYbRM6sUwmM8+RlVBmIEhakKhoDbmidSaT6UH2
CWX6hqTNJK0IkJTOxsDXgGMlLTuhk8tkMvMkWQll+snJpGCA1EbhG8CZwGzglAmcVyaTmUfJgQmZ
fjLV9hPp712AU2xfCFwoacYEziuTycyj5J1Qpp9MldRa2GwD/KLtubzgyWQyY8g3hkw/ORe4XtJj
REj2jQCSXkeY5DKZTGYUOTou01ckvQV4NVEy59l07A3AEq28oUwmk2mRlVAmk8lkJozsE8pkMpnM
hJGVUCaTyWQmjKyEMplMJjNhZCWUyWQymQnj/wMWumRuCEvqvQAAAABJRU5ErkJggg==
"
>
</div>

</div>

</div>
</div>

</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>Looking at the above figure, "boxes" or groups of movie genres automatically emerge!</p>
<p>Intuitively - Crime, Sci-Fi, Mystery, Action, Horror, Drama, Thriller, etc co-occur. 
AND, Romance, Fantasy, Family, Music, Adventure, etc co-occur.</p>
<p>That makes a lot of intuitive sense, right?</p>
<p>One challenge is the broad range of the drama genre. It makes the two clusters highly overlapping. If we merge it together with action thriller, etc. We will end up with nearly all movies just having that label.</p>

</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p><strong>Based on playing around with the stuff above, we can sort the data into the following genre categories - "Drama, Action, ScienceFiction, exciting(thriller, crime, mystery), uplifting(adventure, fantasy, animation, comedy, romance, family), Horror, History"</strong></p>
<p>Note: that this categorization is subjective and by no means the only right solution. One could also just stay with the original labels and only exclude the ones with not enough data. Such tricks are important to balance the dataset, it allows us to increase or decrease the strength of certain signals, making it possible to improve our inferences :)</p>

</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h1 id="Interesting-Questions">Interesting Questions<a class="anchor-link" href="#Interesting-Questions">&#182;</a></h1><p>This really should be a place for you to get creative and hopefully come up with better questions than me.</p>
<p>Here are some of my thoughts:</p>
<ul>
<li>Which actors are bound to a genre, and which can easily hop genres?</li>
<li>Is there a trend in genre popularity over the years?</li>
<li>Can you use sound tracks to identify the genre of a movie?</li>
<li>Are top romance actors higher paid than top action actors?</li>
<li>If you look at release date vs popularity score, which movie genres have a longer shelf life?</li>
</ul>
<p>Ideas to explore specifically for feature correlations:</p>
<ul>
<li>Are title length correlated with movie genre?</li>
<li>Are movie posters darker for horror than for romance end comedy?</li>
<li>Are some genres specifically released more often at a certain time of year? </li>
<li>Is the RPG rating correlated with the genre?</li>
</ul>

</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h1 id="Based-on-this-new-category-set,-we-will-now-pull-posters-from-TMDB-as-our-training-data!">Based on this new category set, we will now pull posters from TMDB as our training data!<a class="anchor-link" href="#Based-on-this-new-category-set,-we-will-now-pull-posters-from-TMDB-as-our-training-data!">&#182;</a></h1>
</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[70]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="c1"># Done before, reading from pickle file now to maintain consistency of data!</span>
<span class="c1"># We now sample 100 movies per genre. Problem is that the sorting is by popular movies, so they will overlap. </span>
<span class="c1"># Need to exclude movies that were already sampled. </span>
<span class="n">movies</span> <span class="o">=</span> <span class="p">[]</span>
<span class="n">baseyear</span> <span class="o">=</span> <span class="mi">2017</span>

<span class="k">print</span><span class="p">(</span><span class="s1">&#39;Starting pulling movies from TMDB. If you want to debug, uncomment the print command. This will take a while, please wait...&#39;</span><span class="p">)</span>
<span class="n">done_ids</span><span class="o">=</span><span class="p">[]</span>
<span class="k">for</span> <span class="n">g_id</span> <span class="ow">in</span> <span class="n">nr_ids</span><span class="p">:</span>
    <span class="c1">#print(&#39;Pulling movies for genre ID &#39;+g_id)</span>
    <span class="n">baseyear</span> <span class="o">-=</span> <span class="mi">1</span>
    <span class="k">for</span> <span class="n">page</span> <span class="ow">in</span> <span class="nb">xrange</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span><span class="mi">6</span><span class="p">,</span><span class="mi">1</span><span class="p">):</span>
        <span class="n">time</span><span class="o">.</span><span class="n">sleep</span><span class="p">(</span><span class="mf">0.5</span><span class="p">)</span>
    
        <span class="n">url</span> <span class="o">=</span> <span class="s1">&#39;https://api.themoviedb.org/3/discover/movie?api_key=&#39;</span> <span class="o">+</span> <span class="n">api_key</span>
        <span class="n">url</span> <span class="o">+=</span> <span class="s1">&#39;&amp;language=en-US&amp;sort_by=popularity.desc&amp;year=&#39;</span> <span class="o">+</span> <span class="nb">str</span><span class="p">(</span><span class="n">baseyear</span><span class="p">)</span> 
        <span class="n">url</span> <span class="o">+=</span> <span class="s1">&#39;&amp;with_genres=&#39;</span> <span class="o">+</span> <span class="nb">str</span><span class="p">(</span><span class="n">g_id</span><span class="p">)</span> <span class="o">+</span> <span class="s1">&#39;&amp;page=&#39;</span> <span class="o">+</span> <span class="nb">str</span><span class="p">(</span><span class="n">page</span><span class="p">)</span>

        <span class="n">data</span> <span class="o">=</span> <span class="n">urllib2</span><span class="o">.</span><span class="n">urlopen</span><span class="p">(</span><span class="n">url</span><span class="p">)</span><span class="o">.</span><span class="n">read</span><span class="p">()</span>

        <span class="n">dataDict</span> <span class="o">=</span> <span class="n">json</span><span class="o">.</span><span class="n">loads</span><span class="p">(</span><span class="n">data</span><span class="p">)</span>
        <span class="n">movies</span><span class="o">.</span><span class="n">extend</span><span class="p">(</span><span class="n">dataDict</span><span class="p">[</span><span class="s2">&quot;results&quot;</span><span class="p">])</span>
    <span class="n">done_ids</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="nb">str</span><span class="p">(</span><span class="n">g_id</span><span class="p">))</span>
<span class="k">print</span><span class="p">(</span><span class="s2">&quot;Pulled movies for genres - &quot;</span><span class="o">+</span><span class="s1">&#39;,&#39;</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">done_ids</span><span class="p">))</span>
</pre></div>

    </div>
</div>
</div>

<div class="output_wrapper">
<div class="output">


<div class="output_area">

    <div class="prompt"></div>


<div class="output_subarea output_stream output_stdout output_text">
<pre>Starting pulling movies from TMDB. If you want to debug, uncomment the print command. This will take a while, please wait...
Pulled movies for genres - 12,14,16,18,27,28,35,36,37,53,80,99,878,9648,10402,10749,10751,10752,10770
</pre>
</div>
</div>

</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[&nbsp;]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="n">f6</span><span class="o">=</span><span class="nb">open</span><span class="p">(</span><span class="s2">&quot;movies_for_posters&quot;</span><span class="p">,</span><span class="s1">&#39;wb&#39;</span><span class="p">)</span>
<span class="n">pickle</span><span class="o">.</span><span class="n">dump</span><span class="p">(</span><span class="n">movies</span><span class="p">,</span><span class="n">f6</span><span class="p">)</span>
<span class="n">f6</span><span class="o">.</span><span class="n">close</span><span class="p">()</span>
</pre></div>

    </div>
</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[36]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="n">f6</span><span class="o">=</span><span class="nb">open</span><span class="p">(</span><span class="s2">&quot;movies_for_posters&quot;</span><span class="p">,</span><span class="s1">&#39;rb&#39;</span><span class="p">)</span>
<span class="n">movies</span><span class="o">=</span><span class="n">pickle</span><span class="o">.</span><span class="n">load</span><span class="p">(</span><span class="n">f6</span><span class="p">)</span>
<span class="n">f6</span><span class="o">.</span><span class="n">close</span><span class="p">()</span>
</pre></div>

    </div>
</div>
</div>

</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>Let's remove any duplicates that we have in the list of movies</p>

</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[37]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="n">movie_ids</span> <span class="o">=</span> <span class="p">[</span><span class="n">m</span><span class="p">[</span><span class="s1">&#39;id&#39;</span><span class="p">]</span> <span class="k">for</span> <span class="n">m</span> <span class="ow">in</span> <span class="n">movies</span><span class="p">]</span>
<span class="k">print</span> <span class="s2">&quot;originally we had &quot;</span><span class="p">,</span><span class="nb">len</span><span class="p">(</span><span class="n">movie_ids</span><span class="p">),</span><span class="s2">&quot; movies&quot;</span>
<span class="n">movie_ids</span><span class="o">=</span><span class="n">np</span><span class="o">.</span><span class="n">unique</span><span class="p">(</span><span class="n">movie_ids</span><span class="p">)</span>
<span class="k">print</span> <span class="nb">len</span><span class="p">(</span><span class="n">movie_ids</span><span class="p">)</span>
<span class="n">seen_before</span><span class="o">=</span><span class="p">[]</span>
<span class="n">no_duplicate_movies</span><span class="o">=</span><span class="p">[]</span>
<span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="nb">len</span><span class="p">(</span><span class="n">movies</span><span class="p">)):</span>
    <span class="n">movie</span><span class="o">=</span><span class="n">movies</span><span class="p">[</span><span class="n">i</span><span class="p">]</span>
    <span class="nb">id</span><span class="o">=</span><span class="n">movie</span><span class="p">[</span><span class="s1">&#39;id&#39;</span><span class="p">]</span>
    <span class="k">if</span> <span class="nb">id</span> <span class="ow">in</span> <span class="n">seen_before</span><span class="p">:</span>
        <span class="k">continue</span>
<span class="c1">#         print &quot;Seen before&quot;</span>
    <span class="k">else</span><span class="p">:</span>
        <span class="n">seen_before</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="nb">id</span><span class="p">)</span>
        <span class="n">no_duplicate_movies</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">movie</span><span class="p">)</span>
<span class="k">print</span> <span class="s2">&quot;After removing duplicates we have &quot;</span><span class="p">,</span><span class="nb">len</span><span class="p">(</span><span class="n">no_duplicate_movies</span><span class="p">),</span> <span class="s2">&quot; movies&quot;</span>
</pre></div>

    </div>
</div>
</div>

<div class="output_wrapper">
<div class="output">


<div class="output_area">

    <div class="prompt"></div>


<div class="output_subarea output_stream output_stdout output_text">
<pre>originally we had  1782  movies
1700
After removing duplicates we have  1700  movies
</pre>
</div>
</div>

</div>
</div>

</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>Also, let's remove movies for which we have no posters!</p>

</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[73]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="n">poster_movies</span><span class="o">=</span><span class="p">[]</span>
<span class="n">counter</span><span class="o">=</span><span class="mi">0</span>
<span class="n">movies_no_poster</span><span class="o">=</span><span class="p">[]</span>
<span class="k">print</span><span class="p">(</span><span class="s2">&quot;Total movies : &quot;</span><span class="p">,</span><span class="nb">len</span><span class="p">(</span><span class="n">movies</span><span class="p">))</span>
<span class="k">print</span><span class="p">(</span><span class="s2">&quot;Started downloading posters...&quot;</span><span class="p">)</span>
<span class="k">for</span> <span class="n">movie</span> <span class="ow">in</span> <span class="n">movies</span><span class="p">:</span>
    <span class="k">if</span> <span class="n">counter</span><span class="o">%</span><span class="k">10</span>==0:
        <span class="k">print</span><span class="p">(</span><span class="n">counter</span><span class="p">)</span>
    <span class="nb">id</span><span class="o">=</span><span class="n">movie</span><span class="p">[</span><span class="s1">&#39;id&#39;</span><span class="p">]</span>
    <span class="n">title</span><span class="o">=</span><span class="n">movie</span><span class="p">[</span><span class="s1">&#39;title&#39;</span><span class="p">]</span>
    <span class="k">if</span> <span class="n">counter</span><span class="o">==</span><span class="mi">1</span><span class="p">:</span>
        <span class="k">print</span><span class="p">(</span><span class="s1">&#39;Downloaded first. Code is working fine. Please wait, this will take quite some time...&#39;</span><span class="p">)</span>
    <span class="k">if</span> <span class="n">counter</span><span class="o">%</span><span class="k">300</span>==0 and counter!=0:
        <span class="k">print</span> <span class="s2">&quot;Done with &quot;</span><span class="p">,</span><span class="n">counter</span><span class="p">,</span><span class="s2">&quot; movies!&quot;</span>
        <span class="k">print</span> <span class="s2">&quot;Trying to get poster for &quot;</span><span class="p">,</span><span class="n">title</span>
    <span class="k">try</span><span class="p">:</span>
        <span class="c1">#grab_poster_tmdb(title)</span>
        <span class="n">poster_movies</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">movie</span><span class="p">)</span>
    <span class="k">except</span><span class="p">:</span>
        <span class="k">try</span><span class="p">:</span>
            <span class="n">time</span><span class="o">.</span><span class="n">sleep</span><span class="p">(</span><span class="mi">7</span><span class="p">)</span>
            <span class="n">grab_poster_tmdb</span><span class="p">(</span><span class="n">title</span><span class="p">)</span>
            <span class="n">poster_movies</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">movie</span><span class="p">)</span>
        <span class="k">except</span><span class="p">:</span>
            <span class="n">movies_no_poster</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">movie</span><span class="p">)</span>
    <span class="n">counter</span><span class="o">+=</span><span class="mi">1</span>
<span class="k">print</span><span class="p">(</span><span class="s2">&quot;Done with all the posters!&quot;</span><span class="p">)</span>
</pre></div>

    </div>
</div>
</div>

<div class="output_wrapper">
<div class="output">


<div class="output_area">

    <div class="prompt"></div>


<div class="output_subarea output_stream output_stdout output_text">
<pre>(&#39;Total movies : &#39;, 1782)
Started downloading posters...
0
Downloaded first. Code is working fine. Please wait, this will take quite some time...
10
20
30
40
50
60
70
80
90
100
110
120
130
140
150
160
170
180
190
200
210
220
230
240
250
260
270
280
290
300
Done with  300  movies!
Trying to get poster for  The Dark Knight Rises
310
320
330
340
350
360
370
380
390
400
410
420
430
440
450
460
470
480
490
500
510
520
530
540
550
560
570
580
590
600
Done with  600  movies!
Trying to get poster for  Back to the Future
610
620
630
640
650
660
670
680
690
700
710
720
730
740
750
760
770
780
790
800
810
820
830
840
850
860
870
880
890
900
Done with  900  movies!
Trying to get poster for  Eastern Promises
910
920
930
940
950
960
970
980
990
1000
1010
1020
1030
1040
1050
1060
1070
1080
1090
1100
1110
1120
1130
1140
1150
1160
1170
1180
1190
1200
Done with  1200  movies!
Trying to get poster for  Good Boy!
1210
1220
1230
1240
1250
1260
1270
1280
1290
1300
1310
1320
1330
1340
1350
1360
1370
1380
1390
1400
1410
1420
1430
1440
1450
1460
1470
1480
1490
1500
Done with  1500  movies!
Trying to get poster for  Joe Dirt
1510
1520
1530
1540
1550
1560
1570
1580
1590
1600
1610
1620
1630
1640
1650
1660
1670
1680
1690
1700
1710
1720
1730
1740
1750
1760
1770
1780
Done with all the posters!
</pre>
</div>
</div>

</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[74]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="k">print</span> <span class="nb">len</span><span class="p">(</span><span class="n">movies_no_poster</span><span class="p">)</span>
<span class="k">print</span> <span class="nb">len</span><span class="p">(</span><span class="n">poster_movies</span><span class="p">)</span>
</pre></div>

    </div>
</div>
</div>

<div class="output_wrapper">
<div class="output">


<div class="output_area">

    <div class="prompt"></div>


<div class="output_subarea output_stream output_stdout output_text">
<pre>0
1782
</pre>
</div>
</div>

</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[75]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="n">f</span><span class="o">=</span><span class="nb">open</span><span class="p">(</span><span class="s1">&#39;poster_movies.pckl&#39;</span><span class="p">,</span><span class="s1">&#39;w&#39;</span><span class="p">)</span>
<span class="n">pickle</span><span class="o">.</span><span class="n">dump</span><span class="p">(</span><span class="n">poster_movies</span><span class="p">,</span><span class="n">f</span><span class="p">)</span>
<span class="n">f</span><span class="o">.</span><span class="n">close</span><span class="p">()</span>
</pre></div>

    </div>
</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[33]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="n">f</span><span class="o">=</span><span class="nb">open</span><span class="p">(</span><span class="s1">&#39;poster_movies.pckl&#39;</span><span class="p">,</span><span class="s1">&#39;r&#39;</span><span class="p">)</span>
<span class="n">poster_movies</span><span class="o">=</span><span class="n">pickle</span><span class="o">.</span><span class="n">load</span><span class="p">(</span><span class="n">f</span><span class="p">)</span>
<span class="n">f</span><span class="o">.</span><span class="n">close</span><span class="p">()</span>
</pre></div>

    </div>
</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[77]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="n">f</span><span class="o">=</span><span class="nb">open</span><span class="p">(</span><span class="s1">&#39;no_poster_movies.pckl&#39;</span><span class="p">,</span><span class="s1">&#39;w&#39;</span><span class="p">)</span>
<span class="n">pickle</span><span class="o">.</span><span class="n">dump</span><span class="p">(</span><span class="n">movies_no_poster</span><span class="p">,</span><span class="n">f</span><span class="p">)</span>
<span class="n">f</span><span class="o">.</span><span class="n">close</span><span class="p">()</span>
</pre></div>

    </div>
</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[34]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="n">f</span><span class="o">=</span><span class="nb">open</span><span class="p">(</span><span class="s1">&#39;no_poster_movies.pckl&#39;</span><span class="p">,</span><span class="s1">&#39;r&#39;</span><span class="p">)</span>
<span class="n">movies_no_poster</span><span class="o">=</span><span class="n">pickle</span><span class="o">.</span><span class="n">load</span><span class="p">(</span><span class="n">f</span><span class="p">)</span>
<span class="n">f</span><span class="o">.</span><span class="n">close</span><span class="p">()</span>
</pre></div>

    </div>
</div>
</div>

</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h1 id="Congratulations,-we-are-done-scraping!">Congratulations, we are done scraping!<a class="anchor-link" href="#Congratulations,-we-are-done-scraping!">&#182;</a></h1>
</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h1 id="Building-a-dataset-out-of-the-scraped-information!">Building a dataset out of the scraped information!<a class="anchor-link" href="#Building-a-dataset-out-of-the-scraped-information!">&#182;</a></h1>
</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>This task is simple, but <strong>extremely</strong> important. It's basically what will set the stage for the whole project. Given that you have the freedom to cast their own project within the framework I am providing, there are many decisions that you must make to finalize <strong>your own version</strong> of the project.</p>

</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>As we are working on a <strong>classification</strong> problem, we need to make two decisions given the data at hand -</p>
<ul>
<li>What do we want to predict, i.e. what's our Y?</li>
<li>What features to use for predicting this Y, i.e. what X should we use?</li>
</ul>

</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>There are many different options possible, and it comes down to you to decide what's most exciting. I will be picking my own version for the example, <strong>but it is imperative that you think this through, and come up with a version which excites you!</strong></p>

</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>As an example, here are some possible ways to frame Y, while still sticking to the problem of genre prediction -</p>
<ul>
<li><p>Assume every movie can have multiple genres, and then it becomes a multi-label classification problem. For example, a movie can be Action, Horror and Adventure simultaneously. Thus, every movie can be more than one genre.</p>
</li>
<li><p>Make clusters of genres as we did in Milestone 1 using biclustering, and then every movie can have only 1 genre. This way, the problem becomes a simpler, multi-class problem. For example, a movie could have the class - Uplifting (refer Milestone 1), or Horror or History. No movie get's more than one class.</p>
</li>
</ul>
<p>For the purposes of this implementation, I'm going with the first case explained above - i.e. a multi-label classification problem.</p>

</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>Similarly, for designing our input features i.e. X, you may pick any features you think make sense, for example, the Director of a movie may be a good predictor for genre. OR, they may choose any features they design using algorithms like PCA. Given the richness of IMDB, TMDB and alternate sources like Wikipedia, there is a plethora of options available. <strong>Be creative here!</strong></p>

</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>Another important thing to note is that in doing so, we must also make many more small implementation decisions on the way. For example, what genres are we going to include? what movies are we going to include? All these are open ended!</p>

</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h2 id="My-Implementation">My Implementation<a class="anchor-link" href="#My-Implementation">&#182;</a></h2>
</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>Implementation decisions made -</p>
<ul>
<li>The problem is framed here as a multi-label problem explained above. </li>
<li>We will try to predict multiple genres associated with a movie. This will be our Y.</li>
<li>We will use 2 different kinds of X - text and images. </li>
<li>For the text part - Input features being used to predict the genre is a form of the movie's plot available from TMDB using the property 'overview'. This will be our X.</li>
<li>For the image part - we will use the scraped poster images as our X. </li>
</ul>
<p>NOTE : We will first look at some conventional machine learning models, which were popular before the recent rise of neural networks and deep learning. For the poster image to genre prediction, I have avoided using this for the reason that conventional ML models are simply not used anymore without using deep learning for feature extraction (all discussed in detail ahead, don't be scared by the jargon). For the movie overview to genre prediction problem we will look at both conventional models and deep learning models.</p>
<p>Now, let's build our X and Y!</p>

</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>First, let's identify movies that have overviews. <strong>Next few steps are going to be a good example on why data cleaning is important!</strong></p>

</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[38]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="n">movies_with_overviews</span><span class="o">=</span><span class="p">[]</span>
<span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="nb">len</span><span class="p">(</span><span class="n">no_duplicate_movies</span><span class="p">)):</span>
    <span class="n">movie</span><span class="o">=</span><span class="n">no_duplicate_movies</span><span class="p">[</span><span class="n">i</span><span class="p">]</span>
    <span class="nb">id</span><span class="o">=</span><span class="n">movie</span><span class="p">[</span><span class="s1">&#39;id&#39;</span><span class="p">]</span>
    <span class="n">overview</span><span class="o">=</span><span class="n">movie</span><span class="p">[</span><span class="s1">&#39;overview&#39;</span><span class="p">]</span>
    
    <span class="k">if</span> <span class="nb">len</span><span class="p">(</span><span class="n">overview</span><span class="p">)</span><span class="o">==</span><span class="mi">0</span><span class="p">:</span>
        <span class="k">continue</span>
    <span class="k">else</span><span class="p">:</span>
        <span class="n">movies_with_overviews</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">movie</span><span class="p">)</span>
        
<span class="nb">len</span><span class="p">(</span><span class="n">movies_with_overviews</span><span class="p">)</span>
</pre></div>

    </div>
</div>
</div>

<div class="output_wrapper">
<div class="output">


<div class="output_area">

    <div class="prompt output_prompt">Out[38]:</div>


<div class="output_text output_subarea output_execute_result">
<pre>1668</pre>
</div>

</div>

</div>
</div>

</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>Now let's store the genre's for these movies in a list that we will later transform into a binarized vector.</p>
<p>Binarized vector representation is a very common and important way data is stored/represented in ML. Essentially, it's a way to reduce a categorical variable with n possible values to n binary indicator variables. What does that mean? For example, let [(1,3),(4)] be the list saying that sample A has two labels 1 and 3, and sample B has one label 4. For every sample, for every possible label, the representation is simply 1 if it has that label, and 0 if it doesn't have that label. So the binarized version of the above list will be -</p>

<pre><code>[(1,0,1,0]),
(0,0,0,1])]</code></pre>

</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[69]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="c1"># genres=np.zeros((len(top1000_movies),3))</span>
<span class="n">genres</span><span class="o">=</span><span class="p">[]</span>
<span class="n">all_ids</span><span class="o">=</span><span class="p">[]</span>
<span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="nb">len</span><span class="p">(</span><span class="n">movies_with_overviews</span><span class="p">)):</span>
    <span class="n">movie</span><span class="o">=</span><span class="n">movies_with_overviews</span><span class="p">[</span><span class="n">i</span><span class="p">]</span>
    <span class="nb">id</span><span class="o">=</span><span class="n">movie</span><span class="p">[</span><span class="s1">&#39;id&#39;</span><span class="p">]</span>
    <span class="n">genre_ids</span><span class="o">=</span><span class="n">movie</span><span class="p">[</span><span class="s1">&#39;genre_ids&#39;</span><span class="p">]</span>
    <span class="n">genres</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">genre_ids</span><span class="p">)</span>
    <span class="n">all_ids</span><span class="o">.</span><span class="n">extend</span><span class="p">(</span><span class="n">genre_ids</span><span class="p">)</span>
</pre></div>

    </div>
</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[70]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="kn">from</span> <span class="nn">sklearn.preprocessing</span> <span class="kn">import</span> <span class="n">MultiLabelBinarizer</span>
<span class="n">mlb</span><span class="o">=</span><span class="n">MultiLabelBinarizer</span><span class="p">()</span>
<span class="n">Y</span><span class="o">=</span><span class="n">mlb</span><span class="o">.</span><span class="n">fit_transform</span><span class="p">(</span><span class="n">genres</span><span class="p">)</span>
</pre></div>

    </div>
</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[71]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="n">genres</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span>
</pre></div>

    </div>
</div>
</div>

<div class="output_wrapper">
<div class="output">


<div class="output_area">

    <div class="prompt output_prompt">Out[71]:</div>


<div class="output_text output_subarea output_execute_result">
<pre>[28, 12, 35]</pre>
</div>

</div>

</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[72]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="k">print</span> <span class="n">Y</span><span class="o">.</span><span class="n">shape</span>
<span class="k">print</span> <span class="n">np</span><span class="o">.</span><span class="n">sum</span><span class="p">(</span><span class="n">Y</span><span class="p">,</span> <span class="n">axis</span><span class="o">=</span><span class="mi">0</span><span class="p">)</span>
</pre></div>

    </div>
</div>
</div>

<div class="output_wrapper">
<div class="output">


<div class="output_area">

    <div class="prompt"></div>


<div class="output_subarea output_stream output_stdout output_text">
<pre>(1668, 19)
[349 253 256 671 226 450 439 135  47 432 241 132 277 187 137 235 294  99
 125]
</pre>
</div>
</div>

</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[73]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="nb">len</span><span class="p">(</span><span class="n">list_of_genres</span><span class="p">)</span>
</pre></div>

    </div>
</div>
</div>

<div class="output_wrapper">
<div class="output">


<div class="output_area">

    <div class="prompt output_prompt">Out[73]:</div>


<div class="output_text output_subarea output_execute_result">
<pre>19</pre>
</div>

</div>

</div>
</div>

</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>This is interesting. We started with only 19 genre labels if you remember. But the shape for Y is 1666,20 while it should be 1666,19 as there are only 19 genres? Let's explore.</p>

</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>Let's find genre IDs that are not present in our original list of genres!</p>

</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[98]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="c1"># Create a tmdb genre object!</span>
<span class="n">genres</span><span class="o">=</span><span class="n">tmdb</span><span class="o">.</span><span class="n">Genres</span><span class="p">()</span>
<span class="c1"># the list() method of the Genres() class returns a listing of all genres in the form of a dictionary.</span>
<span class="n">list_of_genres</span><span class="o">=</span><span class="n">genres</span><span class="o">.</span><span class="n">list</span><span class="p">()[</span><span class="s1">&#39;genres&#39;</span><span class="p">]</span>
<span class="n">Genre_ID_to_name</span><span class="o">=</span><span class="p">{}</span>
<span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="nb">len</span><span class="p">(</span><span class="n">list_of_genres</span><span class="p">)):</span>
    <span class="n">genre_id</span><span class="o">=</span><span class="n">list_of_genres</span><span class="p">[</span><span class="n">i</span><span class="p">][</span><span class="s1">&#39;id&#39;</span><span class="p">]</span>
    <span class="n">genre_name</span><span class="o">=</span><span class="n">list_of_genres</span><span class="p">[</span><span class="n">i</span><span class="p">][</span><span class="s1">&#39;name&#39;</span><span class="p">]</span>
    <span class="n">Genre_ID_to_name</span><span class="p">[</span><span class="n">genre_id</span><span class="p">]</span><span class="o">=</span><span class="n">genre_name</span>
</pre></div>

    </div>
</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[99]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">set</span><span class="p">(</span><span class="n">all_ids</span><span class="p">):</span>
    <span class="k">if</span> <span class="n">i</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">Genre_ID_to_name</span><span class="o">.</span><span class="n">keys</span><span class="p">():</span>
        <span class="k">print</span> <span class="n">i</span>
</pre></div>

    </div>
</div>
</div>

</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>Well, this genre ID wasn't given to us by TMDB when we asked it for all possible genres. How do we go about this now? We can either neglect all samples that have this genre. But if you look up you'll see there's too many of these samples. So, I googled more and went into their documentation and found that this ID corresponds to the genre "Foreign". So, we add it to the dictionary of genre names ourselves. Such problems are ubiquitous in machine learning, and it is up to us to diagnose and correct them. We must always make a decision about what to keep, how to store data and so on.</p>

</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[100]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="n">Genre_ID_to_name</span><span class="p">[</span><span class="mi">10769</span><span class="p">]</span><span class="o">=</span><span class="s2">&quot;Foreign&quot;</span> <span class="c1">#Adding it to the dictionary</span>
</pre></div>

    </div>
</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[101]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="nb">len</span><span class="p">(</span><span class="n">Genre_ID_to_name</span><span class="o">.</span><span class="n">keys</span><span class="p">())</span>
</pre></div>

    </div>
</div>
</div>

<div class="output_wrapper">
<div class="output">


<div class="output_area">

    <div class="prompt output_prompt">Out[101]:</div>


<div class="output_text output_subarea output_execute_result">
<pre>20</pre>
</div>

</div>

</div>
</div>

</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>Now, we turn to building the X matrix i.e. the input features! As described earlier, we will be using the overview of movies as our input vector! Let's look at a movie's overview for example!</p>

</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[78]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="n">sample_movie</span><span class="o">=</span><span class="n">movies_with_overviews</span><span class="p">[</span><span class="mi">5</span><span class="p">]</span>
<span class="n">sample_overview</span><span class="o">=</span><span class="n">sample_movie</span><span class="p">[</span><span class="s1">&#39;overview&#39;</span><span class="p">]</span>
<span class="n">sample_title</span><span class="o">=</span><span class="n">sample_movie</span><span class="p">[</span><span class="s1">&#39;title&#39;</span><span class="p">]</span>
<span class="k">print</span> <span class="s2">&quot;The overview for the movie&quot;</span><span class="p">,</span><span class="n">sample_title</span><span class="p">,</span><span class="s2">&quot; is - </span><span class="se">\n\n</span><span class="s2">&quot;</span>
<span class="k">print</span> <span class="n">sample_overview</span>
</pre></div>

    </div>
</div>
</div>

<div class="output_wrapper">
<div class="output">


<div class="output_area">

    <div class="prompt"></div>


<div class="output_subarea output_stream output_stdout output_text">
<pre>The overview for the movie Zootopia  is - 


Determined to prove herself, Officer Judy Hopps, the first bunny on Zootopia&#39;s police force, jumps at the chance to crack her first case - even if it means partnering with scam-artist fox Nick Wilde to solve the mystery.
</pre>
</div>
</div>

</div>
</div>

</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h2 id="So,-how-do-we-store-this-movie-overview-in-a-matrix?">So, how do we store this movie overview in a matrix?<a class="anchor-link" href="#So,-how-do-we-store-this-movie-overview-in-a-matrix?">&#182;</a></h2><h4 id="Do-we-just-store-the-whole-string?-We-know-that-we-need-to-work-with-numbers,-but-this-is-all-text.-What-do-we-do?!">Do we just store the whole string? We know that we need to work with numbers, but this is all text. What do we do?!<a class="anchor-link" href="#Do-we-just-store-the-whole-string?-We-know-that-we-need-to-work-with-numbers,-but-this-is-all-text.-What-do-we-do?!">&#182;</a></h4>
</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>The way we will be storing the X matrix is called a "Bag of words" representation. The basic idea of this representation in our context is that we can think of all the distinct words that are possible in the movies' reviews as a distinct object. And then every movie overview can be thought as a "Bag" containing a bunch of these possible objects.</p>
<p>For example, in the case of Zootopia the movie above - The "Bag" contains the words ("Determined", "to", "prove", "herself"......"the", "mystery"). We make such lists for all movie overviews. Finally, we binarize again like we did above for Y. scikit-learn makes our job easy here by simply using a function CountVectorizer() because this representation is so often used in Machine Learning.</p>

</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>What this means is that, for all the movies that we have the data on, we will first count all the unique words. Say, there's 30,000 unique words. Then we can represent every movie overview as a 30000x1 vector, where each position in the vector corresponds to the presence or absence of a particular word. If the word corresponding to that position is present in the overview, that position will have 1, otherwise it will be 0.</p>
<p>Ex - if our vocabular was 4 words - "I","am","a","good","boy", then the representation for the sentence "I am a boy" would be [1 1 1 0 1], and for the sentence "I am good" would be [1 1 0 1 0].</p>

</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[79]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="kn">from</span> <span class="nn">sklearn.feature_extraction.text</span> <span class="kn">import</span> <span class="n">CountVectorizer</span>
<span class="kn">import</span> <span class="nn">re</span>
</pre></div>

    </div>
</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[80]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="n">content</span><span class="o">=</span><span class="p">[]</span>
<span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="nb">len</span><span class="p">(</span><span class="n">movies_with_overviews</span><span class="p">)):</span>
    <span class="n">movie</span><span class="o">=</span><span class="n">movies_with_overviews</span><span class="p">[</span><span class="n">i</span><span class="p">]</span>
    <span class="nb">id</span><span class="o">=</span><span class="n">movie</span><span class="p">[</span><span class="s1">&#39;id&#39;</span><span class="p">]</span>
    <span class="n">overview</span><span class="o">=</span><span class="n">movie</span><span class="p">[</span><span class="s1">&#39;overview&#39;</span><span class="p">]</span>
    <span class="n">overview</span><span class="o">=</span><span class="n">overview</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="s1">&#39;,&#39;</span><span class="p">,</span><span class="s1">&#39;&#39;</span><span class="p">)</span>
    <span class="n">overview</span><span class="o">=</span><span class="n">overview</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="s1">&#39;.&#39;</span><span class="p">,</span><span class="s1">&#39;&#39;</span><span class="p">)</span>
    <span class="n">content</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">overview</span><span class="p">)</span>
</pre></div>

    </div>
</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[81]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="k">print</span> <span class="nb">len</span><span class="p">(</span><span class="n">movies_with_overviews</span><span class="p">)</span>
<span class="k">print</span> <span class="n">content</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span>
<span class="k">print</span> <span class="nb">len</span><span class="p">(</span><span class="n">content</span><span class="p">)</span>
</pre></div>

    </div>
</div>
</div>

<div class="output_wrapper">
<div class="output">


<div class="output_area">

    <div class="prompt"></div>


<div class="output_subarea output_stream output_stdout output_text">
<pre>1668
Following the events of Age of Ultron the collective governments of the world pass an act designed to regulate all superhuman activity This polarizes opinion amongst the Avengers causing two factions to side with Iron Man or Captain America which causes an epic battle between former allies
1668
</pre>
</div>
</div>

</div>
</div>

</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h1 id="Are-all-words-equally-important?">Are all words equally important?<a class="anchor-link" href="#Are-all-words-equally-important?">&#182;</a></h1>
</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h4 id="At-the-cost-of-sounding-&quot;Animal-Farm&quot;-inspired,-I-would-say-not-all-words-are-equally-important.">At the cost of sounding "Animal Farm" inspired, I would say not all words are equally important.<a class="anchor-link" href="#At-the-cost-of-sounding-&quot;Animal-Farm&quot;-inspired,-I-would-say-not-all-words-are-equally-important.">&#182;</a></h4><p>For example, let's consider the overview for the Matrix -</p>

</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[82]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="n">get_movie_info_tmdb</span><span class="p">(</span><span class="s1">&#39;The Matrix&#39;</span><span class="p">)[</span><span class="s1">&#39;overview&#39;</span><span class="p">]</span>
</pre></div>

    </div>
</div>
</div>

<div class="output_wrapper">
<div class="output">


<div class="output_area">

    <div class="prompt output_prompt">Out[82]:</div>


<div class="output_text output_subarea output_execute_result">
<pre>u&#39;Set in the 22nd century, The Matrix tells the story of a computer hacker who joins a group of underground insurgents fighting the vast and powerful computers who now rule the earth.&#39;</pre>
</div>

</div>

</div>
</div>

</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>For "The Matrix" a word like "computer" is a stronger indicators of it being a Sci-Fi movie, than words like "who" or "powerful" or "vast". One way computer scientists working with natural language tackled this problem in the past (and it is still used very popularly) is what we call TF-IDF i.e. Term Frequence, Inverse Document Frequency. The basic idea here is that words that are strongly indicative of the content of a single document (every movie overview is a document in our case) are words that occur very frequently in that document, and very infrequently in all other documents. For example, "Computer" occurs twice here but probably will not in most other movie overviews. Hence, it is indicative. On the other hand, generic words like "a","and","the" will occur very often in all documents. Hence, they are not indicative.</p>
<p>So, can we use this information to reduce our insanely high 30,000 dimensional vector representation to a smaller, more handle-able number? But first up, why should we even care? The answer is probably one of the most used phrases in ML - "The Curse of Dimensionality".</p>

</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h1 id="The-Curse-of-Dimensionality">The Curse of Dimensionality<a class="anchor-link" href="#The-Curse-of-Dimensionality">&#182;</a></h1>
</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h4 id="This-section-is-strongly-borrowing-from-one-of-the-greatest-ML-papers-I've-ever-read.">This section is strongly borrowing from one of the greatest <a href="https://homes.cs.washington.edu/~pedrod/papers/cacm12.pdf">ML papers I've ever read.</a><a class="anchor-link" href="#This-section-is-strongly-borrowing-from-one-of-the-greatest-ML-papers-I've-ever-read.">&#182;</a></h4><p>This expression was coined by Bellman in 1961 to refer to the fact that many algorithms that work fine in low dimensions become intractable when the input is high-dimensional. The reason for them not working in high dimensions is very strongly linked to what we discussed earlier - having a representative dataset. Consider this, you have a function $f$ dependent only one dependent variable $x$, and $x$ can only integer values from 1 to 100. Since it's one dimensional, it can be plotted on a line. To get a representative sample, you'd need to sample something like - $f(1),f(20),f(40),f(60),f(80),f(100)$</p>

</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>Now, let's increase the dimensionality i.e. number of dependent variables and see what happens. Say, we have 2 variables $x_1$ and $x_2$, same possible as before - integers between 1 and 100. Now, instead of a line, we'll have a plane with $x_1$ and $x_2$ on the two axes. The interesting bit is that instead of 100 possible values of dependent variables like before, we now have 100,000 possible values! Basically, we can make 100x100 table of possible values of $x_1$ and $x_2$. Wow, that increased exponentially. Not just figuratively, but mathematically exponentially. Needless to say, to cover 5% of the space like we did before, we'd need to sample $f$ at 5000 values.</p>

</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>For 3 variables, it would be 100,000,000, and we'd need to sample at 500,000 points. That's already more than the number of data points we have for most training problems we will ever come across.</p>

</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>Basically, as the dimensionality (number of features) of the examples grows, because a fixed-size training set covers a dwindling fraction of the input space. Even with a moderate dimension of 100 and a huge training set of a trillion examples, the latter covers only a fraction of about $10^{−18}$ of the input space. This is what makes machine learning
both necessary and hard.</p>

</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>So, yes, if some words are unimportant, we want to get rid of them and reduce the dimensionality of our X matrix. And the way we will do it is using TF-IDF to identify un-important words. Python let's us do this with just one line of code (And this is why you should spend more time reading maths, than coding!)</p>

</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[83]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="c1"># The min_df paramter makes sure we exclude words that only occur very rarely</span>
<span class="c1"># The default also is to exclude any words that occur in every movie description</span>
<span class="n">vectorize</span><span class="o">=</span><span class="n">CountVectorizer</span><span class="p">(</span><span class="n">max_df</span><span class="o">=</span><span class="mf">0.95</span><span class="p">,</span> <span class="n">min_df</span><span class="o">=</span><span class="mf">0.005</span><span class="p">)</span>
<span class="n">X</span><span class="o">=</span><span class="n">vectorize</span><span class="o">.</span><span class="n">fit_transform</span><span class="p">(</span><span class="n">content</span><span class="p">)</span>
</pre></div>

    </div>
</div>
</div>

</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>We are excluding all words that occur in too many or too few documents, as these are very unlikely to be discriminative. Words that only occur in one document most probably are names, and words that occur in nearly all documents are probably stop words. Note that the values here were not tuned using a validation set. They are just guesses. It is ok to do, because we didn't evaluate the performance of these parameters. In a strict case, for example for a publication, it would be better to tune these as well.</p>

</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[84]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="n">X</span><span class="o">.</span><span class="n">shape</span>
<span class="n">Y</span><span class="o">.</span><span class="n">shape</span>
</pre></div>

    </div>
</div>
</div>

<div class="output_wrapper">
<div class="output">


<div class="output_area">

    <div class="prompt output_prompt">Out[84]:</div>


<div class="output_text output_subarea output_execute_result">
<pre>(1668, 19)</pre>
</div>

</div>

</div>
</div>

</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>So, each movie's overview gets represented by a 1x1365 dimensional vector.</p>
<p>Now, we are ready for the kill. Our data is cleaned, hypothesis is set (Overview can predict movie genre), and the feature/output vectors are prepped. Let's train some models!</p>

</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[85]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="kn">import</span> <span class="nn">pickle</span>
<span class="n">f4</span><span class="o">=</span><span class="nb">open</span><span class="p">(</span><span class="s1">&#39;X.pckl&#39;</span><span class="p">,</span><span class="s1">&#39;wb&#39;</span><span class="p">)</span>
<span class="n">f5</span><span class="o">=</span><span class="nb">open</span><span class="p">(</span><span class="s1">&#39;Y.pckl&#39;</span><span class="p">,</span><span class="s1">&#39;wb&#39;</span><span class="p">)</span>
<span class="n">pickle</span><span class="o">.</span><span class="n">dump</span><span class="p">(</span><span class="n">X</span><span class="p">,</span><span class="n">f4</span><span class="p">)</span>
<span class="n">pickle</span><span class="o">.</span><span class="n">dump</span><span class="p">(</span><span class="n">Y</span><span class="p">,</span><span class="n">f5</span><span class="p">)</span>
<span class="n">f6</span><span class="o">=</span><span class="nb">open</span><span class="p">(</span><span class="s1">&#39;Genredict.pckl&#39;</span><span class="p">,</span><span class="s1">&#39;wb&#39;</span><span class="p">)</span>
<span class="n">pickle</span><span class="o">.</span><span class="n">dump</span><span class="p">(</span><span class="n">Genre_ID_to_name</span><span class="p">,</span><span class="n">f6</span><span class="p">)</span>
<span class="n">f4</span><span class="o">.</span><span class="n">close</span><span class="p">()</span>
<span class="n">f5</span><span class="o">.</span><span class="n">close</span><span class="p">()</span>
<span class="n">f6</span><span class="o">.</span><span class="n">close</span><span class="p">()</span>
</pre></div>

    </div>
</div>
</div>

</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h1 id="Congratulations,-we-have-our-data-set-ready!">Congratulations, we have our data set ready!<a class="anchor-link" href="#Congratulations,-we-have-our-data-set-ready!">&#182;</a></h1>
</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>A note : As we are building our own dataset, and I didn't want you to spend all your time waiting for poster image downloads to finish, I am working with an EXTREMELY small dataset. That is why, the results we will see for the deep learning portion will not be spectacular as compared to conventional machine learning methods. If you want to see the real power, you should spend some more time scraping something of the order of 100,000 images, as opposed to 1000 odd like I am doing here. Quoting the paper I mentioned above -  MORE DATA BEATS A CLEVERER ALGORITHM.</p>
<h4 id="As-the-TA,-I-saw-that-most-teams-working-on-the-project-had-data-of-the-order-of-100,000-movies.-So,-if-you-want-to-extract-the-power-of-these-models,-consider-scraping-a-larger-dataset-than-me.">As the TA, I saw that most teams working on the project had data of the order of 100,000 movies. So, if you want to extract the power of these models, consider scraping a larger dataset than me.<a class="anchor-link" href="#As-the-TA,-I-saw-that-most-teams-working-on-the-project-had-data-of-the-order-of-100,000-movies.-So,-if-you-want-to-extract-the-power-of-these-models,-consider-scraping-a-larger-dataset-than-me.">&#182;</a></h4>
</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h1 id="Section-5---Non-deep,-Conventional-ML-models-with-above-data">Section 5 - Non-deep, Conventional ML models with above data<a class="anchor-link" href="#Section-5---Non-deep,-Conventional-ML-models-with-above-data">&#182;</a></h1>
</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>Here is a layout of what we will be doing -</p>
<ul>
<li>We will implement two different models</li>
<li>We will decide a performance metric i.e. a quantitative method to be sure about how well difference models are doing. </li>
<li>Discussion of the differences between the models, their strengths, weaknesses, etc. </li>
</ul>

</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>As discussed earlier, there are a LOT of implementation decisions to be made. Between feature engineering, hyper-parameter tuning, model selection and how interpretable do you want your model to be (Read : Bayesian vs Non-Bayesian approaches) a lot is to be decided. For example, some of these models could be:</p>
<ul>
<li>Generalized Linear Models</li>
<li>SVM</li>
<li>Shallow (1 Layer, i.e. not deep) Neural Network</li>
<li>Random Forest</li>
<li>Boosting</li>
<li>Decision Tree</li>
</ul>
<p>Or go more bayesian:</p>
<ul>
<li>Naive Bayes</li>
<li>Linear or Quadratic Discriminant Analysis</li>
<li>Bayesian Hierarchical models</li>
</ul>

</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>The list is endless, and not all models will make sense for the kind of problem you have framed for yourself. <strong> Think about which model best fits for your purpose.</strong></p>

</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>For our purposes here, I will be showing the example of 2 very simple models, one picked from each category above -</p>
<ol>
<li>SVM</li>
<li>Multinomial Naive Bayes</li>
</ol>

</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>A quick overview of the whole pipeline coming below:</p>
<ul>
<li>A little bit of feature engineering</li>
<li>2 different Models </li>
<li>Evaluation Metrics chosen</li>
<li>Model comparisons</li>
</ul>

</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h3 id="Let's-start-with-some-feature-engineering.">Let's start with some feature engineering.<a class="anchor-link" href="#Let's-start-with-some-feature-engineering.">&#182;</a></h3>
</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>Engineering the right features depends on 2 key ideas. Firstly, what is it that you are trying to solve? For example, if you want to guess my music preferences and you try to train a super awesome model while giving it what my height is as input features, you're going to have no luck. On the other hand, giving it my Spotify playlist will solve the problem with any model. So, CONTEXT of the problem plays a role.</p>
<p>Second, you can only represent based on the data at hand. Meaning, if you didn't have access to my Spotify playlist, but to my Facebook statuses - You know all my statuses about Harvard may not be useful. But if you represent me as my Facebook statuses which are YouTube links, that would also solve the problem. So, AVAILABILITY OF DATA at hand is the second factor.</p>
<h4 id="A-nice-way-to-think-of-it-is-to-think-that-you-start-with-the-problem-at-hand,-but-design-features-constrained-by-the-data-you-have-available.-If-you-have-many-independent-features-that-each-correlate-well-with-the-class,-learning-is-easy.-On-the-other-hand,-if-the-class-is-a-very-complex-function-of-the-features,-you-may-not-be-able-to-learn-it.">A nice way to think of it is to think that you start with the problem at hand, but design features constrained by the data you have available. If you have many independent features that each correlate well with the class, learning is easy. On the other hand, if the class is a very complex function of the features, you may not be able to learn it.<a class="anchor-link" href="#A-nice-way-to-think-of-it-is-to-think-that-you-start-with-the-problem-at-hand,-but-design-features-constrained-by-the-data-you-have-available.-If-you-have-many-independent-features-that-each-correlate-well-with-the-class,-learning-is-easy.-On-the-other-hand,-if-the-class-is-a-very-complex-function-of-the-features,-you-may-not-be-able-to-learn-it.">&#182;</a></h4><p>In the context of this problem, we would like to predict the genre of a movie. what we have access to - movie overviews, which are text descriptions of the movie plot. The hypothesis makes sense, overview is a short description of the story and the story is clearly important in assigning genres to movies.</p>
<p>So, let's improve our features by playing with the words in the overviews in our data. One interesting way to go back to what we discussed earlier - TF-IDF. We originally used it to filter words, but we can also assign the tf-idf values as "importance" values to words, as opposed to treating them all equally. Tf-idf simply tries to identify the assign a weightage to each word in the bag of words.</p>

</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>Once again, the way it works is - Most movie descriptions have the word "The" in it. Obviously, it doesn't tell you anything special about it. So weightage should be inversely proportional to how many movies have the word in their description. This is the IDF part.</p>
<p>On the other hand, for the movie interstellar, if the description has the word Space 5 times, and wormhole 2 times, then it's probably more about Space than about wormhole. Thus, space should have a high weightage. This is the TF part.</p>
<p>We simply use TF-IDf to assign weightage to every word in the bag of words. Which makes sense, right? :)</p>

</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[86]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="kn">from</span> <span class="nn">sklearn.feature_extraction.text</span> <span class="kn">import</span> <span class="n">TfidfTransformer</span>
</pre></div>

    </div>
</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[87]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="n">tfidf_transformer</span> <span class="o">=</span> <span class="n">TfidfTransformer</span><span class="p">()</span>
<span class="n">X_tfidf</span> <span class="o">=</span> <span class="n">tfidf_transformer</span><span class="o">.</span><span class="n">fit_transform</span><span class="p">(</span><span class="n">X</span><span class="p">)</span>
<span class="n">X_tfidf</span><span class="o">.</span><span class="n">shape</span>
</pre></div>

    </div>
</div>
</div>

<div class="output_wrapper">
<div class="output">


<div class="output_area">

    <div class="prompt output_prompt">Out[87]:</div>


<div class="output_text output_subarea output_execute_result">
<pre>(1668, 1238)</pre>
</div>

</div>

</div>
</div>

</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>Let's divide our X and Y matrices into train and test split. We train the model on the train split, and report the performance on the test split. Think of this like the questions you do in the problem sets v/s the exam. Of course, they are both (assumed to be) from the same population of questions. And doing well on Problem Sets is a good indicator that you'll do well in exams, but really, you must test before you can make any claims about you knowing the subject.</p>

</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[88]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="n">msk</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">random</span><span class="o">.</span><span class="n">rand</span><span class="p">(</span><span class="n">X_tfidf</span><span class="o">.</span><span class="n">shape</span><span class="p">[</span><span class="mi">0</span><span class="p">])</span> <span class="o">&lt;</span> <span class="mf">0.8</span>
</pre></div>

    </div>
</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[89]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="n">X_train_tfidf</span><span class="o">=</span><span class="n">X_tfidf</span><span class="p">[</span><span class="n">msk</span><span class="p">]</span>
<span class="n">X_test_tfidf</span><span class="o">=</span><span class="n">X_tfidf</span><span class="p">[</span><span class="o">~</span><span class="n">msk</span><span class="p">]</span>
<span class="n">Y_train</span><span class="o">=</span><span class="n">Y</span><span class="p">[</span><span class="n">msk</span><span class="p">]</span>
<span class="n">Y_test</span><span class="o">=</span><span class="n">Y</span><span class="p">[</span><span class="o">~</span><span class="n">msk</span><span class="p">]</span>
<span class="n">positions</span><span class="o">=</span><span class="nb">range</span><span class="p">(</span><span class="nb">len</span><span class="p">(</span><span class="n">movies_with_overviews</span><span class="p">))</span>
<span class="c1"># print positions</span>
<span class="n">test_movies</span><span class="o">=</span><span class="n">np</span><span class="o">.</span><span class="n">asarray</span><span class="p">(</span><span class="n">positions</span><span class="p">)[</span><span class="o">~</span><span class="n">msk</span><span class="p">]</span>
<span class="c1"># test_movies</span>
</pre></div>

    </div>
</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[90]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="kn">from</span> <span class="nn">sklearn.multiclass</span> <span class="kn">import</span> <span class="n">OneVsRestClassifier</span>
<span class="kn">from</span> <span class="nn">sklearn.svm</span> <span class="kn">import</span> <span class="n">SVC</span>
<span class="kn">from</span> <span class="nn">sklearn.model_selection</span> <span class="kn">import</span> <span class="n">GridSearchCV</span>
<span class="kn">from</span> <span class="nn">sklearn.metrics</span> <span class="kn">import</span> <span class="n">f1_score</span>
<span class="kn">from</span> <span class="nn">sklearn.metrics</span> <span class="kn">import</span> <span class="n">make_scorer</span>
<span class="kn">from</span> <span class="nn">sklearn.metrics</span> <span class="kn">import</span> <span class="n">classification_report</span>
</pre></div>

    </div>
</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[93]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="n">parameters</span> <span class="o">=</span> <span class="p">{</span><span class="s1">&#39;kernel&#39;</span><span class="p">:[</span><span class="s1">&#39;linear&#39;</span><span class="p">],</span> <span class="s1">&#39;C&#39;</span><span class="p">:[</span><span class="mf">0.01</span><span class="p">,</span> <span class="mf">0.1</span><span class="p">,</span> <span class="mf">1.0</span><span class="p">]}</span>
<span class="n">gridCV</span> <span class="o">=</span> <span class="n">GridSearchCV</span><span class="p">(</span><span class="n">SVC</span><span class="p">(</span><span class="n">class_weight</span><span class="o">=</span><span class="s1">&#39;balanced&#39;</span><span class="p">),</span> <span class="n">parameters</span><span class="p">,</span> <span class="n">scoring</span><span class="o">=</span><span class="n">make_scorer</span><span class="p">(</span><span class="n">f1_score</span><span class="p">,</span> <span class="n">average</span><span class="o">=</span><span class="s1">&#39;micro&#39;</span><span class="p">))</span>
<span class="n">classif</span> <span class="o">=</span> <span class="n">OneVsRestClassifier</span><span class="p">(</span><span class="n">gridCV</span><span class="p">)</span>

<span class="n">classif</span><span class="o">.</span><span class="n">fit</span><span class="p">(</span><span class="n">X_train_tfidf</span><span class="p">,</span> <span class="n">Y_train</span><span class="p">)</span>
</pre></div>

    </div>
</div>
</div>

<div class="output_wrapper">
<div class="output">


<div class="output_area">

    <div class="prompt output_prompt">Out[93]:</div>


<div class="output_text output_subarea output_execute_result">
<pre>OneVsRestClassifier(estimator=GridSearchCV(cv=None, error_score=&#39;raise&#39;,
       estimator=SVC(C=1.0, cache_size=200, class_weight=&#39;balanced&#39;, coef0=0.0,
  decision_function_shape=None, degree=3, gamma=&#39;auto&#39;, kernel=&#39;rbf&#39;,
  max_iter=-1, probability=False, random_state=None, shrinking=True,
  tol=0.001, verbose=False),...refit=True, return_train_score=True,
       scoring=make_scorer(f1_score, average=micro), verbose=0),
          n_jobs=1)</pre>
</div>

</div>

</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[94]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="n">predstfidf</span><span class="o">=</span><span class="n">classif</span><span class="o">.</span><span class="n">predict</span><span class="p">(</span><span class="n">X_test_tfidf</span><span class="p">)</span>
<span class="k">print</span> <span class="n">X_test_tfidf</span><span class="o">.</span><span class="n">shape</span><span class="p">,</span> <span class="n">Y_test</span><span class="o">.</span><span class="n">shape</span><span class="p">,</span> <span class="n">predstfidf</span><span class="o">.</span><span class="n">shape</span><span class="p">,</span> <span class="nb">len</span><span class="p">(</span><span class="n">genre_names</span><span class="p">)</span>
<span class="k">print</span> <span class="n">genre_names</span>
<span class="k">print</span> <span class="n">classification_report</span><span class="p">(</span><span class="n">Y_test</span><span class="p">,</span> <span class="n">predstfidf</span><span class="p">)</span>
</pre></div>

    </div>
</div>
</div>

<div class="output_wrapper">
<div class="output">


<div class="output_area">

    <div class="prompt"></div>


<div class="output_subarea output_stream output_stdout output_text">
<pre>(363, 1238) (363, 19) (363, 19) 4
[u&#39;Action&#39;, u&#39;Adventure&#39;, u&#39;Animation&#39;, u&#39;Family&#39;]
             precision    recall  f1-score   support

          0       0.41      0.67      0.51        72
          1       0.36      0.51      0.42        59
          2       0.43      0.50      0.46        64
          3       0.53      0.45      0.49       154
          4       0.32      0.49      0.38        41
          5       0.55      0.67      0.60       100
          6       0.43      0.55      0.49        96
          7       0.32      0.36      0.34        36
          8       0.33      0.10      0.15        10
          9       0.24      1.00      0.39        88
         10       0.48      0.61      0.53        51
         11       0.57      0.55      0.56        29
         12       0.36      0.43      0.39        60
         13       0.30      0.45      0.36        38
         14       0.78      0.41      0.54        34
         15       0.00      0.00      0.00        57
         16       0.49      0.56      0.52        71
         17       0.00      0.00      0.00        21
         18       0.11      0.12      0.12        32

avg / total       0.40      0.51      0.43      1113

</pre>
</div>
</div>

</div>
</div>

</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>As you can see, the performance is by and large poorer for movies which are less represented like War and animation, and better for categories like Drama.</p>

</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>Numbers aside, let's look at our model's predictions for a small sample of movies from our test set.</p>

</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[102]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="n">genre_list</span><span class="o">=</span><span class="nb">sorted</span><span class="p">(</span><span class="nb">list</span><span class="p">(</span><span class="n">Genre_ID_to_name</span><span class="o">.</span><span class="n">keys</span><span class="p">()))</span>
</pre></div>

    </div>
</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[105]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="n">predictions</span><span class="o">=</span><span class="p">[]</span>
<span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="n">X_test_tfidf</span><span class="o">.</span><span class="n">shape</span><span class="p">[</span><span class="mi">0</span><span class="p">]):</span>
    <span class="n">pred_genres</span><span class="o">=</span><span class="p">[]</span>
    <span class="n">movie_label_scores</span><span class="o">=</span><span class="n">predstfidf</span><span class="p">[</span><span class="n">i</span><span class="p">]</span>
    <span class="c1">#print len(movie_label_scores), movie_label_scores[18]</span>
<span class="c1">#     print movie_label_scores</span>
    <span class="k">for</span> <span class="n">j</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="nb">len</span><span class="p">(</span><span class="n">movie_label_scores</span><span class="p">)):</span>
        <span class="c1">#print j</span>
        <span class="k">if</span> <span class="n">movie_label_scores</span><span class="p">[</span><span class="n">j</span><span class="p">]</span><span class="o">!=</span><span class="mi">0</span><span class="p">:</span>
            <span class="n">genre</span><span class="o">=</span><span class="n">Genre_ID_to_name</span><span class="p">[</span><span class="n">genre_list</span><span class="p">[</span><span class="n">j</span><span class="p">]]</span>
            <span class="n">pred_genres</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">genre</span><span class="p">)</span>
    <span class="n">predictions</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">pred_genres</span><span class="p">)</span>
</pre></div>

    </div>
</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[96]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="kn">import</span> <span class="nn">pickle</span>
<span class="n">f</span><span class="o">=</span><span class="nb">open</span><span class="p">(</span><span class="s1">&#39;classifer_svc&#39;</span><span class="p">,</span><span class="s1">&#39;wb&#39;</span><span class="p">)</span>
<span class="n">pickle</span><span class="o">.</span><span class="n">dump</span><span class="p">(</span><span class="n">classif</span><span class="p">,</span><span class="n">f</span><span class="p">)</span>
<span class="n">f</span><span class="o">.</span><span class="n">close</span><span class="p">()</span>
</pre></div>

    </div>
</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[57]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="n">X_test_tfidf</span><span class="o">.</span><span class="n">shape</span><span class="p">[</span><span class="mi">0</span><span class="p">]):</span>
    <span class="k">if</span> <span class="n">i</span><span class="o">%</span><span class="k">50</span>==0 and i!=0:
        <span class="k">print</span> <span class="s1">&#39;MOVIE: &#39;</span><span class="p">,</span><span class="n">movies_with_overviews</span><span class="p">[</span><span class="n">i</span><span class="p">][</span><span class="s1">&#39;title&#39;</span><span class="p">],</span><span class="s1">&#39;</span><span class="se">\t</span><span class="s1">PREDICTION: &#39;</span><span class="p">,</span><span class="s1">&#39;,&#39;</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">predictions</span><span class="p">[</span><span class="n">i</span><span class="p">])</span>
</pre></div>

    </div>
</div>
</div>

<div class="output_wrapper">
<div class="output">


<div class="output_area">

    <div class="prompt"></div>


<div class="output_subarea output_stream output_stdout output_text">
<pre>MOVIE:  Alice Through the Looking Glass 	PREDICTION:  Adventure,Fantasy,Animation,Action,Comedy,Science Fiction,Family
MOVIE:  The Hobbit: The Battle of the Five Armies 	PREDICTION:  Adventure,Horror,Thriller
MOVIE:  Attack on Titan II: End of the World 	PREDICTION:  Action,Comedy,Crime,Science Fiction
MOVIE:  Cloudy with a Chance of Meatballs 2 	PREDICTION:  Adventure,Fantasy,Science Fiction
MOVIE:  Zambezia 	PREDICTION:  Drama,Crime
MOVIE:  The Bourne Supremacy 	PREDICTION:  Animation,Drama,Crime,Family
</pre>
</div>
</div>

</div>
</div>

</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>Let's try our second model? The naive bayes model.</p>

</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[58]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="kn">from</span> <span class="nn">sklearn.naive_bayes</span> <span class="kn">import</span> <span class="n">MultinomialNB</span>
<span class="n">classifnb</span> <span class="o">=</span> <span class="n">OneVsRestClassifier</span><span class="p">(</span><span class="n">MultinomialNB</span><span class="p">())</span>
<span class="n">classifnb</span><span class="o">.</span><span class="n">fit</span><span class="p">(</span><span class="n">X</span><span class="p">[</span><span class="n">msk</span><span class="p">]</span><span class="o">.</span><span class="n">toarray</span><span class="p">(),</span> <span class="n">Y_train</span><span class="p">)</span>
<span class="n">predsnb</span><span class="o">=</span><span class="n">classifnb</span><span class="o">.</span><span class="n">predict</span><span class="p">(</span><span class="n">X</span><span class="p">[</span><span class="o">~</span><span class="n">msk</span><span class="p">]</span><span class="o">.</span><span class="n">toarray</span><span class="p">())</span>
</pre></div>

    </div>
</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[59]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="kn">import</span> <span class="nn">pickle</span>
<span class="n">f2</span><span class="o">=</span><span class="nb">open</span><span class="p">(</span><span class="s1">&#39;classifer_nb&#39;</span><span class="p">,</span><span class="s1">&#39;wb&#39;</span><span class="p">)</span>
<span class="n">pickle</span><span class="o">.</span><span class="n">dump</span><span class="p">(</span><span class="n">classifnb</span><span class="p">,</span><span class="n">f2</span><span class="p">)</span>
<span class="n">f2</span><span class="o">.</span><span class="n">close</span><span class="p">()</span>
</pre></div>

    </div>
</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[60]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="n">predictionsnb</span><span class="o">=</span><span class="p">[]</span>
<span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="n">X_test_tfidf</span><span class="o">.</span><span class="n">shape</span><span class="p">[</span><span class="mi">0</span><span class="p">]):</span>
    <span class="n">pred_genres</span><span class="o">=</span><span class="p">[]</span>
    <span class="n">movie_label_scores</span><span class="o">=</span><span class="n">predsnb</span><span class="p">[</span><span class="n">i</span><span class="p">]</span>
    <span class="k">for</span> <span class="n">j</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="nb">len</span><span class="p">(</span><span class="n">movie_label_scores</span><span class="p">)):</span>
        <span class="c1">#print j</span>
        <span class="k">if</span> <span class="n">movie_label_scores</span><span class="p">[</span><span class="n">j</span><span class="p">]</span><span class="o">!=</span><span class="mi">0</span><span class="p">:</span>
            <span class="n">genre</span><span class="o">=</span><span class="n">Genre_ID_to_name</span><span class="p">[</span><span class="n">genre_list</span><span class="p">[</span><span class="n">j</span><span class="p">]]</span>
            <span class="n">pred_genres</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">genre</span><span class="p">)</span>
    <span class="n">predictionsnb</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">pred_genres</span><span class="p">)</span>
</pre></div>

    </div>
</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[61]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="n">X_test_tfidf</span><span class="o">.</span><span class="n">shape</span><span class="p">[</span><span class="mi">0</span><span class="p">]):</span>
    <span class="k">if</span> <span class="n">i</span><span class="o">%</span><span class="k">50</span>==0 and i!=0:
        <span class="k">print</span> <span class="s1">&#39;MOVIE: &#39;</span><span class="p">,</span><span class="n">movies_with_overviews</span><span class="p">[</span><span class="n">i</span><span class="p">][</span><span class="s1">&#39;title&#39;</span><span class="p">],</span><span class="s1">&#39;</span><span class="se">\t</span><span class="s1">PREDICTION: &#39;</span><span class="p">,</span><span class="s1">&#39;,&#39;</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">predictionsnb</span><span class="p">[</span><span class="n">i</span><span class="p">])</span>
</pre></div>

    </div>
</div>
</div>

<div class="output_wrapper">
<div class="output">


<div class="output_area">

    <div class="prompt"></div>


<div class="output_subarea output_stream output_stdout output_text">
<pre>MOVIE:  Alice Through the Looking Glass 	PREDICTION:  Adventure,Action,Science Fiction,Family
MOVIE:  The Hobbit: The Battle of the Five Armies 	PREDICTION:  Adventure,Horror,Action,Thriller,Science Fiction
MOVIE:  Attack on Titan II: End of the World 	PREDICTION:  Action,Thriller,Crime,Science Fiction
MOVIE:  Cloudy with a Chance of Meatballs 2 	PREDICTION:  Adventure,Fantasy,Action,Documentary,Science Fiction,Music
MOVIE:  Zambezia 	PREDICTION:  Drama,Thriller
MOVIE:  The Bourne Supremacy 	PREDICTION:  Drama,TV Movie
</pre>
</div>
</div>

</div>
</div>

</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>As can be seen above, the results seem promising, but how do we really compare the two models? We need to quantify our performance so that we can say which one's better. Takes us back to what we discussed right in the beginning - we're learning a function $g$ which can approximate the original unknown function $f$. For some values of $x_i$, the predictions will be wrong for sure, and we want to minimize it.</p>
<p>For multi label systems, we often keep track of performance using "Precision" and "Recall". These are standard metrics, and you can google to read up more about them if you're new to these terms.</p>

</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h1 id="Evaluation-Metrics">Evaluation Metrics<a class="anchor-link" href="#Evaluation-Metrics">&#182;</a></h1>
</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>We will use the standard precision recall metrics for evaluating our system.</p>

</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[28]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="k">def</span> <span class="nf">precision_recall</span><span class="p">(</span><span class="n">gt</span><span class="p">,</span><span class="n">preds</span><span class="p">):</span>
    <span class="n">TP</span><span class="o">=</span><span class="mi">0</span>
    <span class="n">FP</span><span class="o">=</span><span class="mi">0</span>
    <span class="n">FN</span><span class="o">=</span><span class="mi">0</span>
    <span class="k">for</span> <span class="n">t</span> <span class="ow">in</span> <span class="n">gt</span><span class="p">:</span>
        <span class="k">if</span> <span class="n">t</span> <span class="ow">in</span> <span class="n">preds</span><span class="p">:</span>
            <span class="n">TP</span><span class="o">+=</span><span class="mi">1</span>
        <span class="k">else</span><span class="p">:</span>
            <span class="n">FN</span><span class="o">+=</span><span class="mi">1</span>
    <span class="k">for</span> <span class="n">p</span> <span class="ow">in</span> <span class="n">preds</span><span class="p">:</span>
        <span class="k">if</span> <span class="n">p</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">gt</span><span class="p">:</span>
            <span class="n">FP</span><span class="o">+=</span><span class="mi">1</span>
    <span class="k">if</span> <span class="n">TP</span><span class="o">+</span><span class="n">FP</span><span class="o">==</span><span class="mi">0</span><span class="p">:</span>
        <span class="n">precision</span><span class="o">=</span><span class="mi">0</span>
    <span class="k">else</span><span class="p">:</span>
        <span class="n">precision</span><span class="o">=</span><span class="n">TP</span><span class="o">/</span><span class="nb">float</span><span class="p">(</span><span class="n">TP</span><span class="o">+</span><span class="n">FP</span><span class="p">)</span>
    <span class="k">if</span> <span class="n">TP</span><span class="o">+</span><span class="n">FN</span><span class="o">==</span><span class="mi">0</span><span class="p">:</span>
        <span class="n">recall</span><span class="o">=</span><span class="mi">0</span>
    <span class="k">else</span><span class="p">:</span>
        <span class="n">recall</span><span class="o">=</span><span class="n">TP</span><span class="o">/</span><span class="nb">float</span><span class="p">(</span><span class="n">TP</span><span class="o">+</span><span class="n">FN</span><span class="p">)</span>
    <span class="k">return</span> <span class="n">precision</span><span class="p">,</span><span class="n">recall</span>
</pre></div>

    </div>
</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[63]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="n">precs</span><span class="o">=</span><span class="p">[]</span>
<span class="n">recs</span><span class="o">=</span><span class="p">[]</span>
<span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="nb">len</span><span class="p">(</span><span class="n">test_movies</span><span class="p">)):</span>
    <span class="k">if</span> <span class="n">i</span><span class="o">%</span><span class="k">1</span>==0:
        <span class="n">pos</span><span class="o">=</span><span class="n">test_movies</span><span class="p">[</span><span class="n">i</span><span class="p">]</span>
        <span class="n">test_movie</span><span class="o">=</span><span class="n">movies_with_overviews</span><span class="p">[</span><span class="n">pos</span><span class="p">]</span>
        <span class="n">gtids</span><span class="o">=</span><span class="n">test_movie</span><span class="p">[</span><span class="s1">&#39;genre_ids&#39;</span><span class="p">]</span>
        <span class="n">gt</span><span class="o">=</span><span class="p">[]</span>
        <span class="k">for</span> <span class="n">g</span> <span class="ow">in</span> <span class="n">gtids</span><span class="p">:</span>
            <span class="n">g_name</span><span class="o">=</span><span class="n">Genre_ID_to_name</span><span class="p">[</span><span class="n">g</span><span class="p">]</span>
            <span class="n">gt</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">g_name</span><span class="p">)</span>
<span class="c1">#         print predictions[i],movies_with_overviews[i][&#39;title&#39;],gt</span>
        <span class="n">a</span><span class="p">,</span><span class="n">b</span><span class="o">=</span><span class="n">precision_recall</span><span class="p">(</span><span class="n">gt</span><span class="p">,</span><span class="n">predictions</span><span class="p">[</span><span class="n">i</span><span class="p">])</span>
        <span class="n">precs</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">a</span><span class="p">)</span>
        <span class="n">recs</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">b</span><span class="p">)</span>

<span class="k">print</span> <span class="n">np</span><span class="o">.</span><span class="n">mean</span><span class="p">(</span><span class="n">np</span><span class="o">.</span><span class="n">asarray</span><span class="p">(</span><span class="n">precs</span><span class="p">)),</span><span class="n">np</span><span class="o">.</span><span class="n">mean</span><span class="p">(</span><span class="n">np</span><span class="o">.</span><span class="n">asarray</span><span class="p">(</span><span class="n">recs</span><span class="p">))</span>
</pre></div>

    </div>
</div>
</div>

<div class="output_wrapper">
<div class="output">


<div class="output_area">

    <div class="prompt"></div>


<div class="output_subarea output_stream output_stdout output_text">
<pre>0.4461090040736058 0.5394437420986093
</pre>
</div>
</div>

</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[64]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="n">precs</span><span class="o">=</span><span class="p">[]</span>
<span class="n">recs</span><span class="o">=</span><span class="p">[]</span>
<span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="nb">len</span><span class="p">(</span><span class="n">test_movies</span><span class="p">)):</span>
    <span class="k">if</span> <span class="n">i</span><span class="o">%</span><span class="k">1</span>==0:
        <span class="n">pos</span><span class="o">=</span><span class="n">test_movies</span><span class="p">[</span><span class="n">i</span><span class="p">]</span>
        <span class="n">test_movie</span><span class="o">=</span><span class="n">movies_with_overviews</span><span class="p">[</span><span class="n">pos</span><span class="p">]</span>
        <span class="n">gtids</span><span class="o">=</span><span class="n">test_movie</span><span class="p">[</span><span class="s1">&#39;genre_ids&#39;</span><span class="p">]</span>
        <span class="n">gt</span><span class="o">=</span><span class="p">[]</span>
        <span class="k">for</span> <span class="n">g</span> <span class="ow">in</span> <span class="n">gtids</span><span class="p">:</span>
            <span class="n">g_name</span><span class="o">=</span><span class="n">Genre_ID_to_name</span><span class="p">[</span><span class="n">g</span><span class="p">]</span>
            <span class="n">gt</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">g_name</span><span class="p">)</span>
<span class="c1">#         print predictions[i],movies_with_overviews[i][&#39;title&#39;],gt</span>
        <span class="n">a</span><span class="p">,</span><span class="n">b</span><span class="o">=</span><span class="n">precision_recall</span><span class="p">(</span><span class="n">gt</span><span class="p">,</span><span class="n">predictionsnb</span><span class="p">[</span><span class="n">i</span><span class="p">])</span>
        <span class="n">precs</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">a</span><span class="p">)</span>
        <span class="n">recs</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">b</span><span class="p">)</span>

<span class="k">print</span> <span class="n">np</span><span class="o">.</span><span class="n">mean</span><span class="p">(</span><span class="n">np</span><span class="o">.</span><span class="n">asarray</span><span class="p">(</span><span class="n">precs</span><span class="p">)),</span><span class="n">np</span><span class="o">.</span><span class="n">mean</span><span class="p">(</span><span class="n">np</span><span class="o">.</span><span class="n">asarray</span><span class="p">(</span><span class="n">recs</span><span class="p">))</span>
</pre></div>

    </div>
</div>
</div>

<div class="output_wrapper">
<div class="output">


<div class="output_area">

    <div class="prompt"></div>


<div class="output_subarea output_stream output_stdout output_text">
<pre>0.48468886079505547 0.5216392751790982
</pre>
</div>
</div>

</div>
</div>

</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>The average precision and recall scores for our samples are pretty good! Models seem to be working! Also, we can see that the Naive Bayes performs outperforms SVM. <strong>I strongly suggest you to go read about Multinomial Bayes and think about why it works so well for "Document Classification", which is very similar to our case as every movie overview can be thought of as a document we are assigning labels to.</strong></p>

</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h1 id="Section-6---Deep-Learning-:-an-intuitive-overview">Section 6 - Deep Learning : an intuitive overview<a class="anchor-link" href="#Section-6---Deep-Learning-:-an-intuitive-overview">&#182;</a></h1>
</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>The above results were good, but it's time to bring out the big guns. So first and foremost, let's get a very short idea about what's deep learning. This is for peope who don't have background in this - it's high level and gives just the intuition.</p>

</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>As described above, the two most immportant concepts in doing good classification (or regression) are to 1) use the right representation which captures the right information about the data which is relevant to the problem at hand 2) Using the right model which has the capability of making sense of the representation fed to it.</p>

</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>While for the second part we have complicated and powerful models that we have studied at length, we don't seem to have a principled, mathematical way of doing the first part - i.e. representation. What we did above was to see "What makes sense", and go from there. That is not a good approach for complex data/ complex problems. Is there some way to automate this? Deep Learning, does just this.</p>

</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>To just emphasize the importance of representation in the complex tasks we usually attempt with Deep Learning, let me talk about the original problem which made it famous. The paper is often reffered to as the "Imagenet Challenge Paper", and it was basically working on object recognition in images. Let's try to think about an algorithm that tries to detect a chair.</p>
<h2 id="If-I-ask-you-to-&quot;Define&quot;-a-chair,-how-would-you?---Something-with-4-legs?">If I ask you to "Define" a chair, how would you? - Something with 4 legs?<a class="anchor-link" href="#If-I-ask-you-to-&quot;Define&quot;-a-chair,-how-would-you?---Something-with-4-legs?">&#182;</a></h2>
</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p><img src="chair1.png" height="400" width="400"></p>
<h3><center>All are chairs, none with 4 legs. (Pic Credit: Zoya Bylinskii)</center></h3>
</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h2 id="How-about-some-surface-that-we-sit-on-then?">How about some surface that we sit on then?<a class="anchor-link" href="#How-about-some-surface-that-we-sit-on-then?">&#182;</a></h2>
</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p><img src="chair2.png" height="400" width="400"></p>
<h3><center>All are surfaces we sit on, none are chairs. (Pic Credit: Zoya Bylinskii)</center></h3>
</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>Clearly, these definitions won't work and we need something more complicated. Sadly, we can't come up with a simple text rule that our computer can search for! And we take a more principled approach.</p>

</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>The "Deep" in the deep learning comes from the fact that it was conventionally applied to Neural Networks. Neural Networks, as we all know, are structures organized in layers. Layers of computations. Why do we need layers? Because these layers can be seen as sub-tasks that we do in the complicated task of identifying a chair. It can be thought as a heirarchical break down of a complicated job into smalled sub-tasks.</p>
<p>Mathematically, each layer acts like a space transformation which takes the pixel values to a high dimensional space. When we start out, every pixel in the image is given equal importance in our matrix. With each layer, convolution operations give some parts more importance, and some lesser importance. In doing so, we transform our images to a space in which similar looking objects/object parts are closer (We are basically learning this space transformation in deep learning, nothing else)</p>

</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>What exactly was learnt by these neural networks is hard to know, and an active area of research. But one very crude way to visualize what it does is to think like - It starts by learning very generic features in the first layer. Something as simple as vertical and horizontal lines. In the next layer, it learns that if you combine the vectors representing vertical and horizontal vectors in different ratios, you can make all possible slanted lines. Next layer learns to combine lines to form curves - Say, something like the outline of a face. These curves come together to form 3D objects. And so on. Building sub-modules, combining them in the right way which can give it semantics.</p>

</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p><strong>So, in a nutshell, the first few layers of a "Deep" network learn the right representation of the data, given the problem (which is mathematically described by your objective function trying to minimize difference between ground truth and predicted labels). The last layer simply looks how close or far apart things are in this high dimensional space.</strong></p>

</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>Hence, we can give any kind of data a high dimensional representation using neural networks. Below we will see high dimensional representations of both words in overviews (text) and posters (image). Let's get started with the posters i.e. extracting visual features from posters using deep learning.</p>

</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h1 id="Section-7---Deep-Learning-for-predicting-genre-from-poster">Section 7 - Deep Learning for predicting genre from poster<a class="anchor-link" href="#Section-7---Deep-Learning-for-predicting-genre-from-poster">&#182;</a></h1><p>Once again, we must make an implementation decision. This time, it has more to do with how much time are we willing to spend in return for added accuracy. We are going to use here a technique that is commonly referred to as Pre-Training in Machine Learning Literature.</p>
<p>Instead of me trying to re-invent the wheel here, I am going to borrow this short section on pre-training from Stanford University's lecture on <a href='http://cs231n.github.io/transfer-learning/'> CNN's</a>. To quote -</p>
<p>''In practice, very few people train an entire Convolutional Network from scratch (with random initialization), because it is relatively rare to have a dataset of sufficient size. Instead, it is common to pretrain a ConvNet on a very large dataset (e.g. ImageNet, which contains 1.2 million images with 1000 categories), and then use the ConvNet either as an initialization or a fixed feature extractor for the task of interest. ''</p>
<p>There are three broad ways in which transfer learning or pre-training can be done. (The 2 concepts are different and to understand the difference clearly, I suggest you read the linked lecture thoroughly). The way we are going to about it is by using a pre-trained, released ConvNet as feature extractor. Take a ConvNet pretrained on ImageNet (a popular object detection dataset), remove the last fully-connected layer. After removing the last layer, what we have is just another neural network i.e. a stack of space tranformations. But, originally the output of this stack can be pumped into a single layer which can classify the image into categories like Car, Dog, Cat and so on.</p>
<p>What this means, is that in the space this stack transforms the images to, all images which contain a "dog" are closer to each other, and all images containing a "cat" are closer. Thus, it is a meaningful space where images with similar objects are closer.</p>
<p>Think about it, now if we pump our posters through this stack, it will embed them in a space where posters which contain similar objects are closer. This is a very meaningful feature engineering method! While this may not be ideal for genre prediction, it might be quite meaningful. For example, all posters with a gun or a car are probably action. While a smiling couple would point to romance or drama. The alternative would be to train the CNN from scratch which is fairly computationally intensive and involves a lot of tricks to get the CNN training to converge to the optimal space tranformation.</p>
<p>This way, we can start off with something strong, and then build on top. We pump our images through the pre-trained network to extract the visual features from the posters. Then, using these features as descriptors for the image, and genres as the labels, we train a simpler neural network from scratch which learns to do simply classification on this dataset. These 2 steps are exactly what we are going to do for predicting genres from movie posters.</p>

</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h2 id="Deep-Learning-to-extract-visual-features-from-posters">Deep Learning to extract visual features from posters<a class="anchor-link" href="#Deep-Learning-to-extract-visual-features-from-posters">&#182;</a></h2>
</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>The basic problem here we are answering is that can we use the posters to predict genre. First check - Does this hypothesis make sense? Yes. Because that's what graphic designers do for a living. They leave visual cues to semantics. They make sure that when we look at the poster of a horror movie, we know it's not a happy image. Things like that. Can our deep learning system infer such subtleties? Let's find out!</p>

</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>For Visual features, either we can train a deep neural network ourselves from scratch, or we can use a pre-trained one made available to us from the Visual Geometry Group at Oxford University, one of the most popular methods. This is called the VGG-net. Or as they call it, we will extract the VGG features of an image. Mathematically, as mentioned, it's just a space transformation in the form of layers. So, we simply need to perform this chain of transformations on our image, right? We will be using Pytorch in this notebook. OTher alternatives are Keras and Tensorflow.</p>
<p>We will be working with Keras to keep things simple in code, so that we can spend more time understanding and less time coding. Some common ways people refer to this step are - "Getting the VGG features of an image", or "Forward Propogating the image through VGG and chopping off the last layer". In keras, this is as easy as writing 4 lines.</p>

</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[115]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="c1"># Loading the list of movies we had downloaded posters for eariler - </span>
<span class="kn">import</span> <span class="nn">pickle</span>
<span class="n">f</span><span class="o">=</span><span class="nb">open</span><span class="p">(</span><span class="s1">&#39;poster_movies.pckl&#39;</span><span class="p">,</span><span class="s1">&#39;r&#39;</span><span class="p">)</span>
<span class="n">poster_movies</span><span class="o">=</span><span class="n">pickle</span><span class="o">.</span><span class="n">load</span><span class="p">(</span><span class="n">f</span><span class="p">)</span>
<span class="n">f</span><span class="o">.</span><span class="n">close</span><span class="p">()</span>
</pre></div>

    </div>
</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[29]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="kn">import</span> <span class="nn">torchvision</span>
<span class="kn">import</span> <span class="nn">torchvision.models</span> <span class="kn">as</span> <span class="nn">models</span>
<span class="kn">import</span> <span class="nn">torch.nn</span> <span class="kn">as</span> <span class="nn">nn</span>
<span class="kn">import</span> <span class="nn">numpy</span> <span class="kn">as</span> <span class="nn">np</span>
<span class="kn">from</span> <span class="nn">torch.autograd</span> <span class="kn">import</span> <span class="n">Variable</span>
<span class="kn">import</span> <span class="nn">torchvision.transforms</span> <span class="kn">as</span> <span class="nn">transforms</span>
<span class="kn">import</span> <span class="nn">pickle</span>
<span class="n">vgg16</span> <span class="o">=</span> <span class="n">models</span><span class="o">.</span><span class="n">vgg16</span><span class="p">(</span><span class="n">pretrained</span><span class="o">=</span><span class="bp">True</span><span class="p">)</span>
<span class="n">model</span> <span class="o">=</span> <span class="n">nn</span><span class="o">.</span><span class="n">Sequential</span><span class="p">(</span><span class="o">*</span><span class="nb">list</span><span class="p">(</span><span class="n">vgg16</span><span class="o">.</span><span class="n">children</span><span class="p">())[:</span><span class="o">-</span><span class="mi">1</span><span class="p">])</span>
<span class="n">model</span> <span class="o">=</span> <span class="n">nn</span><span class="o">.</span><span class="n">Sequential</span><span class="p">(</span><span class="o">*</span><span class="nb">list</span><span class="p">(</span><span class="n">model</span><span class="o">.</span><span class="n">children</span><span class="p">())[:</span><span class="o">-</span><span class="mi">2</span><span class="p">])</span>
</pre></div>

    </div>
</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[107]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="kn">from</span> <span class="nn">PIL</span> <span class="kn">import</span> <span class="n">Image</span>
<span class="kn">import</span> <span class="nn">os</span>
<span class="n">poster_folder</span><span class="o">=</span><span class="s1">&#39;posters_final/&#39;</span>
<span class="n">allnames</span><span class="o">=</span><span class="n">os</span><span class="o">.</span><span class="n">listdir</span><span class="p">(</span><span class="n">poster_folder</span><span class="p">)</span>
<span class="n">imnames</span><span class="o">=</span><span class="p">[</span><span class="n">j</span> <span class="k">for</span> <span class="n">j</span> <span class="ow">in</span> <span class="n">allnames</span> <span class="k">if</span> <span class="n">j</span><span class="o">.</span><span class="n">endswith</span><span class="p">(</span><span class="s1">&#39;.jpg&#39;</span><span class="p">)]</span>
<span class="n">feature_list</span><span class="o">=</span><span class="p">[]</span>
<span class="n">genre_list</span><span class="o">=</span><span class="p">[]</span>
<span class="n">file_order</span><span class="o">=</span><span class="p">[]</span>
<span class="k">print</span> <span class="s2">&quot;Starting extracting VGG features for scraped images. This will take time, Please be patient...&quot;</span>
<span class="k">print</span> <span class="s2">&quot;Total images = &quot;</span><span class="p">,</span><span class="nb">len</span><span class="p">(</span><span class="n">imnames</span><span class="p">)</span>
<span class="n">failed_files</span><span class="o">=</span><span class="p">[]</span>
<span class="n">succesful_files</span><span class="o">=</span><span class="p">[]</span>
<span class="n">i</span><span class="o">=</span><span class="mi">0</span>

<span class="k">for</span> <span class="n">mov</span> <span class="ow">in</span> <span class="n">poster_movies</span><span class="p">:</span>
    <span class="n">i</span><span class="o">+=</span><span class="mi">1</span>
    <span class="n">mov_name</span><span class="o">=</span><span class="n">mov</span><span class="p">[</span><span class="s1">&#39;original_title&#39;</span><span class="p">]</span>
    <span class="n">mov_name1</span><span class="o">=</span><span class="n">mov_name</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="s1">&#39;:&#39;</span><span class="p">,</span><span class="s1">&#39;/&#39;</span><span class="p">)</span>
    <span class="n">poster_name</span><span class="o">=</span><span class="n">mov_name</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="s1">&#39; &#39;</span><span class="p">,</span><span class="s1">&#39;_&#39;</span><span class="p">)</span><span class="o">+</span><span class="s1">&#39;.jpg&#39;</span>
    <span class="k">if</span> <span class="n">poster_name</span> <span class="ow">in</span> <span class="n">imnames</span><span class="p">:</span>
        
        <span class="n">img_path</span><span class="o">=</span><span class="n">poster_folder</span><span class="o">+</span><span class="n">poster_name</span>
        <span class="k">try</span><span class="p">:</span>
            <span class="n">img</span> <span class="o">=</span> <span class="n">Image</span><span class="o">.</span><span class="n">open</span><span class="p">(</span><span class="n">img_path</span><span class="p">)</span>
            <span class="n">min_img_size</span> <span class="o">=</span> <span class="mi">224</span>  <span class="c1"># The min size, as noted in the PyTorch pretrained models doc, is 224 px.</span>
            <span class="n">transform_pipeline</span> <span class="o">=</span> <span class="n">transforms</span><span class="o">.</span><span class="n">Compose</span><span class="p">([</span><span class="n">transforms</span><span class="o">.</span><span class="n">Resize</span><span class="p">([</span><span class="n">min_img_size</span><span class="p">,</span><span class="n">min_img_size</span><span class="p">]),</span>
                                                     <span class="n">transforms</span><span class="o">.</span><span class="n">ToTensor</span><span class="p">(),</span>
                                                     <span class="n">transforms</span><span class="o">.</span><span class="n">Normalize</span><span class="p">(</span><span class="n">mean</span><span class="o">=</span><span class="p">[</span><span class="mf">0.485</span><span class="p">,</span> <span class="mf">0.456</span><span class="p">,</span> <span class="mf">0.406</span><span class="p">],</span>
                                                                          <span class="n">std</span><span class="o">=</span><span class="p">[</span><span class="mf">0.229</span><span class="p">,</span> <span class="mf">0.224</span><span class="p">,</span> <span class="mf">0.225</span><span class="p">])])</span>
            <span class="n">x</span> <span class="o">=</span> <span class="n">transform_pipeline</span><span class="p">(</span><span class="n">img</span><span class="p">)</span>
            <span class="n">img</span><span class="o">.</span><span class="n">close</span><span class="p">()</span>
            <span class="c1"># PyTorch pretrained models expect the Tensor dims to be (num input imgs, num color channels, height, width).</span>
            <span class="c1"># Currently however, we have (num color channels, height, width); let&#39;s fix this by inserting a new axis.</span>
<span class="c1">#             print len(img)</span>
<span class="c1">#             print img.shape</span>
<span class="c1">#             print img</span>
<span class="c1">#             break</span>
            <span class="n">x</span> <span class="o">=</span> <span class="n">x</span><span class="o">.</span><span class="n">unsqueeze</span><span class="p">(</span><span class="mi">0</span><span class="p">)</span>  <span class="c1"># Insert the new axis at index 0 i.e. in front of the other axes/dims. </span>
            
            
            <span class="c1"># Now that we have preprocessed our img, we need to convert it into a </span>
            <span class="c1"># Variable; PyTorch models expect inputs to be Variables. A PyTorch Variable is a  </span>
            <span class="c1"># wrapper around a PyTorch Tensor.</span>
            <span class="n">x</span> <span class="o">=</span> <span class="n">Variable</span><span class="p">(</span><span class="n">x</span><span class="p">)</span>
            <span class="n">succesful_files</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">poster_name</span><span class="p">)</span>
            
            <span class="n">features</span> <span class="o">=</span> <span class="n">model</span><span class="p">(</span><span class="n">x</span><span class="p">)</span>
            
            <span class="n">file_order</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">img_path</span><span class="p">)</span>
            <span class="n">feature_list</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">features</span><span class="p">)</span>
            <span class="n">genre_list</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">mov</span><span class="p">[</span><span class="s1">&#39;genre_ids&#39;</span><span class="p">])</span>
            <span class="n">max_prediction</span> <span class="o">=</span> <span class="n">features</span><span class="o">.</span><span class="n">data</span><span class="o">.</span><span class="n">numpy</span><span class="p">()</span><span class="o">.</span><span class="n">argmax</span><span class="p">()</span>  <span class="c1"># Our prediction will be the index of the class label with the largest value.</span>
            <span class="k">if</span> <span class="n">max_prediction</span><span class="o">==</span><span class="mf">0.0</span><span class="p">:</span>
                <span class="k">print</span><span class="p">(</span><span class="s1">&#39;problematic&#39;</span><span class="p">,</span><span class="n">i</span><span class="p">)</span>
            <span class="k">if</span> <span class="n">i</span><span class="o">%</span><span class="k">25</span>==0 or i==1:
                <span class="k">print</span> <span class="s2">&quot;Working on Image : &quot;</span><span class="p">,</span><span class="n">i</span>
        <span class="k">except</span><span class="p">:</span>
            <span class="n">failed_files</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">poster_name</span><span class="p">)</span>
            <span class="k">continue</span>
        
    <span class="k">else</span><span class="p">:</span>
        <span class="k">continue</span>
<span class="k">print</span> <span class="s2">&quot;Done with all features, please pickle for future use!&quot;</span>
</pre></div>

    </div>
</div>
</div>

<div class="output_wrapper">
<div class="output">


<div class="output_area">

    <div class="prompt"></div>


<div class="output_subarea output_stream output_stdout output_text">
<pre>Starting extracting VGG features for scraped images. This will take time, Please be patient...
Total images =  1591
Working on Image :  1
Working on Image :  25
Working on Image :  50
Working on Image :  100
Working on Image :  125
Working on Image :  200
Working on Image :  250
Working on Image :  325
Working on Image :  350
Working on Image :  375
Working on Image :  400
Working on Image :  450
Working on Image :  475
Working on Image :  500
Working on Image :  525
Working on Image :  550
Working on Image :  600
Working on Image :  625
Working on Image :  650
Working on Image :  750
Working on Image :  775
Working on Image :  825
Working on Image :  850
Working on Image :  900
Working on Image :  950
Working on Image :  975
Working on Image :  1000
Working on Image :  1075
Working on Image :  1100
Working on Image :  1150
Working on Image :  1200
Working on Image :  1225
Working on Image :  1250
Working on Image :  1275
Working on Image :  1300
Working on Image :  1350
Working on Image :  1375
Working on Image :  1450
Working on Image :  1500
Working on Image :  1575
Working on Image :  1600
Working on Image :  1625
Working on Image :  1650
Working on Image :  1700
Working on Image :  1725
Working on Image :  1750
Done with all features, please pickle for future use!
</pre>
</div>
</div>

</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[118]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="nb">len</span><span class="p">(</span><span class="n">genre_list</span><span class="p">)</span>
</pre></div>

    </div>
</div>
</div>

<div class="output_wrapper">
<div class="output">


<div class="output_area">

    <div class="prompt output_prompt">Out[118]:</div>


<div class="output_text output_subarea output_execute_result">
<pre>1242</pre>
</div>

</div>

</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[119]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="nb">len</span><span class="p">(</span><span class="n">feature_list</span><span class="p">)</span>
</pre></div>

    </div>
</div>
</div>

<div class="output_wrapper">
<div class="output">


<div class="output_area">

    <div class="prompt output_prompt">Out[119]:</div>


<div class="output_text output_subarea output_execute_result">
<pre>1242</pre>
</div>

</div>

</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[120]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="n">feature_list</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">shape</span>
</pre></div>

    </div>
</div>
</div>

<div class="output_wrapper">
<div class="output">


<div class="output_area">

    <div class="prompt output_prompt">Out[120]:</div>


<div class="output_text output_subarea output_execute_result">
<pre>torch.Size([1, 3, 224, 224])</pre>
</div>

</div>

</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[121]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="c1"># Reading from pickle below, this code is not to be run.</span>
<span class="n">list_pickled</span><span class="o">=</span><span class="p">(</span><span class="n">feature_list</span><span class="p">,</span><span class="n">file_order</span><span class="p">,</span><span class="n">failed_files</span><span class="p">,</span><span class="n">succesful_files</span><span class="p">,</span><span class="n">genre_list</span><span class="p">)</span>
<span class="n">f</span><span class="o">=</span><span class="nb">open</span><span class="p">(</span><span class="s1">&#39;posters_new_features.pckl&#39;</span><span class="p">,</span><span class="s1">&#39;wb&#39;</span><span class="p">)</span>
<span class="n">pickle</span><span class="o">.</span><span class="n">dump</span><span class="p">(</span><span class="n">list_pickled</span><span class="p">,</span><span class="n">f</span><span class="p">)</span>
<span class="n">f</span><span class="o">.</span><span class="n">close</span><span class="p">()</span>
<span class="k">print</span><span class="p">(</span><span class="s2">&quot;Features dumped to pickle file&quot;</span><span class="p">)</span>
</pre></div>

    </div>
</div>
</div>

<div class="output_wrapper">
<div class="output">


<div class="output_area">

    <div class="prompt"></div>


<div class="output_subarea output_stream output_stdout output_text">
<pre>Features dumped to pickle file
</pre>
</div>
</div>

</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[122]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="n">f7</span><span class="o">=</span><span class="nb">open</span><span class="p">(</span><span class="s1">&#39;posters_new_features.pckl&#39;</span><span class="p">,</span><span class="s1">&#39;rb&#39;</span><span class="p">)</span>
<span class="n">list_pickled</span><span class="o">=</span><span class="n">pickle</span><span class="o">.</span><span class="n">load</span><span class="p">(</span><span class="n">f7</span><span class="p">)</span>
<span class="n">f7</span><span class="o">.</span><span class="n">close</span><span class="p">()</span>
<span class="c1"># (feature_list2,file_order2)=list_pickled</span>
</pre></div>

    </div>
</div>
</div>

</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h3 id="Training-a-simple-neural-network-model-using-these-VGG-features.">Training a simple neural network model using these VGG features.<a class="anchor-link" href="#Training-a-simple-neural-network-model-using-these-VGG-features.">&#182;</a></h3>
</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[123]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="p">(</span><span class="n">feature_list</span><span class="p">,</span><span class="n">files</span><span class="p">,</span><span class="n">failed</span><span class="p">,</span><span class="n">succesful</span><span class="p">,</span><span class="n">genre_list</span><span class="p">)</span><span class="o">=</span><span class="n">list_pickled</span>
</pre></div>

    </div>
</div>
</div>

</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>Let's first get the labels on our 1342 samples first! As image download fails on a few instances, the best way to work with the right model is to read the poster names downloaded, and working from there. These posters cannot be uploaded to Github as they are too large, and so are being downloaded and read from my local computer. If you do re-do it, you might have to check and edit the paths in the code to make sure it runs.</p>

</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[124]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="p">(</span><span class="n">a</span><span class="p">,</span><span class="n">b</span><span class="p">,</span><span class="n">c</span><span class="p">,</span><span class="n">d</span><span class="p">)</span><span class="o">=</span><span class="n">feature_list</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">shape</span>
<span class="n">feature_size</span><span class="o">=</span><span class="n">a</span><span class="o">*</span><span class="n">b</span><span class="o">*</span><span class="n">c</span><span class="o">*</span><span class="n">d</span>
</pre></div>

    </div>
</div>
</div>

</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>This looks odd, why are we re-running the loop we ran above again below? The reason is simple, the most important thing to know about numpy is that using vstack() and hstack() are highly sub-optimal. Numpy arrays when created, a fixed size is allocated in the memory and when we stack, a new one is copied and created in a new location. This makes the code really, really slow. The best way to do it (and this remains the same with MATLAB matrices if you work with them), is to create a numpy array of zeros, and over-write it row by row. The above code was just to see what size numpy array we will need!</p>

</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>The final movie poster set for which we have all the information we need, is 1265 movies. In the above code we are making an X numpy array containing the visual features of one image per row. So, the VGG features are reshaped to be in the shape (1,25088) and we finally obtain a matrix of shape (1265,25088)</p>

</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[125]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="n">np_features</span><span class="o">=</span><span class="n">np</span><span class="o">.</span><span class="n">zeros</span><span class="p">((</span><span class="nb">len</span><span class="p">(</span><span class="n">feature_list</span><span class="p">),</span><span class="n">feature_size</span><span class="p">))</span>
<span class="k">print</span> <span class="nb">len</span><span class="p">(</span><span class="n">feature_list</span><span class="p">),</span><span class="n">feature_size</span>
<span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="nb">len</span><span class="p">(</span><span class="n">feature_list</span><span class="p">)):</span>
    <span class="n">feat</span><span class="o">=</span><span class="n">feature_list</span><span class="p">[</span><span class="n">i</span><span class="p">]</span>
    <span class="n">reshaped_feat</span><span class="o">=</span><span class="n">feat</span><span class="o">.</span><span class="n">reshape</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span><span class="o">-</span><span class="mi">1</span><span class="p">)</span>
    <span class="c1">#print reshaped_feat.shape</span>
    <span class="n">np_features</span><span class="p">[</span><span class="n">i</span><span class="p">]</span><span class="o">=</span><span class="n">reshaped_feat</span>
</pre></div>

    </div>
</div>
</div>

<div class="output_wrapper">
<div class="output">


<div class="output_area">

    <div class="prompt"></div>


<div class="output_subarea output_stream output_stdout output_text">
<pre>1242 150528
</pre>
</div>
</div>

</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[126]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="c1"># np_features[-1]</span>
</pre></div>

    </div>
</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[127]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="n">X</span><span class="o">=</span><span class="n">np_features</span>
</pre></div>

    </div>
</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[108]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="kn">from</span> <span class="nn">sklearn.preprocessing</span> <span class="kn">import</span> <span class="n">MultiLabelBinarizer</span>
<span class="n">mlb</span><span class="o">=</span><span class="n">MultiLabelBinarizer</span><span class="p">()</span>
<span class="n">Y</span><span class="o">=</span><span class="n">mlb</span><span class="o">.</span><span class="n">fit_transform</span><span class="p">(</span><span class="n">genre_list</span><span class="p">)</span>
</pre></div>

    </div>
</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[129]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="n">Y</span><span class="o">.</span><span class="n">shape</span>
</pre></div>

    </div>
</div>
</div>

<div class="output_wrapper">
<div class="output">


<div class="output_area">

    <div class="prompt output_prompt">Out[129]:</div>


<div class="output_text output_subarea output_execute_result">
<pre>(1242, 19)</pre>
</div>

</div>

</div>
</div>

</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>Our binarized Y numpy array contains the binarized labels corresponding to the genre IDs of the 1277 movies</p>

</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[130]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="n">visual_problem_data</span><span class="o">=</span><span class="p">(</span><span class="n">X</span><span class="p">,</span><span class="n">Y</span><span class="p">)</span>
<span class="n">f8</span><span class="o">=</span><span class="nb">open</span><span class="p">(</span><span class="s1">&#39;visual_problem_data_clean.pckl&#39;</span><span class="p">,</span><span class="s1">&#39;wb&#39;</span><span class="p">)</span>
<span class="n">pickle</span><span class="o">.</span><span class="n">dump</span><span class="p">(</span><span class="n">visual_problem_data</span><span class="p">,</span><span class="n">f8</span><span class="p">)</span>
<span class="n">f8</span><span class="o">.</span><span class="n">close</span><span class="p">()</span>
</pre></div>

    </div>
</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[131]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="n">f8</span><span class="o">=</span><span class="nb">open</span><span class="p">(</span><span class="s1">&#39;visual_problem_data_clean.pckl&#39;</span><span class="p">,</span><span class="s1">&#39;rb&#39;</span><span class="p">)</span>
<span class="n">visual_features</span><span class="o">=</span><span class="n">pickle</span><span class="o">.</span><span class="n">load</span><span class="p">(</span><span class="n">f8</span><span class="p">)</span>
<span class="n">f8</span><span class="o">.</span><span class="n">close</span><span class="p">()</span>
</pre></div>

    </div>
</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[132]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="p">(</span><span class="n">X</span><span class="p">,</span><span class="n">Y</span><span class="p">)</span><span class="o">=</span><span class="n">visual_features</span>
</pre></div>

    </div>
</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[133]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="n">X</span><span class="o">.</span><span class="n">shape</span>
</pre></div>

    </div>
</div>
</div>

<div class="output_wrapper">
<div class="output">


<div class="output_area">

    <div class="prompt output_prompt">Out[133]:</div>


<div class="output_text output_subarea output_execute_result">
<pre>(1242, 150528)</pre>
</div>

</div>

</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[134]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="n">mask</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">random</span><span class="o">.</span><span class="n">rand</span><span class="p">(</span><span class="nb">len</span><span class="p">(</span><span class="n">X</span><span class="p">))</span> <span class="o">&lt;</span> <span class="mf">0.8</span>
</pre></div>

    </div>
</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[135]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="n">X_train</span><span class="o">=</span><span class="n">X</span><span class="p">[</span><span class="n">mask</span><span class="p">]</span>
<span class="n">X_test</span><span class="o">=</span><span class="n">X</span><span class="p">[</span><span class="o">~</span><span class="n">mask</span><span class="p">]</span>
<span class="n">Y_train</span><span class="o">=</span><span class="n">Y</span><span class="p">[</span><span class="n">mask</span><span class="p">]</span>
<span class="n">Y_test</span><span class="o">=</span><span class="n">Y</span><span class="p">[</span><span class="o">~</span><span class="n">mask</span><span class="p">]</span>
<span class="n">X_train</span>
</pre></div>

    </div>
</div>
</div>

<div class="output_wrapper">
<div class="output">


<div class="output_area">

    <div class="prompt output_prompt">Out[135]:</div>


<div class="output_text output_subarea output_execute_result">
<pre>array([[-2.10077929, -2.10077929, -2.11790395, ..., -1.80444443,
        -1.80444443, -1.80444443],
       [-0.85067213, -1.15891778, -1.56991184, ..., -1.54300654,
        -1.54300654, -1.54300654],
       [-1.72403467, -1.62128615, -1.46716332, ..., -1.43843138,
        -1.43843138, -1.43843138],
       ...,
       [-2.11790395, -2.11790395, -2.11790395, ..., -1.80444443,
        -1.80444443, -1.80444443],
       [ 1.13579929,  1.17004883,  1.1871736 , ...,  0.14762534,
         0.11276696,  0.09533776],
       [ 2.21465874,  2.21465874,  2.21465874, ..., -0.95041388,
        -0.07895417,  1.73368204]])</pre>
</div>

</div>

</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[136]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="n">Y_test</span><span class="o">.</span><span class="n">shape</span>
</pre></div>

    </div>
</div>
</div>

<div class="output_wrapper">
<div class="output">


<div class="output_area">

    <div class="prompt output_prompt">Out[136]:</div>


<div class="output_text output_subarea output_execute_result">
<pre>(236, 19)</pre>
</div>

</div>

</div>
</div>

</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>Now, we create our own keras neural network to use the VGG features and then classify movie genres. Keras makes this super easy.</p>
<p>Neural network architectures have gotten complex over the years. But the simplest ones contain very standard computations organized in layers, as described above. Given the popularity of some of these, Keras makes it as easy as writing out the names of these operations in a sequential order. This way you can make a network while completely avoiding the Mathematics (HIGHLY RECOMMENDED SPENDING MORE TIME ON THE MATH THOUGH)</p>

</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>Sequential() allows us to make models the follow this sequential order of layers. Different kinds of layers like Dense, Conv2D etc can be used, and many activation functions like RELU, Linear etc are also available.</p>

</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h1 id="Important-Question-:-Why-do-we-need-activation-functions?">Important Question : Why do we need activation functions?<a class="anchor-link" href="#Important-Question-:-Why-do-we-need-activation-functions?">&#182;</a></h1><h4 id="Copy-pasting-the-answer-I-wrote-for-this-question-on-Quora-Feel-free-to-leave-comments-there.">Copy pasting the answer I wrote for this question on <a href="https://www.quora.com/Why-do-neural-networks-need-an-activation-function/answer/Spandan-Madan?srid=5ydm">Quora</a> Feel free to leave comments there.<a class="anchor-link" href="#Copy-pasting-the-answer-I-wrote-for-this-question-on-Quora-Feel-free-to-leave-comments-there.">&#182;</a></h4><p>""Sometimes, we tend to get lost in the jargon and confuse things easily, so the best way to go about this is getting back to our basics.</p>
<p>Don’t forget what the original premise of machine learning (and thus deep learning) is - IF the input and output are related by a function y=f(x), then if we have x, there is no way to exactly know f unless we know the process itself. However, machine learning gives you the ability to approximate f with a function g, and the process of trying out multiple candidates to identify the function g best approximating f is called machine learning.</p>
<p>Ok, that was machine learning, and how is deep learning different? Deep learning simply tries to expand the possible kind of functions that can be approximated using the above mentioned machine learning paradigm. Roughly speaking, if the previous model could learn say 10,000 kinds of functions, now it will be able to learn say 100,000 kinds (in actuality both are infinite spaces but one is larger than the other, because maths is cool that ways.)</p>
<p>If you want to know the mathematics of it, go read about VC dimension and how more layers in a network affect it. But I will avoid the mathematics here and rely on your intuition to believe me when I say that not all data can be classified correctly into categories using a linear function. So, we need our deep learning model to be able to approximate more complex functions than just a linear function.</p>
<p>Now, let’s come to your non linearity bit. Imagine a linear function y=2x+3, and another one y=4x+7. What happens if I pool them and take an average? I get another linear function y= 3x+5. So instead of doing those two computations separately and then averaging it out, I could have just used the single linear function y=3x+5. Obviously, this logic holds good if I have more than 2 such linear functions. This is exactly what will happen if you don’t have have non-linearities in your nodes, and also what others have written in their answers.</p>
<p>It simply follows from the definition of a linear function -</p>
<p>(i) If you take two linear functions, AND</p>
<p>(ii)Take a linear combination of them (which is how we combine the outputs of multiple nodes of a network)</p>
<p>You are BOUND to get a linear function because f(x)+g(x)=mx+b+nx+c=(m+n)x+(b+c)= say h(x).</p>
<p>And you could in essence replace your whole network by a simple matrix transformation which accounts for all linear combinations and up/downsamplings.</p>
<p>In a nutshell, you’ll only be trying to learn a linear approximation for original function f relating the input and the output. Which as we discussed above, is not always the best approximation. Adding non-linearities ensures that you can learn more complex functions by approximating every non-linear function as a LINEAR combination of a large number of non-linear functions.</p>
<p>Still new to the field, so if there’s something wrong here please comment below! Hope it helps""</p>

</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h4 id="Let's-train-our-model-then,-using-the-features-we-extracted-from-VGG-net">Let's train our model then, using the features we extracted from VGG net<a class="anchor-link" href="#Let's-train-our-model-then,-using-the-features-we-extracted-from-VGG-net">&#182;</a></h4><p>The model we will use has just 1 hidden layer between the VGG features and the final output layer. The simplest neural network you can get. An image goes into this network with the dimensions (1,25088), the first layer's output is 1024 dimensional. This hidden layer output undergoes a pointwise RELU activation. This output gets transformed into the output layer of 20 dimensions. It goes through a sigmoid.</p>
<p>The sigmoid, or the squashing function as it is often called, is a function which squashes numbers between 0 and 1. What are you reminded of when you think of numebers between 0 and 1? Right, probability.</p>
<p>By squashing the score of each of the 20 output labels between 0 and 1, sigmoid lets us interpret their scores as probabilities. Then, we can just pick the classes with the top 3 or 5 probability scores as the predicted genres for the movie poster! Simple!</p>

</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[30]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="kn">from</span> <span class="nn">__future__</span> <span class="kn">import</span> <span class="n">division</span>
<span class="kn">import</span> <span class="nn">torch</span>
<span class="kn">import</span> <span class="nn">time</span>
<span class="kn">import</span> <span class="nn">copy</span>


<span class="k">class</span> <span class="nc">FeatureDataset</span><span class="p">():</span>
    <span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">X_train</span><span class="p">,</span> <span class="n">Y_train</span><span class="p">,</span> <span class="n">transform</span><span class="o">=</span><span class="bp">None</span><span class="p">):</span>
    
        <span class="bp">self</span><span class="o">.</span><span class="n">X_train</span> <span class="o">=</span> <span class="n">X_train</span>
        <span class="bp">self</span><span class="o">.</span><span class="n">Y_train</span> <span class="o">=</span> <span class="n">Y_train</span>
        <span class="bp">self</span><span class="o">.</span><span class="n">transform</span> <span class="o">=</span> <span class="n">transform</span>

    <span class="k">def</span> <span class="fm">__len__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
        <span class="k">return</span> <span class="nb">len</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">X_train</span><span class="p">)</span>

    <span class="k">def</span> <span class="fm">__getitem__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">idx</span><span class="p">):</span>
        
        <span class="c1">#sample = {&#39;image&#39;: self.X_train[idx], &#39;landmarks&#39;: self.Y_train[idx]}</span>

<span class="c1">#         if self.transform:</span>
<span class="c1">#             sample = self.transform(sample)</span>

        <span class="k">return</span> <span class="n">X_train</span><span class="p">[</span><span class="n">idx</span><span class="p">]</span><span class="o">.</span><span class="n">astype</span><span class="p">(</span><span class="nb">float</span><span class="p">),</span> <span class="n">Y_train</span><span class="p">[</span><span class="n">idx</span><span class="p">]</span><span class="o">.</span><span class="n">astype</span><span class="p">(</span><span class="nb">float</span><span class="p">)</span>

<span class="k">def</span> <span class="nf">train_model</span><span class="p">(</span><span class="n">model</span><span class="p">,</span> <span class="n">criterion</span><span class="p">,</span> <span class="n">optimizer</span><span class="p">,</span> <span class="n">num_epochs</span><span class="p">):</span>
    
    <span class="n">since</span> <span class="o">=</span> <span class="n">time</span><span class="o">.</span><span class="n">time</span><span class="p">()</span>
    <span class="n">USE_TENSORBOARD</span> <span class="o">=</span> <span class="mi">0</span>
    <span class="n">best_model</span> <span class="o">=</span> <span class="n">model</span>
    <span class="n">best_acc</span> <span class="o">=</span> <span class="mf">0.0</span>
    <span class="n">use_gpu</span> <span class="o">=</span> <span class="mi">0</span>
    
    <span class="k">for</span> <span class="n">epoch</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="n">num_epochs</span><span class="p">):</span>
        <span class="k">print</span><span class="p">(</span><span class="s1">&#39;Epoch {}/{}&#39;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">epoch</span><span class="p">,</span> <span class="n">num_epochs</span> <span class="o">-</span> <span class="mi">1</span><span class="p">))</span>
        <span class="k">print</span><span class="p">(</span><span class="s1">&#39;-&#39;</span> <span class="o">*</span> <span class="mi">10</span><span class="p">)</span>

        <span class="c1"># Each epoch has a training and validation phase</span>
        <span class="k">for</span> <span class="n">phase</span> <span class="ow">in</span> <span class="p">[</span><span class="s1">&#39;train&#39;</span><span class="p">,</span> <span class="s1">&#39;test&#39;</span><span class="p">]:</span>
            <span class="k">if</span> <span class="n">phase</span> <span class="o">==</span> <span class="s1">&#39;train&#39;</span><span class="p">:</span>
                <span class="n">mode</span><span class="o">=</span><span class="s1">&#39;train&#39;</span>
                <span class="c1">#optimizer = lr_scheduler(optimizer, epoch)</span>
                <span class="n">model</span><span class="o">.</span><span class="n">train</span><span class="p">()</span>  <span class="c1"># Set model to training mode</span>
                <span class="k">print</span> <span class="s2">&quot;TRAINING STARTED&quot;</span>
            <span class="k">else</span><span class="p">:</span>
                <span class="n">model</span><span class="o">.</span><span class="n">eval</span><span class="p">()</span>
                <span class="n">mode</span><span class="o">=</span><span class="s1">&#39;val&#39;</span>
                <span class="k">print</span> <span class="s2">&quot;TESTING STARTED&quot;</span>

            <span class="n">running_loss</span> <span class="o">=</span> <span class="mf">0.0</span>
            <span class="n">running_corrects</span> <span class="o">=</span> <span class="mi">0</span>

            <span class="n">counter</span><span class="o">=</span><span class="mi">0</span>
            <span class="c1"># Iterate over data.</span>
            <span class="n">answer</span> <span class="o">=</span> <span class="p">[]</span>
            <span class="k">for</span> <span class="n">data</span> <span class="ow">in</span> <span class="n">dset_loaders</span><span class="p">[</span><span class="n">phase</span><span class="p">]:</span>
                <span class="n">inputs</span><span class="p">,</span> <span class="n">labels</span> <span class="o">=</span> <span class="n">data</span> 
                
                <span class="c1"># wrap them in Variable</span>
                <span class="k">if</span> <span class="n">use_gpu</span><span class="p">:</span>
                    <span class="k">try</span><span class="p">:</span>
                        <span class="n">inputs</span><span class="p">,</span> <span class="n">labels</span> <span class="o">=</span> <span class="n">Variable</span><span class="p">(</span><span class="n">inputs</span><span class="o">.</span><span class="n">float</span><span class="p">()</span><span class="o">.</span><span class="n">cuda</span><span class="p">()),</span>                             
                        <span class="n">Variable</span><span class="p">(</span><span class="n">labels</span><span class="o">.</span><span class="n">long</span><span class="p">()</span><span class="o">.</span><span class="n">cuda</span><span class="p">())</span>
                    <span class="k">except</span><span class="p">:</span>
                        <span class="k">print</span><span class="p">(</span><span class="n">inputs</span><span class="p">,</span><span class="n">labels</span><span class="p">)</span>
                <span class="k">else</span><span class="p">:</span>
                    <span class="n">inputs</span><span class="p">,</span> <span class="n">labels</span> <span class="o">=</span> <span class="n">Variable</span><span class="p">(</span><span class="n">Variable</span><span class="p">(</span><span class="n">inputs</span><span class="p">)</span><span class="o">.</span><span class="n">float</span><span class="p">()),</span> <span class="n">Variable</span><span class="p">(</span><span class="n">Variable</span><span class="p">(</span><span class="n">labels</span><span class="p">)</span><span class="o">.</span><span class="n">float</span><span class="p">())</span>

                <span class="c1"># Set gradient to zero to delete history of computations in previous epoch. Track operations so that differentiation can be done automatically.</span>
                <span class="n">optimizer</span><span class="o">.</span><span class="n">zero_grad</span><span class="p">()</span>
                <span class="n">outputs</span> <span class="o">=</span> <span class="n">model</span><span class="p">(</span><span class="n">inputs</span><span class="p">)</span>
                <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="nb">len</span><span class="p">(</span><span class="n">outputs</span><span class="o">.</span><span class="n">data</span><span class="p">)):</span>
                    <span class="n">answer</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">outputs</span><span class="o">.</span><span class="n">data</span><span class="p">[</span><span class="n">i</span><span class="p">])</span>
                <span class="n">_</span><span class="p">,</span> <span class="n">preds</span> <span class="o">=</span> <span class="n">torch</span><span class="o">.</span><span class="n">max</span><span class="p">(</span><span class="n">outputs</span><span class="o">.</span><span class="n">data</span><span class="p">,</span> <span class="mi">1</span><span class="p">)</span>
                <span class="c1">#print outputs.data</span>
                <span class="c1">#print _, preds</span>
                <span class="n">loss</span> <span class="o">=</span> <span class="n">criterion</span><span class="p">(</span><span class="n">outputs</span><span class="p">,</span> <span class="n">labels</span><span class="p">)</span>
                <span class="k">print</span><span class="p">(</span><span class="s1">&#39;loss done&#39;</span><span class="p">)</span>                
                <span class="c1"># Just so that you can keep track that something&#39;s happening and don&#39;t feel like the program isn&#39;t running.</span>
                <span class="k">if</span> <span class="n">counter</span><span class="o">%</span><span class="k">50</span>==0:
                    <span class="k">print</span><span class="p">(</span><span class="s2">&quot;Reached iteration &quot;</span><span class="p">,</span><span class="n">counter</span><span class="p">)</span>
                <span class="n">counter</span><span class="o">+=</span><span class="mi">1</span>

                <span class="c1"># backward + optimize only if in training phase</span>
                <span class="k">if</span> <span class="n">phase</span> <span class="o">==</span> <span class="s1">&#39;train&#39;</span><span class="p">:</span>
                    <span class="k">print</span><span class="p">(</span><span class="s1">&#39;loss backward&#39;</span><span class="p">)</span>
                    <span class="n">loss</span><span class="o">.</span><span class="n">backward</span><span class="p">()</span>
                    <span class="k">print</span><span class="p">(</span><span class="s1">&#39;done loss backward&#39;</span><span class="p">)</span>
                    <span class="n">optimizer</span><span class="o">.</span><span class="n">step</span><span class="p">()</span>
                    <span class="k">print</span><span class="p">(</span><span class="s1">&#39;done optim&#39;</span><span class="p">)</span>
                <span class="c1"># print evaluation statistics</span>
                <span class="k">try</span><span class="p">:</span>
                    <span class="n">running_loss</span> <span class="o">+=</span> <span class="n">loss</span><span class="o">.</span><span class="n">data</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span>

                    <span class="k">print</span> <span class="n">preds</span><span class="o">.</span><span class="n">shape</span><span class="p">,</span> <span class="n">labels</span><span class="o">.</span><span class="n">data</span><span class="o">.</span><span class="n">shape</span>
                    <span class="k">for</span> <span class="n">q</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="nb">len</span><span class="p">(</span><span class="n">labels</span><span class="o">.</span><span class="n">data</span><span class="p">)):</span>
                        <span class="k">if</span> <span class="n">labels</span><span class="o">.</span><span class="n">data</span><span class="p">[</span><span class="n">q</span><span class="p">][</span><span class="n">preds</span><span class="p">[</span><span class="n">q</span><span class="p">]]</span><span class="o">==</span><span class="mi">1</span><span class="p">:</span>
                            <span class="n">running_corrects</span> <span class="o">+=</span> <span class="mi">1</span>
                    
                    <span class="c1">#running_corrects += torch.sum(preds == labels.data.long())</span>
                <span class="k">except</span><span class="p">:</span>
                    <span class="k">print</span><span class="p">(</span><span class="s1">&#39;unexpected error, could not calculate loss or do a sum.&#39;</span><span class="p">)</span>
            <span class="k">print</span><span class="p">(</span><span class="s1">&#39;trying epoch loss&#39;</span><span class="p">)</span>
            <span class="n">epoch_loss</span> <span class="o">=</span> <span class="n">running_loss</span> <span class="o">/</span> <span class="n">dset_sizes</span><span class="p">[</span><span class="n">phase</span><span class="p">]</span>
            <span class="n">epoch_acc</span> <span class="o">=</span> <span class="n">running_corrects</span> <span class="o">/</span> <span class="n">dset_sizes</span><span class="p">[</span><span class="n">phase</span><span class="p">]</span>
            <span class="k">print</span> <span class="n">phase</span><span class="p">,</span> <span class="n">running_corrects</span><span class="p">,</span> <span class="n">dset_sizes</span><span class="p">[</span><span class="n">phase</span><span class="p">]</span>
            <span class="k">print</span><span class="p">(</span><span class="s1">&#39;{} Loss: {:.4f} Acc: {:.4f}&#39;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span>
                <span class="n">phase</span><span class="p">,</span> <span class="n">epoch_loss</span><span class="p">,</span> <span class="n">epoch_acc</span><span class="p">))</span>


            <span class="c1"># deep copy the model</span>
            <span class="k">if</span> <span class="n">phase</span> <span class="o">==</span> <span class="s1">&#39;test&#39;</span><span class="p">:</span>
                <span class="k">if</span> <span class="n">USE_TENSORBOARD</span><span class="p">:</span>
                    <span class="n">foo</span><span class="o">.</span><span class="n">add_scalar_value</span><span class="p">(</span><span class="s1">&#39;epoch_loss&#39;</span><span class="p">,</span><span class="n">epoch_loss</span><span class="p">,</span><span class="n">step</span><span class="o">=</span><span class="n">epoch</span><span class="p">)</span>
                    <span class="n">foo</span><span class="o">.</span><span class="n">add_scalar_value</span><span class="p">(</span><span class="s1">&#39;epoch_acc&#39;</span><span class="p">,</span><span class="n">epoch_acc</span><span class="p">,</span><span class="n">step</span><span class="o">=</span><span class="n">epoch</span><span class="p">)</span>
                <span class="k">if</span> <span class="n">epoch_acc</span> <span class="o">&gt;</span> <span class="n">best_acc</span><span class="p">:</span>
                    <span class="n">best_acc</span> <span class="o">=</span> <span class="n">epoch_acc</span>
                    <span class="n">best_model</span> <span class="o">=</span> <span class="n">copy</span><span class="o">.</span><span class="n">deepcopy</span><span class="p">(</span><span class="n">model</span><span class="p">)</span>
                    <span class="k">print</span><span class="p">(</span><span class="s1">&#39;new best accuracy = &#39;</span><span class="p">,</span><span class="n">best_acc</span><span class="p">)</span>
    <span class="n">time_elapsed</span> <span class="o">=</span> <span class="n">time</span><span class="o">.</span><span class="n">time</span><span class="p">()</span> <span class="o">-</span> <span class="n">since</span>
    <span class="k">print</span><span class="p">(</span><span class="s1">&#39;Training complete in {:.0f}m {:.0f}s&#39;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span>
        <span class="n">time_elapsed</span> <span class="o">//</span> <span class="mi">60</span><span class="p">,</span> <span class="n">time_elapsed</span> <span class="o">%</span> <span class="mi">60</span><span class="p">))</span>
    <span class="k">print</span><span class="p">(</span><span class="s1">&#39;Best val Acc: {:4f}&#39;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">best_acc</span><span class="p">))</span>
    <span class="k">print</span><span class="p">(</span><span class="s1">&#39;returning and looping back&#39;</span><span class="p">)</span>
    <span class="c1">#torch.save(best_model.state_dict(), &#39;fine_tuned_best_model.pt&#39;)</span>
    <span class="k">return</span> <span class="n">best_model</span><span class="p">,</span> <span class="n">answer</span>
</pre></div>

    </div>
</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[138]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="k">print</span><span class="p">(</span><span class="n">X_train</span><span class="p">)</span>
</pre></div>

    </div>
</div>
</div>

<div class="output_wrapper">
<div class="output">


<div class="output_area">

    <div class="prompt"></div>


<div class="output_subarea output_stream output_stdout output_text">
<pre>[[-2.10077929 -2.10077929 -2.11790395 ... -1.80444443 -1.80444443
  -1.80444443]
 [-0.85067213 -1.15891778 -1.56991184 ... -1.54300654 -1.54300654
  -1.54300654]
 [-1.72403467 -1.62128615 -1.46716332 ... -1.43843138 -1.43843138
  -1.43843138]
 ...
 [-2.11790395 -2.11790395 -2.11790395 ... -1.80444443 -1.80444443
  -1.80444443]
 [ 1.13579929  1.17004883  1.1871736  ...  0.14762534  0.11276696
   0.09533776]
 [ 2.21465874  2.21465874  2.21465874 ... -0.95041388 -0.07895417
   1.73368204]]
</pre>
</div>
</div>

</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[&nbsp;]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span> 
</pre></div>

    </div>
</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[139]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="n">dsets</span> <span class="o">=</span> <span class="p">{}</span>
<span class="n">dsets</span><span class="p">[</span><span class="s1">&#39;train&#39;</span><span class="p">]</span> <span class="o">=</span> <span class="n">FeatureDataset</span><span class="p">(</span><span class="n">X_train</span><span class="p">,</span> <span class="n">Y_train</span><span class="p">)</span>
<span class="n">dsets</span><span class="p">[</span><span class="s1">&#39;test&#39;</span><span class="p">]</span> <span class="o">=</span> <span class="n">FeatureDataset</span><span class="p">(</span><span class="n">X_test</span><span class="p">,</span> <span class="n">Y_test</span><span class="p">)</span>
<span class="n">dset_loaders</span> <span class="o">=</span> <span class="p">{</span><span class="n">x</span><span class="p">:</span> <span class="n">torch</span><span class="o">.</span><span class="n">utils</span><span class="o">.</span><span class="n">data</span><span class="o">.</span><span class="n">DataLoader</span><span class="p">(</span><span class="n">dsets</span><span class="p">[</span><span class="n">x</span><span class="p">],</span> <span class="n">batch_size</span><span class="o">=</span><span class="mi">64</span><span class="p">,</span>
                                               <span class="n">shuffle</span><span class="o">=</span><span class="bp">True</span><span class="p">,</span> <span class="n">num_workers</span><span class="o">=</span><span class="mi">25</span><span class="p">)</span>
                <span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="p">[</span><span class="s1">&#39;train&#39;</span><span class="p">,</span> <span class="s1">&#39;test&#39;</span><span class="p">]}</span>

<span class="n">dset_sizes</span> <span class="o">=</span> <span class="p">{</span><span class="n">x</span><span class="p">:</span> <span class="nb">len</span><span class="p">(</span><span class="n">dsets</span><span class="p">[</span><span class="n">x</span><span class="p">])</span> <span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="p">[</span><span class="s1">&#39;train&#39;</span><span class="p">,</span> <span class="s1">&#39;test&#39;</span><span class="p">]}</span>
<span class="k">print</span> <span class="n">dset_sizes</span>
<span class="n">model_visual</span> <span class="o">=</span> <span class="n">nn</span><span class="o">.</span><span class="n">Sequential</span><span class="p">(</span>
    <span class="n">nn</span><span class="o">.</span><span class="n">Linear</span><span class="p">(</span><span class="n">X</span><span class="o">.</span><span class="n">shape</span><span class="p">[</span><span class="mi">1</span><span class="p">],</span><span class="mi">1024</span><span class="p">),</span>  <span class="c1">#150528</span>
    <span class="n">nn</span><span class="o">.</span><span class="n">ReLU</span><span class="p">(),</span>
    <span class="n">nn</span><span class="o">.</span><span class="n">Linear</span><span class="p">(</span><span class="mi">1024</span><span class="p">,</span><span class="mi">256</span><span class="p">),</span>
    <span class="n">nn</span><span class="o">.</span><span class="n">ReLU</span><span class="p">(),</span>
    <span class="n">nn</span><span class="o">.</span><span class="n">Linear</span><span class="p">(</span><span class="mi">256</span><span class="p">,</span><span class="n">Y_test</span><span class="o">.</span><span class="n">shape</span><span class="p">[</span><span class="mi">1</span><span class="p">]),</span>
    <span class="n">nn</span><span class="o">.</span><span class="n">Sigmoid</span><span class="p">()</span>
<span class="p">)</span>
<span class="c1"># #opt = optimizers.rmsprop(lr=0.0001, decay=1e-6)</span>
<span class="n">optimizer</span> <span class="o">=</span> <span class="n">torch</span><span class="o">.</span><span class="n">optim</span><span class="o">.</span><span class="n">RMSprop</span><span class="p">(</span><span class="n">model_visual</span><span class="o">.</span><span class="n">parameters</span><span class="p">(),</span> <span class="n">lr</span><span class="o">=</span><span class="mf">0.0001</span><span class="p">,</span> <span class="n">weight_decay</span><span class="o">=</span><span class="mf">1e-6</span><span class="p">)</span>
<span class="n">criterion</span> <span class="o">=</span> <span class="n">nn</span><span class="o">.</span><span class="n">BCELoss</span><span class="p">()</span>

<span class="c1">#model_visual.load_state_dict(torch.load(&#39;fine_tuned_best_model.pt&#39;))</span>
<span class="n">model_ft</span><span class="p">,</span> <span class="n">Y_preds</span> <span class="o">=</span> <span class="n">train_model</span><span class="p">(</span><span class="n">model_visual</span><span class="p">,</span> <span class="n">criterion</span><span class="p">,</span> <span class="n">optimizer</span><span class="p">,</span> <span class="n">num_epochs</span><span class="o">=</span><span class="mi">50</span><span class="p">)</span>

<span class="c1"># Save model</span>
<span class="n">torch</span><span class="o">.</span><span class="n">save</span><span class="p">(</span><span class="n">model_ft</span><span class="o">.</span><span class="n">state_dict</span><span class="p">(),</span> <span class="s1">&#39;fine_tuned_best_model.pt&#39;</span><span class="p">)</span>
<span class="c1">#model_ft.save_state_dict(&#39;fine_tuned_best_model.pt&#39;)</span>
<span class="c1"># #sgd = optimizers.SGD(lr=0.05, decay=1e-6, momentum=0.4, nesterov=False)</span>
<span class="c1"># model_visual.compile(optimizer=opt,</span>
<span class="c1">#               loss=&#39;binary_crossentropy&#39;,</span>
<span class="c1">#               metrics=[&#39;accuracy&#39;])</span>
</pre></div>

    </div>
</div>
</div>

<div class="output_wrapper">
<div class="output">


<div class="output_area">

    <div class="prompt"></div>


<div class="output_subarea output_stream output_stderr output_text">
<pre>/anaconda3/envs/deeplearningproject/lib/python2.7/site-packages/torchvision-0.2.1-py2.7.egg/torchvision/transforms/transforms.py:563: UserWarning: The use of the transforms.RandomSizedCrop transform is deprecated, please use transforms.RandomResizedCrop instead.
/anaconda3/envs/deeplearningproject/lib/python2.7/site-packages/torchvision-0.2.1-py2.7.egg/torchvision/transforms/transforms.py:188: UserWarning: The use of the transforms.Scale transform is deprecated, please use transforms.Resize instead.
</pre>
</div>
</div>

<div class="output_area">

    <div class="prompt"></div>


<div class="output_subarea output_stream output_stdout output_text">
<pre>{&#39;test&#39;: 236, &#39;train&#39;: 1006}
Epoch 0/49
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
</pre>
</div>
</div>

<div class="output_area">

    <div class="prompt"></div>


<div class="output_subarea output_stream output_stderr output_text">
<pre>/anaconda3/envs/deeplearningproject/lib/python2.7/site-packages/ipykernel_launcher.py:93: UserWarning: invalid index of a 0-dim tensor. This will be an error in PyTorch 0.5. Use tensor.item() to convert a 0-dim tensor to a Python number
</pre>
</div>
</div>

<div class="output_area">

    <div class="prompt"></div>


<div class="output_subarea output_stream output_stdout output_text">
<pre>loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([46]) torch.Size([46, 19])
trying epoch loss
train 332 1006
train Loss: 0.0996 Acc: 0.3300
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([44]) torch.Size([44, 19])
trying epoch loss
test 92 236
test Loss: 0.0958 Acc: 0.3898
(&#39;new best accuracy = &#39;, 0.3898305084745763)
Epoch 1/49
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([46]) torch.Size([46, 19])
trying epoch loss
train 479 1006
train Loss: 0.0638 Acc: 0.4761
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([44]) torch.Size([44, 19])
trying epoch loss
test 156 236
test Loss: 0.0334 Acc: 0.6610
(&#39;new best accuracy = &#39;, 0.6610169491525424)
Epoch 2/49
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([46]) torch.Size([46, 19])
trying epoch loss
train 587 1006
train Loss: 0.0218 Acc: 0.5835
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([44]) torch.Size([44, 19])
trying epoch loss
test 171 236
test Loss: 0.0127 Acc: 0.7246
(&#39;new best accuracy = &#39;, 0.7245762711864406)
Epoch 3/49
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([46]) torch.Size([46, 19])
trying epoch loss
train 701 1006
train Loss: 0.0100 Acc: 0.6968
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([44]) torch.Size([44, 19])
trying epoch loss
test 196 236
test Loss: 0.0074 Acc: 0.8305
(&#39;new best accuracy = &#39;, 0.8305084745762712)
Epoch 4/49
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([46]) torch.Size([46, 19])
trying epoch loss
train 799 1006
train Loss: 0.0061 Acc: 0.7942
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([44]) torch.Size([44, 19])
trying epoch loss
test 215 236
test Loss: 0.0041 Acc: 0.9110
(&#39;new best accuracy = &#39;, 0.9110169491525424)
Epoch 5/49
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([46]) torch.Size([46, 19])
trying epoch loss
train 859 1006
train Loss: 0.0042 Acc: 0.8539
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([44]) torch.Size([44, 19])
trying epoch loss
test 206 236
test Loss: 0.0041 Acc: 0.8729
Epoch 6/49
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([46]) torch.Size([46, 19])
trying epoch loss
train 882 1006
train Loss: 0.0034 Acc: 0.8767
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([44]) torch.Size([44, 19])
trying epoch loss
test 210 236
test Loss: 0.0034 Acc: 0.8898
Epoch 7/49
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([46]) torch.Size([46, 19])
trying epoch loss
train 923 1006
train Loss: 0.0025 Acc: 0.9175
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([44]) torch.Size([44, 19])
trying epoch loss
test 227 236
test Loss: 0.0025 Acc: 0.9619
(&#39;new best accuracy = &#39;, 0.961864406779661)
Epoch 8/49
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([46]) torch.Size([46, 19])
trying epoch loss
train 953 1006
train Loss: 0.0019 Acc: 0.9473
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([44]) torch.Size([44, 19])
trying epoch loss
test 224 236
test Loss: 0.0016 Acc: 0.9492
Epoch 9/49
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([46]) torch.Size([46, 19])
trying epoch loss
train 965 1006
train Loss: 0.0015 Acc: 0.9592
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([44]) torch.Size([44, 19])
trying epoch loss
test 224 236
test Loss: 0.0018 Acc: 0.9492
Epoch 10/49
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([46]) torch.Size([46, 19])
trying epoch loss
train 962 1006
train Loss: 0.0014 Acc: 0.9563
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([44]) torch.Size([44, 19])
trying epoch loss
test 235 236
test Loss: 0.0011 Acc: 0.9958
(&#39;new best accuracy = &#39;, 0.9957627118644068)
Epoch 11/49
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([46]) torch.Size([46, 19])
trying epoch loss
train 968 1006
train Loss: 0.0012 Acc: 0.9622
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([44]) torch.Size([44, 19])
trying epoch loss
test 230 236
test Loss: 0.0012 Acc: 0.9746
Epoch 12/49
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([46]) torch.Size([46, 19])
trying epoch loss
train 984 1006
train Loss: 0.0010 Acc: 0.9781
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([44]) torch.Size([44, 19])
trying epoch loss
test 221 236
test Loss: 0.0021 Acc: 0.9364
Epoch 13/49
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([46]) torch.Size([46, 19])
trying epoch loss
train 956 1006
train Loss: 0.0014 Acc: 0.9503
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([44]) torch.Size([44, 19])
trying epoch loss
test 231 236
test Loss: 0.0011 Acc: 0.9788
Epoch 14/49
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([46]) torch.Size([46, 19])
trying epoch loss
train 984 1006
train Loss: 0.0009 Acc: 0.9781
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([44]) torch.Size([44, 19])
trying epoch loss
test 235 236
test Loss: 0.0007 Acc: 0.9958
Epoch 15/49
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([46]) torch.Size([46, 19])
trying epoch loss
train 996 1006
train Loss: 0.0006 Acc: 0.9901
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([44]) torch.Size([44, 19])
trying epoch loss
test 235 236
test Loss: 0.0006 Acc: 0.9958
Epoch 16/49
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([46]) torch.Size([46, 19])
trying epoch loss
train 999 1006
train Loss: 0.0004 Acc: 0.9930
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([44]) torch.Size([44, 19])
trying epoch loss
test 235 236
test Loss: 0.0005 Acc: 0.9958
Epoch 17/49
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([46]) torch.Size([46, 19])
trying epoch loss
train 1000 1006
train Loss: 0.0004 Acc: 0.9940
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([44]) torch.Size([44, 19])
trying epoch loss
test 236 236
test Loss: 0.0005 Acc: 1.0000
(&#39;new best accuracy = &#39;, 1.0)
Epoch 18/49
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([46]) torch.Size([46, 19])
trying epoch loss
train 992 1006
train Loss: 0.0005 Acc: 0.9861
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([44]) torch.Size([44, 19])
trying epoch loss
test 235 236
test Loss: 0.0006 Acc: 0.9958
Epoch 19/49
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([46]) torch.Size([46, 19])
trying epoch loss
train 997 1006
train Loss: 0.0004 Acc: 0.9911
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([44]) torch.Size([44, 19])
trying epoch loss
test 236 236
test Loss: 0.0004 Acc: 1.0000
Epoch 20/49
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([46]) torch.Size([46, 19])
trying epoch loss
train 990 1006
train Loss: 0.0005 Acc: 0.9841
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([44]) torch.Size([44, 19])
trying epoch loss
test 229 236
test Loss: 0.0024 Acc: 0.9703
Epoch 21/49
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([46]) torch.Size([46, 19])
trying epoch loss
train 867 1006
train Loss: 0.0058 Acc: 0.8618
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([44]) torch.Size([44, 19])
trying epoch loss
test 222 236
test Loss: 0.0037 Acc: 0.9407
Epoch 22/49
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([46]) torch.Size([46, 19])
trying epoch loss
train 963 1006
train Loss: 0.0026 Acc: 0.9573
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([44]) torch.Size([44, 19])
trying epoch loss
test 233 236
test Loss: 0.0011 Acc: 0.9873
Epoch 23/49
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([46]) torch.Size([46, 19])
trying epoch loss
train 998 1006
train Loss: 0.0005 Acc: 0.9920
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([44]) torch.Size([44, 19])
trying epoch loss
test 235 236
test Loss: 0.0004 Acc: 0.9958
Epoch 24/49
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([46]) torch.Size([46, 19])
trying epoch loss
train 1002 1006
train Loss: 0.0003 Acc: 0.9960
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([44]) torch.Size([44, 19])
trying epoch loss
test 235 236
test Loss: 0.0004 Acc: 0.9958
Epoch 25/49
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([46]) torch.Size([46, 19])
trying epoch loss
train 1003 1006
train Loss: 0.0002 Acc: 0.9970
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([44]) torch.Size([44, 19])
trying epoch loss
test 233 236
test Loss: 0.0005 Acc: 0.9873
Epoch 26/49
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([46]) torch.Size([46, 19])
trying epoch loss
train 1002 1006
train Loss: 0.0002 Acc: 0.9960
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([44]) torch.Size([44, 19])
trying epoch loss
test 235 236
test Loss: 0.0003 Acc: 0.9958
Epoch 27/49
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([46]) torch.Size([46, 19])
trying epoch loss
train 1004 1006
train Loss: 0.0002 Acc: 0.9980
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([44]) torch.Size([44, 19])
trying epoch loss
test 235 236
test Loss: 0.0003 Acc: 0.9958
Epoch 28/49
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([46]) torch.Size([46, 19])
trying epoch loss
train 997 1006
train Loss: 0.0005 Acc: 0.9911
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([44]) torch.Size([44, 19])
trying epoch loss
test 228 236
test Loss: 0.0016 Acc: 0.9661
Epoch 29/49
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([46]) torch.Size([46, 19])
trying epoch loss
train 975 1006
train Loss: 0.0015 Acc: 0.9692
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([44]) torch.Size([44, 19])
trying epoch loss
test 227 236
test Loss: 0.0021 Acc: 0.9619
Epoch 30/49
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([46]) torch.Size([46, 19])
trying epoch loss
train 973 1006
train Loss: 0.0022 Acc: 0.9672
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([44]) torch.Size([44, 19])
trying epoch loss
test 226 236
test Loss: 0.0018 Acc: 0.9576
Epoch 31/49
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([46]) torch.Size([46, 19])
trying epoch loss
train 996 1006
train Loss: 0.0007 Acc: 0.9901
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([44]) torch.Size([44, 19])
trying epoch loss
test 235 236
test Loss: 0.0004 Acc: 0.9958
Epoch 32/49
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([46]) torch.Size([46, 19])
trying epoch loss
train 1004 1006
train Loss: 0.0002 Acc: 0.9980
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([44]) torch.Size([44, 19])
trying epoch loss
test 236 236
test Loss: 0.0003 Acc: 1.0000
Epoch 33/49
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([46]) torch.Size([46, 19])
trying epoch loss
train 1004 1006
train Loss: 0.0001 Acc: 0.9980
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([44]) torch.Size([44, 19])
trying epoch loss
test 235 236
test Loss: 0.0004 Acc: 0.9958
Epoch 34/49
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([46]) torch.Size([46, 19])
trying epoch loss
train 1003 1006
train Loss: 0.0001 Acc: 0.9970
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([44]) torch.Size([44, 19])
trying epoch loss
test 235 236
test Loss: 0.0003 Acc: 0.9958
Epoch 35/49
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([46]) torch.Size([46, 19])
trying epoch loss
train 1004 1006
train Loss: 0.0001 Acc: 0.9980
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([44]) torch.Size([44, 19])
trying epoch loss
test 236 236
test Loss: 0.0003 Acc: 1.0000
Epoch 36/49
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([46]) torch.Size([46, 19])
trying epoch loss
train 1004 1006
train Loss: 0.0001 Acc: 0.9980
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([44]) torch.Size([44, 19])
trying epoch loss
test 236 236
test Loss: 0.0002 Acc: 1.0000
Epoch 37/49
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([46]) torch.Size([46, 19])
trying epoch loss
train 965 1006
train Loss: 0.0015 Acc: 0.9592
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([44]) torch.Size([44, 19])
trying epoch loss
test 224 236
test Loss: 0.0040 Acc: 0.9492
Epoch 38/49
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([46]) torch.Size([46, 19])
trying epoch loss
train 964 1006
train Loss: 0.0027 Acc: 0.9583
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([44]) torch.Size([44, 19])
trying epoch loss
test 233 236
test Loss: 0.0008 Acc: 0.9873
Epoch 39/49
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([46]) torch.Size([46, 19])
trying epoch loss
train 998 1006
train Loss: 0.0005 Acc: 0.9920
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([44]) torch.Size([44, 19])
trying epoch loss
test 236 236
test Loss: 0.0005 Acc: 1.0000
Epoch 40/49
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([46]) torch.Size([46, 19])
trying epoch loss
train 1003 1006
train Loss: 0.0002 Acc: 0.9970
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([44]) torch.Size([44, 19])
trying epoch loss
test 236 236
test Loss: 0.0002 Acc: 1.0000
Epoch 41/49
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([46]) torch.Size([46, 19])
trying epoch loss
train 1003 1006
train Loss: 0.0001 Acc: 0.9970
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([44]) torch.Size([44, 19])
trying epoch loss
test 236 236
test Loss: 0.0002 Acc: 1.0000
Epoch 42/49
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([46]) torch.Size([46, 19])
trying epoch loss
train 1005 1006
train Loss: 0.0001 Acc: 0.9990
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([44]) torch.Size([44, 19])
trying epoch loss
test 235 236
test Loss: 0.0003 Acc: 0.9958
Epoch 43/49
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([46]) torch.Size([46, 19])
trying epoch loss
train 1004 1006
train Loss: 0.0001 Acc: 0.9980
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([44]) torch.Size([44, 19])
trying epoch loss
test 235 236
test Loss: 0.0002 Acc: 0.9958
Epoch 44/49
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([46]) torch.Size([46, 19])
trying epoch loss
train 1005 1006
train Loss: 0.0001 Acc: 0.9990
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([44]) torch.Size([44, 19])
trying epoch loss
test 236 236
test Loss: 0.0004 Acc: 1.0000
Epoch 45/49
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([46]) torch.Size([46, 19])
trying epoch loss
train 999 1006
train Loss: 0.0006 Acc: 0.9930
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([44]) torch.Size([44, 19])
trying epoch loss
test 227 236
test Loss: 0.0020 Acc: 0.9619
Epoch 46/49
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([46]) torch.Size([46, 19])
trying epoch loss
train 942 1006
train Loss: 0.0043 Acc: 0.9364
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([44]) torch.Size([44, 19])
trying epoch loss
test 213 236
test Loss: 0.0045 Acc: 0.9025
Epoch 47/49
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([46]) torch.Size([46, 19])
trying epoch loss
train 990 1006
train Loss: 0.0013 Acc: 0.9841
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([44]) torch.Size([44, 19])
trying epoch loss
test 236 236
test Loss: 0.0004 Acc: 1.0000
Epoch 48/49
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([46]) torch.Size([46, 19])
trying epoch loss
train 1003 1006
train Loss: 0.0002 Acc: 0.9970
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([44]) torch.Size([44, 19])
trying epoch loss
test 235 236
test Loss: 0.0004 Acc: 0.9958
Epoch 49/49
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([64]) torch.Size([64, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([46]) torch.Size([46, 19])
trying epoch loss
train 1004 1006
train Loss: 0.0001 Acc: 0.9980
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([64]) torch.Size([64, 19])
loss done
torch.Size([44]) torch.Size([44, 19])
trying epoch loss
test 236 236
test Loss: 0.0003 Acc: 1.0000
Training complete in 35m 4s
Best val Acc: 1.000000
returning and looping back
</pre>
</div>
</div>

</div>
</div>

</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>We train the model using the fit() function. The parameters it takes are - training features and training labels, epochs, batch_size and verbose.</p>
<p>Simplest one - verbose. 0="dont print anything as you work", 1="Inform me as you go".</p>
<p>Often the data set is too large to be loaded into the RAM. So, we load data in batches. For batch_size=32 and epochs=10, the model starts loading rows from X in batches of 32 everytime it calculates the loss and updates the model. It keeps on going till it has covered all the samples 10 times.</p>
<p>So, the no. of times model is updated = (Total Samples/Batch Size) * (Epochs)</p>

</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[140]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="nb">sum</span><span class="p">(</span><span class="nb">sum</span><span class="p">(</span><span class="n">Y_preds</span><span class="p">))</span>
</pre></div>

    </div>
</div>
</div>

<div class="output_wrapper">
<div class="output">


<div class="output_area">

    <div class="prompt output_prompt">Out[140]:</div>


<div class="output_text output_subarea output_execute_result">
<pre>tensor(774.7288)</pre>
</div>

</div>

</div>
</div>

</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h3 id="Let's-look-at-some-of-our-predictions?">Let's look at some of our predictions?<a class="anchor-link" href="#Let's-look-at-some-of-our-predictions?">&#182;</a></h3>
</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[27]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="n">f6</span><span class="o">=</span><span class="nb">open</span><span class="p">(</span><span class="s1">&#39;Genredict.pckl&#39;</span><span class="p">,</span><span class="s1">&#39;rb&#39;</span><span class="p">)</span>
<span class="n">Genre_ID_to_name</span><span class="o">=</span><span class="n">pickle</span><span class="o">.</span><span class="n">load</span><span class="p">(</span><span class="n">f6</span><span class="p">)</span>
<span class="n">f6</span><span class="o">.</span><span class="n">close</span><span class="p">()</span>
</pre></div>

    </div>
</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[142]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="nb">sum</span><span class="p">(</span><span class="n">Y_preds</span><span class="p">[</span><span class="mi">1</span><span class="p">])</span>
</pre></div>

    </div>
</div>
</div>

<div class="output_wrapper">
<div class="output">


<div class="output_area">

    <div class="prompt output_prompt">Out[142]:</div>


<div class="output_text output_subarea output_execute_result">
<pre>tensor(4.9819)</pre>
</div>

</div>

</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[143]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="nb">sum</span><span class="p">(</span><span class="n">Y_preds</span><span class="p">[</span><span class="mi">2</span><span class="p">])</span>
</pre></div>

    </div>
</div>
</div>

<div class="output_wrapper">
<div class="output">


<div class="output_area">

    <div class="prompt output_prompt">Out[143]:</div>


<div class="output_text output_subarea output_execute_result">
<pre>tensor(3.0128)</pre>
</div>

</div>

</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[144]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="n">genre_list</span><span class="o">=</span><span class="nb">sorted</span><span class="p">(</span><span class="nb">list</span><span class="p">(</span><span class="n">Genre_ID_to_name</span><span class="o">.</span><span class="n">keys</span><span class="p">()))</span>
</pre></div>

    </div>
</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[145]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="n">precs</span><span class="o">=</span><span class="p">[]</span>
<span class="n">recs</span><span class="o">=</span><span class="p">[]</span>
<span class="k">print</span> <span class="nb">type</span><span class="p">(</span><span class="n">Y_preds</span><span class="p">)</span>
<span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="nb">len</span><span class="p">(</span><span class="n">Y_preds</span><span class="p">)):</span>
    <span class="n">row</span><span class="o">=</span><span class="n">Y_preds</span><span class="p">[</span><span class="n">i</span><span class="p">]</span>
    <span class="n">gt_genres</span><span class="o">=</span><span class="n">Y_test</span><span class="p">[</span><span class="n">i</span><span class="p">]</span>
    <span class="n">gt_genre_names</span><span class="o">=</span><span class="p">[]</span>
    <span class="k">for</span> <span class="n">j</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="nb">len</span><span class="p">(</span><span class="n">gt_genres</span><span class="p">)):</span>
        <span class="k">if</span> <span class="n">gt_genres</span><span class="p">[</span><span class="n">j</span><span class="p">]</span><span class="o">==</span><span class="mi">1</span><span class="p">:</span>
            <span class="n">gt_genre_names</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">Genre_ID_to_name</span><span class="p">[</span><span class="n">genre_list</span><span class="p">[</span><span class="n">j</span><span class="p">]])</span>
    <span class="n">top_3</span><span class="o">=</span><span class="n">np</span><span class="o">.</span><span class="n">argsort</span><span class="p">(</span><span class="n">row</span><span class="p">)[</span><span class="o">-</span><span class="mi">3</span><span class="p">:]</span>
    <span class="n">predicted_genres</span><span class="o">=</span><span class="p">[]</span>
    <span class="k">for</span> <span class="n">genre</span> <span class="ow">in</span> <span class="n">top_3</span><span class="p">:</span>
        <span class="n">predicted_genres</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">Genre_ID_to_name</span><span class="p">[</span><span class="n">genre_list</span><span class="p">[</span><span class="n">genre</span><span class="p">]])</span>
    <span class="p">(</span><span class="n">precision</span><span class="p">,</span><span class="n">recall</span><span class="p">)</span><span class="o">=</span><span class="n">precision_recall</span><span class="p">(</span><span class="n">gt_genre_names</span><span class="p">,</span><span class="n">predicted_genres</span><span class="p">)</span>
    <span class="n">precs</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">precision</span><span class="p">)</span>
    <span class="n">recs</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">recall</span><span class="p">)</span>
    <span class="k">if</span> <span class="n">i</span><span class="o">%</span><span class="k">50</span>==0:
        <span class="k">print</span> <span class="s2">&quot;Predicted: &quot;</span><span class="p">,</span><span class="s1">&#39;,&#39;</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">predicted_genres</span><span class="p">),</span><span class="s2">&quot; Actual: &quot;</span><span class="p">,</span><span class="s1">&#39;,&#39;</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">gt_genre_names</span><span class="p">)</span>
</pre></div>

    </div>
</div>
</div>

<div class="output_wrapper">
<div class="output">


<div class="output_area">

    <div class="prompt"></div>


<div class="output_subarea output_stream output_stdout output_text">
<pre>&lt;type &#39;list&#39;&gt;
Predicted:  Action,Thriller,Drama  Actual:  Adventure,Fantasy,Action
Predicted:  Animation,Adventure,Family  Actual:  Horror,Thriller
Predicted:  Fantasy,Adventure,Action  Actual:  Drama,History,Documentary
Predicted:  Science Fiction,Action,Adventure  Actual:  Action,Science Fiction
Predicted:  War,Comedy,Drama  Actual:  Fantasy,Animation,Comedy,Family
</pre>
</div>
</div>

</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[146]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="k">print</span> <span class="n">np</span><span class="o">.</span><span class="n">mean</span><span class="p">(</span><span class="n">np</span><span class="o">.</span><span class="n">asarray</span><span class="p">(</span><span class="n">precs</span><span class="p">)),</span><span class="n">np</span><span class="o">.</span><span class="n">mean</span><span class="p">(</span><span class="n">np</span><span class="o">.</span><span class="n">asarray</span><span class="p">(</span><span class="n">recs</span><span class="p">))</span>
</pre></div>

    </div>
</div>
</div>

<div class="output_wrapper">
<div class="output">


<div class="output_area">

    <div class="prompt"></div>


<div class="output_subarea output_stream output_stdout output_text">
<pre>0.2217514124293785 0.216454802259887
</pre>
</div>
</div>

</div>
</div>

</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>So, even with just the poster i.e. visual features we are able to make great predictions! Sure, text outperforms the visual features, but the important thing is that it still works. In more complicated models, we can combine the two to make even better predictions. That is precisely what I work on in my research.</p>

</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>These models were trained on CPU's, and a simple 1 layer model was used to show that there is a lot of information in this data that the models can extract. With a larger dataset, and more training I was able to bring these numbers to as high as 70%, which is the similar to textual features. Some teams in my class outperformed this even more. More data is the first thing you should try if you want better results. Then, you can start playing with training on GPUs, learning rate schedules and other hyperparameters. Finally, you can consider using ResNet, a much more powerful neural network model than VGG. All of these can be tried once you have a working knowledge of machine learning.</p>

</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h1 id="Section-8---Deep-Learning-to-get-Textual-Features">Section 8 - Deep Learning to get Textual Features<a class="anchor-link" href="#Section-8---Deep-Learning-to-get-Textual-Features">&#182;</a></h1>
</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>Let's do the same thing as above with text now?</p>

</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>We will use an off the shelf representation for words - Word2Vec model. Just like VGGnet before, this is a model made available to get a meaningful representation. As the total number of words is small, we don't even need to forward propagate our sample through a network. Even that has been done for us, and the result is stored in the form of a dictionary. We can simply look up the word in the dictionary and get the Word2Vec features for the word.</p>

</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>You can download the dictionary from here - <a href="https://drive.google.com/file/d/0B7XkCwpI5KDYNlNUTTlSS21pQmM/edit">https://drive.google.com/file/d/0B7XkCwpI5KDYNlNUTTlSS21pQmM/edit</a> <br>
Download it to the directory of this tutorial i.e. in the same folder as this ipython notebook.</p>

</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[3]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="kn">from</span> <span class="nn">gensim</span> <span class="kn">import</span> <span class="n">models</span>
<span class="c1"># model2 = models.Word2Vec.load_word2vec_format(&#39;GoogleNews-vectors-negative300.bin&#39;, binary=True) </span>
<span class="n">model2</span> <span class="o">=</span> <span class="n">models</span><span class="o">.</span><span class="n">KeyedVectors</span><span class="o">.</span><span class="n">load_word2vec_format</span><span class="p">(</span><span class="s1">&#39;GoogleNews-vectors-negative300.bin&#39;</span><span class="p">,</span> <span class="n">binary</span><span class="o">=</span><span class="bp">True</span><span class="p">)</span>
</pre></div>

    </div>
</div>
</div>

<div class="output_wrapper">
<div class="output">


<div class="output_area">

    <div class="prompt"></div>


<div class="output_subarea output_stream output_stderr output_text">
<pre>Using TensorFlow backend.
</pre>
</div>
</div>

</div>
</div>

</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>Now, we can simply look up for a word in the above loaded model. For example, to get the Word2Vec representation of the word "King" we just do - model2['king']</p>

</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[67]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="k">print</span> <span class="n">model2</span><span class="p">[</span><span class="s1">&#39;king&#39;</span><span class="p">]</span><span class="o">.</span><span class="n">shape</span>
<span class="k">print</span> <span class="n">model2</span><span class="p">[</span><span class="s1">&#39;dog&#39;</span><span class="p">]</span><span class="o">.</span><span class="n">shape</span>
</pre></div>

    </div>
</div>
</div>

<div class="output_wrapper">
<div class="output">


<div class="output_area">

    <div class="prompt"></div>


<div class="output_subarea output_stream output_stdout output_text">
<pre>(300,)
(300,)
</pre>
</div>
</div>

</div>
</div>

</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>This way, we can represent the words in our overviews using this word2vec model. And then, we can use that as our X representations. So, instead of count of words, we are using a representation which is based on the semantic representation of the word. Mathematically, each word went from 3-4 dimensional (the length) to 300 dimensions!</p>

</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>For the same set of movies above, let's try and predict the genres from the deep representation of their overviews!</p>

</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[39]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="n">final_movies_set</span> <span class="o">=</span> <span class="n">movies_with_overviews</span>
<span class="nb">len</span><span class="p">(</span><span class="n">final_movies_set</span><span class="p">)</span>
</pre></div>

    </div>
</div>
</div>

<div class="output_wrapper">
<div class="output">


<div class="output_area">

    <div class="prompt output_prompt">Out[39]:</div>


<div class="output_text output_subarea output_execute_result">
<pre>1668</pre>
</div>

</div>

</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[40]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="kn">from</span> <span class="nn">nltk.tokenize</span> <span class="kn">import</span> <span class="n">RegexpTokenizer</span>
<span class="kn">from</span> <span class="nn">stop_words</span> <span class="kn">import</span> <span class="n">get_stop_words</span>
<span class="n">tokenizer</span> <span class="o">=</span> <span class="n">RegexpTokenizer</span><span class="p">(</span><span class="sa">r</span><span class="s1">&#39;\w+&#39;</span><span class="p">)</span>

<span class="c1"># create English stop words list</span>
<span class="n">en_stop</span> <span class="o">=</span> <span class="n">get_stop_words</span><span class="p">(</span><span class="s1">&#39;en&#39;</span><span class="p">)</span>
</pre></div>

    </div>
</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[41]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="n">movie_mean_wordvec</span><span class="o">=</span><span class="n">np</span><span class="o">.</span><span class="n">zeros</span><span class="p">((</span><span class="nb">len</span><span class="p">(</span><span class="n">final_movies_set</span><span class="p">),</span><span class="mi">300</span><span class="p">))</span>
<span class="n">movie_mean_wordvec</span><span class="o">.</span><span class="n">shape</span>
</pre></div>

    </div>
</div>
</div>

<div class="output_wrapper">
<div class="output">


<div class="output_area">

    <div class="prompt output_prompt">Out[41]:</div>


<div class="output_text output_subarea output_execute_result">
<pre>(1668, 300)</pre>
</div>

</div>

</div>
</div>

</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>Text needs some pre-processing before we can train the model. The only preprocessing we do here is - we delete commonly occurring words which we know are not informative about the genre. Think of it as the clutter in some sense. These words are often removed and are referred to as "stop words". You can look them up online. These include simple words like "a", "and", "but", "how", "or" and so on. They can be easily removed using the python package NLTK.</p>
<p>From the above dataset, movies with overviews which contain only stop words, or movies with overviews containing no words with word2vec representation are neglected. Others are used to build our Mean word2vec representation. Simply, put for every movie overview -</p>
<ul>
<li>Take movie overview</li>
<li>Throw out stop words</li>
<li>For non stop words:<ul>
<li>If in word2vec - take it's word2vec representation which is 300 dimensional</li>
<li>If not - throw word</li>
</ul>
</li>
<li>For each movie, calculate the arithmetic mean of the 300 dimensional vector representations for all words in the overview which weren't thrown out</li>
</ul>
<p>This mean becomes the 300 dimensional representation for the movie. For all movies, these are stored in a numpy array. So the X matrix becomes (1263,300). And, Y is (1263,20) i.e. binarized 20 genres, as before</p>

</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p><strong>Why do we take the arithmetic mean?</strong>
If you feel that we should have kept all the words separately - Then you're thinking correct, but sadly we're limited by the way current day neural networks work. I will not mull over this for the fear of stressing too much on an otherwise irrelevant detail. But if you're interested, read this awesome paper - 
<a href="https://jiajunwu.com/papers/dmil_cvpr.pdf">https://jiajunwu.com/papers/dmil_cvpr.pdf</a></p>

</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[113]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="n">genres</span><span class="o">=</span><span class="p">[]</span>
<span class="n">rows_to_delete</span><span class="o">=</span><span class="p">[]</span>
<span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="nb">len</span><span class="p">(</span><span class="n">final_movies_set</span><span class="p">)):</span>
    <span class="n">mov</span><span class="o">=</span><span class="n">final_movies_set</span><span class="p">[</span><span class="n">i</span><span class="p">]</span>
    <span class="n">movie_genres</span><span class="o">=</span><span class="n">mov</span><span class="p">[</span><span class="s1">&#39;genre_ids&#39;</span><span class="p">]</span>
    <span class="n">genres</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">movie_genres</span><span class="p">)</span>
    <span class="n">overview</span><span class="o">=</span><span class="n">mov</span><span class="p">[</span><span class="s1">&#39;overview&#39;</span><span class="p">]</span>
    <span class="n">tokens</span> <span class="o">=</span> <span class="n">tokenizer</span><span class="o">.</span><span class="n">tokenize</span><span class="p">(</span><span class="n">overview</span><span class="p">)</span>
    <span class="n">stopped_tokens</span> <span class="o">=</span> <span class="p">[</span><span class="n">k</span> <span class="k">for</span> <span class="n">k</span> <span class="ow">in</span> <span class="n">tokens</span> <span class="k">if</span> <span class="ow">not</span> <span class="n">k</span> <span class="ow">in</span> <span class="n">en_stop</span><span class="p">]</span>
    <span class="n">count_in_vocab</span><span class="o">=</span><span class="mi">0</span>
    <span class="n">s</span><span class="o">=</span><span class="mi">0</span>
    <span class="k">if</span> <span class="nb">len</span><span class="p">(</span><span class="n">stopped_tokens</span><span class="p">)</span><span class="o">==</span><span class="mi">0</span><span class="p">:</span>
        <span class="n">rows_to_delete</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">i</span><span class="p">)</span>
        <span class="n">genres</span><span class="o">.</span><span class="n">pop</span><span class="p">(</span><span class="o">-</span><span class="mi">1</span><span class="p">)</span>
<span class="c1">#         print overview</span>
<span class="c1">#         print &quot;sample &quot;,i,&quot;had no nonstops&quot;</span>
    <span class="k">else</span><span class="p">:</span>
        <span class="k">for</span> <span class="n">tok</span> <span class="ow">in</span> <span class="n">stopped_tokens</span><span class="p">:</span>
            <span class="k">if</span> <span class="n">tok</span><span class="o">.</span><span class="n">lower</span><span class="p">()</span> <span class="ow">in</span> <span class="n">model2</span><span class="o">.</span><span class="n">vocab</span><span class="p">:</span>
                <span class="n">count_in_vocab</span><span class="o">+=</span><span class="mi">1</span>
                <span class="n">s</span><span class="o">+=</span><span class="n">model2</span><span class="p">[</span><span class="n">tok</span><span class="o">.</span><span class="n">lower</span><span class="p">()]</span>
        <span class="k">if</span> <span class="n">count_in_vocab</span><span class="o">!=</span><span class="mi">0</span><span class="p">:</span>
            <span class="n">movie_mean_wordvec</span><span class="p">[</span><span class="n">i</span><span class="p">]</span><span class="o">=</span><span class="n">s</span><span class="o">/</span><span class="nb">float</span><span class="p">(</span><span class="n">count_in_vocab</span><span class="p">)</span>
        <span class="k">else</span><span class="p">:</span>
            <span class="n">rows_to_delete</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">i</span><span class="p">)</span>
            <span class="n">genres</span><span class="o">.</span><span class="n">pop</span><span class="p">(</span><span class="o">-</span><span class="mi">1</span><span class="p">)</span>
<span class="c1">#             print overview</span>
<span class="c1">#             print &quot;sample &quot;,i,&quot;had no word2vec&quot;</span>
</pre></div>

    </div>
</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[114]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="nb">len</span><span class="p">(</span><span class="n">genres</span><span class="p">)</span>
</pre></div>

    </div>
</div>
</div>

<div class="output_wrapper">
<div class="output">


<div class="output_area">

    <div class="prompt output_prompt">Out[114]:</div>


<div class="output_text output_subarea output_execute_result">
<pre>1668</pre>
</div>

</div>

</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[&nbsp;]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span> 
</pre></div>

    </div>
</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[115]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="n">mask2</span><span class="o">=</span><span class="p">[]</span>
<span class="k">for</span> <span class="n">row</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="nb">len</span><span class="p">(</span><span class="n">movie_mean_wordvec</span><span class="p">)):</span>
    <span class="k">if</span> <span class="n">row</span> <span class="ow">in</span> <span class="n">rows_to_delete</span><span class="p">:</span>
        <span class="n">mask2</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="bp">False</span><span class="p">)</span>
    <span class="k">else</span><span class="p">:</span>
        <span class="n">mask2</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="bp">True</span><span class="p">)</span>
</pre></div>

    </div>
</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[116]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="n">X</span><span class="o">=</span><span class="n">movie_mean_wordvec</span><span class="p">[</span><span class="n">mask2</span><span class="p">]</span>
</pre></div>

    </div>
</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[117]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="n">X</span><span class="o">.</span><span class="n">shape</span>
</pre></div>

    </div>
</div>
</div>

<div class="output_wrapper">
<div class="output">


<div class="output_area">

    <div class="prompt output_prompt">Out[117]:</div>


<div class="output_text output_subarea output_execute_result">
<pre>(1668, 300)</pre>
</div>

</div>

</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[118]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="n">Y</span><span class="o">=</span><span class="n">mlb</span><span class="o">.</span><span class="n">fit_transform</span><span class="p">(</span><span class="n">genres</span><span class="p">)</span>
</pre></div>

    </div>
</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[41]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="n">Y</span><span class="o">.</span><span class="n">shape</span>
</pre></div>

    </div>
</div>
</div>

<div class="output_wrapper">
<div class="output">


<div class="output_area">

    <div class="prompt output_prompt">Out[41]:</div>


<div class="output_text output_subarea output_execute_result">
<pre>(1668, 19)</pre>
</div>

</div>

</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[78]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="n">textual_features</span><span class="o">=</span><span class="p">(</span><span class="n">X</span><span class="p">,</span><span class="n">Y</span><span class="p">)</span>
<span class="n">f9</span><span class="o">=</span><span class="nb">open</span><span class="p">(</span><span class="s1">&#39;textual_features.pckl&#39;</span><span class="p">,</span><span class="s1">&#39;wb&#39;</span><span class="p">)</span>
<span class="n">pickle</span><span class="o">.</span><span class="n">dump</span><span class="p">(</span><span class="n">textual_features</span><span class="p">,</span><span class="n">f9</span><span class="p">)</span>
<span class="n">f9</span><span class="o">.</span><span class="n">close</span><span class="p">()</span>
</pre></div>

    </div>
</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[119]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="c1"># textual_features=(X,Y)</span>
<span class="n">f9</span><span class="o">=</span><span class="nb">open</span><span class="p">(</span><span class="s1">&#39;textual_features.pckl&#39;</span><span class="p">,</span><span class="s1">&#39;rb&#39;</span><span class="p">)</span>
<span class="n">textual_features</span><span class="o">=</span><span class="n">pickle</span><span class="o">.</span><span class="n">load</span><span class="p">(</span><span class="n">f9</span><span class="p">)</span>
<span class="n">f9</span><span class="o">.</span><span class="n">close</span><span class="p">()</span>
</pre></div>

    </div>
</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[122]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="p">(</span><span class="n">X</span><span class="p">,</span><span class="n">Y</span><span class="p">)</span><span class="o">=</span><span class="n">textual_features</span>
</pre></div>

    </div>
</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[123]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="n">X</span><span class="o">.</span><span class="n">shape</span>
</pre></div>

    </div>
</div>
</div>

<div class="output_wrapper">
<div class="output">


<div class="output_area">

    <div class="prompt output_prompt">Out[123]:</div>


<div class="output_text output_subarea output_execute_result">
<pre>(1668, 300)</pre>
</div>

</div>

</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[124]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="n">Y</span><span class="o">.</span><span class="n">shape</span>
</pre></div>

    </div>
</div>
</div>

<div class="output_wrapper">
<div class="output">


<div class="output_area">

    <div class="prompt output_prompt">Out[124]:</div>


<div class="output_text output_subarea output_execute_result">
<pre>(1668, 19)</pre>
</div>

</div>

</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[125]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="n">mask_text</span><span class="o">=</span><span class="n">np</span><span class="o">.</span><span class="n">random</span><span class="o">.</span><span class="n">rand</span><span class="p">(</span><span class="nb">len</span><span class="p">(</span><span class="n">X</span><span class="p">))</span><span class="o">&lt;</span><span class="mf">0.8</span>
</pre></div>

    </div>
</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[126]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="n">X_train</span><span class="o">=</span><span class="n">X</span><span class="p">[</span><span class="n">mask_text</span><span class="p">]</span>
<span class="n">Y_train</span><span class="o">=</span><span class="n">Y</span><span class="p">[</span><span class="n">mask_text</span><span class="p">]</span>
<span class="n">X_test</span><span class="o">=</span><span class="n">X</span><span class="p">[</span><span class="o">~</span><span class="n">mask_text</span><span class="p">]</span>
<span class="n">Y_test</span><span class="o">=</span><span class="n">Y</span><span class="p">[</span><span class="o">~</span><span class="n">mask_text</span><span class="p">]</span>
</pre></div>

    </div>
</div>
</div>

</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>Once again, we use a very similar, super simple architecture as before.</p>

</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[127]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="kn">import</span> <span class="nn">torch</span>
<span class="n">dsets</span> <span class="o">=</span> <span class="p">{}</span>
<span class="n">dsets</span><span class="p">[</span><span class="s1">&#39;train&#39;</span><span class="p">]</span> <span class="o">=</span> <span class="n">FeatureDataset</span><span class="p">(</span><span class="n">X_train</span><span class="p">,</span> <span class="n">Y_train</span><span class="p">)</span>
<span class="n">dsets</span><span class="p">[</span><span class="s1">&#39;test&#39;</span><span class="p">]</span> <span class="o">=</span> <span class="n">FeatureDataset</span><span class="p">(</span><span class="n">X_test</span><span class="p">,</span> <span class="n">Y_test</span><span class="p">)</span>
<span class="n">dset_loaders</span> <span class="o">=</span> <span class="p">{</span><span class="n">x</span><span class="p">:</span> <span class="n">torch</span><span class="o">.</span><span class="n">utils</span><span class="o">.</span><span class="n">data</span><span class="o">.</span><span class="n">DataLoader</span><span class="p">(</span><span class="n">dsets</span><span class="p">[</span><span class="n">x</span><span class="p">],</span> <span class="n">batch_size</span><span class="o">=</span><span class="mi">500</span><span class="p">,</span>
                                               <span class="n">shuffle</span><span class="o">=</span><span class="bp">True</span><span class="p">,</span> <span class="n">num_workers</span><span class="o">=</span><span class="mi">25</span><span class="p">)</span>
                <span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="p">[</span><span class="s1">&#39;train&#39;</span><span class="p">,</span> <span class="s1">&#39;test&#39;</span><span class="p">]}</span>

<span class="n">dset_sizes</span> <span class="o">=</span> <span class="p">{</span><span class="n">x</span><span class="p">:</span> <span class="nb">len</span><span class="p">(</span><span class="n">dsets</span><span class="p">[</span><span class="n">x</span><span class="p">])</span> <span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="p">[</span><span class="s1">&#39;train&#39;</span><span class="p">,</span> <span class="s1">&#39;test&#39;</span><span class="p">]}</span>
<span class="k">print</span> <span class="n">dset_sizes</span>
<span class="n">model_textual</span> <span class="o">=</span> <span class="n">nn</span><span class="o">.</span><span class="n">Sequential</span><span class="p">(</span>
    <span class="n">nn</span><span class="o">.</span><span class="n">Linear</span><span class="p">(</span><span class="n">X</span><span class="o">.</span><span class="n">shape</span><span class="p">[</span><span class="mi">1</span><span class="p">],</span><span class="mi">1024</span><span class="p">),</span>  <span class="c1">#300</span>
    <span class="n">nn</span><span class="o">.</span><span class="n">ReLU</span><span class="p">(),</span>
    <span class="n">nn</span><span class="o">.</span><span class="n">Linear</span><span class="p">(</span><span class="mi">1024</span><span class="p">,</span><span class="n">Y</span><span class="o">.</span><span class="n">shape</span><span class="p">[</span><span class="mi">1</span><span class="p">]),</span>
    <span class="n">nn</span><span class="o">.</span><span class="n">Softmax</span><span class="p">()</span>
<span class="p">)</span>

<span class="n">optimizer</span> <span class="o">=</span> <span class="n">torch</span><span class="o">.</span><span class="n">optim</span><span class="o">.</span><span class="n">RMSprop</span><span class="p">(</span><span class="n">model_textual</span><span class="o">.</span><span class="n">parameters</span><span class="p">())</span>
<span class="n">criterion</span> <span class="o">=</span> <span class="n">nn</span><span class="o">.</span><span class="n">BCELoss</span><span class="p">()</span>
<span class="n">textual_model</span><span class="p">,</span> <span class="n">Y_preds</span> <span class="o">=</span> <span class="n">train_model</span><span class="p">(</span><span class="n">model_textual</span><span class="p">,</span> <span class="n">criterion</span><span class="p">,</span> <span class="n">optimizer</span><span class="p">,</span> <span class="n">num_epochs</span><span class="o">=</span><span class="mi">100</span><span class="p">)</span>

<span class="c1"># Save model</span>
<span class="n">torch</span><span class="o">.</span><span class="n">save</span><span class="p">(</span><span class="n">textual_model</span><span class="o">.</span><span class="n">state_dict</span><span class="p">(),</span> <span class="s1">&#39;text_fine_tuned_best_model.pt&#39;</span><span class="p">)</span>
</pre></div>

    </div>
</div>
</div>

<div class="output_wrapper">
<div class="output">


<div class="output_area">

    <div class="prompt"></div>


<div class="output_subarea output_stream output_stdout output_text">
<pre>{&#39;test&#39;: 373, &#39;train&#39;: 1295}
Epoch 0/99
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([295]) torch.Size([295, 19])
trying epoch loss
train 323 1295
train Loss: 0.0088 Acc: 0.2494
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([373]) torch.Size([373, 19])
trying epoch loss
test 106 373
test Loss: 0.0148 Acc: 0.2842
(&#39;new best accuracy = &#39;, 0.28418230563002683)
Epoch 1/99
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([295]) torch.Size([295, 19])
trying epoch loss
train 260 1295
train Loss: 0.0132 Acc: 0.2008
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([373]) torch.Size([373, 19])
trying epoch loss
test 147 373
test Loss: 0.0071 Acc: 0.3941
(&#39;new best accuracy = &#39;, 0.3941018766756032)
Epoch 2/99
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([295]) torch.Size([295, 19])
trying epoch loss
train 317 1295
train Loss: 0.0029 Acc: 0.2448
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([373]) torch.Size([373, 19])
trying epoch loss
test 86 373
test Loss: 0.0013 Acc: 0.2306
Epoch 3/99
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([295]) torch.Size([295, 19])
trying epoch loss
train 341 1295
train Loss: 0.0011 Acc: 0.2633
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([373]) torch.Size([373, 19])
trying epoch loss
test 96 373
test Loss: 0.0013 Acc: 0.2574
Epoch 4/99
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([295]) torch.Size([295, 19])
trying epoch loss
train 402 1295
train Loss: 0.0011 Acc: 0.3104
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([373]) torch.Size([373, 19])
trying epoch loss
test 125 373
test Loss: 0.0014 Acc: 0.3351
Epoch 5/99
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([295]) torch.Size([295, 19])
trying epoch loss
train 589 1295
train Loss: 0.0011 Acc: 0.4548
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([373]) torch.Size([373, 19])
trying epoch loss
test 190 373
test Loss: 0.0013 Acc: 0.5094
(&#39;new best accuracy = &#39;, 0.5093833780160858)
Epoch 6/99
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([295]) torch.Size([295, 19])
trying epoch loss
train 702 1295
train Loss: 0.0010 Acc: 0.5421
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([373]) torch.Size([373, 19])
trying epoch loss
test 146 373
test Loss: 0.0012 Acc: 0.3914
Epoch 7/99
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([295]) torch.Size([295, 19])
trying epoch loss
train 613 1295
train Loss: 0.0010 Acc: 0.4734
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([373]) torch.Size([373, 19])
trying epoch loss
test 159 373
test Loss: 0.0014 Acc: 0.4263
Epoch 8/99
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([295]) torch.Size([295, 19])
trying epoch loss
train 702 1295
train Loss: 0.0010 Acc: 0.5421
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([373]) torch.Size([373, 19])
trying epoch loss
test 241 373
test Loss: 0.0011 Acc: 0.6461
(&#39;new best accuracy = &#39;, 0.646112600536193)
Epoch 9/99
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([295]) torch.Size([295, 19])
trying epoch loss
train 808 1295
train Loss: 0.0009 Acc: 0.6239
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([373]) torch.Size([373, 19])
trying epoch loss
test 230 373
test Loss: 0.0011 Acc: 0.6166
Epoch 10/99
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([295]) torch.Size([295, 19])
trying epoch loss
train 852 1295
train Loss: 0.0009 Acc: 0.6579
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([373]) torch.Size([373, 19])
trying epoch loss
test 236 373
test Loss: 0.0011 Acc: 0.6327
Epoch 11/99
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([295]) torch.Size([295, 19])
trying epoch loss
train 854 1295
train Loss: 0.0009 Acc: 0.6595
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([373]) torch.Size([373, 19])
trying epoch loss
test 234 373
test Loss: 0.0011 Acc: 0.6273
Epoch 12/99
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([295]) torch.Size([295, 19])
trying epoch loss
train 885 1295
train Loss: 0.0009 Acc: 0.6834
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([373]) torch.Size([373, 19])
trying epoch loss
test 259 373
test Loss: 0.0011 Acc: 0.6944
(&#39;new best accuracy = &#39;, 0.6943699731903485)
Epoch 13/99
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([295]) torch.Size([295, 19])
trying epoch loss
train 901 1295
train Loss: 0.0009 Acc: 0.6958
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([373]) torch.Size([373, 19])
trying epoch loss
test 219 373
test Loss: 0.0012 Acc: 0.5871
Epoch 14/99
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([295]) torch.Size([295, 19])
trying epoch loss
train 902 1295
train Loss: 0.0009 Acc: 0.6965
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([373]) torch.Size([373, 19])
trying epoch loss
test 277 373
test Loss: 0.0011 Acc: 0.7426
(&#39;new best accuracy = &#39;, 0.7426273458445041)
Epoch 15/99
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([295]) torch.Size([295, 19])
trying epoch loss
train 977 1295
train Loss: 0.0009 Acc: 0.7544
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([373]) torch.Size([373, 19])
trying epoch loss
test 276 373
test Loss: 0.0010 Acc: 0.7399
Epoch 16/99
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([295]) torch.Size([295, 19])
trying epoch loss
train 960 1295
train Loss: 0.0009 Acc: 0.7413
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([373]) torch.Size([373, 19])
trying epoch loss
test 268 373
test Loss: 0.0011 Acc: 0.7185
Epoch 17/99
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([295]) torch.Size([295, 19])
trying epoch loss
train 976 1295
train Loss: 0.0009 Acc: 0.7537
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([373]) torch.Size([373, 19])
trying epoch loss
test 279 373
test Loss: 0.0011 Acc: 0.7480
(&#39;new best accuracy = &#39;, 0.7479892761394102)
Epoch 18/99
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([295]) torch.Size([295, 19])
trying epoch loss
train 976 1295
train Loss: 0.0009 Acc: 0.7537
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([373]) torch.Size([373, 19])
trying epoch loss
test 285 373
test Loss: 0.0010 Acc: 0.7641
(&#39;new best accuracy = &#39;, 0.7640750670241286)
Epoch 19/99
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([295]) torch.Size([295, 19])
trying epoch loss
train 1015 1295
train Loss: 0.0008 Acc: 0.7838
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([373]) torch.Size([373, 19])
trying epoch loss
test 267 373
test Loss: 0.0010 Acc: 0.7158
Epoch 20/99
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([295]) torch.Size([295, 19])
trying epoch loss
train 961 1295
train Loss: 0.0009 Acc: 0.7421
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([373]) torch.Size([373, 19])
trying epoch loss
test 280 373
test Loss: 0.0011 Acc: 0.7507
Epoch 21/99
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([295]) torch.Size([295, 19])
trying epoch loss
train 1025 1295
train Loss: 0.0008 Acc: 0.7915
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([373]) torch.Size([373, 19])
trying epoch loss
test 282 373
test Loss: 0.0010 Acc: 0.7560
Epoch 22/99
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([295]) torch.Size([295, 19])
trying epoch loss
train 1010 1295
train Loss: 0.0008 Acc: 0.7799
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([373]) torch.Size([373, 19])
trying epoch loss
test 297 373
test Loss: 0.0010 Acc: 0.7962
(&#39;new best accuracy = &#39;, 0.7962466487935657)
Epoch 23/99
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([295]) torch.Size([295, 19])
trying epoch loss
train 1027 1295
train Loss: 0.0008 Acc: 0.7931
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([373]) torch.Size([373, 19])
trying epoch loss
test 298 373
test Loss: 0.0010 Acc: 0.7989
(&#39;new best accuracy = &#39;, 0.7989276139410187)
Epoch 24/99
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([295]) torch.Size([295, 19])
trying epoch loss
train 1049 1295
train Loss: 0.0008 Acc: 0.8100
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([373]) torch.Size([373, 19])
trying epoch loss
test 308 373
test Loss: 0.0010 Acc: 0.8257
(&#39;new best accuracy = &#39;, 0.8257372654155496)
Epoch 25/99
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([295]) torch.Size([295, 19])
trying epoch loss
train 1054 1295
train Loss: 0.0008 Acc: 0.8139
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([373]) torch.Size([373, 19])
trying epoch loss
test 308 373
test Loss: 0.0010 Acc: 0.8257
Epoch 26/99
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([295]) torch.Size([295, 19])
trying epoch loss
train 1054 1295
train Loss: 0.0008 Acc: 0.8139
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([373]) torch.Size([373, 19])
trying epoch loss
test 302 373
test Loss: 0.0010 Acc: 0.8097
Epoch 27/99
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([295]) torch.Size([295, 19])
trying epoch loss
train 1060 1295
train Loss: 0.0008 Acc: 0.8185
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([373]) torch.Size([373, 19])
trying epoch loss
test 315 373
test Loss: 0.0010 Acc: 0.8445
(&#39;new best accuracy = &#39;, 0.8445040214477212)
Epoch 28/99
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([295]) torch.Size([295, 19])
trying epoch loss
train 1055 1295
train Loss: 0.0008 Acc: 0.8147
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([373]) torch.Size([373, 19])
trying epoch loss
test 279 373
test Loss: 0.0010 Acc: 0.7480
Epoch 29/99
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([295]) torch.Size([295, 19])
trying epoch loss
train 1017 1295
train Loss: 0.0008 Acc: 0.7853
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([373]) torch.Size([373, 19])
trying epoch loss
test 314 373
test Loss: 0.0010 Acc: 0.8418
Epoch 30/99
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([295]) torch.Size([295, 19])
trying epoch loss
train 1092 1295
train Loss: 0.0008 Acc: 0.8432
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([373]) torch.Size([373, 19])
trying epoch loss
test 304 373
test Loss: 0.0010 Acc: 0.8150
Epoch 31/99
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([295]) torch.Size([295, 19])
trying epoch loss
train 1062 1295
train Loss: 0.0008 Acc: 0.8201
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([373]) torch.Size([373, 19])
trying epoch loss
test 327 373
test Loss: 0.0009 Acc: 0.8767
(&#39;new best accuracy = &#39;, 0.8766756032171582)
Epoch 32/99
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([295]) torch.Size([295, 19])
trying epoch loss
train 1078 1295
train Loss: 0.0008 Acc: 0.8324
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([373]) torch.Size([373, 19])
trying epoch loss
test 312 373
test Loss: 0.0010 Acc: 0.8365
Epoch 33/99
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([295]) torch.Size([295, 19])
trying epoch loss
train 1090 1295
train Loss: 0.0008 Acc: 0.8417
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([373]) torch.Size([373, 19])
trying epoch loss
test 314 373
test Loss: 0.0009 Acc: 0.8418
Epoch 34/99
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([295]) torch.Size([295, 19])
trying epoch loss
train 1103 1295
train Loss: 0.0008 Acc: 0.8517
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([373]) torch.Size([373, 19])
trying epoch loss
test 300 373
test Loss: 0.0010 Acc: 0.8043
Epoch 35/99
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([295]) torch.Size([295, 19])
trying epoch loss
train 1093 1295
train Loss: 0.0008 Acc: 0.8440
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([373]) torch.Size([373, 19])
trying epoch loss
test 311 373
test Loss: 0.0009 Acc: 0.8338
Epoch 36/99
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([295]) torch.Size([295, 19])
trying epoch loss
train 1091 1295
train Loss: 0.0008 Acc: 0.8425
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([373]) torch.Size([373, 19])
trying epoch loss
test 312 373
test Loss: 0.0010 Acc: 0.8365
Epoch 37/99
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([295]) torch.Size([295, 19])
trying epoch loss
train 1123 1295
train Loss: 0.0008 Acc: 0.8672
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([373]) torch.Size([373, 19])
trying epoch loss
test 328 373
test Loss: 0.0009 Acc: 0.8794
(&#39;new best accuracy = &#39;, 0.8793565683646113)
Epoch 38/99
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([295]) torch.Size([295, 19])
trying epoch loss
train 1127 1295
train Loss: 0.0008 Acc: 0.8703
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([373]) torch.Size([373, 19])
trying epoch loss
test 332 373
test Loss: 0.0009 Acc: 0.8901
(&#39;new best accuracy = &#39;, 0.8900804289544236)
Epoch 39/99
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([295]) torch.Size([295, 19])
trying epoch loss
train 1120 1295
train Loss: 0.0008 Acc: 0.8649
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([373]) torch.Size([373, 19])
trying epoch loss
test 323 373
test Loss: 0.0009 Acc: 0.8660
Epoch 40/99
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([295]) torch.Size([295, 19])
trying epoch loss
train 1122 1295
train Loss: 0.0007 Acc: 0.8664
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([373]) torch.Size([373, 19])
trying epoch loss
test 325 373
test Loss: 0.0009 Acc: 0.8713
Epoch 41/99
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([295]) torch.Size([295, 19])
trying epoch loss
train 1106 1295
train Loss: 0.0008 Acc: 0.8541
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([373]) torch.Size([373, 19])
trying epoch loss
test 331 373
test Loss: 0.0009 Acc: 0.8874
Epoch 42/99
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([295]) torch.Size([295, 19])
trying epoch loss
train 1143 1295
train Loss: 0.0007 Acc: 0.8826
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([373]) torch.Size([373, 19])
trying epoch loss
test 320 373
test Loss: 0.0009 Acc: 0.8579
Epoch 43/99
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([295]) torch.Size([295, 19])
trying epoch loss
train 1122 1295
train Loss: 0.0008 Acc: 0.8664
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([373]) torch.Size([373, 19])
trying epoch loss
test 315 373
test Loss: 0.0009 Acc: 0.8445
Epoch 44/99
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([295]) torch.Size([295, 19])
trying epoch loss
train 1126 1295
train Loss: 0.0007 Acc: 0.8695
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([373]) torch.Size([373, 19])
trying epoch loss
test 339 373
test Loss: 0.0009 Acc: 0.9088
(&#39;new best accuracy = &#39;, 0.9088471849865952)
Epoch 45/99
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([295]) torch.Size([295, 19])
trying epoch loss
train 1159 1295
train Loss: 0.0007 Acc: 0.8950
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([373]) torch.Size([373, 19])
trying epoch loss
test 333 373
test Loss: 0.0009 Acc: 0.8928
Epoch 46/99
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([295]) torch.Size([295, 19])
trying epoch loss
train 1153 1295
train Loss: 0.0007 Acc: 0.8903
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([373]) torch.Size([373, 19])
trying epoch loss
test 339 373
test Loss: 0.0009 Acc: 0.9088
Epoch 47/99
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([295]) torch.Size([295, 19])
trying epoch loss
train 1153 1295
train Loss: 0.0007 Acc: 0.8903
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([373]) torch.Size([373, 19])
trying epoch loss
test 334 373
test Loss: 0.0009 Acc: 0.8954
Epoch 48/99
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([295]) torch.Size([295, 19])
trying epoch loss
train 1172 1295
train Loss: 0.0007 Acc: 0.9050
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([373]) torch.Size([373, 19])
trying epoch loss
test 345 373
test Loss: 0.0009 Acc: 0.9249
(&#39;new best accuracy = &#39;, 0.9249329758713136)
Epoch 49/99
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([295]) torch.Size([295, 19])
trying epoch loss
train 1176 1295
train Loss: 0.0007 Acc: 0.9081
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([373]) torch.Size([373, 19])
trying epoch loss
test 328 373
test Loss: 0.0009 Acc: 0.8794
Epoch 50/99
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([295]) torch.Size([295, 19])
trying epoch loss
train 1146 1295
train Loss: 0.0007 Acc: 0.8849
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([373]) torch.Size([373, 19])
trying epoch loss
test 338 373
test Loss: 0.0009 Acc: 0.9062
Epoch 51/99
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([295]) torch.Size([295, 19])
trying epoch loss
train 1169 1295
train Loss: 0.0007 Acc: 0.9027
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([373]) torch.Size([373, 19])
trying epoch loss
test 335 373
test Loss: 0.0009 Acc: 0.8981
Epoch 52/99
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([295]) torch.Size([295, 19])
trying epoch loss
train 1179 1295
train Loss: 0.0007 Acc: 0.9104
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([373]) torch.Size([373, 19])
trying epoch loss
test 342 373
test Loss: 0.0009 Acc: 0.9169
Epoch 53/99
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([295]) torch.Size([295, 19])
trying epoch loss
train 1179 1295
train Loss: 0.0007 Acc: 0.9104
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([373]) torch.Size([373, 19])
trying epoch loss
test 337 373
test Loss: 0.0009 Acc: 0.9035
Epoch 54/99
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([295]) torch.Size([295, 19])
trying epoch loss
train 1197 1295
train Loss: 0.0007 Acc: 0.9243
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([373]) torch.Size([373, 19])
trying epoch loss
test 344 373
test Loss: 0.0009 Acc: 0.9223
Epoch 55/99
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([295]) torch.Size([295, 19])
trying epoch loss
train 1163 1295
train Loss: 0.0007 Acc: 0.8981
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([373]) torch.Size([373, 19])
trying epoch loss
test 347 373
test Loss: 0.0009 Acc: 0.9303
(&#39;new best accuracy = &#39;, 0.9302949061662198)
Epoch 56/99
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([295]) torch.Size([295, 19])
trying epoch loss
train 1193 1295
train Loss: 0.0007 Acc: 0.9212
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([373]) torch.Size([373, 19])
trying epoch loss
test 352 373
test Loss: 0.0008 Acc: 0.9437
(&#39;new best accuracy = &#39;, 0.9436997319034852)
Epoch 57/99
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([295]) torch.Size([295, 19])
trying epoch loss
train 1207 1295
train Loss: 0.0007 Acc: 0.9320
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([373]) torch.Size([373, 19])
trying epoch loss
test 328 373
test Loss: 0.0009 Acc: 0.8794
Epoch 58/99
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([295]) torch.Size([295, 19])
trying epoch loss
train 1160 1295
train Loss: 0.0007 Acc: 0.8958
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([373]) torch.Size([373, 19])
trying epoch loss
test 351 373
test Loss: 0.0008 Acc: 0.9410
Epoch 59/99
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([295]) torch.Size([295, 19])
trying epoch loss
train 1204 1295
train Loss: 0.0007 Acc: 0.9297
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([373]) torch.Size([373, 19])
trying epoch loss
test 347 373
test Loss: 0.0008 Acc: 0.9303
Epoch 60/99
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([295]) torch.Size([295, 19])
trying epoch loss
train 1212 1295
train Loss: 0.0007 Acc: 0.9359
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([373]) torch.Size([373, 19])
trying epoch loss
test 352 373
test Loss: 0.0008 Acc: 0.9437
Epoch 61/99
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([295]) torch.Size([295, 19])
trying epoch loss
train 1222 1295
train Loss: 0.0007 Acc: 0.9436
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([373]) torch.Size([373, 19])
trying epoch loss
test 353 373
test Loss: 0.0008 Acc: 0.9464
(&#39;new best accuracy = &#39;, 0.9463806970509383)
Epoch 62/99
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([295]) torch.Size([295, 19])
trying epoch loss
train 1212 1295
train Loss: 0.0007 Acc: 0.9359
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([373]) torch.Size([373, 19])
trying epoch loss
test 342 373
test Loss: 0.0009 Acc: 0.9169
Epoch 63/99
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([295]) torch.Size([295, 19])
trying epoch loss
train 1206 1295
train Loss: 0.0007 Acc: 0.9313
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([373]) torch.Size([373, 19])
trying epoch loss
test 348 373
test Loss: 0.0008 Acc: 0.9330
Epoch 64/99
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([295]) torch.Size([295, 19])
trying epoch loss
train 1191 1295
train Loss: 0.0007 Acc: 0.9197
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([373]) torch.Size([373, 19])
trying epoch loss
test 347 373
test Loss: 0.0008 Acc: 0.9303
Epoch 65/99
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([295]) torch.Size([295, 19])
trying epoch loss
train 1218 1295
train Loss: 0.0007 Acc: 0.9405
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([373]) torch.Size([373, 19])
trying epoch loss
test 354 373
test Loss: 0.0008 Acc: 0.9491
(&#39;new best accuracy = &#39;, 0.9490616621983914)
Epoch 66/99
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([295]) torch.Size([295, 19])
trying epoch loss
train 1197 1295
train Loss: 0.0007 Acc: 0.9243
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([373]) torch.Size([373, 19])
trying epoch loss
test 351 373
test Loss: 0.0008 Acc: 0.9410
Epoch 67/99
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([295]) torch.Size([295, 19])
trying epoch loss
train 1216 1295
train Loss: 0.0007 Acc: 0.9390
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([373]) torch.Size([373, 19])
trying epoch loss
test 357 373
test Loss: 0.0008 Acc: 0.9571
(&#39;new best accuracy = &#39;, 0.9571045576407506)
Epoch 68/99
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([295]) torch.Size([295, 19])
trying epoch loss
train 1222 1295
train Loss: 0.0007 Acc: 0.9436
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([373]) torch.Size([373, 19])
trying epoch loss
test 355 373
test Loss: 0.0008 Acc: 0.9517
Epoch 69/99
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([295]) torch.Size([295, 19])
trying epoch loss
train 1236 1295
train Loss: 0.0007 Acc: 0.9544
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([373]) torch.Size([373, 19])
trying epoch loss
test 358 373
test Loss: 0.0008 Acc: 0.9598
(&#39;new best accuracy = &#39;, 0.9597855227882037)
Epoch 70/99
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([295]) torch.Size([295, 19])
trying epoch loss
train 1250 1295
train Loss: 0.0007 Acc: 0.9653
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([373]) torch.Size([373, 19])
trying epoch loss
test 359 373
test Loss: 0.0008 Acc: 0.9625
(&#39;new best accuracy = &#39;, 0.9624664879356568)
Epoch 71/99
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([295]) torch.Size([295, 19])
trying epoch loss
train 1231 1295
train Loss: 0.0007 Acc: 0.9506
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([373]) torch.Size([373, 19])
trying epoch loss
test 343 373
test Loss: 0.0009 Acc: 0.9196
Epoch 72/99
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([295]) torch.Size([295, 19])
trying epoch loss
train 1207 1295
train Loss: 0.0007 Acc: 0.9320
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([373]) torch.Size([373, 19])
trying epoch loss
test 354 373
test Loss: 0.0008 Acc: 0.9491
Epoch 73/99
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([295]) torch.Size([295, 19])
trying epoch loss
train 1234 1295
train Loss: 0.0006 Acc: 0.9529
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([373]) torch.Size([373, 19])
trying epoch loss
test 351 373
test Loss: 0.0008 Acc: 0.9410
Epoch 74/99
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([295]) torch.Size([295, 19])
trying epoch loss
train 1230 1295
train Loss: 0.0007 Acc: 0.9498
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([373]) torch.Size([373, 19])
trying epoch loss
test 346 373
test Loss: 0.0008 Acc: 0.9276
Epoch 75/99
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([295]) torch.Size([295, 19])
trying epoch loss
train 1216 1295
train Loss: 0.0007 Acc: 0.9390
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([373]) torch.Size([373, 19])
trying epoch loss
test 353 373
test Loss: 0.0008 Acc: 0.9464
Epoch 76/99
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([295]) torch.Size([295, 19])
trying epoch loss
train 1233 1295
train Loss: 0.0007 Acc: 0.9521
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([373]) torch.Size([373, 19])
trying epoch loss
test 357 373
test Loss: 0.0008 Acc: 0.9571
Epoch 77/99
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([295]) torch.Size([295, 19])
trying epoch loss
train 1243 1295
train Loss: 0.0006 Acc: 0.9598
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([373]) torch.Size([373, 19])
trying epoch loss
test 362 373
test Loss: 0.0008 Acc: 0.9705
(&#39;new best accuracy = &#39;, 0.9705093833780161)
Epoch 78/99
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([295]) torch.Size([295, 19])
trying epoch loss
train 1252 1295
train Loss: 0.0006 Acc: 0.9668
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([373]) torch.Size([373, 19])
trying epoch loss
test 358 373
test Loss: 0.0008 Acc: 0.9598
Epoch 79/99
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([295]) torch.Size([295, 19])
trying epoch loss
train 1254 1295
train Loss: 0.0006 Acc: 0.9683
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([373]) torch.Size([373, 19])
trying epoch loss
test 357 373
test Loss: 0.0008 Acc: 0.9571
Epoch 80/99
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([295]) torch.Size([295, 19])
trying epoch loss
train 1259 1295
train Loss: 0.0006 Acc: 0.9722
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([373]) torch.Size([373, 19])
trying epoch loss
test 362 373
test Loss: 0.0008 Acc: 0.9705
Epoch 81/99
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([295]) torch.Size([295, 19])
trying epoch loss
train 1264 1295
train Loss: 0.0006 Acc: 0.9761
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([373]) torch.Size([373, 19])
trying epoch loss
test 363 373
test Loss: 0.0008 Acc: 0.9732
(&#39;new best accuracy = &#39;, 0.9731903485254692)
Epoch 82/99
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([295]) torch.Size([295, 19])
trying epoch loss
train 1248 1295
train Loss: 0.0006 Acc: 0.9637
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([373]) torch.Size([373, 19])
trying epoch loss
test 359 373
test Loss: 0.0008 Acc: 0.9625
Epoch 83/99
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([295]) torch.Size([295, 19])
trying epoch loss
train 1249 1295
train Loss: 0.0006 Acc: 0.9645
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([373]) torch.Size([373, 19])
trying epoch loss
test 358 373
test Loss: 0.0008 Acc: 0.9598
Epoch 84/99
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([295]) torch.Size([295, 19])
trying epoch loss
train 1254 1295
train Loss: 0.0006 Acc: 0.9683
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([373]) torch.Size([373, 19])
trying epoch loss
test 362 373
test Loss: 0.0008 Acc: 0.9705
Epoch 85/99
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([295]) torch.Size([295, 19])
trying epoch loss
train 1266 1295
train Loss: 0.0006 Acc: 0.9776
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([373]) torch.Size([373, 19])
trying epoch loss
test 362 373
test Loss: 0.0008 Acc: 0.9705
Epoch 86/99
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([295]) torch.Size([295, 19])
trying epoch loss
train 1252 1295
train Loss: 0.0006 Acc: 0.9668
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([373]) torch.Size([373, 19])
trying epoch loss
test 354 373
test Loss: 0.0008 Acc: 0.9491
Epoch 87/99
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([295]) torch.Size([295, 19])
trying epoch loss
train 1244 1295
train Loss: 0.0006 Acc: 0.9606
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([373]) torch.Size([373, 19])
trying epoch loss
test 364 373
test Loss: 0.0008 Acc: 0.9759
(&#39;new best accuracy = &#39;, 0.9758713136729222)
Epoch 88/99
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([295]) torch.Size([295, 19])
trying epoch loss
train 1263 1295
train Loss: 0.0006 Acc: 0.9753
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([373]) torch.Size([373, 19])
trying epoch loss
test 358 373
test Loss: 0.0008 Acc: 0.9598
Epoch 89/99
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([295]) torch.Size([295, 19])
trying epoch loss
train 1233 1295
train Loss: 0.0006 Acc: 0.9521
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([373]) torch.Size([373, 19])
trying epoch loss
test 354 373
test Loss: 0.0008 Acc: 0.9491
Epoch 90/99
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([295]) torch.Size([295, 19])
trying epoch loss
train 1242 1295
train Loss: 0.0006 Acc: 0.9591
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([373]) torch.Size([373, 19])
trying epoch loss
test 362 373
test Loss: 0.0008 Acc: 0.9705
Epoch 91/99
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([295]) torch.Size([295, 19])
trying epoch loss
train 1266 1295
train Loss: 0.0006 Acc: 0.9776
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([373]) torch.Size([373, 19])
trying epoch loss
test 365 373
test Loss: 0.0008 Acc: 0.9786
(&#39;new best accuracy = &#39;, 0.9785522788203753)
Epoch 92/99
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([295]) torch.Size([295, 19])
trying epoch loss
train 1276 1295
train Loss: 0.0006 Acc: 0.9853
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([373]) torch.Size([373, 19])
trying epoch loss
test 365 373
test Loss: 0.0008 Acc: 0.9786
Epoch 93/99
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([295]) torch.Size([295, 19])
trying epoch loss
train 1281 1295
train Loss: 0.0006 Acc: 0.9892
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([373]) torch.Size([373, 19])
trying epoch loss
test 366 373
test Loss: 0.0008 Acc: 0.9812
(&#39;new best accuracy = &#39;, 0.9812332439678284)
Epoch 94/99
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([295]) torch.Size([295, 19])
trying epoch loss
train 1261 1295
train Loss: 0.0006 Acc: 0.9737
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([373]) torch.Size([373, 19])
trying epoch loss
test 353 373
test Loss: 0.0008 Acc: 0.9464
Epoch 95/99
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([295]) torch.Size([295, 19])
trying epoch loss
train 1253 1295
train Loss: 0.0006 Acc: 0.9676
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([373]) torch.Size([373, 19])
trying epoch loss
test 355 373
test Loss: 0.0008 Acc: 0.9517
Epoch 96/99
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([295]) torch.Size([295, 19])
trying epoch loss
train 1254 1295
train Loss: 0.0006 Acc: 0.9683
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([373]) torch.Size([373, 19])
trying epoch loss
test 365 373
test Loss: 0.0008 Acc: 0.9786
Epoch 97/99
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([295]) torch.Size([295, 19])
trying epoch loss
train 1270 1295
train Loss: 0.0006 Acc: 0.9807
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([373]) torch.Size([373, 19])
trying epoch loss
test 364 373
test Loss: 0.0008 Acc: 0.9759
Epoch 98/99
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([295]) torch.Size([295, 19])
trying epoch loss
train 1264 1295
train Loss: 0.0006 Acc: 0.9761
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([373]) torch.Size([373, 19])
trying epoch loss
test 363 373
test Loss: 0.0008 Acc: 0.9732
Epoch 99/99
----------
TRAINING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([500]) torch.Size([500, 19])
loss done
loss backward
done loss backward
done optim
torch.Size([295]) torch.Size([295, 19])
trying epoch loss
train 1272 1295
train Loss: 0.0006 Acc: 0.9822
TESTING STARTED
loss done
(&#39;Reached iteration &#39;, 0)
torch.Size([373]) torch.Size([373, 19])
trying epoch loss
test 363 373
test Loss: 0.0008 Acc: 0.9732
Training complete in 1m 19s
Best val Acc: 0.981233
returning and looping back
</pre>
</div>
</div>

</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[42]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="n">genre_list</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="mi">10769</span><span class="p">)</span>
</pre></div>

    </div>
</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[43]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="k">print</span> <span class="s2">&quot;Our predictions for the movies are - </span><span class="se">\n</span><span class="s2">&quot;</span>
<span class="n">precs</span><span class="o">=</span><span class="p">[]</span>
<span class="n">recs</span><span class="o">=</span><span class="p">[]</span>
<span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="nb">len</span><span class="p">(</span><span class="n">Y_preds</span><span class="p">)):</span>
    <span class="n">row</span><span class="o">=</span><span class="n">Y_preds</span><span class="p">[</span><span class="n">i</span><span class="p">]</span>
    <span class="n">gt_genres</span><span class="o">=</span><span class="n">Y_test</span><span class="p">[</span><span class="n">i</span><span class="p">]</span>
    <span class="n">gt_genre_names</span><span class="o">=</span><span class="p">[]</span>
    <span class="k">for</span> <span class="n">j</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="nb">len</span><span class="p">(</span><span class="n">gt_genres</span><span class="p">)):</span>
        <span class="k">if</span> <span class="n">gt_genres</span><span class="p">[</span><span class="n">j</span><span class="p">]</span><span class="o">==</span><span class="mi">1</span><span class="p">:</span>
            <span class="n">gt_genre_names</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">Genre_ID_to_name</span><span class="p">[</span><span class="n">genre_list</span><span class="p">[</span><span class="n">j</span><span class="p">]])</span>
    <span class="n">top_3</span><span class="o">=</span><span class="n">np</span><span class="o">.</span><span class="n">argsort</span><span class="p">(</span><span class="n">row</span><span class="p">)[</span><span class="o">-</span><span class="mi">3</span><span class="p">:]</span>
    <span class="n">predicted_genres</span><span class="o">=</span><span class="p">[]</span>
    <span class="k">for</span> <span class="n">genre</span> <span class="ow">in</span> <span class="n">top_3</span><span class="p">:</span>
        <span class="n">predicted_genres</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">Genre_ID_to_name</span><span class="p">[</span><span class="n">genre_list</span><span class="p">[</span><span class="n">genre</span><span class="p">]])</span>
    <span class="p">(</span><span class="n">precision</span><span class="p">,</span><span class="n">recall</span><span class="p">)</span><span class="o">=</span><span class="n">precision_recall</span><span class="p">(</span><span class="n">gt_genre_names</span><span class="p">,</span><span class="n">predicted_genres</span><span class="p">)</span>
    <span class="n">precs</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">precision</span><span class="p">)</span>
    <span class="n">recs</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">recall</span><span class="p">)</span>
    <span class="k">if</span> <span class="n">i</span><span class="o">%</span><span class="k">50</span>==0:
        <span class="k">print</span> <span class="s2">&quot;Predicted: &quot;</span><span class="p">,</span><span class="n">predicted_genres</span><span class="p">,</span><span class="s2">&quot; Actual: &quot;</span><span class="p">,</span><span class="n">gt_genre_names</span>
</pre></div>

    </div>
</div>
</div>

<div class="output_wrapper">
<div class="output">


<div class="output_area">

    <div class="prompt"></div>


<div class="output_subarea output_stream output_stdout output_text">
<pre>Our predictions for the movies are - 

Predicted:  [u&#39;Mystery&#39;, u&#39;Thriller&#39;, u&#39;Horror&#39;]  Actual:  [u&#39;Adventure&#39;, u&#39;Fantasy&#39;, u&#39;Action&#39;, u&#39;Science Fiction&#39;]
Predicted:  [u&#39;Family&#39;, u&#39;Adventure&#39;, u&#39;Fantasy&#39;]  Actual:  [u&#39;Animation&#39;, u&#39;Drama&#39;, u&#39;Action&#39;, u&#39;Science Fiction&#39;]
Predicted:  [u&#39;Comedy&#39;, u&#39;Animation&#39;, u&#39;Family&#39;]  Actual:  [u&#39;Fantasy&#39;, u&#39;Horror&#39;, u&#39;Science Fiction&#39;, u&#39;Mystery&#39;]
Predicted:  [u&#39;Comedy&#39;, u&#39;Family&#39;, u&#39;Fantasy&#39;]  Actual:  [u&#39;Drama&#39;, u&#39;History&#39;, u&#39;Thriller&#39;]
Predicted:  [u&#39;Family&#39;, u&#39;Fantasy&#39;, u&#39;Comedy&#39;]  Actual:  [u&#39;Documentary&#39;, u&#39;Science Fiction&#39;]
Predicted:  [u&#39;Comedy&#39;, u&#39;Animation&#39;, u&#39;Family&#39;]  Actual:  [u&#39;Drama&#39;, u&#39;Crime&#39;, u&#39;Mystery&#39;]
Predicted:  [u&#39;Comedy&#39;, u&#39;Family&#39;, u&#39;Animation&#39;]  Actual:  [u&#39;Adventure&#39;, u&#39;Fantasy&#39;, u&#39;Comedy&#39;, u&#39;Family&#39;]
</pre>
</div>
</div>

</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[44]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span><span class="k">print</span> <span class="n">np</span><span class="o">.</span><span class="n">mean</span><span class="p">(</span><span class="n">np</span><span class="o">.</span><span class="n">asarray</span><span class="p">(</span><span class="n">precs</span><span class="p">)),</span><span class="n">np</span><span class="o">.</span><span class="n">mean</span><span class="p">(</span><span class="n">np</span><span class="o">.</span><span class="n">asarray</span><span class="p">(</span><span class="n">recs</span><span class="p">))</span>
</pre></div>

    </div>
</div>
</div>

<div class="output_wrapper">
<div class="output">


<div class="output_area">

    <div class="prompt"></div>


<div class="output_subarea output_stream output_stdout output_text">
<pre>0.21298949379178606 0.19942693409742118
</pre>
</div>
</div>

</div>
</div>

</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>Even without much tuning of the above model, these results are able to beat our previous results.</p>
<p>Note - I got accuracies as high as 78% when doing classification using plots scraped from Wikipedia. The large amount of information was very suitable for movie genre classification with a deep model. Strongly suggest you to try playing around with architectures.</p>

</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="prompt input_prompt">
</div><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h1 id="Section-9---Upcoming-Tutorials-and-Acknowledgements">Section 9 - Upcoming Tutorials and Acknowledgements<a class="anchor-link" href="#Section-9---Upcoming-Tutorials-and-Acknowledgements">&#182;</a></h1><p>Congrats! This is the end of our pilot project! Needless to say, a lot of the above content may be new to you, or may be things that you know very well. If it's the former, I hope this tutorial would have helped you. If it is the latter and you think I wrote something incorrect or that my understanding can be improved, feel free to create a github issue so that I can correct it!</p>
<p>Writing tutorials can take a lot of time, but it is a great learning experience. I am currently working on a tutorial focussing on word embeddings, which will explore word2vec and other word embeddings in detail. While it will take some time to be up, I will post a link to it's repository on the README for this project so that interested readers can find it.</p>
<p>I would like to thank a few of my friends who had an indispensible role to play in me making this tutorial. Firstly, Professor Hanspeter Pfister and Verena Kaynig at Harvard, who helped guide this tutorial/project and scope it. Secondly, my friends Sahil Loomba and Matthew Tancik for their suggestions and editing the material and the presentation of the storyline. Thirdly, Zoya Bylinskii at MIT for constantly motivating me to put in my effort into this tutorial. Finally, all others who helped me feel confident enough to take up this task and to see it till the end. Thanks all of you!</p>

</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[&nbsp;]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span> 
</pre></div>

    </div>
</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In&nbsp;[&nbsp;]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython2"><pre><span></span> 
</pre></div>

    </div>
</div>
</div>

</div>
    </div>
  </div>
</body>

 
</html>