blog/smartcity_kubeflow_predict_speeding_tickets.html


<!DOCTYPE html>
<!--[if lt IE 7]>      <html class="no-js lt-ie9 lt-ie8 lt-ie7"> <![endif]-->
<!--[if IE 7]>         <html class="no-js lt-ie9 lt-ie8"> <![endif]-->
<!--[if IE 8]>         <html class="no-js lt-ie9"> <![endif]-->
<!--[if gt IE 8]><!--> <html class="no-js"> <!--<![endif]-->
	<head>
<!-- Global site tag (gtag.js) - Google Analytics -->
<script async src="https://www.googletagmanager.com/gtag/js?id=UA-119541534-1"></script>
<script>
  window.dataLayer = window.dataLayer || [];
  function gtag(){dataLayer.push(arguments);}
  gtag('js', new Date());

  gtag('config', 'UA-119541534-1');
</script>
		
	<meta charset="utf-8">
	<meta http-equiv="X-UA-Compatible" content="IE=edge">
	<title>Patterson Consulting: Predicting Speeding Tickets in Chattanooga, TN with Kubeflow and TensorFlow</title>
	<meta name="viewport" content="width=device-width, initial-scale=1">
	<meta name="description" content="blog page for Patterson Consulting" />
	<meta name="keywords" content="blog, patterson consulting, deep learning, machine learning, apache hadoop, apache spark, etl, consulting" />
	<meta name="author" content="Patterson Consulting" />

  	<!-- Facebook and Twitter integration -->
	<meta property="og:title" content="Predicting Speeding Tickets in Chattanooga, TN with Kubeflow and TensorFlow"/>
	<meta property="og:image" content="http://www.pattersonconsultingtn.com/images/exec_strategy_bg.png"/>
	<meta property="og:url" content="http://www.pattersonconsultingtn.com/blog/deploying_computer_vision_object_detection_with_kfserving.html"/>
	<meta property="og:site_name" content=""/>
	<meta property="og:description" content="In this example we ..."/>
	

	<meta name="twitter:title" content="Predicting Speeding Tickets in Chattanooga, TN with Kubeflow and TensorFlow" />
	<meta data-rh="true" property="twitter:description" content="In this example we ..."/>

	<meta name="twitter:image" content="http://www.pattersonconsultingtn.com/images/exec_strategy_bg.png" />
	<meta name="twitter:url" content="http://www.pattersonconsultingtn.com/blog/deploying_computer_vision_object_detection_with_kfserving.html" />
	<meta name="twitter:card" content="summary_large_image" />

	<!-- Place favicon.ico and apple-touch-icon.png in the root directory -->
	<!-- <link rel="shortcut icon" href="favicon.ico"> -->
	
	<link rel="stylesheet" href="../css/animate.css">
	<link rel="stylesheet" href="../css/bootstrap.css">
	<link rel="stylesheet" href="../css/icomoon.css">

	<link rel="stylesheet" href="../css/owl.carousel.min.css">
	<link rel="stylesheet" href="../css/owl.theme.default.min.css">

	<link rel="stylesheet" href="../css/style.css">

	<link rel="stylesheet" href="https://www.w3schools.com/w3css/4/w3.css">

	<link rel="shortcut icon" href="http://www.pattersonconsultingtn.com/pct.ico" type="image/x-icon" />

	<style>
		a { 
			color: #FF0000; 
			text-decoration: underline;
		}

table {
  font-family: arial, sans-serif;
  border-collapse: collapse;
  width: 100%;
}

td, th {
  border: 1px solid #dddddd;
  text-align: left;
  padding: 8px;
}

tr:nth-child(even) {
  background-color: #dddddd;
}

.news_item_row {
	border: 0px solid #999999; 
	padding: 0px; 
	padding-top: 20px; 
	padding-bottom: 24px; 
	margin: 0px; 
	margin-bottom: 6px; 
	background-color: #ffffff;

}

.news_item_label {
	border: 1px solid #cccccc; 
	border-bottom: 0px; 
	width: 50%; 
	padding: 12px; 
	padding-top: 18px; 
	margin: 0px; 
	margin-left: 0px; 
	background-color: #dddddd;
}


.news_item_body {
	border: 2px solid #cccccc; 
	padding: 12px; 
	padding-top: 18px; 
	margin: 20px; 
	margin-left: 0px; 
	margin-top: 0px; 
	background-color: #ffffff;

}

</style>	

	<script src="../js/modernizr-2.6.2.min.js"></script>
	<!--[if lt IE 9]>
	<script src="js/respond.min.js"></script>
	<![endif]-->

	</head>
	<body class="boxed">
	<!-- Loader -->
	<div class="fh5co-loader"></div>

	<div id="wrap">

	<div id="fh5co-page">
		<header id="fh5co-header" role="banner">
			<div class="container">
				<a href="#" class="js-fh5co-nav-toggle fh5co-nav-toggle dark"><i></i></a>
				<div id="fh5co-logo"><a href="index.html"><img src="../images/website_header_top_march2018_v0.png" ></a></div>
				<nav id="fh5co-main-nav" role="navigation">
					<ul>
						<li><a href="../about.html">About</a></li>
						<li class="has-sub">
							<div class="drop-down-menu">
								<a href="#">Services</a>
								<div class="dropdown-menu-wrap">
									<ul>
										<li><a href="../offerings/data_science_offerings.html">Data Science Offerings</a></li>
										<li><a href="../offerings/managed_kafka.html">Managed Kafka</a></li>
										<li><a href="../offerings/managed_kubeflow.html">Managed Kubeflow</a></li>

										<li><a href="../big_data_apps.html">Hadoop Applications</a></li>
										<li><a href="../vision_apps.html">Computer Vision Applications</a></li>
										<li><a href="../sensor_apps.html">Sensor Applications</a></li>
										<li><a href="../exec_strategy.html">Executive Strategy</a></li>

									</ul>
								</div>
							</div>
						</li>
						<!--
						<li><a href="portfolio.html">Portfolio</a></li>
-->
						<li class="has-sub">
							<div class="drop-down-menu">
								<a href="#">Technologies</a>
								<div class="dropdown-menu-wrap">
									<ul>
										<li><a href="../deep_learning.html">Deep Learning</a></li>
										<li><a href="../hadoop.html">Apache Hadoop</a></li>										
									</ul>
								</div>
							</div>
						</li>
						
						<li><a href="../blog/blog_index.html">Blog</a></li>
					
						<li class="cta"><a href="../contact.html">Contact</a></li>
					</ul>
				</nav>
			</div>
		</header>
		
		<div id="fh5co-intro" class="fh5co-section">
			<div class="container">

				
				<div class="row row-bottom-padded-sm">
					<div class="col-md-12" id="fh5co-content">
						<h1>Predicting Speeding Tickets in Chattanooga, TN with Kubeflow and TensorFlow</h1>
						<p>Author: Pete Way</a>
							<br/>
							
							</p>

							<div>
								<img src="https://github.com/pattersonconsulting/pattersonconsulting.github.io/raw/master/blog/images/Citations_TF_Way_CiteDensity.png" class="center" style="width:900px">
							</div>

						<p>

							In this example we demonstrate how to take a multi-layer perceptron neural network out of Tensorflow and Keras libraries, with additional result analysis provided by scikit-learn. Within this tutorial, the reader will begin to understand:</p>
							<ul>
							<li>Import of datasets from CSV, and feather files. </li>
							<li>Addition of temporal specification variables</li>
							<li>Addition of spatial markers to specific GPS coordinates<ul>
							<li>GPS coordinates to Spatial Points</li>
							<li>Adjusting projection of Points. </li>
							</ul>
							</li>
							<li>Creation of negative samples based off of a collection of variable combinations.</li>
							<li>Creation of a variety of neural networks with combinations of input metrics. </li>
							<li>Analysis of the performance of the aforementioned networks based on overall Accuracy and Recall. </li>
							</ul>


						</p>
						

				<h2>Introduction to Tensorflow</h2>
				
				<p>For those who are not already familiar with the Tensorflow Python library, here is the definition of Tensorflow, from their website itself.</p>
				<blockquote class="w3-panel w3-leftbar w3-light-grey" style="padding: 16px; font-size: 12px; border: 1px solid #999999;">
					<p style="font-size: 14px;">"TensorFlow is an end-to-end open source platform for machine learning. 
				It has a comprehensive, flexible ecosystem of tools, libraries and community resources 
				that lets researchers push the state-of-the-art in ML and developers easily build and deploy 
				ML powered applications."
				-<a href="https://www.tensorflow.org/">Tensorflow.org</a></p>
				</blockquote>
				<p>Tensorflow is an amazing backend for many common machine learning problems, and provides easy model building regardless of the coder's experience level. The Tensorflow webpage features many example dataset and code pairings to get one acclimated to the coding involved.</p>
				<p>Furthermore, in this tutorial we will be exploring the Sequential model from Keras:</p>
				<blockquote class="w3-panel w3-leftbar w3-light-grey" style="padding: 16px; font-size: 12px; border: 1px solid #999999;">
					<p style="font-size: 14px;">
						A Sequential model is appropriate for a plain stack of layers where each layer has exactly one input tensor and one output tensor. 
				-<a href="https://keras.io/guides/sequential_model/">Keras.io</a></p>
				</blockquote>
				<p>Sequential models are wonderful for linear questions, such as 'Will it rain today?', where each input is fed through the model without doubling-back. They are not meant for situations with multiple input/multiple output, or if the layers are meant to be shared.</p>
				

				<h2>Use Case Question</h2>
				
				<p>Where in Chattanooga are speeding tickets being issued? 
				Is it possible to predict which roadways are likely to see speeding violations based off of historical reports?</p>
				<h2>Introduction to Data</h2>
				
				<p>Both sets data utilized in this walkthrough can be accessed and are free for public usage.</p>
				<p>The citation data set referenced is available from the ChattaData.org page here: <a href="https://internal.chattadata.org/Public-Safety/City-Court-Citations/th6b-88wc">City Court Citations</a>. </p>
				<p>The citation data includes spatial and temporal data about the location of citations issued from city courts, as well as roughly anonymous data regarding the individual receiving the citation.</p>
				<p>Roadway data for the area is also available via ChattaData, here: <a href="https://internal.chattadata.org/dataset/Chattanooga-Roadways/mw3f-d2mz">Chattanooga Roadways</a>. </p>
				<p>Roadway data includes spatial data regarding to individual segments of roadways within the Chattanooga area. Rough address data is provided, and allows for the creation of a singular 'Segment' column.</p>
				
				
				<img src="https://github.com/pattersonconsulting/pattersonconsulting.github.io/raw/master/blog/images/Citations_TF_Way_Roadways.png" class="center" width="550"/>


				<hr>
				<h2>PreProcessing</h2>
				
				<h3>Importing and Exploring Data</h3>
				
				<p>The first step to understanding any selection of data is taking a look into what data you actually have. Here, we're importing a dataset that includes rough anonymized data regarding Court Citations. First, let's import the dataset CSV using pandas, and take a look at how many entries we have.</p>

		<script charset="UTF-8" src="http://gist-it.appspot.com/github.com/PeteWay/Consulting/blob/master/Citations.py?slice=76:78&footer=minimal"></script>
		
		<pre>
		212812
		</pre>
		
		<p>Next, let's take a look at the types of columns we have. Our data includes temporal and spatial data regarding the citations recorded.</p>
		<p>Now, let's take a look at the first ten records.</p>
		<p>This combination of commands lets us take a closer look into what data we have. Notice that there are multiple 'object' columns, which includes the type and date of the violation, as well as information regarding the individual receiving the citation.</p>
	
		<script charset="UTF-8" src="http://gist-it.appspot.com/github.com/PeteWay/Consulting/blob/master/Citations.py?slice=89:91&footer=minimal"></script>

		<pre>
		Citation Year             int64
		Citation Number          object
		Offense Description      object
		Offense Code             object
		Race of Offender         object
		Sex of Offender          object
		Violation Date           object
		Address                  object
		Longitude               float64
		Latitude                float64
		Location WKT             object
		Citation_Charge_Link      int64
		Agency                   object
		dtype: object
		
		<style scoped>
			.dataframe tbody tr th:only-of-type {
				vertical-align: middle;
			}
		
			.dataframe tbody tr th {
				vertical-align: top;
			}
		
			.dataframe thead th {
				text-align: right;
			}
		

		</style>
		<table class="dataframe" width="50%">
		  <thead>
			<tr style="text-align: right;">
			  <th></th>
			  <th>Citation Year</th>
			  <th>Citation Number</th>
			  <th>Offense Description</th>
			  <th>Offense Code</th>
			  <th>Race of Offender</th>
			  <th>Sex of Offender</th>
			  <th>Violation Date</th>
			  <th>Address</th>
			  <th>Longitude</th>
			  <th>Latitude</th>
			  <th>Location WKT</th>
			  <th>Citation_Charge_Link</th>
			  <th>Agency</th>
			</tr>
		  </thead>
		  <tbody>
			<tr>
			  <th>0</th>
			  <td>2019</td>
			  <td>Q33097</td>
			  <td>RUNNING A RED LIGHT: ACCIDENT</td>
			  <td>O0218A</td>
			  <td>White</td>
			  <td>Female</td>
			  <td>2/17/19 21:30</td>
			  <td>400 N MARKET ST</td>
			  <td>-85.308799</td>
			  <td>35.065202</td>
			  <td>POINT (-85.308798717696 35.065201611634)</td>
			  <td>1187952</td>
			  <td>CHATANOOGA PD</td>
			</tr>
			<tr>
			  <th>1</th>
			  <td>2019</td>
			  <td>Q33097</td>
			  <td>FAILURE TO YIELD RIGHT-OF-WAY: ACCIDENT</td>
			  <td>55-8-130</td>
			  <td>White</td>
			  <td>Female</td>
			  <td>2/17/19 21:30</td>
			  <td>400 N MARKET ST</td>
			  <td>-85.308799</td>
			  <td>35.065202</td>
			  <td>POINT (-85.308798717696 35.065201611634)</td>
			  <td>1187953</td>
			  <td>CHATANOOGA PD</td>
			</tr>
			<tr>
			  <th>2</th>
			  <td>2019</td>
			  <td>Y132034</td>
			  <td>LIGHT LAW VIOLATION</td>
			  <td>55-9-402</td>
			  <td>African American</td>
			  <td>Male</td>
			  <td>1/1/19 21:19</td>
			  <td>1000 N HOLTZCLAW AVE</td>
			  <td>-85.275350</td>
			  <td>35.051747</td>
			  <td>POINT (-85.275350006915 35.05174717652)</td>
			  <td>1220642</td>
			  <td>CHATANOOGA PD</td>
			</tr>
			<tr>
			  <th>3</th>
			  <td>2019</td>
			  <td>Y029161</td>
			  <td>SPEEDING</td>
			  <td>55-8-152</td>
			  <td>White</td>
			  <td>Male</td>
			  <td>1/5/19 21:15</td>
			  <td>5300 HW153 SB</td>
			  <td>-85.242198</td>
			  <td>35.127374</td>
			  <td>POINT (-85.242198247004 35.127374384516)</td>
			  <td>1221039</td>
			  <td>CHATANOOGA PD</td>
			</tr>
			<tr>
			  <th>4</th>
			  <td>2019</td>
			  <td>Y029161</td>
			  <td>FINANCIAL RESPONSIBILITY LAW (INSURANCE LAW)</td>
			  <td>55-12-139</td>
			  <td>White</td>
			  <td>Male</td>
			  <td>1/5/19 21:15</td>
			  <td>5300 HW153 SB</td>
			  <td>-85.242198</td>
			  <td>35.127374</td>
			  <td>POINT (-85.242198247004 35.127374384516)</td>
			  <td>1221040</td>
			  <td>CHATANOOGA PD</td>
			</tr>
			<tr>
			  <th>5</th>
			  <td>2019</td>
			  <td>Y050066</td>
			  <td>FINANCIAL RESPONSIBILITY LAW (INSURANCE LAW)</td>
			  <td>55-12-139</td>
			  <td>African American</td>
			  <td>Male</td>
			  <td>1/4/19 23:59</td>
			  <td>1000 TUNNEL BLVD</td>
			  <td>-85.239521</td>
			  <td>35.040177</td>
			  <td>POINT (-85.239521412228 35.040177111372)</td>
			  <td>1221041</td>
			  <td>CHATANOOGA PD</td>
			</tr>
			<tr>
			  <th>6</th>
			  <td>2019</td>
			  <td>Y050066</td>
			  <td>LIGHT LAW VIOLATION</td>
			  <td>55-9-402</td>
			  <td>African American</td>
			  <td>Male</td>
			  <td>1/4/19 23:59</td>
			  <td>1000 TUNNEL BLVD</td>
			  <td>-85.239521</td>
			  <td>35.040177</td>
			  <td>POINT (-85.239521412228 35.040177111372)</td>
			  <td>1221043</td>
			  <td>CHATANOOGA PD</td>
			</tr>
			<tr>
			  <th>7</th>
			  <td>2019</td>
			  <td>Y066034</td>
			  <td>SPEEDING</td>
			  <td>55-8-152</td>
			  <td>African American</td>
			  <td>Male</td>
			  <td>1/6/19 23:14</td>
			  <td>300 MANUFACTURERS RD</td>
			  <td>-85.313603</td>
			  <td>35.062448</td>
			  <td>POINT (-85.313603119038 35.062448419579)</td>
			  <td>1221114</td>
			  <td>CHATANOOGA PD</td>
			</tr>
			<tr>
			  <th>8</th>
			  <td>2019</td>
			  <td>Y087627</td>
			  <td>FINANCIAL RESPONSIBILITY LAW (INSURANCE LAW)</td>
			  <td>55-12-139</td>
			  <td>African American</td>
			  <td>Male</td>
			  <td>1/9/19 17:22</td>
			  <td>900 N CHAMBERLAIN AVE</td>
			  <td>-85.260147</td>
			  <td>35.045116</td>
			  <td>POINT (-85.260146624969 35.045115634392)</td>
			  <td>1222463</td>
			  <td>CHATANOOGA PD</td>
			</tr>
			<tr>
			  <th>9</th>
			  <td>2019</td>
			  <td>Y082581</td>
			  <td>CHILD RESTRAINT LAW VIOLATION</td>
			  <td>55-9-602</td>
			  <td>African American</td>
			  <td>Female</td>
			  <td>4/11/19 13:40</td>
			  <td>827 WOODMOORE CIR</td>
			  <td>-85.226635</td>
			  <td>35.034660</td>
			  <td>POINT (-85.226635370051 35.03465955868)</td>
			  <td>1231728</td>
			  <td>CHATANOOGA PD</td>
			</tr>
		  </tbody>
		</table>
		</div>
		</pre>

		<p>Let's cut this data down to just the records where the violation was for speeding, to give us a better answer to our main questions we asked above. We'll also reindex the dataset for ease of understanding.</p>

		<script charset="UTF-8" src="http://gist-it.appspot.com/github.com/PeteWay/Consulting/blob/master/Citations.py?slice=98:100&footer=minimal"></script>

		
		<p>Now, we'll be taking a look at our reduced data, now that we've cut it down to just the records that pertain to our question. We can see that there were 20,815 speeding tickets within that larger dataset.</p>
		<p>Again, we'll use the head command to take a look at the first ten records of speeding.</p>
		

		<script charset="UTF-8" src="http://gist-it.appspot.com/github.com/PeteWay/Consulting/blob/master/Citations.py?slice=109:111&footer=minimal"></script>
		<pre>
		20815

		<style scoped>
			.dataframe tbody tr th:only-of-type {
				vertical-align: middle;
			}
		
			.dataframe tbody tr th {
				vertical-align: top;
			}
		
			.dataframe thead th {
				text-align: right;
			}
		</style>
		<table class="dataframe">
		  <thead>
			<tr style="text-align: right;">
			  <th></th>
			  <th>index</th>
			  <th>Citation Year</th>
			  <th>Citation Number</th>
			  <th>Offense Description</th>
			  <th>Offense Code</th>
			  <th>Race of Offender</th>
			  <th>Sex of Offender</th>
			  <th>Violation Date</th>
			  <th>Address</th>
			  <th>Longitude</th>
			  <th>Latitude</th>
			  <th>Location WKT</th>
			  <th>Citation_Charge_Link</th>
			  <th>Agency</th>
			</tr>
		  </thead>
		  <tbody>
			<tr>
			  <th>0</th>
			  <td>3</td>
			  <td>2019</td>
			  <td>Y029161</td>
			  <td>SPEEDING</td>
			  <td>55-8-152</td>
			  <td>White</td>
			  <td>Male</td>
			  <td>1/5/19 21:15</td>
			  <td>5300 HW153 SB</td>
			  <td>-85.242198</td>
			  <td>35.127374</td>
			  <td>POINT (-85.242198247004 35.127374384516)</td>
			  <td>1221039</td>
			  <td>CHATANOOGA PD</td>
			</tr>
			<tr>
			  <th>1</th>
			  <td>7</td>
			  <td>2019</td>
			  <td>Y066034</td>
			  <td>SPEEDING</td>
			  <td>55-8-152</td>
			  <td>African American</td>
			  <td>Male</td>
			  <td>1/6/19 23:14</td>
			  <td>300 MANUFACTURERS RD</td>
			  <td>-85.313603</td>
			  <td>35.062448</td>
			  <td>POINT (-85.313603119038 35.062448419579)</td>
			  <td>1221114</td>
			  <td>CHATANOOGA PD</td>
			</tr>
			<tr>
			  <th>2</th>
			  <td>30</td>
			  <td>2019</td>
			  <td>Y029169</td>
			  <td>SPEEDING</td>
			  <td>55-8-152</td>
			  <td>White</td>
			  <td>Male</td>
			  <td>1/16/19 23:44</td>
			  <td>5400 HW153</td>
			  <td>-85.246361</td>
			  <td>35.136575</td>
			  <td>POINT (-85.246360986015 35.136574900541)</td>
			  <td>1222242</td>
			  <td>CHATANOOGA PD</td>
			</tr>
			<tr>
			  <th>3</th>
			  <td>37</td>
			  <td>2019</td>
			  <td>Y003716</td>
			  <td>SPEEDING</td>
			  <td>55-8-152</td>
			  <td>Asian</td>
			  <td>Male</td>
			  <td>1/9/19 10:12</td>
			  <td>5300 HW153 SB</td>
			  <td>-85.242198</td>
			  <td>35.127374</td>
			  <td>POINT (-85.242198247004 35.127374384516)</td>
			  <td>1221499</td>
			  <td>CHATANOOGA PD</td>
			</tr>
			<tr>
			  <th>4</th>
			  <td>40</td>
			  <td>2019</td>
			  <td>Y029167</td>
			  <td>SPEEDING</td>
			  <td>55-8-152</td>
			  <td>NaN</td>
			  <td>Male</td>
			  <td>1/16/19 21:53</td>
			  <td>5400 HW153</td>
			  <td>-85.246361</td>
			  <td>35.136575</td>
			  <td>POINT (-85.246360986015 35.136574900541)</td>
			  <td>1222238</td>
			  <td>CHATANOOGA PD</td>
			</tr>
		  </tbody>
		</table>
		</div>
		</pre>
		
		<p>We'll need to adjust how our time and date are displayed before splitting the data up, just for simplicity's sake for later usage. Here, we're using a lambda statement to avoid utilizing for loops. For loops are great for assigning variables but they can get slowed down if the dataset becomes too large. While that's not a problem with this smaller dataset, it's good to familarize yourself with time and computation saving code whenever possible.</p>

		<script charset="UTF-8" src="http://gist-it.appspot.com/github.com/PeteWay/Consulting/blob/master/Citations.py?slice=118:124&footer=minimal"></script>


		<p>With the time and date split, let's now get some extra variables to help our model understand those values. Neural network models can't parse string variables, so we'll need to pull out the month, day of the week, and hour in order for the model to understand.</p>
		<p>The weekday function finds the day of the week of a given date, and assigns it a value between 0 and 6, where Monday is zero, and Sunday is six.</p>
		<p>Since the dataset already included a year column, we don't have to find that manually. We're using lambdas here as well.</p>
		<p>Finally, we'll take a look at the head of the dataset again to make sure all of our commands worked correctly.</p>
		
		<script charset="UTF-8" src="http://gist-it.appspot.com/github.com/PeteWay/Consulting/blob/master/Citations.py?slice=137:143&footer=minimal"></script>

		<pre>
		
		<style scoped>
			.dataframe tbody tr th:only-of-type {
				vertical-align: middle;
			}
		
			.dataframe tbody tr th {
				vertical-align: top;
			}
		
			.dataframe thead th {
				text-align: right;
			}
		</style>
		<table class="dataframe">
		  <thead>
			<tr style="text-align: right;">
			  <th></th>
			  <th>index</th>
			  <th>Year</th>
			  <th>Citation Number</th>
			  <th>Offense Description</th>
			  <th>Offense Code</th>
			  <th>Race of Offender</th>
			  <th>Sex of Offender</th>
			  <th>Violation Date</th>
			  <th>Address</th>
			  <th>Longitude</th>
			  <th>Latitude</th>
			  <th>Location WKT</th>
			  <th>Citation_Charge_Link</th>
			  <th>Agency</th>
			  <th>Time</th>
			  <th>Date</th>
			  <th>Month</th>
			  <th>WeekDay</th>
			  <th>Hour</th>
			</tr>
		  </thead>
		  <tbody>
			<tr>
			  <th>0</th>
			  <td>3</td>
			  <td>2019</td>
			  <td>Y029161</td>
			  <td>SPEEDING</td>
			  <td>55-8-152</td>
			  <td>White</td>
			  <td>Male</td>
			  <td>2019-01-05 21:15:00</td>
			  <td>5300 HW153 SB</td>
			  <td>-85.242198</td>
			  <td>35.127374</td>
			  <td>POINT (-85.242198247004 35.127374384516)</td>
			  <td>1221039</td>
			  <td>CHATANOOGA PD</td>
			  <td>21:15:00</td>
			  <td>2019-01-05</td>
			  <td>1</td>
			  <td>5</td>
			  <td>21</td>
			</tr>
			<tr>
			  <th>1</th>
			  <td>7</td>
			  <td>2019</td>
			  <td>Y066034</td>
			  <td>SPEEDING</td>
			  <td>55-8-152</td>
			  <td>African American</td>
			  <td>Male</td>
			  <td>2019-01-06 23:14:00</td>
			  <td>300 MANUFACTURERS RD</td>
			  <td>-85.313603</td>
			  <td>35.062448</td>
			  <td>POINT (-85.313603119038 35.062448419579)</td>
			  <td>1221114</td>
			  <td>CHATANOOGA PD</td>
			  <td>23:14:00</td>
			  <td>2019-01-06</td>
			  <td>1</td>
			  <td>6</td>
			  <td>23</td>
			</tr>
			<tr>
			  <th>2</th>
			  <td>30</td>
			  <td>2019</td>
			  <td>Y029169</td>
			  <td>SPEEDING</td>
			  <td>55-8-152</td>
			  <td>White</td>
			  <td>Male</td>
			  <td>2019-01-16 23:44:00</td>
			  <td>5400 HW153</td>
			  <td>-85.246361</td>
			  <td>35.136575</td>
			  <td>POINT (-85.246360986015 35.136574900541)</td>
			  <td>1222242</td>
			  <td>CHATANOOGA PD</td>
			  <td>23:44:00</td>
			  <td>2019-01-16</td>
			  <td>1</td>
			  <td>2</td>
			  <td>23</td>
			</tr>
			<tr>
			  <th>3</th>
			  <td>37</td>
			  <td>2019</td>
			  <td>Y003716</td>
			  <td>SPEEDING</td>
			  <td>55-8-152</td>
			  <td>Asian</td>
			  <td>Male</td>
			  <td>2019-01-09 10:12:00</td>
			  <td>5300 HW153 SB</td>
			  <td>-85.242198</td>
			  <td>35.127374</td>
			  <td>POINT (-85.242198247004 35.127374384516)</td>
			  <td>1221499</td>
			  <td>CHATANOOGA PD</td>
			  <td>10:12:00</td>
			  <td>2019-01-09</td>
			  <td>1</td>
			  <td>2</td>
			  <td>10</td>
			</tr>
			<tr>
			  <th>4</th>
			  <td>40</td>
			  <td>2019</td>
			  <td>Y029167</td>
			  <td>SPEEDING</td>
			  <td>55-8-152</td>
			  <td>NaN</td>
			  <td>Male</td>
			  <td>2019-01-16 21:53:00</td>
			  <td>5400 HW153</td>
			  <td>-85.246361</td>
			  <td>35.136575</td>
			  <td>POINT (-85.246360986015 35.136574900541)</td>
			  <td>1222238</td>
			  <td>CHATANOOGA PD</td>
			  <td>21:53:00</td>
			  <td>2019-01-16</td>
			  <td>1</td>
			  <td>2</td>
			  <td>21</td>
			</tr>
		  </tbody>
		</table>
		
		</pre>
		<h3>Combining Roadway Data with Citation reports.</h3>
		
		<p>The next section combines our roadway information with the citation information to give each entry a set roadway name.</p>
		<p>First, we'll be creating the geometry column for each of the GPS coordinates in the dataset.</p>
		<p>Then, we'll be setting a coordinate reference system, so that the roadway data and the citation records can be properly matched.</p>
		<p>Finally, we'll take a look at the head of the data again. Notice that the Location WKT column is almost exactly the same as the newly created geometry column. While we could have simply assigned that column as the 'geometry', it is simply a measure of caution to create the geometry column ourselves.</p>
		
		<script charset="UTF-8" src="http://gist-it.appspot.com/github.com/PeteWay/Consulting/blob/master/Citations.py?slice=158:166&footer=minimal"></script>

		<pre>
		<style scoped>
			.dataframe tbody tr th:only-of-type {
				vertical-align: middle;
			}
		
			.dataframe tbody tr th {
				vertical-align: top;
			}
		
			.dataframe thead th {
				text-align: right;
			}
		</style>
		<table class="dataframe">
		  <thead>
			<tr style="text-align: right;">
			  <th></th>
			  <th>index</th>
			  <th>Year</th>
			  <th>Citation Number</th>
			  <th>Offense Description</th>
			  <th>Offense Code</th>
			  <th>Race of Offender</th>
			  <th>Sex of Offender</th>
			  <th>Violation Date</th>
			  <th>Address</th>
			  <th>Longitude</th>
			  <th>Latitude</th>
			  <th>Location WKT</th>
			  <th>Citation_Charge_Link</th>
			  <th>Agency</th>
			  <th>Time</th>
			  <th>Date</th>
			  <th>Month</th>
			  <th>WeekDay</th>
			  <th>Hour</th>
			  <th>geometry</th>
			</tr>
		  </thead>
		  <tbody>
			<tr>
			  <th>0</th>
			  <td>3</td>
			  <td>2019</td>
			  <td>Y029161</td>
			  <td>SPEEDING</td>
			  <td>55-8-152</td>
			  <td>White</td>
			  <td>Male</td>
			  <td>2019-01-05 21:15:00</td>
			  <td>5300 HW153 SB</td>
			  <td>-85.242198</td>
			  <td>35.127374</td>
			  <td>POINT (-85.242198247004 35.127374384516)</td>
			  <td>1221039</td>
			  <td>CHATANOOGA PD</td>
			  <td>21:15:00</td>
			  <td>2019-01-05</td>
			  <td>1</td>
			  <td>5</td>
			  <td>21</td>
			  <td>POINT (-85.24220 35.12737)</td>
			</tr>
			<tr>
			  <th>1</th>
			  <td>7</td>
			  <td>2019</td>
			  <td>Y066034</td>
			  <td>SPEEDING</td>
			  <td>55-8-152</td>
			  <td>African American</td>
			  <td>Male</td>
			  <td>2019-01-06 23:14:00</td>
			  <td>300 MANUFACTURERS RD</td>
			  <td>-85.313603</td>
			  <td>35.062448</td>
			  <td>POINT (-85.313603119038 35.062448419579)</td>
			  <td>1221114</td>
			  <td>CHATANOOGA PD</td>
			  <td>23:14:00</td>
			  <td>2019-01-06</td>
			  <td>1</td>
			  <td>6</td>
			  <td>23</td>
			  <td>POINT (-85.31360 35.06245)</td>
			</tr>
			<tr>
			  <th>2</th>
			  <td>30</td>
			  <td>2019</td>
			  <td>Y029169</td>
			  <td>SPEEDING</td>
			  <td>55-8-152</td>
			  <td>White</td>
			  <td>Male</td>
			  <td>2019-01-16 23:44:00</td>
			  <td>5400 HW153</td>
			  <td>-85.246361</td>
			  <td>35.136575</td>
			  <td>POINT (-85.246360986015 35.136574900541)</td>
			  <td>1222242</td>
			  <td>CHATANOOGA PD</td>
			  <td>23:44:00</td>
			  <td>2019-01-16</td>
			  <td>1</td>
			  <td>2</td>
			  <td>23</td>
			  <td>POINT (-85.24636 35.13657)</td>
			</tr>
			<tr>
			  <th>3</th>
			  <td>37</td>
			  <td>2019</td>
			  <td>Y003716</td>
			  <td>SPEEDING</td>
			  <td>55-8-152</td>
			  <td>Asian</td>
			  <td>Male</td>
			  <td>2019-01-09 10:12:00</td>
			  <td>5300 HW153 SB</td>
			  <td>-85.242198</td>
			  <td>35.127374</td>
			  <td>POINT (-85.242198247004 35.127374384516)</td>
			  <td>1221499</td>
			  <td>CHATANOOGA PD</td>
			  <td>10:12:00</td>
			  <td>2019-01-09</td>
			  <td>1</td>
			  <td>2</td>
			  <td>10</td>
			  <td>POINT (-85.24220 35.12737)</td>
			</tr>
			<tr>
			  <th>4</th>
			  <td>40</td>
			  <td>2019</td>
			  <td>Y029167</td>
			  <td>SPEEDING</td>
			  <td>55-8-152</td>
			  <td>NaN</td>
			  <td>Male</td>
			  <td>2019-01-16 21:53:00</td>
			  <td>5400 HW153</td>
			  <td>-85.246361</td>
			  <td>35.136575</td>
			  <td>POINT (-85.246360986015 35.136574900541)</td>
			  <td>1222238</td>
			  <td>CHATANOOGA PD</td>
			  <td>21:53:00</td>
			  <td>2019-01-16</td>
			  <td>1</td>
			  <td>2</td>
			  <td>21</td>
			  <td>POINT (-85.24636 35.13657)</td>
			</tr>
		  </tbody>
		</table>
		
		</pre>

		<p>Now, we'll be importing our roadway data, double-checking the type of our roadways, and limiting the entries to only that type. Note that sometimes datasets will have corrupted or incomplete data. This was the case with this dataset, where one entry of the roadways was incomplete. We can see that in the totals printed before and after the selection line.</p>
		<p>Next, we join the two datasets together with a geopandas command called sjoin. Sjoin determines spatial matching between datasets of any type (Point, Polygon, Multi-line, etc) and one can select how they would like the new merged set to be set up by selecting 'left', 'right', or 'inner' for the how variable. We are looking to select citations within the roadway data, so we will select 'left', since our citations set is the left variable.</p>
		<p>Something interesting happens when we merge our data, though. notice that the number of records actually increases. This is related to the records that do not fall within a given roadway polygon. The true number of citations retained is shown in our last print line, where we can see the number falling to 18,576.</p>
		<p>Once again, we'll print the head of the data to verify our changes.</p>
		
		<script charset="UTF-8" src="http://gist-it.appspot.com/github.com/PeteWay/Consulting/blob/master/Citations.py?slice=179:196&footer=minimal"></script>

		<pre>
		0        Polygon
		1        Polygon
		2        Polygon
		3        Polygon
		4        Polygon
				  ...   
		11504    Polygon
		11505    Polygon
		11506    Polygon
		11507    Polygon
		11508    Polygon
		Length: 11509, dtype: object
		Roads: 11509
		Roads: 11473
		Citation records before merge: 20815
		Records after merge 54517
		Citations before dropping duplicates: 54517
		Citations after dropping duplicates: 18576
		
		<style scoped>
			.dataframe tbody tr th:only-of-type {
				vertical-align: middle;
			}
		
			.dataframe tbody tr th {
				vertical-align: top;
			}
		
			.dataframe thead th {
				text-align: right;
			}
		</style>
		<table class="dataframe">
		  <thead>
			<tr style="text-align: right;">
			  <th></th>
			  <th>index</th>
			  <th>Year</th>
			  <th>Citation Number</th>
			  <th>Offense Description</th>
			  <th>Offense Code</th>
			  <th>Race of Offender</th>
			  <th>Sex of Offender</th>
			  <th>Violation Date</th>
			  <th>Address</th>
			  <th>Longitude</th>
			  <th>...</th>
			  <th>Agency</th>
			  <th>Time</th>
			  <th>Date</th>
			  <th>Month</th>
			  <th>WeekDay</th>
			  <th>Hour</th>
			  <th>geometry</th>
			  <th>index_right</th>
			  <th>Road</th>
			  <th>Segment</th>
			</tr>
		  </thead>
		  <tbody>
			<tr>
			  <th>0</th>
			  <td>3</td>
			  <td>2019</td>
			  <td>Y029161</td>
			  <td>SPEEDING</td>
			  <td>55-8-152</td>
			  <td>White</td>
			  <td>Male</td>
			  <td>2019-01-05 21:15:00</td>
			  <td>5300 HW153 SB</td>
			  <td>-85.242198</td>
			  <td>...</td>
			  <td>CHATANOOGA PD</td>
			  <td>21:15:00</td>
			  <td>2019-01-05</td>
			  <td>1</td>
			  <td>5</td>
			  <td>21</td>
			  <td>POINT (-85.24220 35.12737)</td>
			  <td>5065.0</td>
			  <td>Highway 153</td>
			  <td>1.0</td>
			</tr>
			<tr>
			  <th>1</th>
			  <td>7</td>
			  <td>2019</td>
			  <td>Y066034</td>
			  <td>SPEEDING</td>
			  <td>55-8-152</td>
			  <td>African American</td>
			  <td>Male</td>
			  <td>2019-01-06 23:14:00</td>
			  <td>300 MANUFACTURERS RD</td>
			  <td>-85.313603</td>
			  <td>...</td>
			  <td>CHATANOOGA PD</td>
			  <td>23:14:00</td>
			  <td>2019-01-06</td>
			  <td>1</td>
			  <td>6</td>
			  <td>23</td>
			  <td>POINT (-85.31360 35.06245)</td>
			  <td>6530.0</td>
			  <td>Manufacturers Rd</td>
			  <td>5.0</td>
			</tr>
			<tr>
			  <th>3</th>
			  <td>37</td>
			  <td>2019</td>
			  <td>Y003716</td>
			  <td>SPEEDING</td>
			  <td>55-8-152</td>
			  <td>Asian</td>
			  <td>Male</td>
			  <td>2019-01-09 10:12:00</td>
			  <td>5300 HW153 SB</td>
			  <td>-85.242198</td>
			  <td>...</td>
			  <td>CHATANOOGA PD</td>
			  <td>10:12:00</td>
			  <td>2019-01-09</td>
			  <td>1</td>
			  <td>2</td>
			  <td>10</td>
			  <td>POINT (-85.24220 35.12737)</td>
			  <td>5065.0</td>
			  <td>Highway 153</td>
			  <td>1.0</td>
			</tr>
			<tr>
			  <th>5</th>
			  <td>50</td>
			  <td>2019</td>
			  <td>Y090161</td>
			  <td>SPEEDING</td>
			  <td>55-8-152</td>
			  <td>White</td>
			  <td>Female</td>
			  <td>2019-01-01 23:10:00</td>
			  <td>200 GEORGIA AVE</td>
			  <td>-85.302488</td>
			  <td>...</td>
			  <td>CHATANOOGA PD</td>
			  <td>23:10:00</td>
			  <td>2019-01-01</td>
			  <td>1</td>
			  <td>1</td>
			  <td>23</td>
			  <td>POINT (-85.30249 35.05758)</td>
			  <td>4413.0</td>
			  <td>Georgia Ave</td>
			  <td>16.0</td>
			</tr>
			<tr>
			  <th>6</th>
			  <td>60</td>
			  <td>2019</td>
			  <td>Y139892</td>
			  <td>SPEEDING</td>
			  <td>55-8-152</td>
			  <td>White</td>
			  <td>Female</td>
			  <td>2019-01-22 16:21:00</td>
			  <td>1600 MCCALLIE AVE</td>
			  <td>-85.278663</td>
			  <td>...</td>
			  <td>CHATANOOGA PD</td>
			  <td>16:21:00</td>
			  <td>2019-01-22</td>
			  <td>1</td>
			  <td>1</td>
			  <td>16</td>
			  <td>POINT (-85.27866 35.03709)</td>
			  <td>6709.0</td>
			  <td>McCallie Ave</td>
			  <td>15.0</td>
			</tr>
		  </tbody>
		</table>
		<p>5 rows × 23 columns</p>
		
		</pre>
		
		<p>Now, let's take the set of roadway strings we just created and convert them into category variables. This takes all the roadways in question and creates a unique numeric tag for each one. This way our roadways can be input into the model, while retaining their unique identities.</p>
		
		<script charset="UTF-8" src="http://gist-it.appspot.com/github.com/PeteWay/Consulting/blob/master/Citations.py?slice=202:204&footer=minimal"></script>


		<pre>
		0        397
		1        455
		3        397
		5        360
		6        461
				... 
		20809     84
		20810    222
		20811    222
		20812    295
		20813    222
		Name: Road_Num, Length: 18576, dtype: int16</pre>
		
		
		<h3>Parring Down Variables and Preparing for Negative Sampling</h3>
		
		<p>Now we can identify which columns need to be removed for our model. Anything that is a string or factor cannot be parsed, so it has to go. Also, there are columns that aren't really necessary for the situation, like information about the person who received the ticket. This list of variables to drop includes:</p>
		<ul>
		<li>Citation Number</li>
		<li>Offense Description</li>
		<li>Offense Code</li>
		<li>Race of Offender</li>
		<li>Sex of Offender</li>
		<li>Violation Date</li>
		<li>Address </li>
		<li>Location WKT</li>
		<li>Citation_Charge_Link</li>
		<li>Agency </li>
		<li>Date</li>
		<li>Index from roadways</li>
		</ul>
		<p>Now, before we remove any of these variables, we'll be making a copy to retain all of our information. Also, we'll be making a smaller dataframe with just our roadways and numeric tags, so we can understand the output of our model.</p>

		<script charset="UTF-8" src="http://gist-it.appspot.com/github.com/PeteWay/Consulting/blob/master/Citations.py?slice=226:230&footer=minimal"></script>

		<script charset="UTF-8" src="http://gist-it.appspot.com/github.com/PeteWay/Consulting/blob/master/Citations.py?slice=235:240&footer=minimal"></script>

		
		<h3>Negative Sampling</h3>
		
		<p>Negative sampling is the creation of negative records from the provided positive records, through the alteration of some if not all of the available variables. Then, the negative samples are compared to the original positive records to remove any possible false negatives. 
		We can roughly determine how many combinations of the remaining variables are possible, minus the records we
		have within the dataset. As shown below, there is an immense amount of negatives possible when considering all combinations of the variables available. However, not all of these options are available, as each roadway has a differing number of segments.</p>

		<script charset="UTF-8" src="http://gist-it.appspot.com/github.com/PeteWay/Consulting/blob/master/Citations.py?slice=250:255&footer=minimal"></script>
		
		
		<pre>
		642279024
		</pre>
		
	
		<p>Since the creation of negative records is not the focus of this post, we'll just highlight a quick example using a very small slice of our overall dataset. We're slicing a copy of the first ten entries in the citations dataset, and expanding on a Itertools method called product in order to mimic the behavior of an R method called expand. Basically, expand takes a dataset and 'expands' the current entries into every possible combination of said entries. Notice that the unique entries in the created negative set test all match to those found in that sliced piece from citations, but the combinations of those variables created almost a million entries, once the existing citations entries were removed.</p>

		<script charset="UTF-8" src="http://gist-it.appspot.com/github.com/PeteWay/Consulting/blob/master/Citations.py?slice=262:286&footer=minimal"></script>

		<pre>     
		Year  Month  WeekDay  Hour  Segment  Road_Num
		0  2019.0    1.0      5.0  21.0      1.0     397.0
		1  2019.0    1.0      6.0  23.0      5.0     455.0
		2  2019.0    1.0      2.0  10.0      1.0     397.0
		3  2019.0    1.0      1.0  23.0     16.0     360.0
		4  2019.0    1.0      1.0  16.0     15.0     461.0
		5  2019.0    4.0      2.0  14.0      9.0     549.0
		6  2019.0    1.0      3.0  19.0      5.0     426.0
		7  2019.0    1.0      1.0  11.0      1.0     398.0
		8  2019.0    1.0      2.0  16.0      2.0     164.0
		9  2019.0    1.0      3.0  10.0     23.0     592.0
		1000000
		992686
		Year  :  [2019.]
		Month  :  [1. 4.]
		WeekDay  :  [5. 6. 2. 1. 3.]
		Hour  :  [21. 19. 14. 23. 11. 10. 16.]
		Segment  :  [ 5.  2. 16. 15.  1. 23.  9.]
		Road_Num  :  [455. 360. 426. 398. 164. 592. 397. 461. 549.]
		
		
		<style scoped>
			.dataframe tbody tr th:only-of-type {
				vertical-align: middle;
			}
		
			.dataframe tbody tr th {
				vertical-align: top;
			}
		
			.dataframe thead th {
				text-align: right;
			}
		</style>
		<table class="dataframe">
		  <thead>
			<tr style="text-align: right;">
			  <th></th>
			  <th>Year</th>
			  <th>Month</th>
			  <th>WeekDay</th>
			  <th>Hour</th>
			  <th>Segment</th>
			  <th>Road_Num</th>
			</tr>
		  </thead>
		  <tbody>
			<tr>
			  <th>1160</th>
			  <td>2019.0</td>
			  <td>1.0</td>
			  <td>5.0</td>
			  <td>21.0</td>
			  <td>5.0</td>
			  <td>455.0</td>
			</tr>
			<tr>
			  <th>1366</th>
			  <td>2019.0</td>
			  <td>1.0</td>
			  <td>5.0</td>
			  <td>19.0</td>
			  <td>5.0</td>
			  <td>360.0</td>
			</tr>
			<tr>
			  <th>1685</th>
			  <td>2019.0</td>
			  <td>1.0</td>
			  <td>5.0</td>
			  <td>14.0</td>
			  <td>2.0</td>
			  <td>426.0</td>
			</tr>
			<tr>
			  <th>1733</th>
			  <td>2019.0</td>
			  <td>1.0</td>
			  <td>5.0</td>
			  <td>23.0</td>
			  <td>16.0</td>
			  <td>398.0</td>
			</tr>
			<tr>
			  <th>1737</th>
			  <td>2019.0</td>
			  <td>1.0</td>
			  <td>5.0</td>
			  <td>11.0</td>
			  <td>16.0</td>
			  <td>398.0</td>
			</tr>
		  </tbody>
		</table>
		
		</pre>

		<p>Now we can import our actual negative data (created outside of this example, for simplicity's sake).</p>
		<p>In order to produce a rough ratio for negatives to positives, we'll find how many times the number of citations fits within the negatives, and then take a given iterative of that number. In this case, The length of the negatives is roughly 142 times the length of the citations, so we'll be saving every 142nd entry in the negatives in order to create a roughly 1:1 ratio.</p>
		<p>The imported negatives have the 'Citation' variable set to zero, since no speeding citation was issued for that temporal/spatial instance. So, we have to add a 'Citation' variable to the actual citation data, and then we are able to merge the sets. All in total, the dataset, once merged, has a length of 37,272 entries with a roughly 1:1 ratio for citations to negatives.</p>
		
		<script charset="UTF-8" src="http://gist-it.appspot.com/github.com/PeteWay/Consulting/blob/master/Citations.py?slice=297:311&footer=minimal"></script>

		<pre>
		2654771
		142
		18696
		18576
		37272
		</pre>
		
		<hr>
		<h2>Neural Network Creation</h2>
		<h3>Splitting Data and preparing for the neural network</h3>
		
		<p>With our data now in a format that our model can understand, let's create the training and testing sets. First, after all of our imports are done, we shuffle the data. This allows the model to properly understand the data, and allows for the best chance of each epoch receiving a different 'chunk' of the data. Next, we'll be dividing the sets by year, with this year's (2020) citations being our testing set. Finally, let's also take a look at how many records are in each. We can see that there are 36,029 entries in our training, and 1,243 in our testing.</p>
		
		<script charset="UTF-8" src="http://gist-it.appspot.com/github.com/PeteWay/Consulting/blob/master/Citations.py?slice=323:349&footer=minimal"></script>

		<pre>
		36029
		1243
		</pre>

		
		<h3>Standardizing Data with MinMaxScaler</h3>
		
		<p>The function below takes in a dataset, removing the 'Y' column and transforming the data using a minimum/maximum scaler. What this means is that for each column in the data, the maximum and minimum values are found, and are reassigned as 1 and 0 respectively. Then, each value in the column is scaled to fall between those two limits. This simplifies the process for the neural network, allowing for faster learning.</p>
		
		<script charset="UTF-8" src="http://gist-it.appspot.com/github.com/PeteWay/Consulting/blob/master/Citations.py?slice=356:366&footer=minimal"></script>

		
		<p>The following code snippet takes the training and testing sets we just created, and standardizes them, splitting into X and Y for both training and testing. Then, we take a look at the resulting datasets.</p>
		<p>Note that in this particular dataset, we don't apply the scaler to the Y column, since it is already on a minmax scale.</p>
		
		<script charset="UTF-8" src="http://gist-it.appspot.com/github.com/PeteWay/Consulting/blob/master/Citations.py?slice=374:382&footer=minimal"></script>

		<pre>

		[&#39;Year&#39; &#39;Month&#39; &#39;WeekDay&#39; &#39;Hour&#39; &#39;Segment&#39; &#39;Road_Num&#39;]
				 Citation
		10275         1.0
		2348822       0.0
		16931         1.0
		758280        0.0
		9939          1.0
			   Year     Month   WeekDay      Hour  Segment  Road_Num
		0  0.571429  0.181818  0.333333  0.869565   0.0250  0.364501
		1  0.285714  0.818182  0.500000  0.695652   0.0000  0.882726
		2  0.142857  0.818182  0.166667  0.608696   0.1000  0.575277
		3  0.714286  0.727273  0.333333  0.782609   0.0250  0.294770
		4  0.571429  0.090909  0.666667  1.000000   0.0875  0.630745
				 Citation
		2087          1.0
		439632        0.0
		1203876       0.0
		1339202       0.0
		1993          1.0
		   Year  Month   WeekDay      Hour   Segment  Road_Num
		0   0.0   0.00  0.333333  0.608696  0.277778  0.636508
		1   0.0   0.00  1.000000  0.521739  0.000000  0.171429
		2   0.0   0.25  0.500000  0.695652  0.000000  0.463492
		3   0.0   0.25  0.500000  0.478261  0.000000  0.503175
		4   0.0   0.25  0.833333  0.695652  0.013889  0.360317
		</pre>
	
		
		<h3>Creating a Neural Network Model</h3>
		
		<p>Now, we can actually create our neural network. For this example, we'll be utilizing a Sequential model, with an input layer, and output layer, and three layers in between, gradually diminishing the node counts. We compile the model with a variety of optimizers and loss functions, to test which of the list is the best combination.</p>
		
		<script charset="UTF-8" src="http://gist-it.appspot.com/github.com/PeteWay/Consulting/blob/master/Citations.py?slice=391:486&footer=minimal"></script>

		<pre style="height: 20pc; overflow-y: scroll;">
		Metrics: rmsprop binary_crossentropy
		Epoch 1/10
		1442/1442 [==============================] - 2s 1ms/step - loss: 0.7022 - recall: 0.8409 - accuracy: 0.5010 - auc: 0.5007 - val_loss: 0.6941 - val_recall: 1.0000 - val_accuracy: 0.4500 - val_auc: 0.5000
		Epoch 2/10
		1442/1442 [==============================] - 2s 1ms/step - loss: 0.6931 - recall: 0.8659 - accuracy: 0.5014 - auc: 0.5010 - val_loss: 0.6930 - val_recall: 1.0000 - val_accuracy: 0.4500 - val_auc: 0.5000
		Epoch 3/10
		1442/1442 [==============================] - 2s 1ms/step - loss: 0.6928 - recall: 0.7408 - accuracy: 0.5222 - auc: 0.5258 - val_loss: 0.6927 - val_recall: 1.0000 - val_accuracy: 0.4500 - val_auc: 0.6467
		Epoch 4/10
		1442/1442 [==============================] - 2s 1ms/step - loss: 0.6912 - recall: 0.7923 - accuracy: 0.5379 - auc: 0.5861 - val_loss: 0.6902 - val_recall: 0.9111 - val_accuracy: 0.4500 - val_auc: 0.6859
		Epoch 5/10
		1442/1442 [==============================] - 2s 1ms/step - loss: 0.6848 - recall: 0.6886 - accuracy: 0.6052 - auc: 0.6519 - val_loss: 0.6837 - val_recall: 0.9111 - val_accuracy: 0.4500 - val_auc: 0.7131
		Epoch 6/10
		1442/1442 [==============================] - 2s 1ms/step - loss: 0.6691 - recall: 0.6104 - accuracy: 0.6289 - auc: 0.6701 - val_loss: 0.6714 - val_recall: 0.8444 - val_accuracy: 0.4700 - val_auc: 0.7356
		Epoch 7/10
		1442/1442 [==============================] - 2s 1ms/step - loss: 0.6492 - recall: 0.5836 - accuracy: 0.6417 - auc: 0.6822 - val_loss: 0.6290 - val_recall: 0.7556 - val_accuracy: 0.6400 - val_auc: 0.7519
		Epoch 8/10
		1442/1442 [==============================] - 2s 1ms/step - loss: 0.6355 - recall: 0.5420 - accuracy: 0.6500 - auc: 0.6880 - val_loss: 0.6296 - val_recall: 0.7778 - val_accuracy: 0.6200 - val_auc: 0.7638
		Epoch 9/10
		1442/1442 [==============================] - 2s 1ms/step - loss: 0.6303 - recall: 0.5237 - accuracy: 0.6513 - auc: 0.6894 - val_loss: 0.6133 - val_recall: 0.7111 - val_accuracy: 0.6200 - val_auc: 0.7644
		Epoch 10/10
		1442/1442 [==============================] - 2s 1ms/step - loss: 0.6290 - recall: 0.5102 - accuracy: 0.6508 - auc: 0.6891 - val_loss: 0.6008 - val_recall: 0.6889 - val_accuracy: 0.6600 - val_auc: 0.7717
		3603/3603 [==============================] - 4s 1ms/step - loss: 0.6285 - recall: 0.5010 - accuracy: 0.6522 - auc: 0.6896
		[0.6285046935081482, 0.5010220408439636, 0.6521968245506287, 0.6895882487297058]
		
		Model Training Accuracy: 65.21968245506287
		Model Training Loss: 0.6677074730396271
		Rounded Test Accuracy: 68.62429605792437
		Test Loss 0.6597847938537598
		AUC: 0.685855
		528 240 150 325
		Saved grid model to disk
		Metrics: rmsprop poisson
		Epoch 1/10
		1442/1442 [==============================] - 2s 1ms/step - loss: 0.8491 - recall: 1.0000 - accuracy: 0.5024 - auc: 0.4972 - val_loss: 0.8125 - val_recall: 1.0000 - val_accuracy: 0.4500 - val_auc: 0.5000
		Epoch 2/10
		1442/1442 [==============================] - 2s 1ms/step - loss: 0.8481 - recall: 0.9077 - accuracy: 0.5134 - auc: 0.5216 - val_loss: 0.8119 - val_recall: 1.0000 - val_accuracy: 0.4500 - val_auc: 0.5899
		Epoch 3/10
		1442/1442 [==============================] - 2s 1ms/step - loss: 0.8476 - recall: 0.6822 - accuracy: 0.5456 - auc: 0.5692 - val_loss: 0.8113 - val_recall: 0.8889 - val_accuracy: 0.4600 - val_auc: 0.6321
		Epoch 4/10
		1442/1442 [==============================] - 2s 1ms/step - loss: 0.8459 - recall: 0.5768 - accuracy: 0.5906 - auc: 0.6207 - val_loss: 0.8101 - val_recall: 0.9111 - val_accuracy: 0.4600 - val_auc: 0.6891
		Epoch 5/10
		1442/1442 [==============================] - 2s 1ms/step - loss: 0.8419 - recall: 0.6034 - accuracy: 0.6114 - auc: 0.6528 - val_loss: 0.8055 - val_recall: 0.8222 - val_accuracy: 0.5200 - val_auc: 0.7190
		Epoch 6/10
		1442/1442 [==============================] - 2s 1ms/step - loss: 0.8348 - recall: 0.5487 - accuracy: 0.6300 - auc: 0.6688 - val_loss: 0.8038 - val_recall: 0.8222 - val_accuracy: 0.5000 - val_auc: 0.7428
		Epoch 7/10
		1442/1442 [==============================] - 2s 1ms/step - loss: 0.8270 - recall: 0.5498 - accuracy: 0.6435 - auc: 0.6814 - val_loss: 0.7912 - val_recall: 0.8000 - val_accuracy: 0.6300 - val_auc: 0.7558
		Epoch 8/10
		1442/1442 [==============================] - 2s 1ms/step - loss: 0.8221 - recall: 0.5320 - accuracy: 0.6493 - auc: 0.6878 - val_loss: 0.7829 - val_recall: 0.7778 - val_accuracy: 0.6300 - val_auc: 0.7586
		Epoch 9/10
		1442/1442 [==============================] - 2s 1ms/step - loss: 0.8201 - recall: 0.5259 - accuracy: 0.6519 - auc: 0.6900 - val_loss: 0.7736 - val_recall: 0.6889 - val_accuracy: 0.6500 - val_auc: 0.7616
		Epoch 10/10
		1442/1442 [==============================] - 2s 1ms/step - loss: 0.8195 - recall: 0.5206 - accuracy: 0.6525 - auc: 0.6897 - val_loss: 0.7749 - val_recall: 0.6889 - val_accuracy: 0.6500 - val_auc: 0.7659
		3603/3603 [==============================] - 4s 1ms/step - loss: 0.8192 - recall: 0.5126 - accuracy: 0.6526 - auc: 0.6903
		[0.8192249536514282, 0.5126236081123352, 0.6526131629943848, 0.6903356313705444]
		
		Model Training Accuracy: 65.26131629943848
		Model Training Loss: 0.8356072545051575
		Rounded Test Accuracy: 67.57843925985519
		Test Loss 0.7977555394172668
		AUC: 0.683416
		500 268 135 340
		Saved grid model to disk
		Metrics: rmsprop mse
		Epoch 1/10
		1442/1442 [==============================] - 2s 1ms/step - loss: 0.2502 - recall: 0.4136 - accuracy: 0.4992 - auc: 0.5005 - val_loss: 0.2504 - val_recall: 1.0000 - val_accuracy: 0.4500 - val_auc: 0.5000
		Epoch 2/10
		1442/1442 [==============================] - 2s 1ms/step - loss: 0.2500 - recall: 0.7103 - accuracy: 0.5063 - auc: 0.5041 - val_loss: 0.2494 - val_recall: 0.0000e+00 - val_accuracy: 0.5500 - val_auc: 0.5000
		Epoch 3/10
		1442/1442 [==============================] - 2s 1ms/step - loss: 0.2499 - recall: 0.6097 - accuracy: 0.5084 - auc: 0.5094 - val_loss: 0.2501 - val_recall: 1.0000 - val_accuracy: 0.4500 - val_auc: 0.6152
		Epoch 4/10
		1442/1442 [==============================] - 2s 1ms/step - loss: 0.2496 - recall: 0.5127 - accuracy: 0.5391 - auc: 0.5489 - val_loss: 0.2499 - val_recall: 1.0000 - val_accuracy: 0.4500 - val_auc: 0.6253
		Epoch 5/10
		1442/1442 [==============================] - 2s 1ms/step - loss: 0.2486 - recall: 0.5544 - accuracy: 0.5832 - auc: 0.6055 - val_loss: 0.2474 - val_recall: 0.6889 - val_accuracy: 0.6700 - val_auc: 0.6982
		Epoch 6/10
		1442/1442 [==============================] - 2s 1ms/step - loss: 0.2452 - recall: 0.4788 - accuracy: 0.6167 - auc: 0.6535 - val_loss: 0.2444 - val_recall: 0.8000 - val_accuracy: 0.5700 - val_auc: 0.7285
		Epoch 7/10
		1442/1442 [==============================] - 2s 1ms/step - loss: 0.2389 - recall: 0.5426 - accuracy: 0.6305 - auc: 0.6678 - val_loss: 0.2384 - val_recall: 0.8000 - val_accuracy: 0.5800 - val_auc: 0.7485
		Epoch 8/10
		1442/1442 [==============================] - 2s 1ms/step - loss: 0.2319 - recall: 0.5585 - accuracy: 0.6398 - auc: 0.6791 - val_loss: 0.2305 - val_recall: 0.8000 - val_accuracy: 0.6300 - val_auc: 0.7537
		Epoch 9/10
		1442/1442 [==============================] - 2s 1ms/step - loss: 0.2265 - recall: 0.5520 - accuracy: 0.6476 - auc: 0.6858 - val_loss: 0.2282 - val_recall: 0.8222 - val_accuracy: 0.6300 - val_auc: 0.7610
		Epoch 10/10
		1442/1442 [==============================] - 2s 1ms/step - loss: 0.2232 - recall: 0.5388 - accuracy: 0.6498 - auc: 0.6882 - val_loss: 0.2229 - val_recall: 0.7778 - val_accuracy: 0.6300 - val_auc: 0.7636
		3603/3603 [==============================] - 4s 1ms/step - loss: 0.2220 - recall: 0.5462 - accuracy: 0.6511 - auc: 0.6899
		[0.22200877964496613, 0.5461576581001282, 0.6511144042015076, 0.6899453401565552]
		
		Model Training Accuracy: 65.11144042015076
		Model Training Loss: 0.2413971558213234
		Rounded Test Accuracy: 65.00402252614641
		Test Loss 0.241142638027668
		AUC: 0.676236
		434 334 101 374
		Saved grid model to disk
		Metrics: adam binary_crossentropy
		Epoch 1/10
		1442/1442 [==============================] - 2s 1ms/step - loss: 0.6932 - recall: 0.8340 - accuracy: 0.5007 - auc: 0.4988 - val_loss: 0.6939 - val_recall: 1.0000 - val_accuracy: 0.4500 - val_auc: 0.5000
		Epoch 2/10
		1442/1442 [==============================] - 2s 1ms/step - loss: 0.6932 - recall: 0.8739 - accuracy: 0.5020 - auc: 0.4972 - val_loss: 0.6934 - val_recall: 1.0000 - val_accuracy: 0.4500 - val_auc: 0.5000
		Epoch 3/10
		1442/1442 [==============================] - 2s 1ms/step - loss: 0.6930 - recall: 0.7185 - accuracy: 0.5093 - auc: 0.5080 - val_loss: 0.6940 - val_recall: 1.0000 - val_accuracy: 0.4500 - val_auc: 0.5000
		Epoch 4/10
		1442/1442 [==============================] - 2s 1ms/step - loss: 0.6921 - recall: 0.9299 - accuracy: 0.5252 - auc: 0.5670 - val_loss: 0.6922 - val_recall: 1.0000 - val_accuracy: 0.4500 - val_auc: 0.6911
		Epoch 5/10
		1442/1442 [==============================] - 2s 1ms/step - loss: 0.6808 - recall: 0.6899 - accuracy: 0.6128 - auc: 0.6525 - val_loss: 0.6757 - val_recall: 0.8889 - val_accuracy: 0.5000 - val_auc: 0.7444
		Epoch 6/10
		1442/1442 [==============================] - 2s 1ms/step - loss: 0.6531 - recall: 0.5752 - accuracy: 0.6423 - auc: 0.6785 - val_loss: 0.6422 - val_recall: 0.8222 - val_accuracy: 0.6600 - val_auc: 0.7588
		Epoch 7/10
		1442/1442 [==============================] - 2s 1ms/step - loss: 0.6359 - recall: 0.5249 - accuracy: 0.6512 - auc: 0.6861 - val_loss: 0.6173 - val_recall: 0.6889 - val_accuracy: 0.6500 - val_auc: 0.7735
		Epoch 8/10
		1442/1442 [==============================] - 2s 1ms/step - loss: 0.6305 - recall: 0.5054 - accuracy: 0.6533 - auc: 0.6875 - val_loss: 0.6024 - val_recall: 0.6889 - val_accuracy: 0.6800 - val_auc: 0.7792
		Epoch 9/10
		1442/1442 [==============================] - 2s 1ms/step - loss: 0.6292 - recall: 0.4966 - accuracy: 0.6542 - auc: 0.6881 - val_loss: 0.5942 - val_recall: 0.6889 - val_accuracy: 0.7000 - val_auc: 0.7798
		Epoch 10/10
		1442/1442 [==============================] - 2s 1ms/step - loss: 0.6289 - recall: 0.4920 - accuracy: 0.6545 - auc: 0.6873 - val_loss: 0.5909 - val_recall: 0.6889 - val_accuracy: 0.7000 - val_auc: 0.7816
		3603/3603 [==============================] - 4s 1ms/step - loss: 0.6287 - recall: 0.4826 - accuracy: 0.6553 - auc: 0.6877
		[0.6286646723747253, 0.482625275850296, 0.655277669429779, 0.6877278089523315]
		
		Model Training Accuracy: 65.5277669429779
		Model Training Loss: 0.662977147102356
		Rounded Test Accuracy: 71.68141592920354
		Test Loss 0.6496215999126435
		AUC: 0.709792
		568 200 152 323
		Saved grid model to disk
		Metrics: adam poisson
		Epoch 1/10
		1442/1442 [==============================] - 2s 1ms/step - loss: 0.8547 - recall: 0.0861 - accuracy: 0.4984 - auc: 0.4971 - val_loss: 0.8122 - val_recall: 1.0000 - val_accuracy: 0.4500 - val_auc: 0.5000
		Epoch 2/10
		1442/1442 [==============================] - 2s 1ms/step - loss: 0.8482 - recall: 0.7375 - accuracy: 0.5014 - auc: 0.5032 - val_loss: 0.8122 - val_recall: 1.0000 - val_accuracy: 0.4500 - val_auc: 0.5000
		Epoch 3/10
		1442/1442 [==============================] - 2s 1ms/step - loss: 0.8479 - recall: 0.5905 - accuracy: 0.5224 - auc: 0.5339 - val_loss: 0.8104 - val_recall: 0.1778 - val_accuracy: 0.6200 - val_auc: 0.6588
		Epoch 4/10
		1442/1442 [==============================] - 2s 1ms/step - loss: 0.8460 - recall: 0.4171 - accuracy: 0.5941 - auc: 0.6306 - val_loss: 0.8078 - val_recall: 0.7778 - val_accuracy: 0.6400 - val_auc: 0.7099
		Epoch 5/10
		1442/1442 [==============================] - 2s 1ms/step - loss: 0.8404 - recall: 0.4682 - accuracy: 0.6281 - auc: 0.6659 - val_loss: 0.8012 - val_recall: 0.8000 - val_accuracy: 0.6300 - val_auc: 0.7388
		Epoch 6/10
		1442/1442 [==============================] - 2s 1ms/step - loss: 0.8314 - recall: 0.5134 - accuracy: 0.6420 - auc: 0.6808 - val_loss: 0.7916 - val_recall: 0.7778 - val_accuracy: 0.6400 - val_auc: 0.7545
		Epoch 7/10
		1442/1442 [==============================] - 2s 1ms/step - loss: 0.8250 - recall: 0.5332 - accuracy: 0.6476 - auc: 0.6884 - val_loss: 0.7852 - val_recall: 0.7778 - val_accuracy: 0.6400 - val_auc: 0.7564
		Epoch 8/10
		1442/1442 [==============================] - 2s 1ms/step - loss: 0.8220 - recall: 0.5391 - accuracy: 0.6515 - auc: 0.6896 - val_loss: 0.7795 - val_recall: 0.7333 - val_accuracy: 0.6500 - val_auc: 0.7638
		Epoch 9/10
		1442/1442 [==============================] - 2s 1ms/step - loss: 0.8208 - recall: 0.5297 - accuracy: 0.6516 - auc: 0.6894 - val_loss: 0.7787 - val_recall: 0.7333 - val_accuracy: 0.6500 - val_auc: 0.7679
		Epoch 10/10
		1442/1442 [==============================] - 2s 1ms/step - loss: 0.8202 - recall: 0.5267 - accuracy: 0.6523 - auc: 0.6898 - val_loss: 0.7801 - val_recall: 0.7333 - val_accuracy: 0.6300 - val_auc: 0.7707
		3603/3603 [==============================] - 4s 1ms/step - loss: 0.8199 - recall: 0.5304 - accuracy: 0.6516 - auc: 0.6903
		[0.8199049830436707, 0.5303574204444885, 0.6516139507293701, 0.6903280019760132]
		
		Model Training Accuracy: 65.16139507293701
		Model Training Loss: 0.8356703877449035
		Rounded Test Accuracy: 66.13032984714401
		Test Loss 0.7959033608436584
		AUC: 0.676917
		469 299 122 353
		Saved grid model to disk
		Metrics: adam mse
		Epoch 1/10
		1442/1442 [==============================] - 2s 1ms/step - loss: 0.2501 - recall: 0.9460 - accuracy: 0.5024 - auc: 0.4974 - val_loss: 0.2499 - val_recall: 0.0000e+00 - val_accuracy: 0.5500 - val_auc: 0.5000
		Epoch 2/10
		1442/1442 [==============================] - 2s 1ms/step - loss: 0.2499 - recall: 0.7777 - accuracy: 0.5090 - auc: 0.5103 - val_loss: 0.2499 - val_recall: 0.8000 - val_accuracy: 0.5500 - val_auc: 0.5111
		Epoch 3/10
		1442/1442 [==============================] - 2s 1ms/step - loss: 0.2495 - recall: 0.7763 - accuracy: 0.5239 - auc: 0.5559 - val_loss: 0.2503 - val_recall: 1.0000 - val_accuracy: 0.4500 - val_auc: 0.6630
		Epoch 4/10
		1442/1442 [==============================] - 2s 1ms/step - loss: 0.2461 - recall: 0.8071 - accuracy: 0.5670 - auc: 0.6385 - val_loss: 0.2425 - val_recall: 0.8000 - val_accuracy: 0.5300 - val_auc: 0.7115
		Epoch 5/10
		1442/1442 [==============================] - 2s 1ms/step - loss: 0.2345 - recall: 0.5906 - accuracy: 0.6341 - auc: 0.6706 - val_loss: 0.2320 - val_recall: 0.8000 - val_accuracy: 0.5800 - val_auc: 0.7453
		Epoch 6/10
		1442/1442 [==============================] - 2s 1ms/step - loss: 0.2241 - recall: 0.5326 - accuracy: 0.6486 - auc: 0.6848 - val_loss: 0.2175 - val_recall: 0.7556 - val_accuracy: 0.6400 - val_auc: 0.7596
		Epoch 7/10
		1442/1442 [==============================] - 2s 1ms/step - loss: 0.2202 - recall: 0.5079 - accuracy: 0.6518 - auc: 0.6880 - val_loss: 0.2110 - val_recall: 0.7111 - val_accuracy: 0.6700 - val_auc: 0.7701
		Epoch 8/10
		1442/1442 [==============================] - 2s 1ms/step - loss: 0.2193 - recall: 0.5002 - accuracy: 0.6528 - auc: 0.6884 - val_loss: 0.2058 - val_recall: 0.6889 - val_accuracy: 0.6800 - val_auc: 0.7764
		Epoch 9/10
		1442/1442 [==============================] - 2s 1ms/step - loss: 0.2191 - recall: 0.4920 - accuracy: 0.6531 - auc: 0.6879 - val_loss: 0.2045 - val_recall: 0.6889 - val_accuracy: 0.6800 - val_auc: 0.7747
		Epoch 10/10
		1442/1442 [==============================] - 2s 1ms/step - loss: 0.2191 - recall: 0.4931 - accuracy: 0.6537 - auc: 0.6881 - val_loss: 0.2032 - val_recall: 0.6889 - val_accuracy: 0.7000 - val_auc: 0.7770
		3603/3603 [==============================] - 4s 1ms/step - loss: 0.2190 - recall: 0.4886 - accuracy: 0.6547 - auc: 0.6890
		[0.21898488700389862, 0.4885917901992798, 0.6546670794487, 0.6889716386795044]
		
		Model Training Accuracy: 65.46670794487
		Model Training Loss: 0.2332095369696617
		Rounded Test Accuracy: 71.44006436041835
		Test Loss 0.22664959579706193
		AUC: 0.708240
		564 204 151 324
		Saved grid model to disk
		Metrics: nadam binary_crossentropy
		Epoch 1/10
		1442/1442 [==============================] - 2s 2ms/step - loss: 0.7036 - recall: 0.1757 - accuracy: 0.4979 - auc: 0.4993 - val_loss: 0.6924 - val_recall: 0.0000e+00 - val_accuracy: 0.5500 - val_auc: 0.5000
		Epoch 2/10
		1442/1442 [==============================] - 2s 1ms/step - loss: 0.6932 - recall: 0.7298 - accuracy: 0.5024 - auc: 0.4991 - val_loss: 0.6939 - val_recall: 1.0000 - val_accuracy: 0.4500 - val_auc: 0.5000
		Epoch 3/10
		1442/1442 [==============================] - 2s 1ms/step - loss: 0.6927 - recall: 0.6330 - accuracy: 0.5204 - auc: 0.5299 - val_loss: 0.6928 - val_recall: 1.0000 - val_accuracy: 0.4500 - val_auc: 0.5954
		Epoch 4/10
		1442/1442 [==============================] - 2s 1ms/step - loss: 0.6900 - recall: 0.5675 - accuracy: 0.5831 - auc: 0.6127 - val_loss: 0.6876 - val_recall: 0.8000 - val_accuracy: 0.6000 - val_auc: 0.6998
		Epoch 5/10
		1442/1442 [==============================] - 2s 1ms/step - loss: 0.6803 - recall: 0.5147 - accuracy: 0.6231 - auc: 0.6583 - val_loss: 0.6732 - val_recall: 0.7778 - val_accuracy: 0.6600 - val_auc: 0.7412
		Epoch 6/10
		1442/1442 [==============================] - 2s 1ms/step - loss: 0.6641 - recall: 0.5436 - accuracy: 0.6375 - auc: 0.6766 - val_loss: 0.6521 - val_recall: 0.7778 - val_accuracy: 0.6600 - val_auc: 0.7473
		Epoch 7/10
		1442/1442 [==============================] - 2s 1ms/step - loss: 0.6490 - recall: 0.5495 - accuracy: 0.6447 - auc: 0.6856 - val_loss: 0.6400 - val_recall: 0.7778 - val_accuracy: 0.6600 - val_auc: 0.7598
		Epoch 8/10
		1442/1442 [==============================] - 2s 1ms/step - loss: 0.6392 - recall: 0.5394 - accuracy: 0.6494 - auc: 0.6885 - val_loss: 0.6388 - val_recall: 0.7778 - val_accuracy: 0.6300 - val_auc: 0.7655
		Epoch 9/10
		1442/1442 [==============================] - 2s 1ms/step - loss: 0.6341 - recall: 0.5246 - accuracy: 0.6518 - auc: 0.6888 - val_loss: 0.6352 - val_recall: 0.7556 - val_accuracy: 0.6300 - val_auc: 0.7721
		Epoch 10/10
		1442/1442 [==============================] - 2s 1ms/step - loss: 0.6315 - recall: 0.5142 - accuracy: 0.6524 - auc: 0.6891 - val_loss: 0.6120 - val_recall: 0.6889 - val_accuracy: 0.6600 - val_auc: 0.7709
		3603/3603 [==============================] - 4s 1ms/step - loss: 0.6304 - recall: 0.5071 - accuracy: 0.6529 - auc: 0.6895
		[0.6304404735565186, 0.5070990324020386, 0.6529462337493896, 0.6895389556884766]
		
		Model Training Accuracy: 65.29462337493896
		Model Training Loss: 0.6677610874176025
		Rounded Test Accuracy: 67.8197908286404
		Test Loss 0.6617873728275299
		AUC: 0.682156
		511 257 143 332
		Saved grid model to disk
		Metrics: nadam poisson
		Epoch 1/10
		1442/1442 [==============================] - 2s 2ms/step - loss: 0.8504 - recall: 0.9106 - accuracy: 0.5015 - auc: 0.4987 - val_loss: 0.8119 - val_recall: 0.0889 - val_accuracy: 0.5900 - val_auc: 0.5000
		Epoch 2/10
		1442/1442 [==============================] - 2s 1ms/step - loss: 0.8482 - recall: 0.6785 - accuracy: 0.5020 - auc: 0.5025 - val_loss: 0.8122 - val_recall: 1.0000 - val_accuracy: 0.4500 - val_auc: 0.5000
		Epoch 3/10
		1442/1442 [==============================] - 2s 1ms/step - loss: 0.8480 - recall: 0.8325 - accuracy: 0.5149 - auc: 0.5262 - val_loss: 0.8113 - val_recall: 0.4444 - val_accuracy: 0.6900 - val_auc: 0.5275
		Epoch 4/10
		1442/1442 [==============================] - 2s 1ms/step - loss: 0.8472 - recall: 0.7624 - accuracy: 0.5607 - auc: 0.6006 - val_loss: 0.8109 - val_recall: 1.0000 - val_accuracy: 0.4600 - val_auc: 0.7271
		Epoch 5/10
		1442/1442 [==============================] - 2s 1ms/step - loss: 0.8441 - recall: 0.6852 - accuracy: 0.6031 - auc: 0.6538 - val_loss: 0.8070 - val_recall: 0.8222 - val_accuracy: 0.4800 - val_auc: 0.7352
		Epoch 6/10
		1442/1442 [==============================] - 2s 1ms/step - loss: 0.8374 - recall: 0.5672 - accuracy: 0.6433 - auc: 0.6779 - val_loss: 0.7917 - val_recall: 0.7111 - val_accuracy: 0.6800 - val_auc: 0.7519
		Epoch 7/10
		1442/1442 [==============================] - 2s 2ms/step - loss: 0.8287 - recall: 0.5117 - accuracy: 0.6493 - auc: 0.6849 - val_loss: 0.7830 - val_recall: 0.6889 - val_accuracy: 0.6400 - val_auc: 0.7566
		Epoch 8/10
		1442/1442 [==============================] - 2s 1ms/step - loss: 0.8230 - recall: 0.5024 - accuracy: 0.6514 - auc: 0.6872 - val_loss: 0.7738 - val_recall: 0.6889 - val_accuracy: 0.6800 - val_auc: 0.7634
		Epoch 9/10
		1442/1442 [==============================] - 2s 2ms/step - loss: 0.8206 - recall: 0.4986 - accuracy: 0.6531 - auc: 0.6882 - val_loss: 0.7720 - val_recall: 0.6889 - val_accuracy: 0.6800 - val_auc: 0.7715
		Epoch 10/10
		1442/1442 [==============================] - 2s 2ms/step - loss: 0.8198 - recall: 0.4963 - accuracy: 0.6536 - auc: 0.6889 - val_loss: 0.7679 - val_recall: 0.6889 - val_accuracy: 0.7000 - val_auc: 0.7774
		3603/3603 [==============================] - 4s 1ms/step - loss: 0.8196 - recall: 0.4827 - accuracy: 0.6554 - auc: 0.6895
		[0.8196445107460022, 0.4827357530593872, 0.6553887128829956, 0.6895128488540649]
		
		Model Training Accuracy: 65.53887128829956
		Model Training Loss: 0.8367379426956176
		Rounded Test Accuracy: 71.44006436041835
		Test Loss 0.7941579639911651
		AUC: 0.707035
		567 201 154 321
		Saved grid model to disk
		Metrics: nadam mse
		Epoch 1/10
		1442/1442 [==============================] - 2s 2ms/step - loss: 0.2504 - recall: 0.2228 - accuracy: 0.4965 - auc: 0.4938 - val_loss: 0.2502 - val_recall: 1.0000 - val_accuracy: 0.4500 - val_auc: 0.5000
		Epoch 2/10
		1442/1442 [==============================] - 2s 1ms/step - loss: 0.2500 - recall: 0.9082 - accuracy: 0.4994 - auc: 0.4958 - val_loss: 0.2500 - val_recall: 1.0000 - val_accuracy: 0.4500 - val_auc: 0.5000
		Epoch 3/10
		1442/1442 [==============================] - 2s 1ms/step - loss: 0.2499 - recall: 0.8549 - accuracy: 0.5136 - auc: 0.5240 - val_loss: 0.2495 - val_recall: 0.6889 - val_accuracy: 0.6900 - val_auc: 0.5519
		Epoch 4/10
		1442/1442 [==============================] - 2s 2ms/step - loss: 0.2479 - recall: 0.5130 - accuracy: 0.6111 - auc: 0.6355 - val_loss: 0.2470 - val_recall: 0.8222 - val_accuracy: 0.5000 - val_auc: 0.7234
		Epoch 5/10
		1442/1442 [==============================] - 2s 1ms/step - loss: 0.2394 - recall: 0.5297 - accuracy: 0.6392 - auc: 0.6711 - val_loss: 0.2373 - val_recall: 0.8000 - val_accuracy: 0.6000 - val_auc: 0.7491
		Epoch 6/10
		1442/1442 [==============================] - 2s 2ms/step - loss: 0.2292 - recall: 0.5336 - accuracy: 0.6490 - auc: 0.6845 - val_loss: 0.2307 - val_recall: 0.7778 - val_accuracy: 0.6200 - val_auc: 0.7628
		Epoch 7/10
		1442/1442 [==============================] - 2s 2ms/step - loss: 0.2237 - recall: 0.5261 - accuracy: 0.6523 - auc: 0.6885 - val_loss: 0.2165 - val_recall: 0.6889 - val_accuracy: 0.6300 - val_auc: 0.7683
		Epoch 8/10
		1442/1442 [==============================] - 2s 1ms/step - loss: 0.2214 - recall: 0.5090 - accuracy: 0.6531 - auc: 0.6875 - val_loss: 0.2149 - val_recall: 0.6889 - val_accuracy: 0.6600 - val_auc: 0.7756
		Epoch 9/10
		1442/1442 [==============================] - 2s 1ms/step - loss: 0.2203 - recall: 0.5011 - accuracy: 0.6537 - auc: 0.6881 - val_loss: 0.2083 - val_recall: 0.6889 - val_accuracy: 0.6800 - val_auc: 0.7798
		Epoch 10/10
		1442/1442 [==============================] - 2s 1ms/step - loss: 0.2197 - recall: 0.4975 - accuracy: 0.6548 - auc: 0.6881 - val_loss: 0.2046 - val_recall: 0.6889 - val_accuracy: 0.7000 - val_auc: 0.7788
		3603/3603 [==============================] - 4s 1ms/step - loss: 0.2195 - recall: 0.4872 - accuracy: 0.6561 - auc: 0.6896
		[0.21951515972614288, 0.48715540766716003, 0.6560548543930054, 0.6895577907562256]
		
		Model Training Accuracy: 65.60548543930054
		Model Training Loss: 0.23518260866403579
		Rounded Test Accuracy: 71.60096540627514
		Test Loss 0.23089875727891923
		AUC: 0.709944
		565 203 150 325
		Saved grid model to disk
		</pre>
		
		<h2>Results Analysis</h2>
		
		<p>Now that the models have completed running, we can analyze their results graphically. Each of the metric combinations is represented by a different color and line type combo, with the colors and linetypes remaining consistent through the following visualization.</p>
	
		<h3>Results - Accuracy Performance</h3>
		
		<p>Here, we can discern that the Nadam optimizer sets begin much lower in accuracy, but catch up to their fellows within one epoch run. The first metric set to begin increase toward the final score is the Adam/MSE pair, with the accuracy reaching 71.89 on the fifth epoch. By the sixth epoch, this metric set had reached its point of convergence. However, this set (72.36) was marginally overtaken by both Nadam/MSE (72.364) and Nadam/Binary Crossentropy (72.397) by the tenth epoch.</p>

		<div>
		<img src="https://github.com/pattersonconsulting/pattersonconsulting.github.io/raw/master/blog/images/Citations_TF_Way_Acc.png" class="center" width="700"/>
		</div>
		
		
		<p>Validation Accuracy proved to be a more chaotic affair, with all sets roughly beginning at 76 percent accuracy. The maximum validation accuracy was provided by the combination of RMSProp and Poisson, finishing at 77 percent where the second position fell to Adam/Poisson.</p>

		
		<div>
		<img src="https://github.com/pattersonconsulting/pattersonconsulting.github.io/raw/master/blog/images/Citations_TF_Way_Valid_Acc.png" class="center" width="700"/>
		</div>

		<p>Once testing of each combination was completed, the RMSProp/Poisson set was found to rank highest in Testing Accuracy, with a score of 78.44 percent. The next was Adam/Poisson, with 78.042.</p>

		
		<div>
		<img src="https://github.com/pattersonconsulting/pattersonconsulting.github.io/raw/master/blog/images/Citations_TF_Way_Test_Acc.png" class="center" width="700"/>
		</div>

		
		<h3>Results - Recall Performance</h3>
		
		<p>Recall is the measure of how many records were correctly predicted positive, or the value found by</p>
		<p>$\frac{True Positive}{(True Positive + False Negative)}$</p>
		<p>Recall of many of the metric sets begins at zero, with all predictions 'No Citation'. However, Nadam/MSE, RMSProp/MSE, and Nadam/Poisson all begin at higher values before falling to zero. While many of the sets began predicting citations on Epoch 4, their performance was most certainly topped by the Adam/MSE set. Note that this set continued to outperform the rest until Epoch 7, where many sets caught up in performance. Epoch 7 is also when the Nadam/Binary Crossentropy set began outperforming, with a Recall of 40.87 percent. This set finished training at 41.67 percent Recall.</p>

		
		<div>
		<img src="https://github.com/pattersonconsulting/pattersonconsulting.github.io/raw/master/blog/images/Citations_TF_Way_Recall.png" class="center" width="700"/>
		</div>

		<p>Most of the sets performed similarly on validation, with only variety in when higher Recall values were achieved. again, Nadam/MSE was the first to approach max Recall, upon Epoch 4, and RMSProp/MSE was the last with Epochs 6/7. All sets finished validation at 58.33 percent Recall.</p>
		
		<div>
		<img src="https://github.com/pattersonconsulting/pattersonconsulting.github.io/raw/master/blog/images/Citations_TF_Way_Valid_Recall.png" class="center" width="700"/>
		</div>

		<p>Recall upon testing of the various sets presented the Adam/Binary Crossentropy, RMSProp/Binary crossentropy, and RMSProp/MSE sets all performing well, all with Recalls of 64.84. The least performing was RMSProp/Poisson, at 62.94.</p>
		
		<div>
		<img src="https://github.com/pattersonconsulting/pattersonconsulting.github.io/raw/master/blog/images/Citations_TF_Way_Test_Recall.png" class="center" width="700"/>
		</div>

		<h3>Creating a prediction set</h3>
		
		<p>Now, we'll be importing a dataset of all citations from the month of April in 2020, and will be suppling them to the model for predictions. Of the 12 citations issued, 8 were correctly identified by the model. The Recall in this case would be roughly 66.7%, somewhat higher than the previously testing recall. Also of note is that even records not correctly identified were above 42.41% probability to see a citation.</p>

		<script charset="UTF-8" src="http://gist-it.appspot.com/github.com/PeteWay/Consulting/blob/master/Citations.py?slice=557:578&footer=minimal"></script>

		<pre>
		[&#39;Year&#39; &#39;Month&#39; &#39;WeekDay&#39; &#39;Hour&#39; &#39;Segment&#39; &#39;Road_Num&#39;]
		Min probability of citation:  42.41
		Max probability of citation:  70.02
		Correctly predicted:  8
		</pre>
	
		<p>However, testing the performance of a model upon only positive data can skew result reporting. Therefore, we must test the model with a given fixed temporal set. In this case, negative samples were specifically selected for Thursdays in April 2020, at 4PM. This means four of the column values within the negatives are set, with only roadway data fluctuating. Then, the citations for that specific timeframe are added and same process is followed as above, passing the data into the predict function and finding the probability of an accident at each of the roadway segments.</p>
		<p>Given that the issuing of citations depends strongly on many other factors not being considered in this brief example, it is understandable that the model created only accurately identified 28 records. Of course, this is considering a loose defined fifty/fifty probability. In many real-world use cases, a fifty/fifty probability would not necessarily be considered in favor of 'higher certainty' scores like 75 or even 90 % probability.</p>
		
		<script charset="UTF-8" src="http://gist-it.appspot.com/github.com/PeteWay/Consulting/blob/master/Citations.py?slice=587:611&footer=minimal"></script>

		<pre>
		Number of records: 4943
		[&#39;Segment&#39; &#39;Road_Num&#39; &#39;Month&#39; &#39;WeekDay&#39; &#39;Year&#39; &#39;Hour&#39;]
		Min probability of citation:  40.72
		Max probability of citation:  66.17
		Correctly predicted:  28
		Citations predicted:  4917.0
		Citations existed:  2
		</pre>

		<h3>Analyzing Results Spatially</h3>
		
		<p>Finally, we can take a look at the specific areas that the model was most certain a citation would occur. The below graphic presents the fifty roadway segments that the model was most certain that a citation would occur. Note that the window of probability presented here is quite small.</p>
		<p>Also of interest is the number of shorter length segments that earned the highest probability. Only two segments of a considerable length were included in the top fifty, with a handful of other segments having longer than a few blocks length.</p>
		<p>Finally, of the entire Chattanooga area the fifty segments with the highest probability of a citation occuring are all located in a small area of the city. Particuarly, the area between 85.325&deg;W and 85.3&deg;W.</p>

		<div>
		<img src="https://github.com/pattersonconsulting/pattersonconsulting.github.io/raw/master/blog/images/Citations_TF_Way_FiftyHighestCitationProbability.png" class="center" width="900"/>
		</div>


	<hr>
		<h2>Summary</h2>
		
		<p>In this example, we've walked through the preprocessing of data, creation of spatial and temporal variables based off of existing variables, 
		and concluded with the creation of multiple Multilayer Perceptron models, made to predict which sections of local roadways would be likely to see 
		speeding citations in 2020 with a variety of metrics. We've also discussed the results of those models, examining which performed best in both Accuracy 
		and Recall, and put the model to the test with a smaller dataset representing April 2020.</p>
		<p>
			In our next post in this series, we'll look how to load this notebook up on Kubeflow 1.0 and accelerate training in the notebook with GPUs.


		</p>

		<br/>
		<br/>
		<br/>
		<br/>
		<br/>
		

					</div>

					<div class="col-md-3" id="fh5co-content" style="margin-bottom: 30px;">
						<a href="http://shop.oreilly.com/product/0636920260929.do">
							<img src="../images/kfops_color_cover.png" style="width: 212px; height: 280px; border: 1px solid;" >
						</a>
					</div>


					<br/>
		<br/>
		<br/>
		<br/>
		
				</div>
				<!-- end of section -->


			</div>
		</div>


	<div class="gototop js-top">
		<a href="#" class="js-gotop"><i class="icon-chevron-down"></i></a>
	</div>
	
	<script src="../js/jquery.min.js"></script>
	<script src="../js/jquery.easing.1.3.js"></script>
	<script src="../js/bootstrap.min.js"></script>
	<script src="../js/owl.carousel.min.js"></script>
	<script src="../js/main.js"></script>

	</body>
</html>