Permalink
Find file Copy path
Fetching contributors…
Cannot retrieve contributors at this time
1 lines (1 sloc) 46.5 KB
{"paragraphs":[{"text":"%md\n### Note\n\nPlease view the [README](https://github.com/deeplearning4j/dl4j-examples/blob/master/tutorials/README.md) to learn about installing, setting up dependencies, and importing notebooks in Zeppelin","dateUpdated":"2018-03-06T20:18:47+0000","config":{"tableHide":false,"editorSetting":{"language":"markdown","editOnDblClick":true},"colWidth":12,"editorMode":"ace/mode/markdown","fontSize":9,"editorHide":true,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"HTML","data":"<h3>Note</h3>\n<p>Please view the <a href=\"https://github.com/deeplearning4j/dl4j-examples/blob/master/tutorials/README.md\">README</a> to learn about installing, setting up dependencies, and importing notebooks in Zeppelin</p>\n"}]},"apps":[],"jobName":"paragraph_1520367527555_1578531571","id":"20180115-062124_1656190287","dateCreated":"2018-03-06T20:18:47+0000","status":"READY","errorMessage":"","progressUpdateIntervalMs":500,"focus":true,"$$hashKey":"object:4662"},{"text":"%md\n### Background","dateUpdated":"2018-03-06T20:18:47+0000","config":{"tableHide":false,"editorSetting":{"language":"markdown","editOnDblClick":true},"colWidth":12,"editorMode":"ace/mode/markdown","fontSize":9,"editorHide":true,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"HTML","data":"<h3>Background</h3>\n"}]},"apps":[],"jobName":"paragraph_1520367527556_1576607827","id":"20180116-040712_1768508310","dateCreated":"2018-03-06T20:18:47+0000","status":"READY","errorMessage":"","progressUpdateIntervalMs":500,"$$hashKey":"object:4663"},{"text":"%md\nIn this tutorial, we will learn how to apply a long-short term memory (LSTM) neural network to a medical time series problem. The data used comes from 4000 intensive care unit (ICU) patients and the goal is to predict the mortality of patients using 6 general descriptor features, such as age, gender, and weight along with 37 sequential features, such as cholesterol level, temperature, pH, and glucose level. Each patient has multiple measurements of the sequential features, with patients having a different amount of measurements taken. Furthermore, the time between measurements also differ among patients as well. \n\nA LSTM is well suited for this type of problem due to the sequential nature of the data. In addition, LSTM networks avoid vanishing and exploding gradients and are able to effectively capture long term dependencies due to its cell state, a feature not present in typical recurrent networks. For a more in depth explanation of LSTM's, see https://deeplearning4j.org/lstm.html.","dateUpdated":"2018-03-06T20:18:47+0000","config":{"tableHide":false,"editorSetting":{"language":"markdown","editOnDblClick":true},"colWidth":12,"editorMode":"ace/mode/markdown","fontSize":9,"editorHide":true,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"HTML","data":"<p>In this tutorial, we will learn how to apply a long-short term memory (LSTM) neural network to a medical time series problem. The data used comes from 4000 intensive care unit (ICU) patients and the goal is to predict the mortality of patients using 6 general descriptor features, such as age, gender, and weight along with 37 sequential features, such as cholesterol level, temperature, pH, and glucose level. Each patient has multiple measurements of the sequential features, with patients having a different amount of measurements taken. Furthermore, the time between measurements also differ among patients as well.</p>\n<p>A LSTM is well suited for this type of problem due to the sequential nature of the data. In addition, LSTM networks avoid vanishing and exploding gradients and are able to effectively capture long term dependencies due to its cell state, a feature not present in typical recurrent networks. For a more in depth explanation of LSTM's, see https://deeplearning4j.org/lstm.html.</p>\n"}]},"apps":[],"jobName":"paragraph_1520367527556_1576607827","id":"20180116-040741_1071141793","dateCreated":"2018-03-06T20:18:47+0000","status":"READY","errorMessage":"","progressUpdateIntervalMs":500,"$$hashKey":"object:4664"},{"text":"%md\n### Imports","dateUpdated":"2018-03-06T20:18:47+0000","config":{"tableHide":false,"editorSetting":{"language":"markdown","editOnDblClick":true},"colWidth":12,"editorMode":"ace/mode/markdown","fontSize":9,"editorHide":true,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"HTML","data":"<h3>Imports</h3>\n"}]},"apps":[],"jobName":"paragraph_1520367527556_1576607827","id":"20180115-062155_892306848","dateCreated":"2018-03-06T20:18:47+0000","status":"READY","errorMessage":"","progressUpdateIntervalMs":500,"$$hashKey":"object:4665"},{"text":"import io.skymind.zeppelin.utils._\nimport io.skymind.modelproviders.history.client.ModelHistoryClient\nimport io.skymind.modelproviders.history.model._\nimport org.datavec.api.records.reader.SequenceRecordReader;\nimport org.datavec.api.records.reader.impl.csv.CSVSequenceRecordReader;\nimport org.datavec.api.split.NumberedFileInputSplit;\nimport org.deeplearning4j.datasets.datavec.SequenceRecordReaderDataSetIterator;\nimport org.deeplearning4j.eval.ROC;\nimport org.deeplearning4j.nn.api.OptimizationAlgorithm;\nimport org.deeplearning4j.nn.conf.ComputationGraphConfiguration;\nimport org.deeplearning4j.nn.conf.NeuralNetConfiguration;\nimport org.deeplearning4j.nn.conf.Updater;\nimport org.deeplearning4j.nn.conf.layers.GravesLSTM;\nimport org.deeplearning4j.nn.conf.layers.RnnOutputLayer;\nimport org.deeplearning4j.nn.graph.ComputationGraph;\nimport org.deeplearning4j.optimize.listeners.ScoreIterationListener;\nimport org.deeplearning4j.nn.conf.graph.rnn.LastTimeStepVertex;\nimport org.deeplearning4j.nn.conf.layers.OutputLayer;\nimport org.nd4j.linalg.api.ndarray.INDArray;\nimport org.deeplearning4j.nn.weights.WeightInit;\nimport org.nd4j.linalg.activations.Activation;\nimport org.nd4j.linalg.dataset.api.DataSet;\nimport org.nd4j.linalg.lossfunctions.LossFunctions;\nimport org.nd4j.linalg.dataset.api.iterator.DataSetIterator;\nimport org.nd4j.linalg.dataset.api.DataSetPreProcessor;\nimport org.nd4j.linalg.factory.Nd4j;\nimport org.nd4j.linalg.indexing.BooleanIndexing;\nimport org.nd4j.linalg.indexing.NDArrayIndex;\nimport org.nd4j.linalg.indexing.conditions.Conditions;\nimport org.nd4j.linalg.primitives.Pair;\nimport org.slf4j.Logger;\nimport org.slf4j.LoggerFactory;\n\nimport java.io.File;\nimport org.apache.commons.io.FileUtils;\nimport org.apache.commons.io.FilenameUtils;\nimport java.io.IOException;\nimport java.util.HashMap;\nimport java.util.Arrays;\nimport java.net.URL;\nimport java.io.BufferedInputStream;\nimport java.io.FileInputStream;\nimport java.io.BufferedOutputStream;\nimport java.io.FileOutputStream;\nimport java.lang.Byte;\n\nimport org.apache.commons.compress.archivers.tar.TarArchiveEntry;\nimport org.apache.commons.compress.archivers.tar.TarArchiveInputStream;\nimport org.apache.commons.compress.compressors.gzip.GzipCompressorInputStream;\n\n\n","user":"admin","dateUpdated":"2018-03-06T20:34:56+0000","config":{"tableHide":false,"editorSetting":{"language":"scala","editOnDblClick":false},"colWidth":12,"editorMode":"ace/mode/scala","fontSize":9,"editorHide":false,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"TEXT","data":"import io.skymind.zeppelin.utils._\nimport io.skymind.modelproviders.history.client.ModelHistoryClient\nimport io.skymind.modelproviders.history.model._\nimport org.datavec.api.records.reader.SequenceRecordReader\nimport org.datavec.api.records.reader.impl.csv.CSVSequenceRecordReader\nimport org.datavec.api.split.NumberedFileInputSplit\nimport org.deeplearning4j.datasets.datavec.SequenceRecordReaderDataSetIterator\nimport org.deeplearning4j.eval.ROC\nimport org.deeplearning4j.nn.api.OptimizationAlgorithm\nimport org.deeplearning4j.nn.conf.ComputationGraphConfiguration\nimport org.deeplearning4j.nn.conf.NeuralNetConfiguration\nimport org.deeplearning4j.nn.conf.Updater\nimport org.deeplearning4j.nn.conf.layers.GravesLSTM\nimport org.deeplearning4j.nn.conf.layers.RnnOutputLayer\nimport org.deeplearning4j.nn.graph.ComputationGraph\nimport org.deeplearning4j.optimize.listeners.ScoreIterationListener\nimport org.deeplearning4j.nn.conf.graph.rnn.LastTimeStepVertex\nimport org.deeplearning4j.nn.conf.layers.OutputLayer\nimport org.nd4j.linalg.api.ndarray.INDArray\nimport org.deeplearning4j.nn.weights.WeightInit\nimport org.nd4j.linalg.activations.Activation\nimport org.nd4j.linalg.dataset.api.DataSet\nimport org.nd4j.linalg.lossfunctions.LossFunctions\nimport org.nd4j.linalg.dataset.api.iterator.DataSetIterator\nimport org.nd4j.linalg.dataset.api.DataSetPreProcessor\nimport org.nd4j.linalg.factory.Nd4j\nimport org.nd4j.linalg.indexing.BooleanIndexing\nimport org.nd4j.linalg.indexing.NDArrayIndex\nimport org.nd4j.linalg.indexing.conditions.Conditions\nimport org.nd4j.linalg.primitives.Pair\nimport org.slf4j.Logger\nimport org.slf4j.LoggerFactory\nimport java.io.File\nimport org.apache.commons.io.FileUtils\nimport org.apache.commons.io.FilenameUtils\nimport java.io.IOException\nimport java.util.HashMap\nimport java.util.Arrays\nimport java.net.URL\nimport java.io.BufferedInputStream\nimport java.io.FileInputStream\nimport java.io.BufferedOutputStream\nimport java.io.FileOutputStream\nimport java.lang.Byte\nimport org.apache.commons.compress.archivers.tar.TarArchiveEntry\nimport org.apache.commons.compress.archivers.tar.TarArchiveInputStream\nimport org.apache.commons.compress.compressors.gzip.GzipCompressorInputStream\n"}]},"apps":[],"jobName":"paragraph_1520367527557_1576223078","id":"20180115-062209_1381660770","dateCreated":"2018-03-06T20:18:47+0000","dateStarted":"2018-03-06T20:34:56+0000","dateFinished":"2018-03-06T20:35:01+0000","status":"FINISHED","progressUpdateIntervalMs":500,"$$hashKey":"object:4666"},{"text":"%md\n\nNow that we have imported everything needed to run this tutorial, we will start with obtaining the data and then converting the data into a format a neural network can understand. ","dateUpdated":"2018-03-06T20:18:47+0000","config":{"tableHide":false,"editorSetting":{"language":"markdown","editOnDblClick":true},"colWidth":12,"editorMode":"ace/mode/markdown","fontSize":9,"editorHide":true,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"HTML","data":"<p>Now that we have imported everything needed to run this tutorial, we will start with obtaining the data and then converting the data into a format a neural network can understand.</p>\n"}]},"apps":[],"jobName":"paragraph_1520367527557_1576223078","id":"20180116-041926_21274632","dateCreated":"2018-03-06T20:18:47+0000","status":"READY","errorMessage":"","progressUpdateIntervalMs":500,"$$hashKey":"object:4667"},{"text":"%md\n### Data Source","dateUpdated":"2018-03-06T20:18:47+0000","config":{"tableHide":false,"editorSetting":{"language":"markdown","editOnDblClick":true},"colWidth":12,"editorMode":"ace/mode/markdown","fontSize":9,"editorHide":true,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"HTML","data":"<h3>Data Source</h3>\n"}]},"apps":[],"jobName":"paragraph_1520367527557_1576223078","id":"20180115-063620_1089188478","dateCreated":"2018-03-06T20:18:47+0000","status":"READY","errorMessage":"","progressUpdateIntervalMs":500,"$$hashKey":"object:4668"},{"text":"%md\nThe data is contained in a compressed tar.gz file. We will have to download the data from the url below and then extract csv files containing the ICU data. Each patient will have a separate csv file for the features and labels. The features will be contained in a directory called sequence and the labels will be contained in a directory called mortality. The features are contained in a single csv file with the columns representing the features and the rows representing different time steps. The labels are contained in a single csv file which contains a value of 0 indicating death and a value of 1 indicating survival.","dateUpdated":"2018-03-06T20:18:47+0000","config":{"tableHide":false,"editorSetting":{"language":"markdown","editOnDblClick":true},"colWidth":12,"editorMode":"ace/mode/markdown","fontSize":9,"editorHide":true,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"HTML","data":"<p>The data is contained in a compressed tar.gz file. We will have to download the data from the url below and then extract csv files containing the ICU data. Each patient will have a separate csv file for the features and labels. The features will be contained in a directory called sequence and the labels will be contained in a directory called mortality. The features are contained in a single csv file with the columns representing the features and the rows representing different time steps. The labels are contained in a single csv file which contains a value of 0 indicating death and a value of 1 indicating survival.</p>\n"}]},"apps":[],"jobName":"paragraph_1520367527557_1576223078","id":"20180116-042022_82417057","dateCreated":"2018-03-06T20:18:47+0000","status":"READY","errorMessage":"","progressUpdateIntervalMs":500,"$$hashKey":"object:4669"},{"text":"val DATA_URL = \"https://skymindacademy.blob.core.windows.net/physionet2012/physionet2012.tar.gz\"\nval DATA_PATH = FilenameUtils.concat(System.getProperty(\"java.io.tmpdir\"), \"dl4j_physionet/\")","user":"admin","dateUpdated":"2018-03-06T20:35:00+0000","config":{"tableHide":false,"editorSetting":{"language":"scala","editOnDblClick":false},"colWidth":12,"editorMode":"ace/mode/scala","fontSize":9,"editorHide":false,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"TEXT","data":"DATA_URL: String = https://skymindacademy.blob.core.windows.net/physionet2012/physionet2012.tar.gz\nDATA_PATH: String = /tmp/dl4j_physionet/\n"}]},"apps":[],"jobName":"paragraph_1520367527557_1576223078","id":"20180115-063627_2071788954","dateCreated":"2018-03-06T20:18:47+0000","dateStarted":"2018-03-06T20:35:00+0000","dateFinished":"2018-03-06T20:35:01+0000","status":"FINISHED","progressUpdateIntervalMs":500,"$$hashKey":"object:4670"},{"text":"%md\n### Download Data","dateUpdated":"2018-03-06T20:18:47+0000","config":{"tableHide":false,"editorSetting":{"language":"markdown","editOnDblClick":true},"colWidth":12,"editorMode":"ace/mode/markdown","fontSize":9,"editorHide":true,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"HTML","data":"<h3>Download Data</h3>\n"}]},"apps":[],"jobName":"paragraph_1520367527558_1577377325","id":"20180115-063817_834797484","dateCreated":"2018-03-06T20:18:47+0000","status":"READY","errorMessage":"","progressUpdateIntervalMs":500,"$$hashKey":"object:4671"},{"text":"%md\nTo download the data, we will create a temporary directory that will store the data files, extract the tar.gz file from the url, and place it in the specified directory.","dateUpdated":"2018-03-06T20:18:47+0000","config":{"tableHide":false,"editorSetting":{"language":"markdown","editOnDblClick":true},"colWidth":12,"editorMode":"ace/mode/markdown","fontSize":9,"editorHide":true,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"HTML","data":"<p>To download the data, we will create a temporary directory that will store the data files, extract the tar.gz file from the url, and place it in the specified directory.</p>\n"}]},"apps":[],"jobName":"paragraph_1520367527558_1577377325","id":"20180116-042108_1540744847","dateCreated":"2018-03-06T20:18:47+0000","status":"READY","errorMessage":"","progressUpdateIntervalMs":500,"$$hashKey":"object:4672"},{"text":"val directory = new File(DATA_PATH)\ndirectory.mkdir() // create new directory at specified path\n\nval archizePath = DATA_PATH + \"physionet2012.tar.gz\" // set path for tar.gz file\nval archiveFile = new File(archizePath) // create tar.gz file\nval extractedPath = DATA_PATH + \"physionet2012\" \nval extractedFile = new File(extractedPath)\n\nFileUtils.copyURLToFile(new URL(DATA_URL), archiveFile) // copy data from URL to file","user":"admin","dateUpdated":"2018-03-06T20:35:02+0000","config":{"tableHide":false,"editorSetting":{"language":"scala","editOnDblClick":false},"colWidth":12,"editorMode":"ace/mode/scala","fontSize":9,"editorHide":false,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"TEXT","data":"directory: java.io.File = /tmp/dl4j_physionet\nres49: Boolean = false\narchizePath: String = /tmp/dl4j_physionet/physionet2012.tar.gz\narchiveFile: java.io.File = /tmp/dl4j_physionet/physionet2012.tar.gz\nextractedPath: String = /tmp/dl4j_physionet/physionet2012\nextractedFile: java.io.File = /tmp/dl4j_physionet/physionet2012\n"}]},"apps":[],"jobName":"paragraph_1520367527558_1577377325","id":"20180115-063829_1644657611","dateCreated":"2018-03-06T20:18:47+0000","dateStarted":"2018-03-06T20:35:02+0000","dateFinished":"2018-03-06T20:35:05+0000","status":"FINISHED","progressUpdateIntervalMs":500,"$$hashKey":"object:4673"},{"text":"%md \n\nNext, we must extract the data from the tar.gz file, recreate directories within the tar.gz file into our temporary directory, and copy the files into our temporary directory. ","dateUpdated":"2018-03-06T20:18:47+0000","config":{"tableHide":false,"editorSetting":{"language":"markdown","editOnDblClick":true},"colWidth":12,"editorMode":"ace/mode/markdown","fontSize":9,"editorHide":true,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"HTML","data":"<p>Next, we must extract the data from the tar.gz file, recreate directories within the tar.gz file into our temporary directory, and copy the files into our temporary directory.</p>\n"}]},"apps":[],"jobName":"paragraph_1520367527558_1577377325","id":"20180115-064020_988667885","dateCreated":"2018-03-06T20:18:47+0000","status":"READY","errorMessage":"","progressUpdateIntervalMs":500,"$$hashKey":"object:4674"},{"text":"var fileCount = 0\nvar dirCount = 0\nval BUFFER_SIZE = 4096\n\nval tais = new TarArchiveInputStream(new GzipCompressorInputStream( new BufferedInputStream( new FileInputStream(archizePath))))\n\nvar entry = tais.getNextEntry().asInstanceOf[TarArchiveEntry]\n\nwhile(entry != null){\n if (entry.isDirectory()) {\n new File(DATA_PATH + entry.getName()).mkdirs()\n dirCount = dirCount + 1\n fileCount = 0\n }\n else {\n \n val data = new Array[scala.Byte](4 * BUFFER_SIZE)\n\n val fos = new FileOutputStream(DATA_PATH + entry.getName());\n val dest = new BufferedOutputStream(fos, BUFFER_SIZE);\n var count = tais.read(data, 0, BUFFER_SIZE)\n \n while (count != -1) {\n dest.write(data, 0, count)\n count = tais.read(data, 0, BUFFER_SIZE)\n }\n \n dest.close()\n fileCount = fileCount + 1\n }\n if(fileCount % 1000 == 0){\n print(\".\")\n }\n \n entry = tais.getNextEntry().asInstanceOf[TarArchiveEntry]\n}","user":"admin","dateUpdated":"2018-03-06T20:35:04+0000","config":{"tableHide":false,"editorSetting":{"language":"scala","editOnDblClick":false},"colWidth":12,"editorMode":"ace/mode/scala","fontSize":9,"editorHide":false,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"TEXT","data":"fileCount: Int = 0\ndirCount: Int = 0\nBUFFER_SIZE: Int = 4096\ntais: org.apache.commons.compress.archivers.tar.TarArchiveInputStream = org.apache.commons.compress.archivers.tar.TarArchiveInputStream@685a202f\nentry: org.apache.commons.compress.archivers.tar.TarArchiveEntry = org.apache.commons.compress.archivers.tar.TarArchiveEntry@221cf6b\n........................................."}]},"apps":[],"jobName":"paragraph_1520367527558_1577377325","id":"20180115-070134_320419844","dateCreated":"2018-03-06T20:18:47+0000","dateStarted":"2018-03-06T20:35:04+0000","dateFinished":"2018-03-06T20:35:47+0000","status":"FINISHED","progressUpdateIntervalMs":500,"$$hashKey":"object:4675"},{"text":"%md\n### DataSetIterators","dateUpdated":"2018-03-06T20:18:47+0000","config":{"tableHide":false,"editorSetting":{"language":"markdown","editOnDblClick":true},"colWidth":12,"editorMode":"ace/mode/markdown","fontSize":9,"editorHide":true,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"HTML","data":"<h3>DataSetIterators</h3>\n"}]},"apps":[],"jobName":"paragraph_1520367527558_1577377325","id":"20180115-070220_1700945870","dateCreated":"2018-03-06T20:18:47+0000","status":"READY","errorMessage":"","progressUpdateIntervalMs":500,"$$hashKey":"object:4676"},{"text":"%md\nOur next goal is to convert the raw data (csv files) into a DataSetIterator, which can then be fed into a neural network for training. Our training data will have 3200 examples which will be represented by a single DataSetIterator, and the testing data will have 800 examples which will be represented by a separate DataSet Iterator.","dateUpdated":"2018-03-06T20:18:47+0000","config":{"tableHide":false,"editorSetting":{"language":"markdown","editOnDblClick":true},"colWidth":12,"editorMode":"ace/mode/markdown","fontSize":9,"editorHide":true,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"HTML","data":"<p>Our next goal is to convert the raw data (csv files) into a DataSetIterator, which can then be fed into a neural network for training. Our training data will have 3200 examples which will be represented by a single DataSetIterator, and the testing data will have 800 examples which will be represented by a separate DataSet Iterator.</p>\n"}]},"apps":[],"jobName":"paragraph_1520367527559_1576992576","id":"20180116-043157_768466617","dateCreated":"2018-03-06T20:18:47+0000","status":"READY","errorMessage":"","progressUpdateIntervalMs":500,"$$hashKey":"object:4677"},{"text":"val NB_TRAIN_EXAMPLES = 2000 // number of training examples\nval NB_TEST_EXAMPLES = 800 // number of testing examples","user":"admin","dateUpdated":"2018-03-06T20:35:07+0000","config":{"tableHide":false,"editorSetting":{"language":"scala","editOnDblClick":false},"colWidth":12,"editorMode":"ace/mode/scala","fontSize":9,"editorHide":false,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"TEXT","data":"NB_TRAIN_EXAMPLES: Int = 2000\nNB_TEST_EXAMPLES: Int = 800\n"}]},"apps":[],"jobName":"paragraph_1520367527559_1576992576","id":"20180115-071757_4986863","dateCreated":"2018-03-06T20:18:47+0000","dateStarted":"2018-03-06T20:35:07+0000","dateFinished":"2018-03-06T20:35:47+0000","status":"FINISHED","progressUpdateIntervalMs":500,"$$hashKey":"object:4678"},{"text":"%md\nIn order to obtain DataSetIterators, we must first initialize CSVSequenceRecordReaders, which will parse the raw data into record-like format. We will first set the directories for the features and labels and initialize the CSVSequenceRecordReaders.\n\nNext, we can initialize the SequenceRecordReaderDataSetIterator using the previously created CSVSequenceRecordReaders. We will use an alignment mode of ALIGN_END. This alignment mode is needed due to the fact that the number of time steps differs between different patients. Because the mortality label is always at the end of the sequence, we need all the sequences aligned so that the time step with the mortality label is the last time step for all patients. For a more in depth explanation of alignment modes, see https://deeplearning4j.org/usingrnns. ","dateUpdated":"2018-03-06T20:18:47+0000","config":{"tableHide":false,"editorSetting":{"language":"markdown","editOnDblClick":true},"colWidth":12,"editorMode":"ace/mode/markdown","fontSize":9,"editorHide":true,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"HTML","data":"<p>In order to obtain DataSetIterators, we must first initialize CSVSequenceRecordReaders, which will parse the raw data into record-like format. We will first set the directories for the features and labels and initialize the CSVSequenceRecordReaders.</p>\n<p>Next, we can initialize the SequenceRecordReaderDataSetIterator using the previously created CSVSequenceRecordReaders. We will use an alignment mode of ALIGN_END. This alignment mode is needed due to the fact that the number of time steps differs between different patients. Because the mortality label is always at the end of the sequence, we need all the sequences aligned so that the time step with the mortality label is the last time step for all patients. For a more in depth explanation of alignment modes, see https://deeplearning4j.org/usingrnns.</p>\n"}]},"apps":[],"jobName":"paragraph_1520367527559_1576992576","id":"20180115-070711_769734985","dateCreated":"2018-03-06T20:18:47+0000","status":"READY","errorMessage":"","progressUpdateIntervalMs":500,"$$hashKey":"object:4679"},{"text":"val path = FilenameUtils.concat(DATA_PATH, \"physionet2012/\") // set parent directory\n\nval featureBaseDir = FilenameUtils.concat(path, \"sequence\") // set feature directory\nval mortalityBaseDir = FilenameUtils.concat(path, \"mortality\") // set label directory\n\n// Load training data\n\nval trainFeatures = new CSVSequenceRecordReader(1, \",\")\ntrainFeatures.initialize( new NumberedFileInputSplit(featureBaseDir + \"/%d.csv\", 0, NB_TRAIN_EXAMPLES - 1))\n\nval trainLabels = new CSVSequenceRecordReader()\ntrainLabels.initialize(new NumberedFileInputSplit(mortalityBaseDir + \"/%d.csv\", 0, NB_TRAIN_EXAMPLES - 1))\n\nval trainData = new SequenceRecordReaderDataSetIterator(trainFeatures, trainLabels,\n 1, 2, false, SequenceRecordReaderDataSetIterator.AlignmentMode.ALIGN_END)\n\n \n// Load testing data\nval testFeatures = new CSVSequenceRecordReader(1, \",\");\ntestFeatures.initialize(new NumberedFileInputSplit(featureBaseDir + \"/%d.csv\", NB_TRAIN_EXAMPLES, NB_TRAIN_EXAMPLES + 50));\n \nval testLabels = new CSVSequenceRecordReader();\ntestLabels.initialize(new NumberedFileInputSplit(mortalityBaseDir + \"/%d.csv\", NB_TRAIN_EXAMPLES, NB_TRAIN_EXAMPLES + 50));\n\nval testData = new SequenceRecordReaderDataSetIterator(testFeatures, testLabels,\n 1, 2, false, SequenceRecordReaderDataSetIterator.AlignmentMode.ALIGN_END)\n","user":"admin","dateUpdated":"2018-03-06T20:35:09+0000","config":{"tableHide":false,"editorSetting":{"language":"scala","editOnDblClick":false},"colWidth":12,"editorMode":"ace/mode/scala","fontSize":9,"editorHide":false,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"TEXT","data":"path: String = /tmp/dl4j_physionet/physionet2012/\nfeatureBaseDir: String = /tmp/dl4j_physionet/physionet2012/sequence\nmortalityBaseDir: String = /tmp/dl4j_physionet/physionet2012/mortality\ntrainFeatures: org.datavec.api.records.reader.impl.csv.CSVSequenceRecordReader = org.datavec.api.records.reader.impl.csv.CSVSequenceRecordReader@ab1017\ntrainLabels: org.datavec.api.records.reader.impl.csv.CSVSequenceRecordReader = org.datavec.api.records.reader.impl.csv.CSVSequenceRecordReader@5e7cb566\ntrainData: org.deeplearning4j.datasets.datavec.SequenceRecordReaderDataSetIterator = org.deeplearning4j.datasets.datavec.SequenceRecordReaderDataSetIterator@17e69dca\ntestFeatures: org.datavec.api.records.reader.impl.csv.CSVSequenceRecordReader = org.datavec.api.records.reader.impl.csv.CSVSequenceRecordReader@47cfdf34\ntestLabels: org.datavec.api.records.reader.impl.csv.CSVSequenceRecordReader = org.datavec.api.records.reader.impl.csv.CSVSequenceRecordReader@7d0e265f\ntestData: org.deeplearning4j.datasets.datavec.SequenceRecordReaderDataSetIterator = org.deeplearning4j.datasets.datavec.SequenceRecordReaderDataSetIterator@58693049\n"}]},"apps":[],"jobName":"paragraph_1520367527559_1576992576","id":"20180115-070257_386614995","dateCreated":"2018-03-06T20:18:47+0000","dateStarted":"2018-03-06T20:35:47+0000","dateFinished":"2018-03-06T20:35:48+0000","status":"FINISHED","progressUpdateIntervalMs":500,"$$hashKey":"object:4680"},{"text":"%md\n### Neural Network Configuration","dateUpdated":"2018-03-06T20:18:47+0000","config":{"tableHide":false,"editorSetting":{"language":"markdown","editOnDblClick":true},"colWidth":12,"editorMode":"ace/mode/markdown","fontSize":9,"editorHide":true,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"HTML","data":"<h3>Neural Network Configuration</h3>\n"}]},"apps":[],"jobName":"paragraph_1520367527560_1575068831","id":"20180115-062914_1963273192","dateCreated":"2018-03-06T20:18:47+0000","status":"READY","errorMessage":"","progressUpdateIntervalMs":500,"$$hashKey":"object:4681"},{"text":"%md\nNow we can finally configure and then initialize the neural network for this problem. We will be using the ComputationGraph class of DL4J.","dateUpdated":"2018-03-06T20:18:47+0000","config":{"tableHide":false,"editorSetting":{"language":"markdown","editOnDblClick":true},"colWidth":12,"editorMode":"ace/mode/markdown","fontSize":9,"editorHide":true,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"HTML","data":"<p>Now we can finally configure and then initialize the neural network for this problem. We will be using the ComputationGraph class of DL4J.</p>\n"}]},"apps":[],"jobName":"paragraph_1520367527560_1575068831","id":"20180116-045323_1278918662","dateCreated":"2018-03-06T20:18:47+0000","status":"READY","errorMessage":"","progressUpdateIntervalMs":500,"$$hashKey":"object:4682"},{"text":"// Set neural network parameters\nval NB_INPUTS = 86\nval NB_EPOCHS = 10\nval RANDOM_SEED = 1234\nval LEARNING_RATE = 0.005\nval BATCH_SIZE = 32\nval LSTM_LAYER_SIZE = 200\nval NUM_LABEL_CLASSES = 2 ","user":"admin","dateUpdated":"2018-03-06T20:35:13+0000","config":{"tableHide":false,"editorSetting":{"language":"scala","editOnDblClick":false},"colWidth":12,"editorMode":"ace/mode/scala","fontSize":9,"editorHide":false,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"TEXT","data":"NB_INPUTS: Int = 86\nNB_EPOCHS: Int = 10\nRANDOM_SEED: Int = 1234\nLEARNING_RATE: Double = 0.005\nBATCH_SIZE: Int = 32\nLSTM_LAYER_SIZE: Int = 200\nNUM_LABEL_CLASSES: Int = 2\n"}]},"apps":[],"jobName":"paragraph_1520367527560_1575068831","id":"20180115-062305_1051366040","dateCreated":"2018-03-06T20:18:47+0000","dateStarted":"2018-03-06T20:35:48+0000","dateFinished":"2018-03-06T20:35:49+0000","status":"FINISHED","progressUpdateIntervalMs":500,"$$hashKey":"object:4683"},{"text":"\nval conf = new NeuralNetConfiguration.Builder()\n .optimizationAlgo(OptimizationAlgorithm.STOCHASTIC_GRADIENT_DESCENT)\n .l2(0.01)\n .graphBuilder()\n .addInputs(\"in\")\n .addLayer(\"lstm\", new GravesLSTM.Builder().nIn(NB_INPUTS).nOut(30).build(), \"in\")\n .addVertex(\"lastStep\", new LastTimeStepVertex(\"in\"), \"lstm\")\n .addLayer(\"out\", new OutputLayer.Builder().activation(Activation.SOFTMAX).nIn(30).nOut(2)\n .build(), \"lastStep\")\n .setOutputs(\"out\")\n .build();\n\n\nval model = new ComputationGraph(conf);\nmodel.init();","user":"admin","dateUpdated":"2018-03-06T20:54:58+0000","config":{"tableHide":false,"editorSetting":{"language":"scala","editOnDblClick":false},"colWidth":12,"editorMode":"ace/mode/scala","editorHide":false,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"TEXT","data":"conf: org.deeplearning4j.nn.conf.ComputationGraphConfiguration = \n{\n \"backprop\" : true,\n \"backpropType\" : \"Standard\",\n \"cacheMode\" : \"NONE\",\n \"defaultConfiguration\" : {\n \"cacheMode\" : \"NONE\",\n \"epochCount\" : 0,\n \"iterationCount\" : 0,\n \"l1ByParam\" : { },\n \"l2ByParam\" : { },\n \"layer\" : null,\n \"maxNumLineSearchIterations\" : 5,\n \"miniBatch\" : true,\n \"minimize\" : true,\n \"optimizationAlgo\" : \"STOCHASTIC_GRADIENT_DESCENT\",\n \"pretrain\" : false,\n \"seed\" : 1520369698297,\n \"stepFunction\" : null,\n \"variables\" : [ ]\n },\n \"epochCount\" : 0,\n \"inferenceWorkspaceMode\" : \"SEPARATE\",\n \"iterationCount\" : 0,\n \"networkInputs\" : [ \"in\" ],\n \"networkOutputs\" : [ \"out\" ],\n \"pretrain\" : false,\n \"tbpttBackLength\" : 20,\n \"tbpttFwdLength\" : 20,\n \"trainingWorksp...model: org.deeplearning4j.nn.graph.ComputationGraph = org.deeplearning4j.nn.graph.ComputationGraph@1298742\n"}]},"apps":[],"jobName":"paragraph_1520367527560_1575068831","id":"20180203-035206_1227695564","dateCreated":"2018-03-06T20:18:47+0000","dateStarted":"2018-03-06T20:54:58+0000","dateFinished":"2018-03-06T20:54:58+0000","status":"FINISHED","progressUpdateIntervalMs":500,"$$hashKey":"object:4684"},{"text":"%md\n\n### Training","dateUpdated":"2018-03-06T20:18:47+0000","config":{"tableHide":false,"editorSetting":{"language":"markdown","editOnDblClick":true},"colWidth":12,"editorMode":"ace/mode/markdown","fontSize":9,"editorHide":true,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"HTML","data":"<h3>Training</h3>\n"}]},"apps":[],"jobName":"paragraph_1520367527560_1575068831","id":"20180115-200258_1843146082","dateCreated":"2018-03-06T20:18:47+0000","status":"READY","errorMessage":"","progressUpdateIntervalMs":500,"$$hashKey":"object:4685"},{"text":"%md\nBefore we train the neural network, we first need to preprocess the data so that only the last step of the labels array is used for training. Thus, we will define the LastStepPreProc class to do this, which is an extension of the DataSetPreProcessor.","dateUpdated":"2018-03-06T20:18:47+0000","config":{"tableHide":false,"editorSetting":{"language":"markdown","editOnDblClick":true},"colWidth":12,"editorMode":"ace/mode/markdown","fontSize":9,"editorHide":true,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"HTML","data":"<p>Before we train the neural network, we first need to preprocess the data so that only the last step of the labels array is used for training. Thus, we will define the LastStepPreProc class to do this, which is an extension of the DataSetPreProcessor.</p>\n"}]},"apps":[],"jobName":"paragraph_1520367527561_1574684082","id":"20180116-045440_285367027","dateCreated":"2018-03-06T20:18:47+0000","status":"READY","errorMessage":"","progressUpdateIntervalMs":500,"$$hashKey":"object:4686"},{"text":"class LastStepPreProc extends DataSetPreProcessor {\n\n override def preProcess(in : DataSet) {\n\n val origLabels = in.getLabels();\n val lMask = in.getLabelsMaskArray();\n\n //On master: use TimeSeriesUtils.pullLastTimeSteps(origLabels, lMask);\n \n val labels2d = pullLastTimeSteps(origLabels, lMask);\n \n in.setLabels(labels2d);\n in.setLabelsMaskArray(null);\n }\n\n def pullLastTimeSteps( pullFrom : INDArray, mask : INDArray) : INDArray = {\n if (mask == null) {\n //No mask array -> extract same (last) column for all\n var lastTS = pullFrom.size(2) - 1;\n var out = pullFrom.get(NDArrayIndex.all(), NDArrayIndex.all(), NDArrayIndex.point(lastTS));\n var fwdPassTimeSteps = null; //Null -> last time step for all examples\n out\n \n } else {\n var outShape = new Array[Double](2);\n outShape(0) = pullFrom.size(0);\n outShape(1) = pullFrom.size(1);\n \n var out = Nd4j.create(outShape);\n\n //Want the index of the last non-zero entry in the mask array\n var lastStepArr = BooleanIndexing.lastIndex(mask, Conditions.epsNotEquals(0.0), 1);\n var fwdPassTimeSteps = lastStepArr.data().asInt();\n \n for ( i <- 0 to fwdPassTimeSteps.length-1) {\n out.putRow(i, pullFrom.get(NDArrayIndex.point(i), NDArrayIndex.all(),\n NDArrayIndex.point(fwdPassTimeSteps(i))));\n }\n out\n }\n }\n}","user":"admin","dateUpdated":"2018-03-06T20:35:18+0000","config":{"tableHide":false,"editorSetting":{"language":"scala","editOnDblClick":true},"colWidth":12,"editorMode":"ace/mode/scala","editorHide":true,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"TEXT","data":"defined class LastStepPreProc\n"}]},"apps":[],"jobName":"paragraph_1520367527561_1574684082","id":"20180212-075949_266693958","dateCreated":"2018-03-06T20:18:47+0000","dateStarted":"2018-03-06T20:35:49+0000","dateFinished":"2018-03-06T20:35:50+0000","status":"FINISHED","progressUpdateIntervalMs":500,"$$hashKey":"object:4687"},{"text":"%md \n\nTo actually train the neural network, we use a for loop for the number of epochs to train. We then extract each DataSet, preprocess it, and fit it to the model.","dateUpdated":"2018-03-06T20:18:47+0000","config":{"tableHide":false,"editorSetting":{"language":"markdown","editOnDblClick":true},"colWidth":12,"editorMode":"ace/mode/markdown","editorHide":true,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"HTML","data":"<p>To actually train the neural network, we use a for loop for the number of epochs to train. We then extract each DataSet, preprocess it, and fit it to the model.</p>\n"}]},"apps":[],"jobName":"paragraph_1520367527561_1574684082","id":"20180212-222213_2027748665","dateCreated":"2018-03-06T20:18:47+0000","status":"READY","errorMessage":"","progressUpdateIntervalMs":500,"$$hashKey":"object:4688"},{"text":"val proc = new LastStepPreProc()\n\ntrainData.reset()\n\nfor( i <- 1 to 5){\n println(\"Epoch:\")\n println(i)\n while(trainData.hasNext()){\n val batch = trainData.next()\n proc.preProcess(batch) \n model.fit(batch)\n }\n trainData.reset()\n}\n","user":"admin","dateUpdated":"2018-03-06T21:01:02+0000","config":{"tableHide":false,"editorSetting":{"language":"scala","editOnDblClick":false},"colWidth":12,"editorMode":"ace/mode/scala","editorHide":false,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"TEXT","data":"proc: LastStepPreProc = $iwC$$iwC$LastStepPreProc@342839b3\nEpoch:\n1\n"}]},"apps":[],"jobName":"paragraph_1520367527561_1574684082","id":"20180203-035511_669197466","dateCreated":"2018-03-06T20:18:47+0000","dateStarted":"2018-03-06T20:36:51+0000","dateFinished":"2018-03-06T20:42:47+0000","status":"FINISHED","progressUpdateIntervalMs":500,"$$hashKey":"object:4689"},{"text":"%md \n\n### Adding Model to SKIL Experiment\n","dateUpdated":"2018-03-06T20:18:47+0000","config":{"tableHide":false,"editorSetting":{"language":"markdown","editOnDblClick":true},"colWidth":12,"editorMode":"ace/mode/markdown","editorHide":true,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"HTML","data":"<h3>Adding Model to SKIL Experiment</h3>\n"}]},"apps":[],"jobName":"paragraph_1520367527561_1574684082","id":"20180212-210749_1826023024","dateCreated":"2018-03-06T20:18:47+0000","status":"READY","errorMessage":"","progressUpdateIntervalMs":500,"$$hashKey":"object:4690"},{"text":"%md\nTo finally add the model to the SKIL experiment, we will initialize a skilContext and use the addModelToExperiment method.","dateUpdated":"2018-03-06T20:18:47+0000","config":{"tableHide":false,"editorSetting":{"language":"markdown","editOnDblClick":true},"colWidth":12,"editorMode":"ace/mode/markdown","editorHide":true,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"HTML","data":"<p>To finally add the model to the SKIL experiment, we will initialize a skilContext and use the addModelToExperiment method.</p>\n"}]},"apps":[],"jobName":"paragraph_1520367527561_1574684082","id":"20180213-003721_1171320857","dateCreated":"2018-03-06T20:18:47+0000","status":"READY","errorMessage":"","progressUpdateIntervalMs":500,"$$hashKey":"object:4691"},{"text":"val skilContext = new SkilContext()\nval client = skilContext.client\nval model_id = skilContext.addModelToExperiment(z, model)\n","user":"admin","dateUpdated":"2018-03-06T20:43:33+0000","config":{"tableHide":false,"editorSetting":{"language":"scala","editOnDblClick":false},"colWidth":12,"editorMode":"ace/mode/scala","fontSize":9,"editorHide":false,"results":{},"enabled":true},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"TEXT","data":"skilContext: io.skymind.zeppelin.utils.SkilContext = io.skymind.zeppelin.utils.SkilContext@2bcd8d34\nclient: io.skymind.modelproviders.history.client.ModelHistoryClient = io.skymind.modelproviders.history.client.ModelHistoryClient@6a52240e\nmodel_id: String = 77452c5b-eab7-4403-9725-0092832b3aec\n"}]},"apps":[],"jobName":"paragraph_1520367527562_1575838329","id":"20180116-045740_857601022","dateCreated":"2018-03-06T20:18:47+0000","dateStarted":"2018-03-06T20:43:33+0000","dateFinished":"2018-03-06T20:43:33+0000","status":"FINISHED","progressUpdateIntervalMs":500,"$$hashKey":"object:4692"},{"user":"admin","config":{"colWidth":12,"enabled":true,"results":{},"editorSetting":{"language":"markdown","editOnDblClick":true},"editorMode":"ace/mode/markdown","editorHide":true,"tableHide":false},"settings":{"params":{},"forms":{}},"apps":[],"jobName":"paragraph_1520370049641_-944962838","id":"20180306-210049_1604586517","dateCreated":"2018-03-06T21:00:49+0000","status":"FINISHED","progressUpdateIntervalMs":500,"$$hashKey":"object:4693","text":"%md\nWe then initialize an Evaluation class in order to evalute how well our model performs.","dateUpdated":"2018-03-06T21:02:27+0000","dateFinished":"2018-03-06T21:02:28+0000","dateStarted":"2018-03-06T21:02:27+0000","results":{"code":"SUCCESS","msg":[{"type":"HTML","data":"<p>We then initialize an Evaluation class in order to evalute how well our model performs.</p>\n"}]}},{"text":"testData.reset()\nval eval = new Evaluation(2);","user":"admin","dateUpdated":"2018-03-06T21:00:40+0000","config":{"colWidth":12,"enabled":true,"results":{},"editorSetting":{"language":"scala"},"editorMode":"ace/mode/scala"},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"TEXT","data":"eval: org.deeplearning4j.eval.Evaluation = \n\n\nWarning: 2 classes were never predicted by the model and were excluded from average precision\nClasses excluded from average precision: [0, 1]\nWarning: 2 classes were never predicted by the model and were excluded from average recall\nClasses excluded from average recall: [0, 1]\n\n==========================Scores========================================\n # of classes: 2\n Accuracy: 0.0000\n Precision: 0.0000\n Recall: 0.0000\n F1 Score: NaN\n========================================================================\n"}]},"apps":[],"jobName":"paragraph_1520368098724_463970665","id":"20180306-202818_278879978","dateCreated":"2018-03-06T20:28:18+0000","dateStarted":"2018-03-06T21:00:40+0000","dateFinished":"2018-03-06T21:00:40+0000","status":"FINISHED","progressUpdateIntervalMs":500,"$$hashKey":"object:4694"},{"user":"admin","config":{"colWidth":12,"enabled":true,"results":{},"editorSetting":{"language":"markdown","editOnDblClick":true},"editorMode":"ace/mode/markdown","editorHide":true,"tableHide":false},"settings":{"params":{},"forms":{}},"apps":[],"jobName":"paragraph_1520370150445_-1718153241","id":"20180306-210230_1794475520","dateCreated":"2018-03-06T21:02:30+0000","status":"FINISHED","progressUpdateIntervalMs":500,"focus":true,"$$hashKey":"object:7060","text":"%md\nWe can then evaluate our test set using a while loop.","dateUpdated":"2018-03-06T21:03:24+0000","dateFinished":"2018-03-06T21:03:24+0000","dateStarted":"2018-03-06T21:03:24+0000","results":{"code":"SUCCESS","msg":[{"type":"HTML","data":"<p>We can then evaluate our test set using a while loop.</p>\n"}]}},{"text":"while(testData.hasNext()){\n val batch = testData.next()\n proc.preProcess(batch) \n val myOutput = model.output(batch.getFeatures())\n eval.eval(batch.getLabels(),myOutput(0))\n}\n","user":"admin","dateUpdated":"2018-03-06T21:01:02+0000","config":{"colWidth":12,"enabled":true,"results":{},"editorSetting":{"language":"scala"},"editorMode":"ace/mode/scala"},"settings":{"params":{},"forms":{}},"apps":[],"jobName":"paragraph_1520370036838_-1455555277","id":"20180306-210036_1924140819","dateCreated":"2018-03-06T21:00:36+0000","status":"READY","progressUpdateIntervalMs":500,"$$hashKey":"object:4695"},{"user":"admin","config":{"colWidth":12,"enabled":true,"results":{},"editorSetting":{"language":"markdown","editOnDblClick":true},"editorMode":"ace/mode/markdown","editorHide":true,"tableHide":false},"settings":{"params":{},"forms":{}},"apps":[],"jobName":"paragraph_1520370166576_874700245","id":"20180306-210246_502773764","dateCreated":"2018-03-06T21:02:46+0000","status":"FINISHED","progressUpdateIntervalMs":500,"focus":true,"$$hashKey":"object:7143","text":"%md\nFinally we add the evaluation to the model using the addEvaluationToModel method of the skilContext.","dateUpdated":"2018-03-06T21:03:10+0000","dateFinished":"2018-03-06T21:03:10+0000","dateStarted":"2018-03-06T21:03:10+0000","results":{"code":"SUCCESS","msg":[{"type":"HTML","data":"<p>Finally we add the evaluation to the model using the addEvaluationToModel method of the skilContext.</p>\n"}]}},{"text":"skilContext.addEvaluationToModel(z, model_id, eval)","user":"admin","dateUpdated":"2018-03-06T20:58:41+0000","config":{"colWidth":12,"editorMode":"ace/mode/scala","results":{},"enabled":true,"editorSetting":{"language":"scala"}},"settings":{"params":{},"forms":{}},"results":{"code":"SUCCESS","msg":[{"type":"TEXT","data":"res206: String = c72ce563-fbd4-4593-a427-81869df871a1\n"}]},"apps":[],"jobName":"paragraph_1520367527562_1575838329","id":"20180207-090259_714890604","dateCreated":"2018-03-06T20:18:47+0000","dateStarted":"2018-03-06T20:58:41+0000","dateFinished":"2018-03-06T20:58:41+0000","status":"FINISHED","progressUpdateIntervalMs":500,"$$hashKey":"object:4696"},{"user":"admin","config":{"colWidth":12,"enabled":true,"results":{},"editorSetting":{"language":"scala"},"editorMode":"ace/mode/scala"},"settings":{"params":{},"forms":{}},"apps":[],"jobName":"paragraph_1520370019555_-578343109","id":"20180306-210019_1017301605","dateCreated":"2018-03-06T21:00:19+0000","status":"READY","progressUpdateIntervalMs":500,"$$hashKey":"object:4697"}],"name":"ClinicalTimeSeriesLSTMExample","id":"2DA58UTWC","angularObjects":{"2D9577PF1:existing_process":[],"2D86VT151:existing_process":[],"2D7PUMBZ1:existing_process":[],"2D7V8B4SM:existing_process":[],"2D6JWT1U6:existing_process":[],"2D8S4RU7Z:existing_process":[],"2D9YYJMG9:existing_process":[],"2D86WT5A7:existing_process":[]},"config":{"looknfeel":"default","personalizedMode":"false"},"info":{}}