Conference Paper

Anomaly Detection in Time Series Medical Data with LSTM

Michael Bowie, Edmon Begoli

Introduction

Background

To Do: Research and understand the problem better: Why are deep learning architectures, such as an LSTM, important to doctors and hospitals?

Potential User Stories:

As a doctor, I made a wrong diagnosis/prescription because the decision was based on undetected erroneous / anomalous data. Since manually combing through potentially thousands or millions of data points is expensive and time-consuming, I need a system that can automatically detect anomalies in data.

As a clinical data scientist, I am not sure how good my machine learning models are at finding anomalies compared to the state-of-the-art. I need a high-quality, industry-standard dataset where models’ performance are benchmarked, compared, and published. This way, I know exactly how good or bad my models are and how to improve them.

As a clinical data scientist, I need an unsupervised model that is great at detecting anomalies in time-series data, i.e electronic health records (EHRs).

Hypthosis:

As the first recorded anomaly detection benchmark for the MIMIC-III dataset, LSTMs will more accurately classify anomalous records in an ICU event than a regular logistic regression or k-nearest neighbors algorithm will. With this benchmark being set, all three of the previous user stories are provided an LSTM based anomaly detection solution that performs better than a standard logistic regression or k-nearest neighbors based solution.

The prediction based anomaly detection model of the LSTM (see below) will perform better than the encoder-decoder based anomaly detection model.

Research Question:

There are two main state-of-the-art techniques for anomaly detection with LSTMs. First is a prediction based anomaly detection. The second is an encoder-decoder based anomaly detection. With the first technique, prediction based anomaly detection, the LSTM is trained on an anomaly-free dataset. The model is trained to predict the very next sequence in a time-series. Once the model is very accurate at predicting the next sequence of a time-series, the model can then be tested on an erroneous dataset.
In order to flag the suspicious record, the model compares the error between the prediction and the actual value. If the error is quite large, then this record could be classified as a potentially anomalous record. However in order for this to work, the model’s accuracy in predictions needs to be very high.
With the second technique, encoder-decoder based anomaly detection, the LSTM is once again trained on an anomaly-free dataset. The model is trained to encode the ICU event and then accurately reconstruct, or decode, the same event. The idea is when the model is presented with an anomalous or erroneous record, the model will perform poorly in decoding the record since the model has never seen a record like this before. This error can be measured and thus classifies the record as a potential anomaly.

The research question here would be which model performs better?

Approach

Experiments

• Univariate
  -- Logistic regression
  -- K-nearest neighbor
  -- Prediction based AD-LSTM
  -- Encoder-Decoder based AD-LSTM

• Multivariate
  -- Logistic regression
  -- K-nearest neighbor
  -- Prediction based AD-LSTM
  -- Encoder-Decoder based AD-LSTM

Results

Summary

Future Work