1 - Introduction - Structured vs Unstructured.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Workshop 1 <br> Gaining Clinical Insight from Text <br> 1.0 - Introduction - Structured vs Unstructured Data\n",
    "### Christina Chen and Team  <br> 2/2018\n",
    "## Objectives\n",
    "### Participants should leave this workshop with ability to:\n",
    "* Understand the difference between structured and unstructured data\n",
    "* Understand what regular expression means and the basics of how to write one\n",
    "* Apply regular expressions to data from MIMIC-III\n",
    "* Get an idea for more complicated concepts we can extract from data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## What is Structured Data?\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Structured data has a high degree of organization, easier to search, often seen in a relational databases.<br>\n",
    "    For example: \n",
    "\n",
    "Patient | Lab | Value\n",
    "--- | --- | ---\n",
    "12333                      |   Creatinine | 1.2\n",
    "73442                      |   White Blood Count | 20.3\n",
    "83423                      |   Creatinine | 2.5\n",
    "34524                      |   Lactate | 25\n",
    "74234                      |   Sodium | 132\n",
    "12537                      |   BUN | 103\n",
    "12537                      |   Creatinine | 3.5\n",
    "83462                      |   pH | 7.04\n",
    "88293                      |   AST | 394\n",
    "33423                      |   Hematocrit | 23.1\n",
    "\n",
    "* As you can see, each line has a patient ID, lab type, and lab value.\n",
    "* If you wanted to pull out patients and their lab values when the creatinine was > 2, you would be able to write a statement to get the following:\n",
    "\n",
    "Patient | Lab | Value\n",
    "--- | --- | ---\n",
    "83423                      |   Creatinine | 2.5\n",
    "12537                      |   Creatinine | 3.5\n",
    "\n",
    "* You can imagine that you would be able to write a statement to get this type of information for thousands of lab values.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## What is Untructured Data?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Unstructured data is everything that isn't structured.  In medicine, this includes things like nursing notes, radiology reports, discharge summaries.\n",
    "    \n",
    "    ```\n",
    "    AP UPRIGHT PORTABLE CHEST RADIOGRAPH. \n",
    "    INDICATION:  Hypoxia. \n",
    "    \n",
    "    There are patchy opacities throughout both lungs, but more prominent on the right.  \n",
    "    The heart size, mediastinal and hilar contours are normal.  The pulmonary vascularity \n",
    "    is normal.  Soft tissue and osseous structures are unremarkable. \n",
    "    ```\n",
    "    \n",
    "* As you can see, there is plenty of information here, but it is less easy to pull out, especially if we have thousands of reports."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## MIMIC-III\n",
    "This stands for the Medical Information Mart for Intensive Care III (MIMIC-III) database.  \n",
    "- Patients admitted to the untensive care units (ICUs) at Beth Israel Deaconess Medical Center, Boston MA. \n",
    "- Patients admitted from 2002 to the beginning of 2012\n",
    "- Includes over 46,000 patients\n",
    "<br><br>\n",
    "### MIMIC has over 2 million notes including the following types of notes:\n",
    "\n",
    "Type of Note | Number of Notes\n",
    "--- | --- \n",
    "Case Management |\t967\n",
    "Consult\t| 98\n",
    "Discharge summary |\t59,652\n",
    "ECG |\t209,051\n",
    "Echo |\t45,794\n",
    "General |\t8,301\n",
    "Nursing |\t223,556\n",
    "Nursing/other |  \t822,497\n",
    "Nutrition |\t9,418\n",
    "Pharmacy | \t103\n",
    "Physician |\t141,624\n",
    "Radiology |\t522,279\n",
    "Rehab Services |\t5,431\n",
    "Respiratory | \t31,739\n",
    "Social Work | \t2,670\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}