Merge pull request #88 from OHDSI/CDM_v5.2.0

CDM v5.2 closes #71, #73, #75, #83, #84, #69, #85

Documentation/CommonDataModel_Wiki_Files/Home.md

@@ -1,4 +1,5 @@
-***OMOP Common Data Model v5.1.1 Specifications***
+***OMOP Common Data Model v5.2 Specifications***
+
 <br>*Authors: Christian Reich, Patrick Ryan, Rimma Belenkaya, Karthik Natarajan, Clair Blacketer*
 <br>*12 July 2017*
 
@@ -44,6 +45,7 @@ Welcome to the Common Data Model wiki! This wiki houses all of the documentation
 <br>[CONDITION_OCCURRENCE](wiki/CONDITION_OCCURRENCE)
 <br>[MEASUREMENT](wiki/MEASUREMENT)
 <br>[NOTE](wiki/NOTE)
+<br>[NOTE_NLP](wiki/NOTE_NLP)
 <br>[OBSERVATION](wiki/OBSERVATION)
 <br>[FACT_RELATIONSHIP](wiki/FACT_RELATIONSHIP)
 <br>

Documentation/CommonDataModel_Wiki_Files/StandardizedClinicalDataTables/CONDITION_OCCURRENCE.md

@@ -16,8 +16,10 @@ Field|Required|Type|Description
 | stop_reason                    | No        | varchar(20)  | The reason that the condition was no longer present, as indicated in the source data.                                                                                                                            |
 | provider_id                    | No        | integer      | A foreign key to the Provider in the PROVIDER table who was responsible for capturing (diagnosing) the Condition.                                                                                                |
 | visit_occurrence_id            | No        | integer      | A foreign key to the visit in the VISIT table during which the Condition was determined (diagnosed).                                                                                                             |
-| condition_source_concept_id    | No        | integer      | A foreign key to a Condition Concept that refers to the code used in the source.                                                                                                                                 |
 | condition_source_value         | No        | varchar(50)  | The source code for the condition as it appears in the source data. This code is mapped to a standard condition concept in the Standardized Vocabularies and the original code is stored here for reference.     |
+| condition_source_concept_id    | No        | integer      | A foreign key to a Condition Concept that refers to the code used in the source.                                                                                                                                 |
+| condition_status_source_value  | No        | varchar(50)  | The source code for the condition status as it appears in the source data.    |
+| condition_status_concept_id    | No        | integer      | A foreign key to the predefined concept in the standard vocabulary reflecting the condition status |                                                                                                                               |
 
 ### Conventions 
 
@@ -31,4 +33,8 @@ Field|Required|Type|Description
     * ICD-9-CM Secondary Diagnoses from inpatient and outpatient Claims
     * Diagnoses or problems recorded in an EHR.
   * The Stop Reason indicates why a Condition is no longer valid with respect to the purpose within the source data. Typical values include "Discharged", "Resolved", etc.  Note that a Stop Reason does not necessarily imply that the condition is no longer occurring.
-  * Condition source codes are typically ICD-9-CM, Read or ICD-10 diagnosis codes from medical claims or discharge status/visit diagnosis codes from EHRs.
+  * Condition source codes are typically ICD-9-CM, Read or ICD-10 diagnosis codes from medical claims or discharge status/visit diagnosis codes from EHRs.
+  * Presently, there is no designated vocabulary, domain, or class that represents condition status. The following concepts from SNOMED are recommended:
+    * Admitting diagnosis: 4203942
+    * Final diagnosis: 4230359 � should also be used for �Discharge diagnosis�
+    * Preliminary diagnosis: 4033240

Documentation/CommonDataModel_Wiki_Files/StandardizedClinicalDataTables/DRUG_EXPOSURE.md

@@ -14,17 +14,16 @@ Field|Required|Type|Description
 |drug_concept_id|Yes|integer|A foreign key that refers to a Standard Concept identifier in the Standardized Vocabularies for the Drug concept.|
 |drug_exposure_start_date|Yes|date|The start date for the current instance of Drug utilization. Valid entries include a start date of a prescription, the date a prescription was filled, or the date on which a Drug administration procedure was recorded.|
 |drug_exposure_start_datetime|No|datetime|The start date and time for the current instance of Drug utilization. Valid entries include a start date of a prescription, the date a prescription was filled, or the date on which a Drug administration procedure was recorded.|
-|drug_exposure_end_date|No|date|The end date for the current instance of Drug utilization. It is not available from all sources.|
+|drug_exposure_end_date|Yes|date|The end date for the current instance of Drug utilization. It is not available from all sources.|
 |drug_exposure_end_datetime|No|datetime|The end date and time for the current instance of Drug utilization. It is not available from all sources.|
+|verbatim_end_date|No|date|The known end date of a drug_exposure as provided by the source|
 |drug_type_concept_id|Yes|integer| A foreign key to the predefined Concept identifier in the Standardized Vocabularies reflecting the type of Drug Exposure recorded. It indicates how the Drug Exposure was represented in the source data.|
 |stop_reason|No|varchar(20)|The reason the Drug was stopped. Reasons include regimen completed, changed, removed, etc.|
 |refills|No|integer|The number of refills after the initial prescription. The initial prescription is not counted, values start with 0.|
 |quantity |No|float|The quantity of drug as recorded in the original prescription or dispensing record.|
 |days_supply|No|integer|The number of days of supply of the medication as recorded in the original prescription or dispensing record.|
 |sig|No|clob|The directions ("signetur") on the Drug prescription as recorded in the original prescription (and printed on the container) or dispensing record.|
 |route_concept_id|No|integer|A foreign key to a predefined concept in the Standardized Vocabularies reflecting the route of administration.|
-|effective_drug_dose|No|float|Numerical value of Drug dose for this Drug Exposure record.|
-|dose_unit_concept_ id|No|integer|A foreign key to a predefined concept in the Standardized Vocabularies reflecting the unit the effective_drug_dose value is expressed.|
 |lot_number|No|varchar(50)|An identifier assigned to a particular quantity or lot of Drug product from the manufacturer.|
 |provider_id|No|integer|A foreign key to the provider in the provider table who initiated (prescribed or administered) the Drug Exposure.|
 |visit_occurrence_id|No|integer|A foreign key to the visit in the visit table during which the Drug Exposure was initiated.|
@@ -41,7 +40,7 @@ Field|Required|Type|Description
   * A Drug Type is assigned to each Drug Exposure to track from what source the information was drawn or inferred from. The valid domain_id for these Concepts is "Drug Type".
   * The content of the refills field determines the current number of refills, not the number of remaining refills. For example, for a drug prescription with 2 refills, the content of this field for the 3 Drug Exposure events are null, 1 and 2.
   * The route_concept_id refers to a Standard Concepts of the "Route" domain. Note: Route information can also be inferred from the Drug product itself by determining the Drug Form of the Concept, creating some partial overlap of the same type of information. However, the route_concept_id could resolve ambiguities of how a certain Drug Form is actually applied. For example, a "Solution" could be used orally or parentherally, and this field will make this determination. 
-  * The Effective Drug Dose and the Dose Unit Concepts are provided in cases when dose information is explicitly provided, as it is typically for pediatric and chemotherapeutic treatments. The domain_id for the Dose Unit Concept is "Unit". Note: this information can only be present if the Drug contains a single active ingredient. Combination products which have doses for each ingredient need to be recorded as separate records.
   * The lot_number field contains an identifier assigned from the manufacturer of the Drug product. 
   * If possible, the visit in which the drug was prescribed or delivered is recorded in the visit_occurrence_id field through a reference to the visit table.
-  * If possible, the prescribing or administering provider (physician or nurse) is recorded in the provider_id field through a reference to the provider table.
+  * If possible, the prescribing or administering provider (physician or nurse) is recorded in the provider_id field through a reference to the provider table.
+  * The drug_exposure_end_date denotes the day the drug exposure ended for the patient. This could be that the duration of drug_supply was reached (in which case drug_exposure_end_date = drug_exposure_start_date + days_supply -1), or because the exposure was stopped (medication changed, medication discontinued, etc.)

Documentation/CommonDataModel_Wiki_Files/StandardizedClinicalDataTables/NOTE.md

@@ -7,7 +7,11 @@ Field|Required|Type|Description
 |note_date |Yes|date|The date the note was recorded.|
 |note_datetime|No|datetime|The date and time the note was recorded.|
 |note_type_concept_id|Yes|integer|A foreign key to the predefined Concept in the Standardized Vocabularies reflecting the type, origin or provenance of the Note.|
+|note_class_concept_id|	Yes|	integer|	A foreign key to the predefined Concept in the Standardized Vocabularies reflecting the HL7 LOINC Document Type Vocabulary classification of the note.|
+|note_title	|No|	varchar(250)|	The title of the Note as it appears in the source.|
 |note_text|Yes|RBDMS dependent text|The content of the Note.|
+|encoding_concept_id	|Yes	|integer|	A foreign key to the predefined Concept in the Standardized Vocabularies reflecting the note character encoding type|
+|language_concept_id	|Yes	|integer	|A foreign key to the predefined Concept in the Standardized Vocabularies reflecting the language of the note|
 |provider_id|No|integer|A foreign key to the Provider in the PROVIDER table who took the Note.|
 |note_source_value|No|varchar(50)|The source value associated with the origin of the note|
 |visit_occurrence_id|No|integer|Foreign key to the Visit in the VISIT_OCCURRENCE table when the Note was taken.|
@@ -16,4 +20,112 @@ Field|Required|Type|Description
   * The NOTE table contains free text (in ASCII, or preferably in UTF8 format) taken by a healthcare Provider.
   * The Visit during which the note was written is recorded through a reference to the VISIT_OCCURRENCE table. This information is not always available.
   * The Provider making the note is recorded through a reference to the PROVIDER table. This information is not always available.
-  * The type of note_text is CLOB or string(MAX) depending on RDBMS
+  * The type of note_text is CLOB or varchar(MAX) depending on RDBMS
+  * note_class_concept_id is a foreign key to the CONCEPT table to describe a standardized combination of five LOINC axes (role, domain, setting, type of service, and document kind). See below for description.
+
+### Mapping of clinical documents to Clinical Document Ontology (CDO) and standard terminology
+
+HL7/LOINC CDO is a standard for consistent naming of documents to support a range of use cases: retrieval, organization, display, and exchange. It guides the creation of LOINC codes for clinical notes. CDO annotates each document with 5 dimensions:
+
+* **Kind of Document:** Characterizes the generalc structure of the document at a macro level (e.g. Anesthesia Consent)
+* **Type of Service**: Characterizes the kind of service or activity (e.g. evaluations, consultations, and summaries). The notion of time sequence, e.g., at the beginning (admission) at the end (discharge) is subsumed in this axis. Example: Discharge Teaching.
+* **Setting:** Setting is an extension of CMS�s definitions (e.g. Inpatient, Outpatient)
+* **Subject Matter Domain (SMD):** Characterizes the subject matter domain of a note (e.g. Anesthesiology)
+* **Role:** Characterizes the training or professional level of the author of the document, but does not break down to specialty or subspecialty (e.g. Physician)
+
+Each combination of these 5 dimensions should roll up to a unique LOINC code. For example, Dentistry Hygienist Outpatient Progress note (LOINC code 34127-1) has the following dimensions:
+
+* According to CDO requirements, only 2 of the 5 dimensions are required to properly annotate a document: Kind of Document and any one of the other 4 dimensions.
+* However, not all the permutations of the CDO dimensions will necessarily yield an existing LOINC code.2 HL7/LOINC workforce is committed to establish new LOINC codes for each new encountered combination of CDO dimensions. 3
+
+Automation of mapping of clinical notes to a standard terminology based on the note title is possible when it is driven by ontology (aka CDO). Mapping to individual LOINC codes which may or may not exist for a particular note type cannot be fully automated. To support mapping of clinical notes to CDO in OMOP CDM, we propose the following approach:
+
+#### 1.  Add all LOINC concepts representing 5 CDO dimensions to the Concept table. For example:
+
+Field | Record 1 | Record 2
+:-- | :-- | :--
+concept_id | 55443322132 | 55443322175
+concept_name | Administrative note | Against medical advice note
+concept_code | LP173418-7 | LP173388-2
+vocabulary_id | LOINC | LOINC
+
+#### 2. Represent CDO hierarchy in the Concept_Relationship table using the �Subsumes� � �Is a� relationship pair. For example:
+
+Field | Record 1 | Record 2
+:-- | :-- | :--
+concept_id_1 | 55443322132 | 55443322175
+concept_id_2 | 55443322175 | 55443322132
+relationship_id | Subsumes | Is a
+
+#### 3. Add LOINC document codes to the Concept table (e.g. Dentistry Hygienist Outpatient Progress note, LOINC code 34127-1). For example:
+
+Field | Record 1 | Record 2
+:-- | :-- | :--
+concept_id | 193240 | 193241
+concept_name | Dentistry Hygienist Outpatient Progress note | Consult note
+concept_code | 34127-1 | 11488-4
+vocabulary_id | LOINC | LOINC
+
+#### 4.  Represent dimensions of each document concept in Concept_Relationship table by its relationships to the respective concepts from CDO. 
+
+* Use the �Member Of� � �Has Member�  (new) relationship pair. 
+* Using example from the Dentistry Hygienist Outpatient Progress note (LOINC code 34127-1):
+
+concept_id_1 | concept_id_1 | relationship_id
+:-- | :-- | :--
+193240 | 55443322132 | Member Of
+55443322132 | 193240 | Has Member
+193240 | 55443322175 | Member Of
+55443322175 | 193240 | Has Member
+193240 | 55443322166 | Member Of
+55443322166 | 193240 | Has Member
+193240 | 55443322107 | Member Of
+55443322107 | 193240 | Has Member
+193240 | 55443322146 | Member Of
+55443322146 | 193240 | Has Member
+
+Where concept codes represent the following concepts:
+
+Content | Description
+:---------- | :--------------------------------------------------------------------
+193240 | Corresponds to LOINC 34127-1, Dentistry Hygienist Outpatient Progress note
+55443322132 | Corresponds to LOINC LP173418-7, Kind of Document = Note
+55443322175 | Corresponds to LOINC LP173213-2, Type of Service = Progress
+55443322166 | Corresponds to LOINC LP173051-6, Setting = Outpatient
+55443322107 | Corresponds to LOINC LP172934-4, Subject Matter Domain �= Dentistry
+55443322146 | Corresponds to LOINC LP173071-4, Role = Hygienist
+
+Most of the codes will not have all 5 dimensions. Therefore, they may be represented by 2-5 relationship pairs.
+
+#### 5. If LOINC does not have a code corresponding to a permutation of the 5 CDO encountered in the source, this code will be generated as OMOP vocabulary code. 
+
+* Its relationships to the CDO dimensions will be represented exactly as those of existing LOINC concepts (as described above). If/when a proper LOINC code for this permutation is released, the old code should be deprecated.  Transition between the old and new codes should be represented by �Concept replaces� � �Concept replaced by� pairs.
+
+#### 6.  Mapping from the source data will be performed to the 2-5 CDO dimensions.
+
+Query below finds LOINC code for Dentistry Hygienist Outpatient Progress note (see example above) that has all 5 dimensions:
+
+```sql	
+	SELECT   
+	FROM Concept_Relationship  
+	WHERE relationship_id = �Has Member� AND
+		(concept_id_1 = 55443322132 
+		OR concept_id_1 = 55443322175
+		OR concept_id_1 = 55443322166
+		OR concept_id_1 = 55443322107
+		OR concept_id_1 = 55443322146)
+	GROUP BY concept_ID_2
+```
+
+If less than 5 dimensions are available, `HAVING COUNT(n)` clause should be added to get a unique record at the intersection of these dimensions. n is the number of dimensions available:
+
+```sql
+	SELECT  
+	FROM Concept_Relationship  
+	WHERE relationship_id = �Has Member� AND  
+		(concept_id_1 = 55443322132  
+		OR concept_id_1 = 55443322175  
+		OR concept_id_1 = 55443322146)  
+	GROUP BY concept_ID_2  
+	HAVING COUNT(*) = 3  
+```