A CLI tool for extracting event logs out of MIMIC Databases. This branch is for MIMIC-IV 1.0. If you use MIMIC-IV 2.0 or 2.2, please pull from the respective branch: https://github.com/bptlab/mimic-log-extraction/tree/mimic-2.0 , https://github.com/bptlab/mimic-log-extraction/blob/mimic-2.2/
- requires python 3.8.10 (newer versions might be fine, though)
- using a python virtual environment seems like a good idea
The official python documentation provides a good overview on how to create virtual environments. We recommend having the environment either in this directory, or one level above.
usage: extract_log.py [-h] [--db_name DB_NAME] [--db_host DB_HOST] [--db_user DB_USER] [--db_pw DB_PW] [--subject_ids SUBJECT_IDS]
[--hadm_ids HADM_IDS] [--icd ICD] [--icd_version ICD_VERSION] [--icd_sequence_number ICD_SEQUENCE_NUMBER] [--drg DRG]
[--drg_type DRG_TYPE] [--age AGE] [--type TYPE] [--tables TABLES] [--tables_activities TABLES_ACTIVITIES]
[--tables_timestamps TABLES_TIMESTAMPS] [--notion NOTION] [--case_attribute_list CASE_ATTRIBUTE_LIST] [--config CONFIG]
[--save_intermediate] [--ignore_intermediate]
optional arguments:
-h, --help show this help message and exit
--db_name DB_NAME Database Name
--db_host DB_HOST Database Host
--db_user DB_USER Database User
--db_pw DB_PW Database Password
--subject_ids SUBJECT_IDS
Subject IDs of cohort
--hadm_ids HADM_IDS Hospital Admission IDs of cohort
--icd ICD ICD code(s) of cohort
--icd_codes_intersection Optional argument, if one wants to filter for disease combinations, such that patients have to have an icd code from icd_codes and from icd_codes_intersection
--icd_version ICD_VERSION
ICD version
--icd_sequence_number ICD_SEQUENCE_NUMBER
Ranking threshold of diagnosis
--drg DRG DRG code(s) of cohort
--drg_type DRG_TYPE DRG type (HCFA, APR)
--age AGE Patient Age of cohort
--type TYPE Event Type
--tables TABLES Low level tables
--tables_activities TABLES_ACTIVITIES
Activity Columns for Low level tables
--tables_timestamps TABLES_TIMESTAMPS
Timestamp Columns for Low level tables
--notion NOTION Case Notion
--case_attribute_list CASE_ATTRIBUTE_LIST
Case Attributes
--config CONFIG Config file for providing all options via file
--save_intermediate Store intermediate extraction results as csv. For debugging purposes.
--ignore_intermediate
Explicitly disable storing of intermediate results.
--csv_log Store resulting log as a .csv file instead of as an .xes event log
Call the tool via
python3 -m extract_log <...>passing the required parameters.
If you installed the tool via cloning this repository, you should instead execute
python3 ./extract_log.py <...>For providing parameters via a .yml config file, provide the path to that file via the --config flag.
This will override any setting provided via prompt or input flag, so be careful. Refer to the example_config.yml file for how to provide options. The config keys icd_codes, drg_codes, and additional_event_attributes need to be explicitly set to [] in order to not be prompted for during extraction. include_medications only needs to be set for POE event logs to avoid the prompt. When case_attributes is set to [], the respective default attributes are used. If the key is not provided, no case attributes are added. To be prompted for it during execution, prompt_case_attributes needs to be set to true.
db:
name: mimic
host: 127.0.0.1
user: some_db_user
pw: some_db_password
save_intermediate: True # True, False
csv_log: False # True, defaults to False
cohort:
subject_ids: # Omitting does not consider subject_ids
- some subject_ids
- ...
hadm_ids: # Omitting does not consider hadm_ids
- some hadm_ids
- ...
icd_codes: # could also be [] to avoid ICD filtering. Omitting makes the tool prompt for input.
- some ICD code
- ...
icd_codes_intersection: # optional argument, if one wants to filter for disease combinations, such that patients have to have an icd code from icd_codes and from icd_codes_intersection
- some ICD code
- ...
icd_version: 10 # 9, 10, 0
icd_seq_num: 1
drg_codes: [] # could also contain keys to filter for DRG codes. Omitting makes the tool prompt for input.
drg_ontology: APR # APR, HCFA
age: # could also be [] to avoid age range filtering. Omitting makes the tool prompt for input.
- 0:25
- 50:90
event_type: admission # admission, transfer, poe
include_medications: False # False, True. Only needed if POE event_type
case_notion: hospital admission # subject, hospital admission
case_attributes: [] # could also be None. [] uses default case attributes for case notion.
prompt_case_attributes: False # False, True. Setting True forces case attributes to be determined if not provided
low_level_tables: # only if event type OTHER
- pharmacy
- labevents
low_level_activities:
- medication
- label
low_level_timestamps:
- starttime
- charttime
additional_event_attributes: # Can be set to []. Omitting makes the tool prompt for input
-
start_column: a
end_column: b
time_column: c
table_to_aggregate: d
column_to_aggregate: f
aggregation_method: g
filter_column: h # can be omitted
filter_values:
- one
- other
-
start_column: a
end_column: b
time_column: c
table_to_aggregate: d
column_to_aggregate: f
aggregation_method: g
filter_column: h # can be omittedSimply run the pip installation command to install the extraction tool:
pip install git+https://github.com/bptlab/mimic-log-extraction/Alternatively, clone this repo and execute
pip install -e .For development and testing, all dev dependencies can be installed using
pip install -e .[dev]If you're using zsh, escape the square brackets: pip install -e .\[dev\]
After installing all required dev dependencies, make sure to regularly call
pylint extract_log.py extractor --rcfile .pylintrc
mypy --config-file mypy.ini .to ensure linted and typechecked code.