Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add variables to PICDS #8

Merged
merged 29 commits into from
Mar 22, 2024
Merged

Add variables to PICDS #8

merged 29 commits into from
Mar 22, 2024

Conversation

mlondschien
Copy link
Member

@mlondschien mlondschien commented Mar 22, 2024

cai

The config description is "calcium iodized". The corresponding row in the picdb d_labitems is

131,5215,钙(离子) Ca2+,"Calcium, Total",Blood,Chemistry,36739

cai

The config description is "calcium". The corresponding row in the picdb d_labitems is

23,5034,钙,"Calcium, Total",Blood,Chemistry,36739

Note the english description is equal to that of ca. The entries in picdb labevents are mmol/L, so we need to transform.

dbp and sbp

14,1015,舒张压,Diastolic Pressure,chartevents,Routine Vital Signs,mmHg
15,1016,收缩压,Systolic Pressure,chartevents,Routine Vital Signs,mmHg

Note, there are also

462,SV3,动脉收缩压,Systolic Blood Pressure,surgery_vital_signs,,mmHg
463,SV4,动脉舒张压,Diastolic Blood Pressure,surgery_vital_signs,,mmHg
464,SV5,动脉平均压,Mean Arterial Pressure,surgery_vital_signs,,mmHg
465,SV6,中心静脉压,Central Venous Pressure,surgery_vital_signs,,mmHg
466,SV7,肺动脉收缩压,Pulmonary Arterial Systolic Pressure,surgery_vital_signs,,mmHg
467,SV8,肺动脉舒张压,Pulmonary Arterial Diastolic Pressure,surgery_vital_signs,,mmHg
468,SV9,肺动脉平均压,Mean Pulmonary Arterial Pressure,surgery_vital_signs,,mmHg
469,SV10,无创收缩压,Noninvasive Systolic Blood Pressure,surgery_vital_signs,,mmHg
470,SV11,无创舒张压,Noninvasive Diastolic Blood Pressure,surgery_vital_signs,,mmHg
471,SV12,无创平均压,Noninvasive Mean Arterial Pressure,surgery_vital_signs,,mmHg
472,SV13,呼吸,Respiratory Rate,surgery_vital_signs,,insp/min

in the surgery tables.

etco2

Only exists for surgery tables

475,SV16,呼气末二氧化碳分压,End-Tidal CO2,surgery_vital_signs,,mmHg

tco2

165,5256,tCO2(P),tCO2(P),Blood,Chemistry,

Note that the unit of tco2(P) in picdb is mmol/L, different to mEq/L. But this is probably the same, possibly as the charge is +-1? @martin

Also, means appear to agree between mimic / picdb. mimic is meq/l.

In [46]: pd.to_numeric(labevents_mimic[lambda x: x["ITEMID"].eq(50804)]["VALUE"], errors="coerce").mean()
Out[46]: 26.041371577681282

In [47]: pd.to_numeric(labevents[lambda x: x["ITEMID"].eq(5256)]["VALUE"], errors="coerce").mean()
Out[47]: 25.695453843778942

pco2

147,5235,二氧化碳分压 pCO2,pCO2,Blood,Blood Gas,11557-6
148,5236,二氧化碳分压 pCO2(T),pCO2,Blood,Blood Gas,11557-6

What is pco2(T) vs pco2?

map

This does not appear in picdb, but can be computed as 2/3 * DBP + 1/3 SBP. Is this correct @martin? How to implement this in ricu?

hr

already there. Also

460,SV1,心率,Heart Rate,surgery_vital_signs,,bpm

We also add

2,1002,脉搏,Pulse,chartevents,,bpm

temp

There is an entry in labevents. Why? Values appear to make sense though:

In [61]: pd.to_numeric(chartevents[lambda x: x["ITEMID"].eq(1001)]["VALUE"], errors="coerce").median()
Out[61]: 36.9

In [63]: pd.to_numeric(labevents[lambda x: x["ITEMID"].eq(5253)]["VALUE"], errors="coerce").median()
Out[63]: 37.0

We leave as is.

fio2

135,5222,FO2(I),FO2(I),Blood,Chemistry,

Note that this appears to be used in

152,5241,pO2(A)/FO2(I),pO2(A)/FO2(I),Blood,Chemistry,
153,5242,"pO2(A,T)","pO2(A,T)",Blood,Chemistry,
154,5243,pO2(T)/FO2(I),pO2(T)/FO2(I),Blood,Chemistry,

value make sense:

In [64]: pd.to_numeric(labevents[lambda x: x["ITEMID"].eq(5222)]["VALUE"], errors="coerce").median()
Out[64]: 21.0

In [65]: labevents[lambda x: x["ITEMID"].eq(5222)]["VALUEUOM"].value_counts()
Out[65]: 
VALUEUOM
%    27816
Name: count, dtype: int64

There's also FO2Hb, with mean value

In [68]: pd.to_numeric(labevents[lambda x: x["ITEMID"].eq(5231)]["VALUE"], errors="coerce").median()
Out[68]: 96.1
144,5231,FO2Hb,FO2Hb,Blood,Chemistry,

hbco

In [69]: pd.to_numeric(labevents[lambda x: x["ITEMID"].eq(5219)]["VALUE"], errors="coerce").median()
Out[69]: 1.2

appears to make sense - leave as is

lact

In [70]: pd.to_numeric(labevents[lambda x: x["ITEMID"].eq(5227)]["VALUE"], errors="coerce").median()
Out[70]: 1.4

appears to make sense - leave as is

methb

In [71]: pd.to_numeric(labevents[lambda x: x["ITEMID"].eq(5228)]["VALUE"], errors="coerce").median()
Out[71]: 0.8

appears to make sense

ph

removed everything except

149,5237,酸碱度 pH,pH,Blood,Blood Gas,11558-4
150,5238,酸碱度 pH(T),pH,Blood,Blood Gas,11558-4

What is "(T)" here? @martin

In [80]: labevents[lambda x: x["ITEMID"].isin([
    ...:                         5237,
    ...:                         5238,
    ...:                         5374,
    ...:                         5386
    ...:                     ])].groupby("ITEMID")["VALUE"].apply(lambda x: pd.to_numeric(x, errors="coerce").notna().sum())
Out[80]: 
ITEMID
5237    165021
5238     27979
5374         1
5386         0
Name: VALUE, dtype: int64
In [82]: labevents[lambda x: x["ITEMID"].isin([
    ...:                         5237,
    ...:                         5238,
    ...:                         5374,
    ...:                         5386
    ...:                     ])].groupby("ITEMID")["VALUE"].apply(lambda x: pd.to_numeric(x, errors="coerce").median())
/Users/mlondschien/mambaforge/envs/dev/lib/python3.10/site-packages/numpy/lib/nanfunctions.py:1215: RuntimeWarning: Mean of empty slice
  return np.nanmean(a, axis, out=out, keepdims=keepdims)
Out[82]: 
ITEMID
5237    7.408
5238    7.394
5374    4.000
5386      NaN
Name: VALUE, dtype: float64

po2

In [85]: pd.to_numeric(labevents_mimic[lambda x: x["ITEMID"].eq(50821)]["VALUE"], errors="coerce").median()
Out[85]: 109.0

In [86]: pd.to_numeric(labevents[lambda x: x["ITEMID"].eq(5239)]["VALUE"], errors="coerce").median()
Out[86]: 122.0

In [87]: pd.to_numeric(labevents[lambda x: x["ITEMID"].eq(5244)]["VALUE"], errors="coerce").median()
Out[87]: 103.0
126,5208,"pO2(a/A,T)","pO2(a/A,T)",Blood,Chemistry,
127,5210,"pO2(A-a,T)","pO2(A-a,T)",Blood,Chemistry,

5210 also contains strings "alkaline" etc., however the unit of 5208 is %. Also, the median is ~30.

sex

gender column in patients table.

ethnic

Everyone's asian.

age

patients table has a dob variable, same as mimic.

death

same as mimic.

death_icu

TODO

death_hosp

TODO

adm

did not find anything.

weight

Get rid of labevents entry

height

Get rid of labevents entry

alb

14,5024,白蛋白,Albumin,Blood,Chemistry,1751-7
531,6465,血清白蛋白,Serum Albumin,Blood,Chemistry,

alp

gsehd guet us

alt

16,5026,丙氨酸氨基转移酶,Alanine Aminotransferase (ALT),Blood,Chemistry,1742-6
116,5195,丙氨酸氨基转移酶(干片法),Alanine Aminotransferase (ALT),Blood,Chemistry,1742-6

both U/L

we excluded

471,6338,丙氨酸,Alanine,Blood,Chemistry,

What is the difference @martin? This has unit μmol/L.

ast

This is probably a typo (asparate instead of aspartate)

20,5031,天门冬氨酸氨基转移酶,Asparate Aminotransferase (AST),Blood,Chemistry,7519

bicar

looks good

bili

164,5255,总胆红素 tBil,"Bilirubin, Total",Blood,Chemistry,27426
54,5075,总胆红素,"Bilirubin, Total",Blood,Chemistry,27426

both times μmol/L.

bili_dir

31,5042,直接胆红素,"Bilirubin, Direct",Blood,Chemistry,25020

again in mumol/L

bun

found nothing

What is

539,6477,尿素(干片法),Urea,Blood,Chemistry,
22,5033,尿素,Urea,Blood,Chemistry,

ck

27,5038,肌酸激酶,Creatine Kinase (CK),Blood,Chemistry,94021

ckmb

28,5039,肌酸激酶-MB活性,"Creatine Kinase, MB Isoenzyme",Blood,Chemistry,1779980

conversion factor from https://onlineconversion.vbulletin.net/forum/main-forums/convert-and-calculate/4310-u-l-to-ng-ml-conversion?p=86907#post86907.

Medians between mimic/picdb are similar after conversion.

cl

132,5218,氯 Cl-,"Chloride, Whole Blood",Blood,Blood Gas,61788

crea

21,5032,肌酐(干片法),Creatinine,Blood,Chemistry,2160-0
777,6954,血肌酐,Creatinine,Blood,Chemistry,2160-0
30,5041,肌酐,Creatinine,Blood,Chemistry,2160-0

added conversion.

crp

looks good

glu

added conversion

k

40,5059,镁,Magnesium,Blood,Chemistry,256096
819,7023,全血镁,whole blood Magnesium,Blood,Chemistry,

7023 has 2 entries without units, so we excluded it.

mg

We added the same conversion as hirid. The eicu conversion does not make sense, should be multiplication instead of division. However, the densities look reasonable. What gives? TODO: Check densities of picdb.

na

143,5230,钠 Na+,"Sodium, Whole Blood",Blood,Blood Gas,2947-0
43,5062,钠,Sodium,Blood,Chemistry,383903
In [167]: labevents[lambda x: x["ITEMID"].eq(5230)]["VALUE"].median()
Out[167]: 137.0

In [168]: labevents[lambda x: x["ITEMID"].eq(5062)]["VALUE"].median()
Out[168]: 67.1

The non-whole blood values are lower @martin, so we removed them.

phos

added conversion from hirid

tnt

Need to convert from picogram to nanogram. Why are values often similar? Also, often "<0.003" in "VALUE" and nan in VALUENUM.

tri

looks good

samp

import pandas as pd

d_items = pd.read_csv("data/picdb/V1.1.0/D_ITEMS.csv")[lambda x: x["ITEMID"].str.startswith("MI")]
microbio = pd.read_csv("data/picdb/V1.1.0/MICROBIOLOGYEVENTS.csv")

out = {}

regex = [
    "not detected",
    "no pathogenic haemophilus cultured",
    "no bacteria",
    "without fungal growth",
    "no ureaplasma urealyticum growth",
    "no acid-fast bacilli found",
    "no fungus",
    "without pathogenic bacteria",
    "without bacteria",
    "almost no bacteria",
    "no pathogenic bacteria growth"
    "no urea and human mycoplasma growth",
    "no ureaplasma urealyticum growth",
    "no Mycoplasma hominis  growth",
]

output = ""

for col in d_items.iterrows():
    if any (x in col[1]["LABEL"].lower() for x in regex):
        output += f"'{col[1]['ITEMID']}', "

print(output)

@manuelburger
Copy link
Collaborator

manuelburger commented Mar 22, 2024

Done files:

  • blood_gas
  • chemistry
  • vitals
  • demographics
  • microbiology

To-Do

  • outcome
  • respiratory
  • nematology
  • medications
  • neurological
  • output

@manuelburger manuelburger mentioned this pull request Mar 22, 2024
10 tasks
@manuelburger manuelburger merged commit 7824352 into main Mar 22, 2024
1 of 7 checks passed
manuelburger added a commit that referenced this pull request Mar 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants