Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Initial Restructure of code #59

Merged
merged 45 commits into from
May 17, 2021
Merged

Conversation

yevgenybulochnik
Copy link
Member

Fixes coderxio/medication-diversification#ISSUE NUMBER

Explanation

This is a very large PR to get going with a different code structure. I would still consider this setup a monkey patch to get things working as a python package. We will need to further refactor code.

src/mdt
├── fda
│   ├── __init__.py
│   └── utils.py
├── meps
│   ├── sql
│   │   ├── __init__.py
│   │   └── meps_reference.sql
│   ├── __init__.py
│   ├── columns.py
│   └── utils.py
├── rxnorm
│   ├── sql
│   │   ├── __init__.py
│   │   ├── dfg_df.sql
│   │   └── rxcui_ndc.sql
│   ├── __init__.py
│   ├── rxclass.py
│   └── utils.py
├── __init__.py
├── config.py
├── database.py
├── run_mdt.py
└── utils.py

Rationale

This setup allows our codebase to be installed as a python package using setup.py and provides more structure in general. The FDA, RxNorm, Meps packages are all designed to be free standing so we can pull them out and reuse them. We also leverage package namespaces to make the code syntax more expressive. This needs to be further refactored.

Tests

  1. What testing did you do?
  • Execute python -m mdt.run_mdt D007037 may_treat
  1. Attach testing logs inside a summary block:
testing logs

mdt output

➜ python -m mdt.run_mdt D007037 may_treat
rxnconso table created in DB
rxnrel table created in DB
rxnsat table created in DB
rxcui_ndc table created in DB
dfg_df table created in DB
meps_prescription table created in DB
meps_demographics table created in DB
meps_region_states table created in DB
meps_reference table created in DB
DB table meps_prescription  has 319666 records
DB table meps_demographics has 30461 records
DB table meps_reference has 101844 records
DB table meps_region_states has 51 records
product table created in DB
package table created in DB
DB table product has 120907 records
DB table package has 237938 records
Payload built with base URL: https://rxnav.nlm.nih.gov/REST/rxclass/class/byId.json? and parameters: classId=D007037
GET Request sent to URL: https://rxnav.nlm.nih.gov/REST/rxclass/class/byId.json?classId=D007037
Response HTTP Code: 200
['Hypothyroidism']
Payload built with base URL: https://rxnav.nlm.nih.gov/REST/rxclass/classMembers.json? and parameters: classId=D007037&relaSource=MEDRT&ttys=IN+MIN&rela=may_treat
GET Request sent to URL: https://rxnav.nlm.nih.gov/REST/rxclass/classMembers.json?classId=D007037&relaSource=MEDRT&ttys=IN+MIN&rela=may_treat
Response HTTP Code: 200
['285155', '10582', '10814', '10565', '325521', '10572']
RXCUI list matched on 1556 NDCs
RXCUI list matched on 1561 NDCs

Data dir

data
├── Hypothyroidism_ingredient_distrib.csv
├── Hypothyroidism_product_levothyroxine_distrib.csv
├── Hypothyroidism_product_liothyronine_distrib.csv
├── Hypothyroidism_product_thyroid_USP_distrib.csv
├── MDT.db
├── df_output.csv
└── json_output.json

0 directories, 7 files

yevgenybulochnik and others added 30 commits May 16, 2021 13:59
@yevgenybulochnik yevgenybulochnik requested review from Bridg109 and jrlegrand and removed request for Bridg109 and jrlegrand May 16, 2021 23:15
@kristentaytok
Copy link
Contributor

kristentaytok commented May 17, 2021

@yevgenybulochnik I don't know if this is just for me, but when I try running files with "from . import xxx" in VSCode, I get the error message "ImportError: attempted relative import with no known parent package"..

oops nvm -- I realized this is more of a VScode vs. command line thing, so when I run in command line it's fine.

kristentaytok added a commit that referenced this pull request May 17, 2021
Taking Eugene's restructure branch (from PR #59), this updates the following:
1. Added cleaned version of Rob's code to database.by --> creates an ingredient_rxcui_year table & a product_rxcui_year table, for their respective distributions in generate_module
2. added 'year' column to the generate_module dataframes/CSV files.
3. fixed default_probability typo in utils.py (if idx == 1 --> changed to if idx == 0)
@jrlegrand
Copy link
Member

I tested this branch and everything appears to have worked. I also ran a Synthea population of 1000 patients and got the following meds:

START STOP PATIENT PAYER ENCOUNTER CODE DESCRIPTION BASE_COST PAYER_COVERAGE DISPENSES TOTALCOST REASONCODE REASONDESCRIPTION
2018-03-04T00:31:19Z   661aff27-a5bd-dc83-1a78-5a2ac2d7d62e 7caa7254-5050-3b5e-9eae-bd5ea30e809c 74cc073b-bb8e-9d9f-56c1-6518d8410059 966221 levothyroxine_sodium_0_05_MG_Oral_Tablet 263.49 0 39 10276.11    
2010-09-09T10:12:38Z   7ab806d7-9ea2-adc1-7246-2c513921b051 6e2f1a2d-27bd-3701-8d08-dae202c58632 5baff77c-5068-b27c-1014-b0c444f669af 892251 levothyroxine_sodium_0_2_MG_Oral_Tablet 263.49 0 130 34253.7    
2020-09-12T19:46:59Z   c3f7f647-6a9a-08a8-3b9a-f815b50e6ad8 4d71f845-a6a9-3c39-b242-14d25ef86a8d 8a65fc9a-74a8-881d-553e-c9147088f237 966253 levothyroxine_sodium_0_088_MG_Oral_Tablet 263.49 0 8 2107.92    
2018-02-26T10:14:04Z   4bab0710-ee37-a376-7fad-cb76b517a2c2 d47b3510-2895-3b70-9897-342d681c769d 13871793-9487-0d9a-ab5d-048b518ca36c 892246 levothyroxine_sodium_0_1_MG_Oral_Tablet 263.49 0 39 10276.11    
2019-06-08T19:02:05Z   27b81300-7229-8381-c79d-c955cbf4cb29 b1c428d6-4f07-31e0-90f0-68ffa6ff8c76 42eff404-e2c5-c544-00ea-cefa11455fee 892246 levothyroxine_sodium_0_1_MG_Oral_Tablet 263.49 0 23 6060.27    
2012-04-10T07:31:37Z   0d6d9d43-a6d4-1d61-15fa-69d3a4389622 7c4411ce-02f1-39b5-b9ec-dfbea9ad3c1a f362182f-0a8d-a5f0-c83a-ca3caa589243 966224 levothyroxine_sodium_0_125_MG_Oral_Tablet 263.49 0 111 29247.39    
2006-03-27T06:27:53Z   0524fa28-0c80-b714-9739-df7a3c4ff5b3 7caa7254-5050-3b5e-9eae-bd5ea30e809c 7b1e9f5a-ac00-00aa-88d3-f526ad6dd611 966222 levothyroxine_sodium_0_075_MG_Oral_Tablet 0.3 0 184 55.2    
2011-08-23T03:22:48Z   2e9916bf-3e3f-5acf-304b-173a14349d6b d47b3510-2895-3b70-9897-342d681c769d fd698264-ba95-7b21-bb35-8a4eb2b56dbb 966221 levothyroxine_sodium_0_05_MG_Oral_Tablet 263.49 0 118 31091.82    
2009-09-04T13:11:55Z   05104ecc-a956-7571-9cdc-d5f41ccf8c18 6e2f1a2d-27bd-3701-8d08-dae202c58632 dfa3874d-09bb-419f-49ad-b5d50b1e0732 966248 levothyroxine_sodium_0_112_MG_Oral_Tablet 263.49 0 142 37415.58    

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
4 participants