Cadio Catch Diseases is a company specializing in early-stage heart disease detection. Its business model is of the service type, that is, the company offers the early diagnosis of a cardiovascular disease for a certain price.
Currently, the diagnosis of a cardiovascular disease is done manually by a team of specialists. The current accuracy of the diagnosis varies between 55% and 65%, due to the complexity of the diagnosis and also to the fatigue of the team that takes turns to minimize the risks. The cost of each diagnosis, including the equipment and the analysts' payroll, is around R$ 1,000.00.
With a Model, get a better precison on cardiovascular diagnosis.
- What is the precision and accuracy of this new tool?
- How mutch profit the Cardio Catch Diseases will earn with this new tool?
- What is the confidence interval of this new tool?
Service is a business model like consultory, the company make a work and receive a profit based on her work results. For example, Cardio Catch Diseases, the price of the diagnosis, paid by the client, varies according to the precision achieved by the time of specialists, the client pays R$500.00 for every 5% of accuracy above 50%. For example, for an accuracy of 55%, the diagnosis is R$500.00 for the client, for an accuracy of 60%, the value is R$1000.00, and so on. If the diagnostic accuracy is 50%, the customer does not pay for it.
Other example is terrain analysis, based on terrain size and terrain quality (terrain analysis) the price of the analysis can change severely.
Cardiovascular Diseases (CVD's) are a group of disorders of the heart and blood vessels.
Heart attacks and strokes are usually acute events and are mainly caused by a blockage that prevents blood from flowing to the heart or brain. The most common reason for this is a build-up of fatty deposits on the inner walls of the blood vessels that supply the heart or brain.
The most important behavioural risk factors of heart disease and stroke are unhealthy diet, physical inactivity, tobacco use and harmful use of alcohol. The effects of behavioural risk factors may show up in individuals as raised blood pressure, raised blood glucose, raised blood lipids, and overweight and obesity. These “intermediate risks factors” can be measured in primary care facilities and indicate an increased risk of heart attack, stroke, heart failure and other complications.
In addition, drug treatment of hypertension, diabetes and high blood lipids are necessary to reduce cardiovascular risk and prevent heart attacks and strokes among people with these conditions.
A heart attack occurs when the blood flow to a part of the heart is blocked by a blood clot, fat or other substances. If this clot cuts off the blood flow completely, the part of the heart muscle supplied by that artery begins to die, if the blood flow is interrupted can damage or destroy part of heart muscle.
The medications and lifestyle changes that your doctor recommends may vary according to how badly your heart was damaged, and to what degree of heart disease caused the heart attack.
Occurs when the heart muscle doesn't pump blood as well as it should. When this happens, blood often backs up and fluid can build up in the lungs, causing shortness of breath. In heart failure, the main pumping chambers of the heart (the ventricles) may become stiff and not fill properly between beats. In some people, the heart muscle may become damaged and weakened. The ventricles may stretch to the point that the heart can't pump enough blood through the body.
One way to prevent heart failure is to prevent and control conditions that can cause it, such as coronary artery disease, high blood pressure, diabetes and obesity.
Your heart has four valves that keep blood flowing in the correct direction. In some cases, one or more of the valves don't open or close properly. This can cause the blood flow through your heart to your body to be disrupted.
- Regurgitation.
- Stenosis.
- Atresia.
The valve flaps don't close properly, causing blood to leak backward in your heart. This commonly occurs due to valve flaps bulging back, a condition called prolapse.
The valve flaps become thick or stiff and possibly fuse together. This results in a narrowed valve opening and reduced blood flow through the valve.
The valve isn't formed, and a solid sheet of tissue blocks the blood flow between the heart chambers.
Have two types of Stroke:
- Ischemic stroke
- Hemorrhagic stroke
It happens when the brain's blood vessels become narrowed or blocked, causing severely reduced blood flow (ischemia). Are caused by fatty deposits that build up in blood vessels or by blood clots or other debris that travel through the bloodstream, most often from the heart, and lodge in the blood vessels in the brain.
Occurs when a blood vessel within the brain bursts. This is most often caused by uncontrolled hypertension (high blood pressure).
Arrhythmia refers to an abnormal heart rhythm. There are various types of arrhythmias, The heart can beat too slow, too fast or irregularly, an arrhythmia can affect how well your heart works. With an irregular heartbeat, your heart may not be able to pump enough blood to meet your body’s needs
- Bradycardia
- Tachycardia
Heart rate that’s too slow, is when the heart rate is less than 60 beats per minute.
Heart rate that’s too fast, refers to a heart rate of more than 100 beats per minute.
The blood pressure is other important thing to check heart health, itsn very important to check the systolic and diastolic, like a fraction of blood pressure on mm Hg ( 120 systolic / 60 diastolic ).
- Systolic Pressure
- Diastolic Pressure
The top number refers to the amount of pressure in your arteries during the contraction of your heart muscle.
The bottom number refers to your blood pressure when your heart muscle is between beats.
The Dataset Base Cardiovascular Disease.
The Deployment of the model is on Google Sheets, the cardio team can check the probability of the new users on base have or no a 'Cardio Diseases'.
- Data Cleaning & Descriptive Statistical.
- First real step is download the dataset, import in jupyter and start in seven steps to change data types, data dimension, fillout na... At first statistic dataframe, i used simple statistic descriptions to check how my data is organized, and check, in dataset have only numerical attirbutes!
- Feature Engineering.
- In this step, with coggle.it to make a mind map and use the mind map to create some hypothesis list, after this list, i created some new features based on blood, like blood volume, blood systolic and diastolic pressure, pulse pressure and bmi, but on dataset do not have other features for more feature engineering.
- Data Filtering.
- On Dataset Have some Outliers, height, weight, blood pressure, extreme negative diastolic pressure, etc,to work with this i have tried to get a "medical intuition" and removed extreme negative diastolic and systolic blood pressure, and a little height and weight threshold.
- Data Balance.
- On Next Cycle i like to use SMOTEEN to clean data overlapping for better model accuracy and precision.
- Exploratory Data Analysis.
- With this dataset is hard to define a class limit, need much deeper feature engineering.
- Data Preparation.
- Used MinMiaxScaler, Robust Scaler & Frequency Encoding for Rescaling some features and drop "alco" & "smoke", because XGBoost and RF did not classify these two features as relevant.
- ML Models.
- I try 7 models on total, four are Tree-based models.
- Data Balance.
- I used SMOTEEN and SMOTETOMEK for Dataset Balance.
- Data Preparation.
- Used MinMiaxScaler, Robust Scaler & Frequency Encoding for Rescaling both datasets (Smoteen Dataset and Smotetomek Dataset) some features and drop "alco" & "smoke", because XGBoost and RF did not classify these two features as relevant.
- ML Models.
- I Used SGD and Ada, focus on SGD classifier.
- Feature Space Study.
- In this new cycle, i have try some Embedding spaces using UMAP, PCA, tSNE and Tree-Based Embedding.
- In this step, I used all the datasets that I had created previously, like the entire dataset, Smoteen and Smotetomeklinks dataset, to analyze all these data behaviors with different tools.
EDA is the most important step on Data Science projects, in this step you "deep dive" on data and work with univariable, bivariable and multivariable data analysis.
In Univariable Analisys
- The Dataset have some identical and normal features, good for machine learning model.
- Have a good balance between classes.
Bivariate Analysis
- The Features based on class, it's hard to find a separating boundary.
With Pearson's correlation method, i get aprox .50 positive correlation with height and gender!
People who suffer from dwarfism have 25% higher cholesterol than a normal adult person.
Alcoholic people have a greater chance of developing cardiovascular disease than people who smoke.
People over 45 are 70% more likely to develop cardiovascular disease.
For Rescaling i used both, MinMax and RobustScaler and Frequency Encoding for numerical features like Gluc Level.
On first cycle i did not used Smoteen for cleaning data overlapping, in next cycles i will go try more things like better feature engineering, PCA, Smoteen...
On Next Cycle i will try balance Dataset with Smoteen and Smotetomeklinks.
I Try both, Smoteen and Smotetomek on Cardio Dataset.
Smoteen removed a lot of data overlapping on Dataset and Smotetomek do not work much well than Smoteen for Balance.
The Feature Space study or Embedding study, in this step i have dedicated a complete Cycle, the Third Cycle, to analyze all data behaviors with different datasets that i have already created the Smoteen Dataset, Smotetomeklinks Dataset and Full Dataset with diferent tools and methods.
For Rescaling i used Both, StandardScaler and MinMaxScaler to analyze this differences.
With this both Rescaling methods i performed the Umap, tSNE, PCA and Tree-Based Embedding on all Three Datasets, you can check the Notebook of Third Cycle to see the different behaviors.
I'm not going to share all the spaces in the README so as not to make it too big.
- Support Vector Machines
- XGBoost
- Random Forest
- K Nearest Neighbors
- Stochastic Gradient Descent
- Light GBM
- Ada Boost
I studied about the power of SVM, but in training I didn't see it that powerful, maybe I'll proceed with this model to tune.
My personal favorite model, fast, light and haved a normal results on training with this dataset.
Random Forest get less results than XGBoost, but, rf have selected some important features thai i selected.
First time I trained a KNN, I really liked the result with 10 neighbors.
This is a good "linear" model, maybe i use on tuning too.
Similar to XGBoost, but significantly better with this only one train dataset.
First time trained AdaBoost.
This is the principal step on this Data Science Project, because, there aren't many ways to create features or collect new data, so a very very *very* detailed data preparation and a good tuning in these cases is very important.
For First Cycle i used SGD and Ada Boosting for Tuning, because Ada have great performace and SGD is a ""linear model"" with linear coeficients. But after tuning, i chosed the SGD because he is it is much lighter on HD than ada, '5Kb' of Disk Space.
I used a simple Random Search to find the best params for model.
On Cross Validation the model have a good performace (Precision).
This step is after tuning the model, to calibrate the super and sub estimation adjustments.
The Calibrated SGD simple performace.
This is the last step of the step of tuning the machine learning model, in this step the confidence intervals are calculated using a ready-made formula from MachineLearningMastery
I Using SGD and Ada on Second Cycle Too for tuning on Smoteen and Smotetomek Dataset. But after some tests i prefer to use ADA to production.
Final ADA Model Performace on Cycle II for Production.
The calibration curve of Raw ADA model
The calibration curve of Tuned ADA model
The Bootstrap of Tuned Only ADA Model.
I do not selected calibrated + tuned model because on bootstrap eith calibrated + tuned model i get an insignificantly larger error. I only used Tuned Model to Deploy.
Need to answer the Questions
- What is the precision and accuracy of this new tool?
- How mutch profit the Cardio Catch Diseases will earn with this new tool?
- What is the confidence interval of this new tool?
At Cross Validation Between ( Mean + / - Std )
- Accuracy ( 0.724 + / - 0.0006 )
- Precision ( 0.7874 + / - 0.0048)
Based on All Dataset (68k Patients).
| % of Precision | 50 % | 55 % | 60 % | 65 % | 70 % | 75 % | 80 % |
| Money / Precision | FREE | $ 500 | $ 1000 | $ 1500 | $ 2000 | $ 2500 | $ 3000 |
| Actual Money / Precision | $ 0 | $ 343,530.00 | $ 687,060.00 | $ 1,030,590.00 | $ --#-- | $ --#-- | $ --#-- |
| Model Money / Precision | $ --#-- | $ 343,530.00 | $ 687,060.00 | $ 1,030,590.00 | $ 1,374,120.00 | $ 1,717,650.00 | $ --#-- |
| __ / __ | Best Scenario | Worst Scenario |
|---|---|---|
| Model | +/- $ 1,717,650.00 | +/- $ 1,374,120.00 |
| Actual | +/- $ 1,030,590.00 | +/- $ 343,530.00 |
- 25% confidence interval of Model Performace ( 77.81% & 78.34% )
- 50% confidence interval of Model Performace ( 77.44% & 78.59% )
- 75% confidence interval of Model Performace ( 77.02% & 78.95% )
The Model is on Google Sheets, because it's a good deploy strategy and it also has a good benefit as for the study of cardiovascular disease cases and checking if the model needs improvement, example, the cardio team like to change the values to study new behaviors and how the model will classify against this new behavior
This is the step for user use the Model Prediction to make better decisions, i chose to deploy on Heroku and make a JS Request on Google Scripts.