Shew! Congratulations! You're really coming along! In this section, you conducted a full data-science pipeline from data acquisition through modeling and interpretation.
You will be able to:
- Understand and explain what was covered in this section
- Understand and explain why this section will help you become a data scientist
This section gave you an opportunity to review some of the wide-ranging skills you've acquired to date and conduct a full data science project! With that, you should have seen that a good data science process requires careful thought not just to technical details, but to the general structure and story behind the data itself. Indeed, substantial business value comes from asking the right questions and persuasively communicating the results. Similarly, even when modeling, much of the predictive value comes from thoughtful selection, and creative feature engineering through exploration of the data.
To further summarize:
- The most common data science frameworks are CRISP-DM, KDD and OSEMiN
- The process of finding, filtering and loading the appropriate data to answer a question is non-trivial
- Decisions made in the data munging/scrubbing phase can have a huge impact on the accuracy of your predictions
- Visualization is a key phase in EDA.
- Analyzing Regression Models:
- Check p-values to determine whether features are significant
- Use Q-Q plots to check for normality
- Plot residuals against the target variable to check for homoscedasticity (and rule out heteroscedasticity)
- Use the Variance Inflation Factor to assess Multicollinearity between independent variables
Again, well done! You put a lot of your skills to work and went through a full process of collecting data, cleaning it, analyzing and using it to answer questions.