The Logic of Logistic Regression: A Tutorial
This might just be the most important lesson in machine learning you will ever see. Why? Because in the process of learning about Logistic Regression, you will encounter several powerful techniques and ideas that are the backbone of the entire field of AI (artificial intelligence). One of these ideas is the method of Gradient Descent -- which is the key to all of deep learning with neural networks.
As you progress along your machine learning journey, you'll notice that ideas you encountered in this lesson will recur again and again. Even if you don't completely grasp all the ideas today, you'll get a solid introduction, and your understanding will grow with time. These ideas will become familiar and cherished tools that you will use to crack open one machine learning problem afer another.
Sections 1 - 3 of this notebook present a concise introduction to Logistic Regression and its application to binary classification problems. Section 4 is a Wrap-up and Summary, and Section 5 is an Assessment to be done in a group setting. The References section lists a few helpful resources.
The Appendix contains further discussion of classification problems, and of the assumptions, strengths and weaknesses of the Logistic Regression classifier.
The Supplement: A Complete Introduction to Logistic Regression extends and completes our discussion, for those who want to go further. Section S.1 The Mathematics of Logistic Regression explains how a Logistic Regression model is trained. In the process, we encounter several key concepts in Machine Learning: the Maximum Likelihood Method, the Binary Cross-Entropy function, and the Method of Gradient Descent. In section S.2 we show how to implement the ideas of Section S.1 to build our own Logistic Regression Classifier in Python. Finally, in Section S.3, we apply our Logistic Regression Classifier to an instructive Case Study of the Pima Indians Diabetes Dataset.