Skip to content

2. Logistic Regression

Jump to bottom
QiHong Lu edited this page Dec 5, 2016 · 13 revisions

Compare logistic loss versus least square loss (UNDER CONSTRUCTION)
[simulation code]

What I want to say:

  • classification - as simple as least square can be used as a classifier - sign(X * beta)
  • what is logistic regression
  • logistic loss is more appropriate for classification (than least square linear prediction)
  • all the regularization techniques work for logistic regression

1.Compare the decision boundaries: least square loss vs. logistic loss

Here, the OLS boundary is suboptimal (having a smaller margin), since OLS loss care about the distance between the prediction and the truth, even when the prediction is on the right side. On the other hand, logistic regression behaves more like an support vector machine. (explain more)

Thought question: Explain the plot above by comparing the logistic loss versus the squared error loss.

  • Hint 1: logistic loss is less sensitive to far away points.
  • Hint 2: square error loss has penalty for "easy" points.
Clone this wiki locally