HW 2 review by OttoS

[ostegm]

@jawilliams3000

Hey Jamie,

Hope all is well. I've been running through your HW2 submission and wanted to give you some feedback.

Firstly - the "official" solutions are posted now. You should be able to do a git pull and find those for review. They're also located here: https://github.com/ga-students/DAT_SF_12/tree/gh-pages/Solutions

So... some comments, in chronological order running through your homework.

1. Your application of K-means and cross-validation is great. You arrived at the correct answer, however I wanted to point out a few subtle points. It looks like you used the code from class (which is awesome) - but be sure to think about what the inputs are. For example, you looped through the list of odd integers from 1 to 51.

n_neighbors = range(1, 51, 2)

Why 51? That was the number of observations we had in the demo example. In this case 51 isn't a bad choice, but I wanted to make sure it was conscious. In this data set we have 178 observations, so technically you could go to 177, although I think 51 is a better choice.... Anyway, small point, but wanted to make sure you understood why we used 51.

1. When you chose the number of neighbors (27) I see that you based this off of the graph. Which optimized the score at 27. Remember that this graph was built off of only one slice of the data:

If you were to run the code below (with a random seed of 1, you'd find a different value for K.

X_train, X_test, y_train, y_test = train_test_split(wine_variables, wine['Wine'], test_size=0.3, random_state=1)

The point is, be careful picking K values based off of a random slice of data. Sometimes you can end up overfitting the model based on a single slice of the data. Another way to do this would be to fit the model, and the score based on cross-validation before choosing your K value.

1. This is kind of bonus material, but did you notice that proline is of a bigger magnitude than all the other variables? It ranges from 0-168 when some of the other variables a much much smaller. This scale problem over amplifies the effect of proline. If you scale the data. you can get an accuracy of 96%.

from sklearn.preprocessing import StandardScaler
features_scalar = StandardScaler()
X_train_scaled = features_scalar.fit_transform(X_train)

from sklearn import neighbors
clf_scaled = neighbors.KNeighborsClassifier(3, weights='uniform')
clf_scaled.fit(X_train_scaled, y_train)

1. For part two, clustering - awesome job - I really like that you picked out the two most important features and ran with the predictions in 2 dimensions so you could visualize it. It is possible to run those algorithms with more features, and to do that, just review the solution set. Also keep in mind the scaling issue.
2. At the end, I like that you tried to compare the prediction accuracy across models. It looked like you ran into some trouble with getting the columns to match up on the index. When you do the predictions, they're all still indexed according to the original data, so you can just set new columns for each prediction:

wine_predict = pd.DataFrame(Y_hat_kmeans, columns = ['Kmeans Predicted Class'])
wine_predict['Heir Predicted Class'] = Y_hat_hier
wine_predict['Actual Class'] = wine2.Wine
wine_predict

Just remember that the class labels won't match up. A zero won't be a zero in all three. But you can still see the patterns.

Let me know if you have questions!

Thanks

[jawilliams3000]

@ostegm

Thanks for the feedback. I'll incorporate it moving forward.