Add InformativeFeatures class

[rebeccabilbro]

Need to add a class to enable the user to evaluate the features that were most informative for a fitted model. This class will inherit from ScoreVisualizer

[rebeccabilbro]

http://scikit-learn.org/stable/auto_examples/ensemble/plot_forest_importances.html

[rebeccabilbro]

In Scikit-Learn, it looks like most supervised methods have a coef_ or feature_importances_ method to determine the most important features.

Examples that use feature_importances_:

• http://scikit-learn.org/stable/modules/generated/sklearn.ensemble.GradientBoostingRegressor.html
• http://scikit-learn.org/stable/modules/generated/sklearn.tree.DecisionTreeClassifier.html
• http://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomForestClassifier.html
• http://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomForestRegressor.html
• http://scikit-learn.org/stable/modules/generated/sklearn.ensemble.AdaBoostClassifier.html

Examples that use coef_:

• http://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html
• http://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LinearRegression.html
• http://scikit-learn.org/stable/modules/generated/sklearn.linear_model.Lasso.html#sklearn.linear_model.Lasso
• http://scikit-learn.org/stable/modules/generated/sklearn.linear_model.ElasticNet.html#sklearn.linear_model.ElasticNet
• http://scikit-learn.org/stable/modules/generated/sklearn.svm.LinearSVC.html

[bbengfort]

@rebeccabilbro adding the comments I wrote on this from the paper. Also; I'd say this is a feature, not technical debt?

Generalized linear models compute a predicted independent variable via the linear combination of an array of coefficients with an array of dependent variables. GLMs are fit by modifying the coefficients so as to minimize error and regularization techniques specify how the model modifies coefficients in relation to each other. As a result, an opportunity presents itself: larger coefficients are necessarily "more informative" because they contribute a greater weight to the final prediction in most cases. Additionally we may say that instance features may also be more or less
"informative" depending on the product of the instance feature value with the feature coefficient. This creates two possibilities:

1. We can compare models based on ranking of coefficients, such that a higher coefficient is "more informative".
2. We can compare instances based on ranking of feature/coefficient products such that a higher product is "more informative".

In both cases, because the coefficient may be negative (indicating a strong negative correlation) we must rank features by the absolute values of their coefficients. Visualizing a model or multiple models by most informative feature is usually done via bar chart where the y-axis is the feature names and the x-axis is numeric value of the coefficient such that the x-axis has both a positive and negative quadrant. The bigger the size of the bar, the more informative that feature is.

This method may also be used for instances; but generally there are very many instances relative to the number models being compared. Instead a heatmap grid is a better choice to inspect the influence of features on individual instances. Here the grid is constructed such that the x-axis represents individual features, and the y-axis represents individual instances. The color of each cell (an instance, feature pair) represents the magnitude of the product of the instance value with the feature's coefficient for a single model. Visual inspection of this diagnostic may reveal a set of instances for which one feature is more predictive than another; or other types of regions of information in the model itself.

[rebeccabilbro]

@bbengfort is that a way of saying that you want to take this one?

[bbengfort]

@rebeccabilbro just saying that I had been thinking about it and had written some notes; if you'd like to take a first crack at it - please feel free! I'm still pushing on Radviz and Paralllel coords (plus all the style and architecture stuff).

[bbengfort]

Closed by #317