Add Q-Q plot to the ResidualsPlot (#853) #1042

VladSkripniuk · 2020-02-18T11:37:15Z

This PR fixes #853 which requested a feature to allow the user to create Q-Q plot beside the residual plot as a mean of normality checking of the residuals.

I have made the following changes:

Plot Q-Q plot using scipy.stats.probplot
Add tests.

Sample Code and Plot

from yellowbrick.regressor.residuals import ResidualsPlot
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LinearRegression
from yellowbrick.datasets import load_concrete

X, y = load_concrete()

X_train, X_test, y_train, y_test = train_test_split(X, y)
viz = ResidualsPlot(LinearRegression(), hist=False, qqplot=True)

viz.fit(X_train, y_train)
viz.score(X_test, y_test)
viz.show()

CHECKLIST

Is the commit message formatted correctly?
Have you noted the new functionality/bugfix in the release notes of the next release?

Included a sample plot to visually illustrate your changes?
Do all of your functions and methods have docstrings?
Have you added/updated unit tests where appropriate?
Have you updated the baseline images if necessary?
Have you run the unit tests using pytest?
Is your code style correct (are you using PEP8, pyflakes)?
Have you documented your new feature/functionality in the docs?

Have you built the docs using make html?

Adds functionality to plot Q-Q plot side by side with residuals plot and verifies that it works as expected

lwgray

Thank you for submitting this PR. I had a question about your use of the probplot from scipy and whether it is valid for creating a Q-Q plot. Can you give your reason as to why you choose probplot

yellowbrick/regressor/residuals.py

tests/test_regressor/test_residuals.py

lwgray · 2020-03-14T14:08:45Z

@VladSkripniuk For each visualizer there is a "Quick Method". It allows for the Visualizer to be called on a single line. Since you modified the ResidualsPlot Class you will need to alter the quick method also. The quickmethod begins as Line 766

Tasks

Add the qqplot parameter Lines 771 to 786
Add qqplot to the instantiation of ResidualsPlot Lines 878 to 889
Add qqplot to the docstring

lwgray · 2020-03-14T14:09:53Z

@VladSkripniuk Another, note... Your branch is out-of-date with the base branch... Please Update this branch before pushing any changes

lwgray

@VladSkripniuk Great job on this PR. There are just a couple of things that need to be done before I can test and merge this PR.

Update Quick method - See comments on main PR page
Update Documentation to showcase Q-Q Plot here

yellowbrick/regressor/residuals.py

VladSkripniuk · 2020-03-15T08:29:17Z

@lwgray thank you for your help with this PR!

lwgray · 2020-03-15T12:26:16Z

yellowbrick/regressor/residuals.py

@@ -719,6 +776,7 @@ def residuals_plot(
    y_test=None,
    ax=None,
    hist=True,
+    qqplot=False,


Thanks for this addition!

lwgray · 2020-03-15T12:26:54Z

yellowbrick/regressor/residuals.py

@@ -772,6 +830,12 @@ def residuals_plot(
        If set to 'density', the probability density function will be plotted.
        If set to True or 'frequency' then the frequency will be plotted.

+    qqplot : {True, False}, default: False


Thanks for adding this here. We do our best to keep our docstring up-to-date because this is how our docs are generated.

lwgray · 2020-03-15T12:28:00Z

docs/api/regressor/residuals.rst

@@ -52,6 +52,19 @@ Note that if the histogram is not desired, it can be turned off with the ``hist=

 .. warning:: The histogram on the residuals plot requires matplotlib 2.0.2 or greater. If you are using an earlier version of matplotlib, simply set the ``hist=False`` flag so that the histogram is not drawn.

+Histogram can be replaced with a Q-Q plot, which is a common way to check that residuals are normally distributed. If the residuals are normally distributed, then their quantiles when plotted against quantiles of normal distribution should form a straight line. The example below shows, how Q-Q plot can be drawn with a ``qqplot=True`` flag. Notice that ``hist`` has to be set to ``False`` in this case.


This is great explanation. Awesome on notifying the user that hist has to be set to False

lwgray · 2020-03-15T19:35:52Z

Your PR is currently blocked because of PR #1050 which resolves some bugs in the test but yours will be merged once #1050 is merged

lwgray · 2020-04-04T05:13:31Z

We fixed the test failures.... i will do a final review and if test past then I will merge this PR. I have updated your branch so do a pull before you make anymore commits

lwgray · 2020-04-04T06:30:23Z

@bbengfort or @rebeccabilbro Travis has Image Comparison failures for the Q-Q plot related tests although they are passing locally. I had a couple of questions:

The local test for Q-Q plot xfail. Is it necessary to have the pytest xfail decorator with the Q-Qplot test ?
Would an appropriate fix be to clear the actual & base images, run residual plots tests, then move the actual images to the baseline images (ie)

python -m tests.images -C tests/test_regressor/test_residuals.py
pytest -vs tests/test_regressor/test_residuals.py
python -m tests.images tests/test_regressor/test_residuals.py

bbengfort · 2020-04-06T12:38:02Z

tests/test_regressor/test_residuals.py

+
+        visualizer.fit(self.data.X.train, self.data.y.train)
+        visualizer.score(self.data.X.test, self.data.y.test)
+        visualizer.finalize()


@lwgray I recommend removing this call to finalize and seeing if that helps fix the tests on Travis - there is a known problem that the image similarity tests aren't removing text from the alternate axes (as you can see in the baseline image) - and text is one of the primary reasons we have a divergence between AppVeyor and Travis.

Were the baseline images created on a Windows machine or Mac/Linux? That might also contribute to this; we have an xfail for windows or conda, but if the image were generated on Windows than it would fail anyway on Travis.

@VladSkripniuk can you remove the call to finalize as described in the above comment from @bbengfort ?
I updated your branch so make sure you pull before you commit and new changes.
Also, did you create the baseline images on Windows or Mac?

@lwgray Sure, I just removed the call to finalize. I created tests/baseline_images/test_regressor/test_residuals/test_residuals_plot_QQ_plot.png in commit 71f83bb using Ubuntu 18.04.

Hi @VladSkripniuk your baseline image is out-of-date. We have since merged fixes issues we were having with matplotlib, so you will need to recreate your baseline images. You can perform the following commands to reset your baseline images.

python -m tests.images -C tests/test_regressor/test_residuals.py
pytest -vs tests/test_regressor/test_residuals.py
python -m tests.images tests/test_regressor/test_residuals.py

@lwgray I executed the commands you mentioned:

python -m tests.images -C tests/test_regressor/test_residuals.py pytest -vs tests/test_regressor/test_residuals.py python -m tests.images tests/test_regressor/test_residuals.py

and ran python -m pytest tests/test_regressor/test_residuals.py and tests passed successfully on my machine. I upload the updated version of tests/baseline_images/test_regressor/test_residuals/test_residuals_plot_QQ_plot.png

@VladSkripniuk great call on upgrading mpl to 3.2.1 - I forgot to tell you to do that.

…als_plot_QQ_plot.png

…ment.

lwgray · 2020-04-06T23:48:57Z

@VladSkripniuk Thank you for your patience and your contribution. I will start working on your other PRs.

Add Q-Q plot to the ResidualsPlot (DistrictDataLabs#853)

71f83bb

Adds functionality to plot Q-Q plot side by side with residuals plot and verifies that it works as expected

lwgray requested changes Mar 10, 2020

View reviewed changes

Add qqplot to docstring of ResidualsPlot.

71c57d1

VladSkripniuk requested a review from lwgray March 10, 2020 21:49

VladSkripniuk added 2 commits March 14, 2020 16:08

Merge remote-tracking branch 'yellowbrick/develop' into fix853

415586f

Add qqplot to the Quick Method of ResidualsPlot.

7e95c79

lwgray requested changes Mar 14, 2020

View reviewed changes

yellowbrick/regressor/residuals.py Show resolved Hide resolved

Add qqplot to documentatition of ResidualsPlot.

58f4d1d

lwgray reviewed Mar 15, 2020

View reviewed changes

Merge branch 'develop' into fix853

eeb8b89

lwgray approved these changes Apr 4, 2020

View reviewed changes

bbengfort reviewed Apr 6, 2020

View reviewed changes

VladSkripniuk added 4 commits April 6, 2020 17:52

remove finalize() in test_residuals_plot_QQ_plot

a70fd83

Reset tests/baseline_images/test_regressor/test_residuals/test_residu…

aa5491d

…als_plot_QQ_plot.png

Reset test_residuals_plot_QQ_plot.png using different virtual environ…

9f0594a

…ment.

Reset test_residuals_plot_QQ_plot.png using mpl==3.2.1

377edb6

lwgray merged commit bc2c316 into DistrictDataLabs:develop Apr 6, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add Q-Q plot to the ResidualsPlot (#853) #1042

Add Q-Q plot to the ResidualsPlot (#853) #1042

VladSkripniuk commented Feb 18, 2020

lwgray left a comment

lwgray commented Mar 14, 2020

lwgray commented Mar 14, 2020 •

edited

lwgray left a comment

VladSkripniuk commented Mar 15, 2020

lwgray Mar 15, 2020

lwgray Mar 15, 2020

lwgray Mar 15, 2020

lwgray commented Mar 15, 2020

lwgray commented Apr 4, 2020

lwgray commented Apr 4, 2020

bbengfort Apr 6, 2020

lwgray Apr 6, 2020

VladSkripniuk Apr 6, 2020

lwgray Apr 6, 2020

VladSkripniuk Apr 6, 2020

lwgray Apr 6, 2020

lwgray commented Apr 6, 2020

		@@ -52,6 +52,19 @@ Note that if the histogram is not desired, it can be turned off with the ``hist=

		.. warning:: The histogram on the residuals plot requires matplotlib 2.0.2 or greater. If you are using an earlier version of matplotlib, simply set the ``hist=False`` flag so that the histogram is not drawn.

		Histogram can be replaced with a Q-Q plot, which is a common way to check that residuals are normally distributed. If the residuals are normally distributed, then their quantiles when plotted against quantiles of normal distribution should form a straight line. The example below shows, how Q-Q plot can be drawn with a ``qqplot=True`` flag. Notice that ``hist`` has to be set to ``False`` in this case.

Add Q-Q plot to the ResidualsPlot (#853) #1042

Add Q-Q plot to the ResidualsPlot (#853) #1042

Conversation

VladSkripniuk commented Feb 18, 2020

Sample Code and Plot

CHECKLIST

lwgray left a comment

Choose a reason for hiding this comment

lwgray commented Mar 14, 2020

lwgray commented Mar 14, 2020 • edited

lwgray left a comment

Choose a reason for hiding this comment

VladSkripniuk commented Mar 15, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lwgray commented Mar 15, 2020

lwgray commented Apr 4, 2020

lwgray commented Apr 4, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lwgray commented Apr 6, 2020

lwgray commented Mar 14, 2020 •

edited