New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[MRG] Migrate raising errors from nose to pytest #321

Merged
merged 7 commits into from Aug 24, 2017

Conversation

Projects
None yet
3 participants
@massich
Contributor

massich commented Aug 17, 2017

Reference Issue

Partial migration from nose to pytest (see #323)

What does this implement/fix? Explain your changes.

Things related to raising. The idea is to migrate or change the following:

  • from sklearn.utils.testing import assert_raises
  • from sklearn.utils.testing import assert_raises_regex
  • from sklearn.utils.testing import assert_raise_message
  • from sklearn.utils.testing import assert_warns
  • from sklearn.utils.testing import assert_warns_message

* [ ] from sklearn.utils.testing import assert_no_warnings

Without touching any testing logic.

Any other comments?

Do not squash this PR (I've reworked the git history for easier inspection). And I would recommend to review each commit of the PR independently.

@codecov

This comment has been minimized.

codecov bot commented Aug 17, 2017

Codecov Report

Merging #321 into master will increase coverage by 0.18%.
The diff coverage is 100%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #321      +/-   ##
==========================================
+ Coverage      98%   98.19%   +0.18%     
==========================================
  Files          66       66              
  Lines        3860     3980     +120     
==========================================
+ Hits         3783     3908     +125     
+ Misses         77       72       -5
Impacted Files Coverage Δ
imblearn/ensemble/tests/test_balance_cascade.py 100% <100%> (ø) ⬆️
..._sampling/prototype_selection/tests/test_allknn.py 100% <100%> (ø) ⬆️
imblearn/utils/testing.py 100% <100%> (ø) ⬆️
..._selection/tests/test_edited_nearest_neighbours.py 100% <100%> (ø) ⬆️
imblearn/datasets/tests/test_imbalance.py 100% <100%> (ø) ⬆️
imblearn/combine/tests/test_smote_enn.py 100% <100%> (ø) ⬆️
imblearn/tests/test_exceptions.py 100% <100%> (ø) ⬆️
imblearn/combine/tests/test_smote_tomek.py 100% <100%> (ø) ⬆️
imblearn/utils/estimator_checks.py 92.76% <100%> (+3.15%) ⬆️
...election/tests/test_instance_hardness_threshold.py 100% <100%> (ø) ⬆️
... and 14 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a2de53a...4be2547. Read the comment docs.

@massich massich referenced this pull request Aug 18, 2017

Closed

pytest follow up #323

9 of 15 tasks complete

@massich massich force-pushed the massich:nose_to_pytest branch from deec149 to c995c8b Aug 18, 2017

@pep8speaks

This comment has been minimized.

pep8speaks commented Aug 18, 2017

Hello @massich! Thanks for updating the PR.

Line 149:5: E722 do not use bare except'

Comment last updated on August 24, 2017 at 15:40 Hours UTC

@massich massich force-pushed the massich:nose_to_pytest branch from c995c8b to 8d2a1dd Aug 18, 2017

@massich massich changed the title from [MRG] change assert_raise for raises(xxx) to [MRG] Migrate raising errors from nose to pytest Aug 18, 2017

@massich

This comment has been minimized.

Contributor

massich commented Aug 18, 2017

Regarding from sklearn.utils.testing import assert_raise_message, shall keep using message or we can see it like a particular case of regex were everything matches? And as a related thought, shall we move the tests that use regex with the whole error message and use only particularities?

cc: @glemaitre, @chkoar

@glemaitre

This comment has been minimized.

Member

glemaitre commented Aug 18, 2017

Regarding from sklearn.utils.testing import assert_raise_message, shall keep using message or we can see it like a particular case of regex were everything matches? And as a related thought, shall we move the tests that use regex with the whole error message and use only particularities?

move to regex. for the second thing, only the semantic is necessary. that's why I like regex.

Assert that a code block/function call warns ``expected_warning``
and raise a failure exception otherwise.
If using Python 2.5 or above, you may use this function as a

This comment has been minimized.

@massich

massich Aug 21, 2017

Contributor

This should change to 2.7. cc: @glemaitre

This comment has been minimized.

@massich

massich Aug 21, 2017

Contributor

and the docstring lines should be truncated to 80 characters. But I don't know if this would kill the doctest. Does anyone know it?

This comment has been minimized.

@glemaitre

glemaitre Aug 21, 2017

Member

you can use +NORMALIZE_WHITESPACE

This comment has been minimized.

@glemaitre

glemaitre Aug 21, 2017

Member

you need also to use a more numpydoc way using Parameters Returns and Examples.
I would add a narrative documentation in the user guide instead of the docstring.

This comment has been minimized.

@massich

massich Aug 22, 2017

Contributor

I think that the structure of this narrative documentation should be done in a different PR (merge it before this one) and add the relevant information regarding warns in this PR.

edit: I've created #325 for such purpose

This comment has been minimized.

@glemaitre

glemaitre Aug 22, 2017

Member

you can make a PR adding warns + documentation if you which

Assert that a code block/function call warns ``expected_warning``
and raise a failure exception otherwise.
If using Python 2.5 or above, you may use this function as a

This comment has been minimized.

@glemaitre

glemaitre Aug 21, 2017

Member

you can use +NORMALIZE_WHITESPACE

Assert that a code block/function call warns ``expected_warning``
and raise a failure exception otherwise.
If using Python 2.5 or above, you may use this function as a

This comment has been minimized.

@glemaitre

glemaitre Aug 21, 2017

Member

you need also to use a more numpydoc way using Parameters Returns and Examples.
I would add a narrative documentation in the user guide instead of the docstring.

@massich massich force-pushed the massich:nose_to_pytest branch from 6014b7e to 8da92de Aug 21, 2017

@glemaitre glemaitre changed the title from [MRG] Migrate raising errors from nose to pytest to [WIP] Migrate raising errors from nose to pytest Aug 22, 2017

@glemaitre glemaitre referenced this pull request Aug 22, 2017

Merged

[MRG] EHN handling sparse matrices whenever possible #316

26 of 26 tasks complete

@massich massich force-pushed the massich:nose_to_pytest branch 4 times, most recently from a46ade0 to 09a744a Aug 22, 2017

@massich

This comment has been minimized.

Contributor

massich commented Aug 23, 2017

I think I'll finish this PR here and ask for reviews (cc:@glemaitre, @chkoar, @mrastgoo).
And move the deprecation warnings and assert_no_warngings in different PRs.

Refer to this for assert_no_warnings

@massich massich force-pushed the massich:nose_to_pytest branch from 09a744a to e161fa1 Aug 23, 2017

@massich massich force-pushed the massich:nose_to_pytest branch from e161fa1 to 66f7ea4 Aug 23, 2017

@massich massich changed the title from [WIP] Migrate raising errors from nose to pytest to [MRG] Migrate raising errors from nose to pytest Aug 23, 2017

...
Failed: DID NOT WARN. No warnings of type (<class 'RuntimeWarning'>,) was emitted. The list of emitted warnings is: [UserWarning(<class 'UserWarning'>,)].


This comment has been minimized.

@glemaitre

glemaitre Aug 23, 2017

Member

remove two blanks lines

@@ -96,3 +96,45 @@ same information as the deprecation warning as explained above. Use the
On the top of all the functionality provided by scikit-learn. Imbalance-learn
provides :func:`deprecate_parameter`: which is used to deprecate a sampler's
parameter (attribute) by another one.

Warning management when testing

This comment has been minimized.

@glemaitre

glemaitre Aug 23, 2017

Member

Change to "Testing utilities"


Warning management when testing
===============================

This comment has been minimized.

@glemaitre

glemaitre Aug 23, 2017

Member
Currently, imbalanced-learn provide a warning management utility.
This feature is going to be merge in pytest and will be removed when the pytest release will have it.

@massich massich force-pushed the massich:nose_to_pytest branch from 66f7ea4 to d206aa4 Aug 23, 2017

@glemaitre

This comment has been minimized.

Member

glemaitre commented Aug 24, 2017

You should check my previous comment. Since that you pushed they get folded and outdated but they need to be addressed.

@glemaitre

Apart of the import ordered and the doc of warns it LGTM

... pass
Traceback (most recent call last):
...
Failed: DID NOT WARN. No warnings of type (<.*RuntimeWarning.*>,) was emitted. The list of emitted warnings is: [].

This comment has been minimized.

@glemaitre

glemaitre Aug 24, 2017

Member

I did not think that doctest is an exact matching. You probably have to check in the doctest doc how to make regular expression or use the ellipsis as work around

from sklearn.utils.testing import assert_warns_message
from imblearn.utils.testing import warns

from pytest import raises

This comment has been minimized.

@glemaitre

glemaitre Aug 24, 2017

Member

this import need to be before the imblearn import

This comment has been minimized.

@glemaitre

glemaitre Aug 24, 2017

Member

I would prefer under numpy in fact

from sklearn.ensemble import RandomForestClassifier

from imblearn.ensemble import BalanceCascade

from pytest import raises

This comment has been minimized.

@glemaitre

glemaitre Aug 24, 2017

Member

before imblearn import (the best would be under numpy)

@@ -7,7 +7,7 @@

import numpy as np
from sklearn.utils.testing import assert_allclose, assert_array_equal
from sklearn.utils.testing import assert_raises_regex
from pytest import raises

This comment has been minimized.

@glemaitre

glemaitre Aug 24, 2017

Member

move it under numpy

@@ -7,7 +7,7 @@

import numpy as np
from sklearn.utils.testing import assert_allclose, assert_array_equal
from sklearn.utils.testing import assert_raises_regex
from pytest import raises

This comment has been minimized.

@glemaitre

glemaitre Aug 24, 2017

Member

move under numpy

@@ -18,7 +18,9 @@
as sklearn_yield_all_checks, check_estimator \
as sklearn_check_estimator, check_parameters_default_constructible
from sklearn.exceptions import NotFittedError
from sklearn.utils.testing import assert_warns, assert_raises_regex
from pytest import raises

This comment has been minimized.

@glemaitre

glemaitre Aug 24, 2017

Member

above the scikit-learn import

@@ -14,6 +14,11 @@

from sklearn.base import BaseEstimator

from pytest import warns as _warns
from contextlib import contextmanager
from re import compile

This comment has been minimized.

@glemaitre

glemaitre Aug 24, 2017

Member

the order will be:

  • standard library
  • 3rd party library
  • own library

then

move context and re on the top
then pytest
then scikit-learn
then imblearn


@contextmanager
def warns(expected_warning, match=None):
"""

This comment has been minimized.

@glemaitre

glemaitre Aug 24, 2017

Member

you need to improve the docstring

def warns(expected_warning, match=None):
    """Assert a warning is raised with an optional pattern.

       Assert that a code block/function call warns ``expected_warning``
       and raise a failure exception otherwise. It can be used within a
       context manager ``with``.

        Parameters
        ----------
        expected_warning : Warning
            Warning type.

        match : regex str or None, optional
            The pattern to be checked. By default, no check is done.

        Returns
        -------
        None
       
    """

Check the english because it is on the top of the head

This comment has been minimized.

@glemaitre

glemaitre Aug 24, 2017

Member

You can add a small example as well.

from sklearn.utils.validation import check_X_y, check_array

from imblearn.utils.estimator_checks import check_estimator

from pytest import raises

This comment has been minimized.

@glemaitre

glemaitre Aug 24, 2017

Member

above sklearn

from sklearn.utils.testing import assert_raises_regex
from sklearn.utils.testing import assert_warns_message
from imblearn.utils.testing import warns
from pytest import raises

This comment has been minimized.

@glemaitre

glemaitre Aug 24, 2017

Member

above sklearn

@massich massich force-pushed the massich:nose_to_pytest branch from d206aa4 to a127230 Aug 24, 2017

@massich massich force-pushed the massich:nose_to_pytest branch from a127230 to 2077fca Aug 24, 2017

@massich massich force-pushed the massich:nose_to_pytest branch from 47ea290 to dd2b0db Aug 24, 2017

@massich massich force-pushed the massich:nose_to_pytest branch from 1acfd30 to 4be2547 Aug 24, 2017

@massich

This comment has been minimized.

Contributor

massich commented Aug 24, 2017

ping @glemaitre

@glemaitre

This comment has been minimized.

Member

glemaitre commented Aug 24, 2017

@glemaitre

This comment has been minimized.

Member

glemaitre commented Aug 24, 2017

good to be merge when green

@glemaitre glemaitre merged commit 488a0e8 into scikit-learn-contrib:master Aug 24, 2017

2 of 3 checks passed

continuous-integration/appveyor/pr Waiting for AppVeyor build to complete
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
lgtm analysis: Python No alert changes
Details
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment