Upgrade pandas to 1.1.0 and update .week usage #1094

waynethewizard · 2020-07-31T01:22:00Z

Pull Request Description

Addresses resolving the deprecation warning in the primitive cited in this issue: #1091. Tested via ipython (images below) and message is resolved.

Before

After

After creating the pull request: in order to pass the changelog_updated check you will need to update the "Future Release" section of docs/source/changelog.rst to include this pull request.

CLAassistant · 2020-07-31T01:22:07Z

All committers have signed the CLA.

codecov · 2020-07-31T01:31:46Z

Codecov Report

Merging #1094 into main will decrease coverage by 0.01%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##             main    #1094      +/-   ##
==========================================
- Coverage   98.36%   98.35%   -0.02%     
==========================================
  Files         126      126              
  Lines       13267    13275       +8     
==========================================
+ Hits        13050    13056       +6     
- Misses        217      219       +2

Impacted Files	Coverage Δ
...retools/primitives/standard/transform_primitive.py	`100.00% <100.00%> (ø)`
.../tests/primitive_tests/test_transform_primitive.py	`100.00% <100.00%> (ø)`
featuretools/synthesis/deep_feature_synthesis.py	`97.34% <0.00%> (-0.49%)`	⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6b9b5a8...27dd733. Read the comment docs.

thehomebrewnerd

@wsankey Thank you for the contribution! After some discussion, we have a different idea on how we would like to handle this situation for now so that we can suppress this warning without bumping the pandas minimum version.

It seems as if the current method of accessing the week will remain until pandas 2.0, so we can keep using our current code for the time being.

Here is an outline of what we would like to see:

Keep the week(vals) function as it is currently: return vals.dt.week
Add a warnings filter immediately prior to the return statement to suppress the warning coming from Featuretools. The following code should work after importing warnings at the top of the file:

warnings.filterwarnings("ignore", message="Series.dt.weekofyear and Series.dt.week have been deprecated.", module="featuretools")

Change the pandas version restriction to be pandas>=0.24.1,<2.0.0 since we expect this code will break with version 2.0
Update the changelog comment to better reflect these changes, and add your username to the list of contributors

We would like to include this in the next release of Featuretools, which we are planning to make later today. Sorry for the short notice, but do you think you would be able to make these updates today?

Thanks again!

thehomebrewnerd · 2020-07-31T16:27:51Z

docs/source/changelog.rst

@@ -17,6 +17,7 @@ Changelog
        * Change default branch to ``main`` (:pr:`1038`)
        * Raise TypeError if improper input is supplied to ``Entity.delete_variables()`` (:pr:`1064`)
        * Updates for compatibility with pandas 1.1.0 (:pr:`1079`, :pr:`1089`)
+        * Upgrade pandas to >=1.1.0 (:pr: `1094`)


Also, remove the space between :pr: and 1094

Ah that's what it was, thank you!

featuretools/primitives/standard/transform_primitive.py

waynethewizard · 2020-07-31T17:17:23Z

The message persists for me in the ipython terminal with this change. Changing the line to warnings.filterwarnings("ignore", category=FutureWarning) resolves it within the ipython terminal. This may be filtered with that line within the module but I haven't checked in that context.

…testing

waynethewizard · 2020-07-31T17:31:59Z

thehomebrewnerd · 2020-07-31T17:37:14Z

The message persists for me in the ipython terminal with this change. Changing the line to warnings.filterwarnings("ignore", category=FutureWarning) resolves it within the ipython terminal. This may be filtered with that line within the module but I haven't checked in that context.

I think that change will cause any FutureWarning afterward to be suppressed - so if the user did something outside of featuretools that would generate a different FutureWarning, the would never see it. We were trying to limit scope to the specific message within featuretools. I tested from a command line script and it seemed to work, but didn't check in ipython.

rwedge · 2020-07-31T17:40:23Z

@wsankey when you were testing before were you defining the Week primitive in the ipython or importing it from the featuretools module? It's possible if you defined it in memory it did not get filtered due to the module="featuretools" part of the filter

waynethewizard · 2020-07-31T17:43:59Z

Understood. Yes that was the issue. Setting the specific warning message and importing the class resolved this issue. Thank you for the guidance @thehomebrewnerd and @rwedge!

Updated PR to be limited in scope.

rwedge · 2020-07-31T17:55:16Z

We should probably add a test case to confirm the warning is being filtered out

I believe this tests that a warning does not occur

with pytest.warns(None) as record:
    function()

assert not record

@wsankey would you be willing to write a test case?

waynethewizard · 2020-07-31T18:33:58Z

Yes, sure @rwedge I just added a unit test. Not certain this is the proper spot but I manually checked it failed, then passed.

rwedge

@wsankey thanks for the test case, that file is a good place for it.

Just a few small comments

rwedge · 2020-07-31T19:08:11Z

docs/source/changelog.rst

@@ -17,6 +17,7 @@ Changelog
        * Change default branch to ``main`` (:pr:`1038`)
        * Raise TypeError if improper input is supplied to ``Entity.delete_variables()`` (:pr:`1064`)
        * Updates for compatibility with pandas 1.1.0 (:pr:`1079`, :pr:`1089`)
+        * Set pandas version to pandas>=0.24.1,<2.0.0. Filter week and weekofyear primitive warning. (:pr:`1094`)


Let's make this "Set pandas version to pandas>=0.24.1,<2.0.0. Filter pandas deprecation warning in Week primitive."

rwedge · 2020-07-31T19:13:18Z

featuretools/primitives/standard/transform_primitive.py

@@ -212,6 +214,7 @@ class Week(TransformPrimitive):

    def get_function(self):
        def week(vals):
+            warnings.filterwarnings("ignore", message="Series.dt.weekofyear and Series.dt.week have been deprecated.", module="featuretools")


this line is a little long (142), can we reduce line length

rwedge · 2020-07-31T19:14:23Z

featuretools/tests/primitive_tests/test_transform_primitive.py

@@ -83,3 +83,14 @@ def test_age_nan():
    ages = age(dates, time=datetime(2020, 2, 26))
    correct_ages = [10.159, np.nan, 8.159]
    np.testing.assert_array_almost_equal(ages, correct_ages, decimal=3)
+
+
+def test_no_deprecation_message():


let's call this "test_week_no_deprecation_message"

rwedge

Looks good! Thanks @wsankey for the contribution!

waynethewizard · 2020-07-31T19:57:06Z

Thanks! Anyway to rerun these tests, I think it was a coincidence they failed..

waynethewizard · 2020-07-31T19:57:31Z

OK thank you!

rwedge · 2020-07-31T20:03:33Z

Thanks! Anyway to rerun these tests, I think it was a coincidence they failed..

Only the code coverage test failed and that was due to a known issue, wouldn't have been impacted by these code changes

waynethewizard added 2 commits July 30, 2020 21:10

Upgrade pandas to >1.1.0

c5f1985

Use dt.isocalendar().week per pandas docs

815a1bf

waynethewizard added 2 commits July 30, 2020 21:23

Update changelog

62d223f

Greater than or equal to 1.1.0

4930736

thehomebrewnerd suggested changes Jul 31, 2020

View reviewed changes

thehomebrewnerd reviewed Jul 31, 2020

View reviewed changes

waynethewizard added 4 commits July 31, 2020 12:51

Import warnings

fa250b7

Lint the file

5db0036

Set pandas pandas>=0.24.1,<2.0.0

4c405f8

Update changelog to reflect pandas version difference and filtering

0ef9dcb

waynethewizard changed the base branch from main to 0.13.0.dev July 31, 2020 16:58

waynethewizard changed the base branch from 0.13.0.dev to main July 31, 2020 16:59

rwedge reviewed Jul 31, 2020

View reviewed changes

featuretools/primitives/standard/transform_primitive.py Outdated Show resolved Hide resolved

Import warnings, filter .week deprecation message

d78950d

Ignore the FutureWarning here resolves the issue in ipython terminal …

2f97f21

…testing

Ignore the specific deprecation warning

20ab46f

Add test checking no warning is thrown

4ab73e3

lint test

0fa92b6

rwedge reviewed Jul 31, 2020

View reviewed changes

waynethewizard added 2 commits July 31, 2020 15:22

Break lines

f82a432

Update name per review

b508c51

Update changelog per review

27dd733

rwedge approved these changes Jul 31, 2020

View reviewed changes

thehomebrewnerd approved these changes Jul 31, 2020

View reviewed changes

rwedge merged commit 111d0c4 into alteryx:main Jul 31, 2020

waynethewizard deleted the upgrade-pandas110 branch July 31, 2020 19:57

thehomebrewnerd mentioned this pull request Jul 31, 2020

Deprecation warning using Week primitive with pandas 1.1.0 installed #1091

Closed

rwedge mentioned this pull request Jul 31, 2020

v0.18.0 #1098

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Upgrade pandas to 1.1.0 and update .week usage #1094

Upgrade pandas to 1.1.0 and update .week usage #1094

waynethewizard commented Jul 31, 2020 •

edited

Loading

CLAassistant commented Jul 31, 2020 •

edited

Loading

codecov bot commented Jul 31, 2020 •

edited

Loading

thehomebrewnerd left a comment

thehomebrewnerd Jul 31, 2020

waynethewizard Jul 31, 2020

waynethewizard commented Jul 31, 2020 •

edited

Loading

waynethewizard commented Jul 31, 2020

thehomebrewnerd commented Jul 31, 2020

rwedge commented Jul 31, 2020

waynethewizard commented Jul 31, 2020

rwedge commented Jul 31, 2020 •

edited

Loading

waynethewizard commented Jul 31, 2020

rwedge left a comment

rwedge Jul 31, 2020

rwedge Jul 31, 2020

waynethewizard Jul 31, 2020

rwedge Jul 31, 2020

waynethewizard Jul 31, 2020

rwedge left a comment

waynethewizard commented Jul 31, 2020

waynethewizard commented Jul 31, 2020

rwedge commented Jul 31, 2020

Upgrade pandas to 1.1.0 and update .week usage #1094

Upgrade pandas to 1.1.0 and update .week usage #1094

Conversation

waynethewizard commented Jul 31, 2020 • edited Loading

Pull Request Description

Before

After

CLAassistant commented Jul 31, 2020 • edited Loading

codecov bot commented Jul 31, 2020 • edited Loading

Codecov Report

thehomebrewnerd left a comment

Choose a reason for hiding this comment

thehomebrewnerd Jul 31, 2020

Choose a reason for hiding this comment

waynethewizard Jul 31, 2020

Choose a reason for hiding this comment

waynethewizard commented Jul 31, 2020 • edited Loading

waynethewizard commented Jul 31, 2020

thehomebrewnerd commented Jul 31, 2020

rwedge commented Jul 31, 2020

waynethewizard commented Jul 31, 2020

rwedge commented Jul 31, 2020 • edited Loading

waynethewizard commented Jul 31, 2020

rwedge left a comment

Choose a reason for hiding this comment

rwedge Jul 31, 2020

Choose a reason for hiding this comment

rwedge Jul 31, 2020

Choose a reason for hiding this comment

waynethewizard Jul 31, 2020

Choose a reason for hiding this comment

rwedge Jul 31, 2020

Choose a reason for hiding this comment

waynethewizard Jul 31, 2020

Choose a reason for hiding this comment

rwedge left a comment

Choose a reason for hiding this comment

waynethewizard commented Jul 31, 2020

waynethewizard commented Jul 31, 2020

rwedge commented Jul 31, 2020

waynethewizard commented Jul 31, 2020 •

edited

Loading

CLAassistant commented Jul 31, 2020 •

edited

Loading

codecov bot commented Jul 31, 2020 •

edited

Loading

waynethewizard commented Jul 31, 2020 •

edited

Loading

rwedge commented Jul 31, 2020 •

edited

Loading