CLN/DEPR: Final panel removal #27101

WillAyd · 2019-06-28T18:03:18Z

closes CLN: final Panel cleanup #25632
tests added / passed
passes git diff upstream/master -u -- "*.py" | flake8 --diff
whatsnew entry

jorisvandenbossche · 2019-06-28T18:28:29Z

That looked easy ;)

pandas/tests/api/test_api.py

jreback · 2019-06-28T19:30:58Z

https://travis-ci.org/pandas-dev/pandas/jobs/551916327

xarray is explicity referencing Panel. not sure if if this is actually failing on the current one though.

jorisvandenbossche · 2019-06-28T19:41:37Z

So we will need to keep something like

class Panel:

    def __init__(self, *args, **kwargs):
        raise ImportError("pandas.Panel is removed")

in core/panel.py and still import that in the main pandas namespace to avoid breaking released version of statsmodels and xarray.

WillAyd · 2019-06-28T21:35:37Z

Brief glance at the xarray source this is used for introspection so not sure a dummy class is that useful. Perhaps something more cleanly handled downstream? cc @shoyer

jreback · 2019-06-28T21:41:23Z

statsmodels is doing the same type of checking ?

cc @jbrockmendel

jbrockmendel · 2019-06-28T21:45:56Z

statsmodels is doing the same type of checking ?

Yes. Opened statsmodels/statsmodels#5892 to remove it.

shoyer · 2019-06-28T23:20:17Z

I can fix this prior to the next xarray bug fix release

…

On Fri, Jun 28, 2019 at 2:46 PM jbrockmendel ***@***.***> wrote: statsmodels is doing the same type of checking ? Yes. Opened statsmodels/statsmodels#5892 <statsmodels/statsmodels#5892> to remove it. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#27101?email_source=notifications&email_token=AAJJFVTTGYVMVBEINENYJQLP42BBXA5CNFSM4H4HN7FKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODY3HTBI#issuecomment-506886533>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAJJFVXFKORSKIFQSMW6WBDP42BBXANCNFSM4H4HN7FA> .

shoyer · 2019-06-28T23:32:13Z

For what it's worth, this is probably an indication that the deprecation notice for Panel was not entirely effective, because it could still be used only for type checks without an error. In the future it would probably make sense to use module level __getattr__ when deprecating classes.

It's being removed in the next pandas 0.25 release (see pandas-dev/pandas#27101).

jreback · 2019-06-28T23:33:47Z

For what it's worth, this is probably an indication that the deprecation notice for Panel was not entirely effective, because it could still be used only for type checks without an error. In the future it would probably make sense to use module level __getattr__ when deprecating classes.

@shoyer well that's very hard with < 3.7 (we did actually have some machinery to do this but blew it away a while back as it was complex) and not worth the effort IMHO.

jorisvandenbossche · 2019-06-30T16:59:19Z

I would "be nice" for now and don't knowingly break other projects. As @shoyer said, it is hard to catch this as the specific use case does not result in a warning message (so there will probably be other projects as well). If we want to release a RC next week, I think it is better if that RC does not break importing statsmodels or xarray.

Or, @shoyer, do you plan to do a xarray bugfix release soonish?

Keeping a dummy class is not that hard (only a couple of lines of code). For Python 3.7 we could maybe actually use the getattribute trick (if it is possible to do that depending on the python version)

jorisvandenbossche · 2019-06-30T17:09:28Z

Something like this seems to work (at least on Python 3.7):

diff --git a/pandas/__init__.py b/pandas/__init__.py
index eafb89c52..d1d4cf678 100644
--- a/pandas/__init__.py
+++ b/pandas/__init__.py
@@ -158,3 +158,18 @@ Here are just a few of the things that pandas does well:
     conversion, moving window statistics, moving window linear regressions,
     date shifting and lagging, etc.
 """
+
+if pandas.compat.PY37:
+    def __getattr__(name):
+        if name == 'Panel':
+            import warnings
+            warnings.warn(
+                "The Panel class is removed from pandas. Accessing it from the "
+                "top-level namespace will also be removed in the next version",
+                FutureWarning, stacklevel=2)
+            return None
+        raise AttributeError(
+            "module 'pandas' has no attribute {}".format(name))
+else:
+    class Panel:
+        pass

It's being removed in the next pandas 0.25 release (see pandas-dev/pandas#27101).

jorisvandenbossche · 2019-07-01T17:20:39Z

@WillAyd In order to get this merged, are you ok with adding something like the above? (#27101 (comment))

WillAyd · 2019-07-01T17:23:28Z

I would prefer not to add something like that since we have been targeting removal on 0.25 for a while and we still then wouldn't be removing it.

jorisvandenbossche · 2019-07-01T17:42:14Z

To be clear, I don't propose to keep the 1000+ lines of panel code. We are removing the actual Panel class. I only propose to add a few lines to give a more gracious deprecation for a use case we have not been raising a warning for.

jbrockmendel · 2019-07-01T20:27:13Z

I would prefer not to add something like that since we have been targeting removal on 0.25 for a while and we still then wouldn't be removing it.

Agreed. Especially if it is mainly xarray and statsmodels we're worried about, we can help them get quick bugfix releases out and have this over Once And For All.

TomAugspurger · 2019-07-02T20:14:14Z

I think a compatibility shim like #27101 (comment) is good to have for a bit.

@WillAyd do you have time to do that? If not, do you mind if I do it sometime tonight?

shoyer · 2019-07-02T20:19:35Z

Even if xarray issues a bug fix release in the next week (which is quite likely), there are still going to be combinations of pandas with statsmodels/xarray that don't even import. This sort of strict version compatibility requirements is best avoided if possible, and in this case it looks like it would be pretty easy/painless to avoid.

…

On Tue, Jul 2, 2019 at 1:14 PM Tom Augspurger ***@***.***> wrote: I think a compatibility shim like #27101 (comment) <#27101 (comment)> is good to have for a bit. @WillAyd <https://github.com/WillAyd> do you have time to do that? If not, do you mind if I do it sometime tonight? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#27101?email_source=notifications&email_token=AAJJFVRXWJVPGS4ORMCELQ3P5OZJ5A5CNFSM4H4HN7FKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODZCNMPY#issuecomment-507827775>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAJJFVUI3ZLLGGHMMWHVFQDP5OZJ5ANCNFSM4H4HN7FA> .

jreback · 2019-07-02T20:54:33Z

yeah i think we need this shim.

WillAyd · 2019-07-02T21:06:51Z

Sure will probably get to this tonight

…

Sent from my iPhone

On Jul 2, 2019, at 4:54 PM, Jeff Reback ***@***.***> wrote: yeah i think we need this shim. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

jreback · 2019-07-03T02:34:31Z

@WillAyd I pushed some compat code; let's see.

jorisvandenbossche · 2019-07-03T02:41:22Z

@jreback I think we should do the getattr check on Python 3.7 as I showed above in #27101 (comment).
That actually gives a deprecation warning for the use case that we are keeping a dummy class for (only if you are using py 3.7, but that is at least something).

jreback · 2019-07-03T02:44:02Z

@jreback I think we should do the getattr check on Python 3.7 as I showed above in #27101 (comment).
That actually gives a deprecation warning for the use case that we are keeping a dummy class for (only if you are using py 3.7, but that is at least something).

@jorisvandenbossche ok sure

pep8speaks · 2019-07-03T02:49:48Z

Hello @WillAyd! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

There are currently no PEP 8 issues detected in this Pull Request. Cheers! 🍻

Comment last updated at 2019-07-03 15:44:44 UTC

WillAyd · 2019-07-03T08:21:11Z

Thanks Jeff! Traveling back west today so feel free to move forward on this without me if a blocker. Otherwise can pick up later this evening

jreback · 2019-07-03T11:35:25Z

@TomAugspurger I think this needs a conditional in pandas/tests/api/tests_api.py where if < py37 then add to the class list (deprecated), otherwise its not there. I can do later today unless you want to push a patch.

TomAugspurger · 2019-07-03T11:42:33Z

I can take care of it.

…

On Wed, Jul 3, 2019 at 6:35 AM Jeff Reback ***@***.***> wrote: @TomAugspurger <https://github.com/TomAugspurger> I think this needs a condition in pandas/tests/api/tests_api.py where if < py37 then add to the class list (deprecated), otherwise its not there. I can do later today unless you want to push a patch. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#27101?email_source=notifications&email_token=AAKAOIWNQP4AEA4O7ZSD433P5SFILA5CNFSM4H4HN7FKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODZEFF6A#issuecomment-508056312>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAKAOIRJCVNFZ4CEPNMLVCTP5SFILANCNFSM4H4HN7FA> .

jreback · 2019-07-03T13:19:18Z

doc/source/whatsnew/v0.15.0.rst(734,6): error E999: SyntaxError: invalid syntax
doc/source/whatsnew/v0.11.0.rst(392,12): error E999: SyntaxError: invalid syntax
Bash exited with code '1'.

TomAugspurger · 2019-07-03T14:06:24Z

Fixing now.

TomAugspurger · 2019-07-03T14:11:10Z

pandas/__init__.py

+                "from the top-level namespace will also be removed in "
+                "the next version",
+                FutureWarning, stacklevel=2)
+            return None


Downstream are expecting this to be a type so they can do isinstance(thing, pd.Panel). Will make another dummy class Panel here (not outside the getattr, so that we don't pollute the namespace on py37+).

TomAugspurger · 2019-07-03T16:51:28Z

@jreback the CI failure is unrelated (the categorical ASV we fixed in the other PR).

Should be good to go.

Final panel removal

53abf20

jorisvandenbossche changed the title ~~Final panel removal~~ CLN/DEPR: Final panel removal Jun 28, 2019

jorisvandenbossche added this to the 0.25.0 milestone Jun 28, 2019

jreback added the Panel label Jun 28, 2019

jbrockmendel reviewed Jun 28, 2019

View reviewed changes

pandas/tests/api/test_api.py Outdated Show resolved Hide resolved

jorisvandenbossche mentioned this pull request Jun 28, 2019

Remove remaining usage of pandas Panel pydata/xarray#3057

Closed

shoyer added a commit to shoyer/xarray that referenced this pull request Jun 28, 2019

Ensure xarray imports (and docs build) even without pandas.Panel

e82c12f

It's being removed in the next pandas 0.25 release (see pandas-dev/pandas#27101).

shoyer mentioned this pull request Jun 28, 2019

Ensure xarray imports (and docs build) even without pandas.Panel pydata/xarray#3058

Merged

2 tasks

shoyer added a commit to pydata/xarray that referenced this pull request Jun 30, 2019

Ensure xarray imports (and docs build) even without pandas.Panel (#3058)

76aa4f0

It's being removed in the next pandas 0.25 release (see pandas-dev/pandas#27101).

Merge remote-tracking branch 'upstream/master' into remove-panel

9ec5917

WillAyd force-pushed the remove-panel branch from 603b713 to 9ec5917 Compare July 1, 2019 16:44

Fixed list issue

0d0800b

jreback added 2 commits July 2, 2019 22:28

Merge branch 'master' into PR_TOOL_MERGE_PR_27101

5974f74

compat

cfaad48

test compat

b597d87

use module level getattr to give a nice warning for Panel

d7ff286

flake8

efcd001

TomAugspurger added 2 commits July 3, 2019 06:45

35, 36 compat

cc585ef

whatsnew fixups

5227932

Merge remote-tracking branch 'upstream/master' into WillAyd-remove-panel

7bd3061

TomAugspurger reviewed Jul 3, 2019

View reviewed changes

TomAugspurger added 3 commits July 3, 2019 09:11

downstream compat

38c675d

fixup docs

e43a74d

fixups

ad61f61

jorisvandenbossche approved these changes Jul 3, 2019

View reviewed changes

jorisvandenbossche merged commit 4e185fc into pandas-dev:master Jul 3, 2019

jreback mentioned this pull request Jul 26, 2019

DEPR: deprecations log for removed issues #13777

Closed

WillAyd deleted the remove-panel branch January 16, 2020 00:35

rs2 mentioned this pull request Jan 20, 2021

DOC: Fix inaccurate statement in 1.2.0 release notes #39291

Closed

rs2 added a commit to rs2/pandas that referenced this pull request Jan 21, 2021

Leave pandas-dev#27101 reference due to its relevance

295a32b

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

CLN/DEPR: Final panel removal #27101

CLN/DEPR: Final panel removal #27101

WillAyd commented Jun 28, 2019 •

edited by jreback

Loading

jorisvandenbossche commented Jun 28, 2019

jreback commented Jun 28, 2019

jorisvandenbossche commented Jun 28, 2019

WillAyd commented Jun 28, 2019

jreback commented Jun 28, 2019

jbrockmendel commented Jun 28, 2019

shoyer commented Jun 28, 2019 via email

shoyer commented Jun 28, 2019

jreback commented Jun 28, 2019

jorisvandenbossche commented Jun 30, 2019

jorisvandenbossche commented Jun 30, 2019

jorisvandenbossche commented Jul 1, 2019

WillAyd commented Jul 1, 2019

jorisvandenbossche commented Jul 1, 2019

jbrockmendel commented Jul 1, 2019

TomAugspurger commented Jul 2, 2019

shoyer commented Jul 2, 2019 via email

jreback commented Jul 2, 2019

WillAyd commented Jul 2, 2019 via email

jreback commented Jul 3, 2019

jorisvandenbossche commented Jul 3, 2019 •

edited

Loading

jreback commented Jul 3, 2019

pep8speaks commented Jul 3, 2019 •

edited

Loading

WillAyd commented Jul 3, 2019

jreback commented Jul 3, 2019 •

edited

Loading

TomAugspurger commented Jul 3, 2019 via email

jreback commented Jul 3, 2019

TomAugspurger commented Jul 3, 2019

TomAugspurger Jul 3, 2019

TomAugspurger commented Jul 3, 2019

CLN/DEPR: Final panel removal #27101

CLN/DEPR: Final panel removal #27101

Conversation

WillAyd commented Jun 28, 2019 • edited by jreback Loading

jorisvandenbossche commented Jun 28, 2019

jreback commented Jun 28, 2019

jorisvandenbossche commented Jun 28, 2019

WillAyd commented Jun 28, 2019

jreback commented Jun 28, 2019

jbrockmendel commented Jun 28, 2019

shoyer commented Jun 28, 2019 via email

shoyer commented Jun 28, 2019

jreback commented Jun 28, 2019

jorisvandenbossche commented Jun 30, 2019

jorisvandenbossche commented Jun 30, 2019

jorisvandenbossche commented Jul 1, 2019

WillAyd commented Jul 1, 2019

jorisvandenbossche commented Jul 1, 2019

jbrockmendel commented Jul 1, 2019

TomAugspurger commented Jul 2, 2019

shoyer commented Jul 2, 2019 via email

jreback commented Jul 2, 2019

WillAyd commented Jul 2, 2019 via email

jreback commented Jul 3, 2019

jorisvandenbossche commented Jul 3, 2019 • edited Loading

jreback commented Jul 3, 2019

pep8speaks commented Jul 3, 2019 • edited Loading

Comment last updated at 2019-07-03 15:44:44 UTC

WillAyd commented Jul 3, 2019

jreback commented Jul 3, 2019 • edited Loading

TomAugspurger commented Jul 3, 2019 via email

jreback commented Jul 3, 2019

TomAugspurger commented Jul 3, 2019

TomAugspurger Jul 3, 2019

Choose a reason for hiding this comment

TomAugspurger commented Jul 3, 2019

WillAyd commented Jun 28, 2019 •

edited by jreback

Loading

jorisvandenbossche commented Jul 3, 2019 •

edited

Loading

pep8speaks commented Jul 3, 2019 •

edited

Loading

jreback commented Jul 3, 2019 •

edited

Loading