DOC: Deprecate null_counts parameter of DataFrame.info #37999

hongshaoyang · 2020-11-22T10:15:24Z

closes DOC: unclear kwarg null_counts #36805
tests added / passed
passes black pandas
passes git diff upstream/master -u -- "*.py" | flake8 --diff

Background.

Might be worth it actually - I think people are only likely to use .info in an interactive session, rather than as part of production code, so perhaps it is worth going through the deprecation process to make this argument clearer

xref DOC: unclear kwarg null_counts #36805 (comment)
Deprecate null_counts parameter of DataFrame.info in favor of show_counts and add warning.
Add whatsnew and add tests to test for FutureWarning. Move usages of null_counts in tests to show_counts.

Doc preview:

arw2019

Thanks @hongshaoyang for the PR!

We'll want to catch this warning in any tests that hit this.

We'll also need a whatsnew

hongshaoyang · 2020-11-23T06:12:25Z

Thanks @arw2019 , I added tests and a whatsnew entry.

jreback · 2020-11-24T03:11:30Z

looks fine, @jorisvandenbossche ok here?

jreback · 2020-11-24T14:11:38Z

just merged some changes that will affect this PR, can you merge master and update: #37868

# Conflicts: # pandas/io/formats/info.py

ivanovmg · 2020-11-24T18:11:59Z

pandas/tests/io/formats/test_format.py

@@ -194,6 +194,11 @@ def check(null_counts, result):
            check(True, False)
            check(False, False)

+        with tm.assert_produces_warning(FutureWarning):  # GH37999


tm.assert_produces_warning now has an optional match kwarg to match the warning message.
Could use it here to ensure that the proper warning message is issues.

ivanovmg · 2020-11-24T18:21:33Z

pandas/core/frame.py

+        if null_counts is not None and show_counts is None:
+            warnings.warn(
+                "null_counts is deprecated. Use show_counts instead",
+                FutureWarning,
+                stacklevel=2,
+            )


The user will not get this warning when setting both arguments at the same time.
For example, df.info(null_counts = True, show_counts=False).
This type of invocation is incorrect and should raise an error.

ivanovmg

One minor comment. Otherwise good.

ivanovmg · 2020-11-25T07:20:33Z

pandas/core/frame.py

        null_counts: Optional[bool] = None,
    ) -> None:
+        if null_counts is not None:
+            if show_counts is not None:
+                raise ValueError("null_counts used with show_counts")


I suggest adding Use show_counts.
Otherwise looks good to me.

jorisvandenbossche · 2020-11-25T09:35:12Z

doc/source/whatsnew/v1.2.0.rst

@@ -480,6 +480,7 @@ Deprecations
 - The ``how`` keyword in :meth:`PeriodIndex.astype` is deprecated and will be removed in a future version, use ``index.to_timestamp(how=how)`` instead (:issue:`37982`)
 - Deprecated :meth:`Index.asi8` for :class:`Index` subclasses other than :class:`DatetimeIndex`, :class:`TimedeltaIndex`, and :class:`PeriodIndex` (:issue:`37877`)
 - The ``inplace`` parameter of :meth:`Categorical.remove_unused_categories` is deprecated and will be removed in a future version (:issue:`37643`)
+- The ``null_counts`` parameter of :meth:`DataFrame.info` is deprecated and will be removed in a future version (:issue:`37999`)


Can you indicate here that the keyword is renamed to show_counts ?

There's an unrelated test on Travis that fails intermittently

FAILED pandas/tests/plotting/frame/test_frame.py::TestDataFramePlots::test_area_lim

jreback · 2020-11-25T20:59:30Z

thanks @hongshaoyang very nice

jbrockmendel · 2022-10-29T01:46:03Z

@hongshaoyang want to make a PR enforcing this deprecation?

### What changes were proposed in this pull request? Remove `null_counts` from info() ### Why are the changes needed? Pandas 2.0 _Removed deprecated null_counts argument in [DataFrame.info()](https://pandas.pydata.org/pandas-docs/version/2.0/reference/api/pandas.DataFrame.info.html#pandas.DataFrame.info). Use show_counts instead ([GH37999](pandas-dev/pandas#37999 ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Tested local ### Before this PR `F05.info()` ``` TypeError Traceback (most recent call last) Cell In[12], line 1 ----> 1 F05.info() File /opt/spark/python/pyspark/pandas/frame.py:12167, in DataFrame.info(self, verbose, buf, max_cols, null_counts) 12163 count_func = self.count 12164 self.count = ( # type: ignore[assignment] 12165 lambda: count_func()._to_pandas() # type: ignore[assignment, misc, union-attr] 12166 ) > 12167 return pd.DataFrame.info( 12168 self, # type: ignore[arg-type] 12169 verbose=verbose, 12170 buf=buf, 12171 max_cols=max_cols, 12172 memory_usage=False, 12173 null_counts=null_counts, 12174 ) 12175 finally: 12176 del self._data TypeError: DataFrame.info() got an unexpected keyword argument 'null_counts' ``` ### With this PR `F05.info()` ``` <class 'pyspark.pandas.frame.DataFrame'> Int64Index: 5257 entries, 0 to 5256 Data columns (total 203 columns): # Column Non-Null Count Dtype --- ------ -------------- ----- 0 DOFFIN_APPENDIX:EXPRESSION_OF_INTEREST_URL 471 non-null object (...) ``` Closes #40913 from bjornjorgensen/remove-null_counts. Authored-by: bjornjorgensen <bjornjorgensen@gmail.com> Signed-off-by: Hyukjin Kwon <gurwls223@apache.org>

DOC: Deprecate null_counts

ef5d8f6

arw2019 reviewed Nov 22, 2020

View reviewed changes

Add test and whatsnew

f1aafa0

jreback added the Deprecate Functionality to remove in pandas label Nov 24, 2020

jreback added this to the 1.2 milestone Nov 24, 2020

jreback mentioned this pull request Nov 24, 2020

DEPR: log of deprecations in 1.x (to be removed in 2.0) #30228

Closed

hongshaoyang added 2 commits November 24, 2020 22:40

Merge branch 'master' into 36805-deprecate-nullcounts

5a253a5

# Conflicts: # pandas/io/formats/info.py

Fix merge

d647cd7

hongshaoyang mentioned this pull request Nov 24, 2020

ENH: enable Series.info() #37320

Merged

5 tasks

ivanovmg reviewed Nov 24, 2020

View reviewed changes

Add tests for bad combination of null_counts + show_counts

09f8438

ivanovmg approved these changes Nov 25, 2020

View reviewed changes

jorisvandenbossche reviewed Nov 25, 2020

View reviewed changes

Clarify whatsnew

aa0c57f

jreback merged commit 08913f8 into pandas-dev:master Nov 25, 2020

hongshaoyang deleted the 36805-deprecate-nullcounts branch November 26, 2020 00:05

mroeschke mentioned this pull request Oct 31, 2022

DEPR: Remove df.info(null_counts=) #49430

Merged

bjornjorgensen mentioned this pull request Apr 22, 2023

[SPARK-43239][PS] Remove null_counts from info() apache/spark#40913

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

DOC: Deprecate null_counts parameter of DataFrame.info #37999

DOC: Deprecate null_counts parameter of DataFrame.info #37999

hongshaoyang commented Nov 22, 2020 •

edited

arw2019 left a comment

hongshaoyang commented Nov 23, 2020

jreback commented Nov 24, 2020

jreback commented Nov 24, 2020

ivanovmg Nov 24, 2020

hongshaoyang Nov 25, 2020

ivanovmg Nov 24, 2020

hongshaoyang Nov 25, 2020

ivanovmg left a comment

ivanovmg Nov 25, 2020

jorisvandenbossche Nov 25, 2020

hongshaoyang Nov 25, 2020

hongshaoyang Nov 25, 2020

jreback commented Nov 25, 2020

jbrockmendel commented Oct 29, 2022

DOC: Deprecate null_counts parameter of DataFrame.info #37999

DOC: Deprecate null_counts parameter of DataFrame.info #37999

Conversation

hongshaoyang commented Nov 22, 2020 • edited

Background.

arw2019 left a comment

Choose a reason for hiding this comment

hongshaoyang commented Nov 23, 2020

jreback commented Nov 24, 2020

jreback commented Nov 24, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ivanovmg left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jreback commented Nov 25, 2020

jbrockmendel commented Oct 29, 2022

hongshaoyang commented Nov 22, 2020 •

edited