Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DEPR: be stricter in assert_almost_equal #52081

Merged
merged 18 commits into from
May 24, 2023

Conversation

jbrockmendel
Copy link
Member

  • closes #xxxx (Replace xxxx with the GitHub issue number)
  • Tests added and passed if fixing a bug or adding a new feature
  • All code checks passed.
  • Added type annotations to new arguments/methods/functions.
  • Added an entry in the latest doc/source/whatsnew/vX.X.X.rst file if fixing a bug or adding a new feature.

xref #18463 this doesn't get us all the way there since assert_index_equal and assert_numpy_array_equal go through different paths.

Note the many tests that needed to be changed; some of these likely represent undesired behaviors.

@phofl i found a usecase for downcast=False in fillna!

@mroeschke mroeschke added Testing pandas testing functions or related to the test suite Deprecate Functionality to remove in pandas labels Mar 20, 2023
elif checknull(b):
# GH#18463
warnings.warn(
f"Mismatched null-like values {a} and {b} found. In a future "
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need a whatsnew entry for this change since the testing functions are public?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea, will update

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated+green

@mroeschke mroeschke added this to the 2.1 milestone May 24, 2023
@mroeschke mroeschke merged commit 75daea4 into pandas-dev:main May 24, 2023
@mroeschke
Copy link
Member

Thanks @jbrockmendel

@jbrockmendel jbrockmendel deleted the bug-assert_almost_equal-nas branch May 24, 2023 20:54
topper-123 pushed a commit to topper-123/pandas that referenced this pull request May 27, 2023
* DEPR: be stricter in assert_almost_equal

* 32bit builds

* Fix transform test

* ignore warning i cant reproduce localy

* pylint fixup

* Fix AarrayManager and CoW builds

* fix tests

* Whatsnew

---------

Co-authored-by: Matthew Roeschke <10647082+mroeschke@users.noreply.github.com>
topper-123 pushed a commit to topper-123/pandas that referenced this pull request Jun 5, 2023
* DEPR: be stricter in assert_almost_equal

* 32bit builds

* Fix transform test

* ignore warning i cant reproduce localy

* pylint fixup

* Fix AarrayManager and CoW builds

* fix tests

* Whatsnew

---------

Co-authored-by: Matthew Roeschke <10647082+mroeschke@users.noreply.github.com>
@jorisvandenbossche
Copy link
Member

This turns out to be a bit annoying while updating pyarrow tests for certain cases. For example, we have tests where we check that in the input you can have mixed NA-likes values. To assert the result, with this PR we need to construct the expected data with a consistent NA value. For small data it's typically straightforward to redefine it correctly. For larger data, that's a bit more annoying though.

One thing that also doesn't help is that we don't allow filling NAs with None (pyarrow uses None for object dtype arrays when converting to numpy/pandas): df.fillna(None) errors.

Should we also consider adding a keyword for relaxing this, so you can override the strict default for cases you don't care about the difference? (which was also suggested in #18463)

Daquisu pushed a commit to Daquisu/pandas that referenced this pull request Jul 8, 2023
* DEPR: be stricter in assert_almost_equal

* 32bit builds

* Fix transform test

* ignore warning i cant reproduce localy

* pylint fixup

* Fix AarrayManager and CoW builds

* fix tests

* Whatsnew

---------

Co-authored-by: Matthew Roeschke <10647082+mroeschke@users.noreply.github.com>
@CaselIT
Copy link

CaselIT commented Dec 6, 2023

Any chance of getting a flag in assert_X_equal() to restore the old behaviour? It's very useful when using check_dtype=False.

Or maybe it could be connected to check_dtype (restoring the old behaviour when False)?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Deprecate Functionality to remove in pandas Testing pandas testing functions or related to the test suite
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants