Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BUG: datetime64 hash. #14622

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Conversation

walshb
Copy link
Contributor

@walshb walshb commented Oct 1, 2019

@eric-wieser
Copy link
Member

Can you add a test for this comment?

@eric-wieser
Copy link
Member

Sorry for the conflict, should be straightforward to resolve

*/
NPY_NO_EXPORT npy_int64
datetime_hash(PyArray_DatetimeMetaData *meta, npy_datetime dt)
{
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this handle NaT?

@walshb
Copy link
Contributor Author

walshb commented Oct 19, 2019

That requirement for hash(datetime.datetime) == hash(np.datetime64) meant lots of changes.
EDIT: (for #3836 (comment))

@eric-wieser
Copy link
Member

Indeed it does. It might be worth discussing in that issue whether we care.

I suppose we could merge this without resolving that, but I don't think we can consider this completely fixed until we address that too.

@eric-wieser
Copy link
Member

Oh, I see you already made those changes - thanks!

Copy link
Member

@eric-wieser eric-wieser left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This patch is looking pretty good, thanks! Only small comments.

@eric-wieser
Copy link
Member

eric-wieser commented Oct 20, 2019

Some of the CI failures look real. @mattip, any idea what's up with PyPy?

@walshb walshb force-pushed the bw-datetime-hash-3836 branch 2 times, most recently from 8f24882 to 16fdbcb Compare October 20, 2019 07:04

if (meta->base == NPY_FR_GENERIC) {
/* XXX generic compares equal to *every* other base, so no single hash works. */
return (td == -1) ? -2 : td; /* avoid -1 (error) */
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the path I am suggesting should raise.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah. Yes. I agree, but that breaks another test, "test_correct_hash_dict" (in test_regression.py). So I left it as a XXX comment for now.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that test is just not written well enough - it should be comparing hash(x) and x.__hash__(), which it had a close enough approximation to until this change

@walshb
Copy link
Contributor Author

walshb commented Oct 20, 2019

Re: CL failures, it seems to be some funny business with pypy. In pypy:

np.timedelta64 == datetime.timedelta, but datetime.timedelta != np.timedelta64.

Very strange. This doesn't affect np.datetime64 in pypy.

@mattip
Copy link
Member

mattip commented Oct 20, 2019

PyPy uses the pure-python implementation of datetime, but it might have some issues. Let me take a look

@mattip
Copy link
Member

mattip commented Oct 20, 2019

The timedelta._eq issue was fixed for CPython 3.7, but not backported. PyPy is backporting that now.

@walshb
Copy link
Contributor Author

walshb commented Oct 20, 2019

@mattip Cool. Thanks!

@Qiyu8
Copy link
Member

Qiyu8 commented May 26, 2020

@walshb Are you still working on this? I think this is a good PR, just need some conflicting fix.

@walshb
Copy link
Contributor Author

walshb commented May 26, 2020

Travis problem, before the build started: "Unable to download 3.7 archive. The archive may not exist. Please consider a different version.".

@walshb
Copy link
Contributor Author

walshb commented May 27, 2020

@Qiyu8 I rebased onto master, so there are no conflicts. But the build failed due to a missing Python interpreter for the s390x architecture on Travis.
This error seems to happen occasionally. For example, here it happens on numpy master:
https://travis-ci.org/github/numpy/numpy/jobs/691548492
What can we do? Is it possible to kick off the Travis build just to confirm my changes are ok? Do you know if anyone is working on the Travis s390x problem? Thanks.

@rossbar
Copy link
Contributor

rossbar commented May 27, 2020

The CI problem you mentioned is a known issue (#16186) and not related to the changes in the PR, so nothing specifically needs to be done here.

Base automatically changed from master to main March 4, 2021 02:04
@mattip mattip mentioned this pull request Aug 22, 2021
@mattip
Copy link
Member

mattip commented Apr 21, 2022

@walshb where does this PR stand?

@charris
Copy link
Member

charris commented Jun 9, 2022

@mattip Seems this is ready for final review.

@walshb
Copy link
Contributor Author

walshb commented Jun 9, 2022

@mattip Seems this is ready for final review.

Sorry for this late reply -- I've been trying to find some time to make one more change.
I'm not happy with the way that I hash the npy_datetimestruct via PyBytes_FromStringAndSize in some cases.
Please can I have a few more days to make this nicer?
Thanks.

@InessaPawson InessaPawson added the 52 - Inactive Pending author response label Apr 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
00 - Bug 52 - Inactive Pending author response component: numpy.datetime64 (and timedelta64)
Projects
Status: Pending author's response
Development

Successfully merging this pull request may close these issues.

None yet

7 participants