-
-
Notifications
You must be signed in to change notification settings - Fork 17.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
BUG: Index.get_indexer with mixed-reso datetime64s #50690
Comments
The correct place is to fix the issue is the |
Agreed. Unfortunately numpy/numpy#3836 doesn't seem to be a high priority for numpy.
|
@jbrockmendel you probably mean PyDatetimeScalarObject (https://github.com/numpy/numpy/blob/548bc6826b597ab79b9c1451b79ec8d23db9d444/numpy/core/include/numpy/arrayscalars.h#L119-L123)? Really, it doesn't look too bad for performance (I wrongly assumed we would need to go via |
I wish. |
@jbrockmendel Oh, you are correct. I guess we could do something for "sane" cases.
As you can see the objects represent quite different time, but are equal (probably due to an overflow in the compare-operator)... to ensure, that the hash in this case is the same would be quite a challenge. A simply idea would to transform all times in attosecond ('as'), which will be in range |
I'd want to do roughly the same thing we did in
|
@WillAyd im trying to implement this in khash_python.h and struggling to make Thoughts? |
Given your issue is with If that doesn't work feel free to push up a PR and can take a look. Also might help if you post the gcc command that setuptools is generating. |
bingo, thanks. |
IIUC the issue is in the hashing of the datetime64 objects, which do not follow the invariance
x == y \Rightarrow hash(x) == hash(y)
(xref numpy/numpy#3836)When implementing non-nanosecond support for Timestamp/Timedelta, we implemented
__hash__
to retain this invariance (at the cost of performance).Unless numpy changes its behavior, I think to fix this we need to patch how we treat datetime64 objects in our khash code, likely mirroring
Timestamp.__hash__
. cc @realead thoughts?The text was updated successfully, but these errors were encountered: