Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Default fill value of -9999 is incompatible with numpy 2.0 after integration of NEP 50 #98

Open
lukecampbell opened this issue Jun 21, 2024 · 1 comment

Comments

@lukecampbell
Copy link

https://numpy.org/neps/nep-0050-scalar-promotion.html#nep50

fill_value=series.dtype.type(CFDataset.default_fill_value),

default_fill_value = -9999.9

With the release of numpy 2.0 adopting NEP 50 by default, type promotion has changed and any attempts to coerce python scalar values into data types that can't represent that value now raise an OverflowError.

Before, if you had a dtype of uint8 the fill value would wind up being 241 after overflowing.

I recommend creating a more sophisticated mapping of appropriate fill values based on dtypes:

  • uint8 - 255
  • int8 - -128
  • ...
  • float32 - nan
  • float64 - nan
  • datetime64 - nan (depends on context... numpy supported contexts are NaT, float-only contexts would be NaN)

I'll try to take a look at a PR if I can find the time.

@ocefpaf
Copy link
Member

ocefpaf commented Jun 21, 2024

You can use np._set_promotion_state("legacy") for now until this is properly fixed, in case you have broken workflows.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants