Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Invalid conversion of generic time coordinates #10071

Closed
4 of 5 tasks
Dave-Allured opened this issue Feb 22, 2025 · 4 comments · Fixed by #10072
Closed
4 of 5 tasks

Invalid conversion of generic time coordinates #10071

Dave-Allured opened this issue Feb 22, 2025 · 4 comments · Fixed by #10072
Labels
bug

Comments

@Dave-Allured
Copy link

What happened?

Using xarray.open_dataset. Trying to read a Netcdf dataset with generic time coordinates, not CF, with time:units = "seconds". Trying to apply the conversion "time_unit='us'" as directed by a warning message.

  • Got an invalid "SerializationWarning" message.
  • Argument "time_unit='us'" seems to be ignored. Time is converted to nanosecs rather than microsecs.

What did you expect to happen?

  • After input, time coordinates should be microsecs rather than nanosecs.
  • Clean execution with no warning message.

Minimal Complete Verifiable Example

#!/usr/bin/env python3
import xarray as xr
coder = xr.coders.CFDatetimeCoder (use_cftime=False, time_unit='us')
IN = xr.open_dataset ('xtime.nc', decode_timedelta=coder)
print (IN)

# See below for source code to generate the simple "xtime.nc" test file.

MVCE confirmation

  • Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray.
  • Complete example — the example is self-contained, including all data and the text of any traceback.
  • Verifiable example — the example copy & pastes into an IPython prompt or Binder notebook, returning the result.
  • New issue — a search of GitHub Issues suggests this is not a duplicate.
  • Recent environment — the issue occurs with the latest version of xarray and its dependencies.

Relevant log output

(test_env) > ./demo.time.2.py
/home/xxxxxx/demo.time.2.py:4: SerializationWarning: Can't decode floating point timedelta to 's' without precision loss, decoding to 'us' instead. To silence this warning use time_unit='us' in call to decoding function.
  IN = xr.open_dataset ('xtime.nc', decode_timedelta=coder)
<xarray.Dataset> Size: 40B
Dimensions:  (time: 5)
Coordinates:
  * time     (time) timedelta64[ns] 40B 00:00:48.561388 ... 00:01:47.403320
Data variables:
    *empty*

Anything else we need to know?

Source code for xtime.nc:

netcdf xtime {
dimensions:
	time = 5 ;
variables:
	float time(time) ;
		time:units = "seconds" ;
data:
 time = 48.56139, 63.27169, 77.98219, 92.6929, 107.4033 ;
}

Generate the actual test file using this command:

ncgen -o xtime.nc xtime.txt

Environment

INSTALLED VERSIONS ------------------ commit: None python: 3.13.1 | packaged by conda-forge | (main, Jan 13 2025, 09:53:10) [GCC 13.3.0] python-bits: 64 OS: Linux OS-release: 4.18.0-553.36.1.el8_10.x86_64 machine: x86_64 processor: x86_64 byteorder: little LC_ALL: None LANG: C LOCALE: ('C', 'UTF-8') libhdf5: 1.14.4 libnetcdf: 4.9.2 xarray: 2025.1.2 pandas: 2.2.3 numpy: 2.2.2 scipy: 1.15.1 netCDF4: 1.7.2 pydap: None h5netcdf: None h5py: None zarr: None cftime: 1.6.4 nc_time_axis: None iris: None bottleneck: None dask: 2025.1.0 distributed: 2025.1.0 matplotlib: 3.10.0 cartopy: 0.24.0 seaborn: None numbagg: None fsspec: 2025.2.0 cupy: None pint: 0.24.4 sparse: None flox: None numpy_groupies: None setuptools: None pip: 25.0 conda: None pytest: None mypy: None IPython: None sphinx: None
@Dave-Allured Dave-Allured added bug needs triage Issue that has not been reviewed by xarray team member labels Feb 22, 2025
@welcome Welcome
Copy link

welcome bot commented Feb 22, 2025

Thanks for opening your first issue here at xarray! Be sure to follow the issue template!
If you have an idea for a solution, we would really welcome a Pull Request with proposed changes.
See the Contributing Guide for more.
It may take us a while to respond here, but we really value your contribution. Contributors like you help make xarray better.
Thank you!

@spencerkclark
Copy link
Member

spencerkclark commented Feb 22, 2025 •
edited
Loading

Thanks for raising this @Dave-Allured. That confusing SerializationWarning should not be emitted in the first place, since we still decode to nanosecond-resolution timedeltas by default. We should fix that.

However, if you do want microsecond-resolution timedeltas, be sure to pass a CFTimedeltaCoder to decode_timedelta instead of a CFDatetimeCoder:

coder = xr.coders.CFTimedeltaCoder(time_unit="us")
ds = xr.open_dataset("xtime.nc", decode_timedelta=coder)

which works as expected for me.

@spencerkclark spencerkclark removed the needs triage Issue that has not been reviewed by xarray team member label Feb 22, 2025
@Dave-Allured
Copy link
Author

@spencerkclark Thanks for explaining. As novice to Xarray, I was not aware of CFTimedeltaCoder. With better understanding, I think I will just omit the coder and go with decode_timedelta=True and default nanoseconds. That seems to be working for me now.

Please fix that invalid SerializationWarning. Otherwise this issue is resolved for me. Feel free to leave open this issue as a reminder, if you like.

@spencerkclark
Copy link
Member

It's new to experienced users too! We have only recently enabled the use of non-nanosecond timedeltas and datetimes, so there are obviously still some kinks to sort out—thanks for your patience / understanding. It will be good to get this false warning issue fixed.

@spencerkclark spencerkclark mentioned this issue Feb 23, 2025
Merged
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix false timedelta decoding SerializationWarning and improve warning message spencerkclark/xarray
2 participants
@spencerkclark @Dave-Allured