Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ENH: xarray grid output #1477

Open
wants to merge 7 commits into
base: main
Choose a base branch
from
Open

ENH: xarray grid output #1477

Changes from 5 commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
3acf03c
ENH: xarray grid compatibility
syedhamidali Oct 19, 2023
8ad98a9
ENH: xarray grid output
syedhamidali Oct 20, 2023
6f8a260
adding back xarray availablity check
syedhamidali Oct 20, 2023
116a31c
pre-commit run
syedhamidali Oct 20, 2023
2f73ad0
fix minor error in grid_to_xarary
syedhamidali Oct 20, 2023
461de4e
Merge branch 'ARM-DOE:main' into master
syedhamidali Feb 29, 2024
defa65e
Merge branch 'main' into master
zssherman May 9, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
121 changes: 93 additions & 28 deletions pyart/core/grid.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -326,47 +326,112 @@ def to_xarray(self):
y = self.y["data"]
x = self.x["data"]

time = np.array([num2date(self.time["data"][0], self.time["units"])])
time = np.array(
[num2date(self.time["data"][0], units=self.time["units"])],
)

ds = xarray.Dataset()
for field in list(self.fields.keys()):
field_data = self.fields[field]["data"]
for field, field_info in self.fields.items():
field_data = field_info["data"]
data = xarray.DataArray(
np.ma.expand_dims(field_data, 0),
dims=("time", "z", "y", "x"),
coords={
"time": (["time"], time),
"z": (["z"], z),
"time": time,
"z": z,
"lat": (["y", "x"], lat),
"lon": (["y", "x"], lon),
"y": (["y"], y),
"x": (["x"], x),
"y": y,
"x": x,
},
)
for meta in list(self.fields[field].keys()):

for meta, value in field_info.items():
if meta != "data":
data.attrs.update({meta: self.fields[field][meta]})
data.attrs.update({meta: value})

ds[field] = data
ds.lon.attrs = [
("long_name", "longitude of grid cell center"),
("units", "degree_E"),
("standard_name", "Longitude"),
]
ds.lat.attrs = [
("long_name", "latitude of grid cell center"),
("units", "degree_N"),
("standard_name", "Latitude"),
]

ds.z.attrs = get_metadata("z")
ds.y.attrs = get_metadata("y")
ds.x.attrs = get_metadata("x")

ds.z.encoding["_FillValue"] = None
ds.lat.encoding["_FillValue"] = None
ds.lon.encoding["_FillValue"] = None
ds.close()

ds.lon.attrs = [
("long_name", "longitude of grid cell center"),
("units", "degree_E"),
("standard_name", "Longitude"),
]
ds.lat.attrs = [
("long_name", "latitude of grid cell center"),
("units", "degree_N"),
("standard_name", "Latitude"),
]

ds.z.attrs = get_metadata("z")
ds.y.attrs = get_metadata("y")
ds.x.attrs = get_metadata("x")

for attr in [ds.z, ds.lat, ds.lon]:
attr.encoding["_FillValue"] = None

# Delayed import
from ..io.grid_io import _make_coordinatesystem_dict

ds["ProjectionCoordinateSystem"] = xarray.DataArray(
data=np.array(1, dtype="int32"),
attrs=_make_coordinatesystem_dict(self),
)

# write the projection dictionary as a scalar
projection = self.projection.copy()
# NetCDF does not support boolean attribute, covert to string
if "_include_lon_0_lat_0" in projection:
include = projection["_include_lon_0_lat_0"]
projection["_include_lon_0_lat_0"] = ["false", "true"][include]
ds["projection"] = xarray.DataArray(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this variable, along with the other origin_latitude, etc. variables belong as coordinates - can you add those to the list of coordinates? I think that is what is causing the failure in CI here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mgrover1 My apologies for the late response; I've been tied up with exams recently and have another busy week ahead. I'll get to it as soon as possible. Thanks for your suggestion.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No worries! Take your time - just let me know when you are ready for another review!

data=np.array(1, dtype="int32"),
dims=None,
attrs=projection,
)

for attr_name in [
"origin_latitude",
"origin_longitude",
"origin_altitude",
"radar_altitude",
"radar_latitude",
"radar_longitude",
"radar_time",
]:
if hasattr(self, attr_name):
attr_data = getattr(self, attr_name)
if attr_data is not None:
if "radar_time" not in attr_name:
attr_value = np.ma.expand_dims(attr_data["data"][0], 0)
else:
attr_value = [
np.array(
num2date(
attr_data["data"][0],
units=attr_data["units"],
),
dtype="datetime64[ns]",
)
]
dims = ("nradar",)
ds[attr_name] = xarray.DataArray(
attr_value, dims=dims, attrs=get_metadata(attr_name)
)

if "radar_time" in ds.variables:
ds.radar_time.attrs.pop("calendar")

if self.radar_name is not None:
radar_name = self.radar_name["data"]
ds["radar_name"] = xarray.DataArray(
np.array([b"".join(radar_name)]),
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the reason for using a byte-encoded name here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because some radar data use the following convention:

print(grid.radar_name)
{'long_name': 'Name of radar used to make the grid',
 'data': masked_array(data=[[b'D', b'O', b'W', b'8']],
              mask=False,
        fill_value=b'N/A',
             dtype='|S1')}

To make it look like b'DOW8, because if you read a pyart or radx generated grid using xarray, it shows up like array([b'DOW8'], dtype='|S4'). That's why, I am using byte-encoded name so the result looks like

ds["radar_name"] = xarray.DataArray(
                np.array([b"".join(grid.radar_name)])
print(ds["radar_name"])
array([b'DOW8'], dtype='|S4')

dims=("nradar"),
attrs=get_metadata("radar_name"),
)

ds.attrs = self.metadata
ds.close()
return ds

def add_field(self, field_name, field_dict, replace_existing=False):
Expand Down