Skip to content

Commit

Permalink
Update data_tools.onc_json_to_dataset() to ONC API v3 JSON schema a…
Browse files Browse the repository at this point in the history
…nd add tests (#93)

* Update onc_json_to_dataset() to v3 API and add tests

Update the onc_json_to_dataset() function to handle ONC data web service API v3
response JSON schema. Also add unit tests for the function.

* Add "station" dataset attribute from JSON locationCode

Modified onc_json_to_dataset() to include "station" as a dataset attribute from
the `parameters.locationCode` item in the JSON response. This is necessary
because "station" is the key that the `get_onc_ferry` worker expects to find an
element of the metadata it needs under. Updated unit tests to verify that the
"station" attribute exists and has the correct value after conversion.
  • Loading branch information
douglatornell committed Feb 7, 2024
1 parent 9e5fe54 commit d059105
Show file tree
Hide file tree
Showing 2 changed files with 238 additions and 84 deletions.
22 changes: 13 additions & 9 deletions SalishSeaTools/salishsea_tools/data_tools.py
Original file line number Diff line number Diff line change
Expand Up @@ -342,12 +342,12 @@ def onc_json_to_dataset(onc_json, teos=True):
:arg dict onc_json: Data structure returned from an ONC data web service
API request.
Typically produces by calling the :py:meth:`json`
Typically produced by calling the :py:meth:`json`
method on the :py:class:`~requests.Response` object
produced by calling :py:meth:`requests.get`.
:arg boolean teos: Convert salinity data from PSU
(Practical Salinity Units) to TEOS-10 reference
(Practical Salinity Units) to TEOS-10 reference
salinity in g/kg.
Defaults to :py:obj:`True`.
Expand All @@ -357,28 +357,32 @@ def onc_json_to_dataset(onc_json, teos=True):
data_vars = {}
for sensor in onc_json["sensorData"]:
if sensor["sensorName"] == "Practical Salinity" and teos:
data = teos_tools.psu_teos([d["value"] for d in sensor["data"]])
data = teos_tools.psu_teos([d for d in sensor["data"]["values"]])
sensor["sensorName"] = "Reference Salinity"
sensor["unitOfMeasure"] = "g/kg"
else:
data = [d["value"] for d in sensor["data"]]
data_vars[sensor["sensor"]] = xarray.DataArray(
name=sensor["sensor"],
data = [d for d in sensor["data"]["values"]]
sensor_code = sensor["sensorCode"].lower()
data_vars[sensor_code] = xarray.DataArray(
name=sensor_code,
data=data,
coords={
"sampleTime": [
arrow.get(d["sampleTime"]).naive for d in sensor["data"]
arrow.get(d).naive for d in sensor["data"]["sampleTimes"]
],
},
dims=("sampleTime",),
attrs={
"qaqcFlag": np.array([d["qaqcFlag"] for d in sensor["data"]]),
"qaqcFlag": np.array([d for d in sensor["data"]["qaqcFlags"]]),
"sensorName": sensor["sensorName"],
"unitOfMeasure": sensor["unitOfMeasure"],
"actualSamples": sensor["actualSamples"],
},
)
return xarray.Dataset(data_vars, attrs=onc_json["serviceMetadata"])
dataset_attrs = {
"station": onc_json["parameters"]["locationCode"]
}
return xarray.Dataset(data_vars, attrs=dataset_attrs)


def get_chs_tides(
Expand Down

0 comments on commit d059105

Please sign in to comment.