New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add WW3 Stations Spec reader #92
Conversation
This looks great @mpiannucci thanks for this PR! Minor comments before merging this, shall we also support files specified by strings rather than a file object in case those are available locally? I think it would be nice in fact to set this up more broadly for other file types as well as you suggested in #91. Also, could you please run black in the new code you have added, we use it to ensure all the code reads the same. |
Thanks for the feedback! I will make those changes and then take a stab at making the changes to support fsspec more broadly. |
I took care of the low hanging fruit, but the methods that expect file globs for ascii data are a decent bit harder |
wavespectra/input/__init__.py
Outdated
dset = xr.open_mfdataset( | ||
filename_or_fileglob, chunks=_chunks, combine="by_coords" | ||
) | ||
except ValueError: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I get an IndexError
rather than a ValueError
when trying to open a file-like object with xr.open_mfdataset - do you get ValueError
in wavespectra?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This gives me a ValueError
, saying that it does not know which engine to use because it is just a file like object passed in.
import xarray as xr
with open('./sample_files/swanfile.nc', 'rb') as file:
xr.open_mfdataset(file, chunks={}, combine="by_coords")
If I specify the engine, that seems to give me a ValueError
as well:
with open('./sample_files/swanfile.nc', 'rb') as file:
xr.open_mfdataset(file, chunks={}, combine="by_coords", engine="netcdf4")
gives:
ValueError: can only read bytes or file-like objects with engine='scipy' or 'h5netcdf'
Switching to h5netcdf
also throws a ValueError
for me:
ValueError: can't open netCDF4/HDF5 as bytes try passing a path or file-like object
Which doesnt totally make sense, but throwing an error does. I am unable to get an IndexError
, can you show me the code to reproduce the IndexError
? I'm also happy to just catch any exception, but better to be explicit
I am not sure if the error is messed up between versions of xarray or netcdf, but for now I just changed it to Otherwise I think this is as far as I want to refactor for this PR, it doesnt remove any existing functionality, but adds flexibility to many of the readers. |
* Split open_netcdf into a different function so it can be reused, only raise ValueError and IndexError * New read_ascii_or_binary function to read str or fileobj, change input functions to use these generic openers when possible
This PR adds support for reading WW3 Station Spectral Wave Files. These files are distributed with GFS Wave output by NOAA. An example location of these files is here: https://noaa-gfs-bdp-pds.s3.amazonaws.com/index.html#gfs.20230820/18/wave/station/bulls.t18z/ . They are also available from NOMADS and GCP.
Because these files are available from cloud providers, the reader accepts a file like object instead of a filename to parse out the dataset.
Example usage with
fsspec
: