-
Notifications
You must be signed in to change notification settings - Fork 3
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Improve reading model output #194
Conversation
- add kwarg delayed, if False, load data into memory, else return data array with delayed dask arrays - add kwarg chunked, if True, chunk data array with chunks="auto" - add x,y data to vertex grid data array
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good job!
I was thinking if we should make this functions cache-able using the cache_netcdf
decorator but I think it is too hard (and to little gains) because the cache_netcdf
requires a dataset as input and for this function it is optional. However if these functions do become annoyingly slow we could evaluate again because it is certainly possible to use the cache here.
- fix comments - add docstrings
- fix where command for dry/noflow - improve support for loading output without gwf or ds
- new folder mfoutput - add new flopy binary read functions that support multithreading (binaryfile.py) - separate reading budget and head output - modify gwf.output and gwt.output to use new methods - split logic into multiple reusable functions - add support for grb files - add/improve tests - add method to obtain dims, coords from modelgrid object
Alright, the idea is still the same but I refactored the code significantly to increase readability and simplify things. The idea now is that the binary output file (
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
see my comments
- codacy - fix comments @OnnoEbbens - some additional fixes
Use dask.delayed to load model output. This avoids reading all data into memory if
delayed=True
. Optionally chunk data array for doing memory efficient calculations on large data arrays.Adds two kwargs to output methods:
delayed
, if True, do not load data into memory, default is Falsechunked
, if True, chunk data array usingda.chunk("auto")
, default is FalseDefault behavior is same as before, for memory intensive output, use
delayed=True
, and optionallychunked=True
, e.g.: