Support from reading unformatted Fortran files

[tomchor]

I was wondering if there is interest in developing support for reading unformatted binary files into DataArrays/Datasets.

I have been using a couple of models for a while that output unformatted Fortran binary files that I have to read into DataArrays and Datasets. I have developed my own custom functions to do this and they work quite well for my purposes, but I was wondering is the community thinks this is something that would be useful for more people. I know the syntax could be a bit cumbersome, but I was thinking it could be implemented like

da = xr.DataArray.from_binary(filename, access="direct", dtype=np.float64, coords=coords, dims=dims,)

From there we could figure out the counts to read, etc. If there's interest in this feature we could discuss it and I'd be happy to implement.

Cheers

[darothen]

Hi @tomchor, it's not too difficult to take the readers that you already have and two wrap them in such a way that you can interact with them via xarray; you can check out the packages xgcm or xbpch for examples of this can work in practice. I'm not sure if a more generic reader is within or beyond the scope of the core xarray project, though... although example implementations and writeups would make a great contribution to the community!

[dcherian]

example implementations and writeups would make a great contribution to the community!

+1. Maybe put up your example implementation on github/gitlab, and we can link to it from the documentation?

[tomchor]

@dcherian That's a good idea. At the moment my implementation is very problem-specific so it's not good to upload that. Generalizing it is very straightforward though (albeit a bit of work), but I'll try to do that soon. I wonder what the best place for that in the docs would be.

[dcherian]

I think "Reading and writing files" would be most appropriate 😃
https://xarray.pydata.org/en/stable/io.html

Adding a new section on "binary fortran files" and linking to xgcm, xbpch and your code would be great.

[tomchor]

Wrote a simple function here.
I'm not sure what the best to link this to the docs is, since I can't guarantee that this function will change in the future. Maybe link the commit? Or maybe just straight up pasting that into the docs with a small example script using it?

[sjvrijn]

@tomchor For small snippets including it directly into the docs seems best to me, but an explicit link to the commit/file seems fine too. I've seen links to github issues and blog posts in the docs, so linking to a commit for a larger piece of code doesn't seem out of the ordinary to me.