Do sql joins work?

[ahuang11]

New to this library, but curious if joins work? I don't see mentions of it, except #5

[alxmrs]

Do they work? Yes and no. Yes, in the sense that the feature is implemented, taken care of by DataFusion. No, they do not work, in the sense that the xarray.Dataset loading portion is not (yet) sufficiently lazy to be performant.

If your data is relatively small (say, can fit in memory) then joins work today.

[ahuang11]

Thanks! Do you think it'll be feasibly efficient in the future?

[alxmrs]

I am investigating a few separate paths to make this so. I have a lot of faith in @maximedion2's https://github.com/datafusion-contrib/arrow-zarr as a fallback option for Zarr stores. There are other approaches on the Python side that seem promising, but may not work.