Resampling followed by to_array().values

[tjvandal]

I'm having an issue accessing data as an array after resampling. Before resampling there are no problems using to_array().values but after resampling this same method takes 10+ minutes to complete. The dataset isn't large with dimensions (shape=(1, 672, 180, 360)). I've replicated this with another dataset. I've also tried copying the dataset to another variable but that didn't work either. Does anybody have any thoughts?

cmip5_file = "/Users/tj/data/cmip5/access1-0/*.nc"

# climate model data
cmip5 = xray.open_mfdataset(cmip5_file)
cmip5.time = pandas.to_datetime(cmip5.  time.values)
print cmip5.to_array().values   ## accesses the data just fine

resampled = cmip5.resample('MS', 'time', how='mean')
print resampled.to_array().values  ## this takes 10+ minutes

[shoyer]

When you use open_mfdataset, the data is loaded with dask and all computation is lazy. That means that the resampling doesn't happen until you actually look at the data or call .values on the line below.

If the data is small enough that it fits into memory, one thing you could do is to call .load() before doing the resample. Then the computation will be done on all NumPy arrays, which can be faster in some circumstances.