generating/evaluating coordinates only when required

[telegraphic]

Hi all, I have a data array with a very long axis, and would like to avoid having to pre-compute coordinates to attach. Are there any ways I can attach a simple function to generate coordinates?

The coordinates are all equally spaced, so can be generated easily from a start value, step size, and number of steps:

start_val = 1000.0
step_size = 0.25
N_steps = 2**30
coordinate_array = start_val + step_size * np.arange(0, N_steps)

However it is really slow to compute all 2^30 values, and we only ever need to access a small slice from the data array. Can I somehow attach a function that computes index values as required, instead of an array of coordinates?

Here's an example with coords attached to the array:

import numpy as np
import xarray as xr
N, M = 10, 2**20
data = np.zeros(shape=(N, M))
time_coords = np.arange(0, N)
freq_coords = freq0 + freq_step * np.arange(0, M)
data_array = xr.DataArray(data, dims=('time', 'frequency'), coords={'timestep': time_coords, 'frequency': freq_coords} )

and the question is: can I replace the very large freq_coords array with a class/function that computes indexes?

Thanks!

[keewis]

There's a feature request for this (#3620) which has more discussion, but I think the recommendation there was to use dask to lazily compute the values. Of course, this only works with non-dimension coordinates right now, but the ongoing index refactor will add support for dimension coordinate in the future.

[telegraphic]

Thanks @keewis, I am indeed looking for something similar to WCS (but simpler!). I hadn't appreciated that a non-dimension coordinates could be lazy evaluated. If there's an ongoing refactor I will bide my time.

FWIW I ended up implementing a duck array that takes a start, stop and step value (like slice) and pretends to be a numpy/astropy array. I had some success attaching this to an xarray as a dimension, but it gets evaluated
https://github.com/UCBerkeleySETI/hyperseti/blob/master/hyperseti/dimension_scale.py#L46. Nevertheless a good learning experience :)

Tagging FRBs/sigpyproc3#15

[keewis]

but it gets evaluated

that's because it is missing some important properties. Have a look at Integrating with duck arrays and Working with numpy-like arrays for more information.