Rolling window #31

Closed
wants to merge 1 commit into
from

Conversation

Projects
None yet
8 participants

rigtorp commented Jan 1, 2011

A useful stride trick that not many people are aware of.

dwf commented on 0eff7d0 Jan 11, 2011

Looks good to me. One question: does this assume C-contiguity?

Owner

rigtorp replied Jan 11, 2011

No

Can this work for other axes too, or only the last one? That feels a bit too specific.

It would also needs a few tests before it can be added.

dwf replied Mar 11, 2011

The last axis seems like the natural place for it, though I'm having a hard time finding the words as to why.

+1 on tests though.

I don't really get why the last axis would be more useful, it depends on the data in your array. If I want a rolling window view of the first axis, it would be annoying to first have to roll axes around because this function only works with the last axis.

Owner

rigtorp replied Mar 11, 2011

It can work with any axis if I add a swapaxis call in there. Problem is where should the new axis go?

New axis always in the same place I'd think, it becomes either the first or last axis.

I suggest a new keyword "axis", with the standard default value (always forget if that's -1 or None, easy to check with other function signatures).

In the docstring it would be good where the extra axis goes and what the shape is of the returned array. It took me a little while to figure out why the new shape was (2, 3, 3) for an input of shape (2, 5) and window=1.

Finally, I think it would be easier if window is a keyword with a default value of 1.

Contributor

WarrenWeckesser commented Jun 5, 2011

It seems natural to me for the axis to change "in place"; that is, the window axis is immediately after the axis being "windowed". I'd also like a "step" keyword that determines the increment of the start of each window. This should do it:

def rolling_window(a, window=1, step=1, axis=-1):
    if window < 1:  
        raise ValueError("`window` must be at least 1.")
    if window > a.shape[axis]:
        raise ValueError("`window` is too long.")
    if step < 1:
        raise ValueError("`step` must be at least 1.")
    axis = axis % a.ndim
    num_windows = (a.shape[axis] - window + step) / step
    shape = a.shape[:axis] + (num_windows, window) + a.shape[axis + 1:]
    strides = a.strides[:axis] + (step * a.strides[axis], a.strides[axis]) + a.strides[axis + 1:]
    return as_strided(a, shape=shape, strides=strides)

For example,

In [74]: a
Out[74]: 
array([[ 0,  1,  2,  3,  4,  5],
       [ 6,  7,  8,  9, 10, 11],
       [12, 13, 14, 15, 16, 17],
       [18, 19, 20, 21, 22, 23]])

In [75]: rolling_window(a, window=3, step=2)
Out[75]: 
array([[[ 0,  1,  2],
        [ 2,  3,  4]],

       [[ 6,  7,  8],
        [ 8,  9, 10]],

       [[12, 13, 14],
        [14, 15, 16]],

       [[18, 19, 20],
        [20, 21, 22]]])

In [76]: rolling_window(a, window=2, axis=0)
Out[76]: 
array([[[ 0,  1,  2,  3,  4,  5],
        [ 6,  7,  8,  9, 10, 11]],

       [[ 6,  7,  8,  9, 10, 11],
        [12, 13, 14, 15, 16, 17]],

       [[12, 13, 14, 15, 16, 17],
        [18, 19, 20, 21, 22, 23]]])

In [77]: _.shape
Out[77]: (3, 2, 6)

In [78]: rolling_window(a, window=2, step=2, axis=0)
Out[78]: 
array([[[ 0,  1,  2,  3,  4,  5],
        [ 6,  7,  8,  9, 10, 11]],

       [[12, 13, 14, 15, 16, 17],
        [18, 19, 20, 21, 22, 23]]])

In [79]: _.shape
Out[79]: (2, 2, 6)
Owner

pv commented Sep 6, 2011

Additional feature suggestion: How about 2-D (and higher dimensional) windows? These may also useful, and it would be nice to have everything done by a single function.

Member

seberg commented Aug 17, 2012

Just to note, something like this is very easy to generalize for n-dimensions, of course that would make an axis argument a bit ill defined as to what it would do, I guess it would be possible to add it when window is scalar, and maybe a choice as to add the new dimensions at the start or end. If window == step, it creates non overlapping "tiles".

EDIT: I created a new very general version which allows for pretty much all of the above (I could imagine adding axis argument to build a up the correct window), also there is no support for negative steps, but I am not sure that matters. I guess the comments are not understandable probably...

sorry for the spam, have put that code here: https://gist.github.com/3866040 instead.

Contributor

insertinterestingnamehere commented Jun 19, 2015

What's the status of this? It looks like @seberg already has a working implementation. What's left?

Owner

njsmith commented Jun 19, 2015

Also relevant: the c-level neighborhood iterator. (One of the few things
that nditer can't do.)
On Jun 18, 2015 8:58 PM, "Ian Henriksen" notifications@github.com wrote:

What's the status of this? It looks like @seberg
https://github.com/seberg already has a working implementation. What's
left?


Reply to this email directly or view it on GitHub
#31 (comment).

Member

seberg commented Aug 9, 2015

Maybe a bit harsh, but I am going to just close this due to age, and the fact that I would like to see such a function be more general, so a few changes would be needed here still. I would be happy to see someone pick it up though!
Yes, I do think my code might be a base to actually implement something like this. I also personally think this could be nice addition to numpy, I certainly do these kind of things every new and then.

@seberg seberg closed this Aug 9, 2015

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment