# is sm.tsa.filters.arfilter an AR filter? #1478

Closed
opened this issue Mar 14, 2014 · 7 comments

Projects
None yet
2 participants
Member

### jseabold commented Mar 14, 2014

 In the terminology R uses in their `filter` function [1], what we have as `arfilter` is just the convolution method of their filter, which is just a centered moving average. Their recursive method is autoregressive and we would do the same using `scipy.signal.lfilter`. I was going to add a method argument to arfilter for convolution vs. recursive, but I thought to maybe just rename it to `linear_filter` first? Thoughts?
Member Author

### jseabold commented Mar 14, 2014

 Or arfilter -> convolution_filter and arfilter uses signal.lfilter internally.
Member

### josef-pkt commented Mar 14, 2014

 In terms of general terminology. I had big problems keeping apart which is which, especially also with the signal processing terminology in scipy.signal (num, denom), and I didn't read R at the time. I tried to translate things into ARMA terminology, ar_filter, ma_filter for things that signal.lfilter can do, ARMA terminology is more mnemonic for me. (assuming I got it correct) edit: I'm not sure about the following after 30 minutes: I think the term arfilter is wrong, a moving average, convolution filter should be an mafilter. assume y is observed y_t for ar(L) y_t = e_t versus y_t = ma(L) e_t ma(L) would be a FIR filter (finite impulse response) in scipy.signal ar or arma is a IIR filter I thought I had an arma_filter wrapper around lfilter, but cannot find anything like that. ArmaFft in the sandbox has a frequency domain arma filter method. one of the things I got confused about in signal processing terminology is transfer function filter, where x is the input and y is the output linear_filter sounds fine, related: for using lfilter we should have the correct choice of starting values, so we get identical results to recursion with fixed initial conditions.
Member Author

### jseabold commented Mar 14, 2014

 The one-sided convolution filter maps to MA using lfilter. These should give the same, but it doesn't look like lfilter properly handles NA values in this case. Looks like a bug to me. ``````signal.convolve(dta.co2.values, [.75, .25], mode='valid') signal.lfilter([0, .75, .25], [1.], dta.co2.values) `````` I made a quick mapping between convolve, lfilter and R's filter methods, optionally using initial conditions. For the one-sided convolution, I might just do a one-linear using `pd.rolling_apply` since it handles NAs correctly.
Member Author

### jseabold commented Mar 14, 2014

 Sorry, no those shouldn't give the same. The first line should be ``````filter(dta\$co2, c(.75, .25), sides=1) `````` in R.
Member

### josef-pkt commented Mar 14, 2014

 `lfilter` should have leading ones in both polynomials, and the sign is polynomial not regression `signal.lfilter([1, .75, .25], [1.], dta.co2.values)`
Member Author

### jseabold commented Mar 14, 2014

 If you want a one-sided convolution filter, then the 0 lag for MA is 0. I.e., don't include the current term if you're doing a moving average that should only include past terms.
Member

### josef-pkt commented Mar 14, 2014

 ok, right for shifted to use only past information, the ar part needs (or needed) to have a strictly positive 1st value, IIRC. I haven't looked at these things in a long time.

### jseabold added a commit to jseabold/statsmodels that referenced this issue Mar 15, 2014

``` REF: Refactor arfilter. Correct name and split. Closes statsmodels#1478. ```
``` 1a1434e ```

### jseabold added a commit to jseabold/statsmodels that referenced this issue Mar 23, 2014

``` REF: Refactor arfilter. Correct name and split. Closes statsmodels#1478. ```
``` 006a1e7 ```

### jseabold added a commit to jseabold/statsmodels that referenced this issue Mar 28, 2014

``` REF: Refactor arfilter. Correct name and split. Closes statsmodels#1478. ```
``` 86d5c69 ```

### PierreBdR pushed a commit to PierreBdR/statsmodels that referenced this issue Sep 2, 2014

``` REF: Refactor arfilter. Correct name and split. Closes statsmodels#1478. ```
``` 2801165 ```