pandas interpolate inconsistent results with axis and method ffill

[saarahrasheed]

Problem:
I am trying to interpolate a dataframe of mxn with na values. The default axis is 0 (index), so if I were to interpolate at axis 0, df should fill missing values (say at index n) for each column with values from index n-1 and n+1 (or n+p, where n+p is the closest index with a valid value). This holds for default linear method but not for ffill method.

Code:
data = np.array([[1,2,3,4, np.nan, 5], [2,4,6,np.nan, 8, 10], [3, 6, 9, np.nan, np.nan, 30]]).T
d = pd.DataFrame(data, columns=['A', 'B', 'C'])
d
A B C
0 1.0 2.0 3.0
1 2.0 4.0 6.0
2 3.0 6.0 9.0
3 4.0 NaN NaN
4 NaN 8.0 NaN
5 5.0 10.0 30.0

d.interpolate(method='ffill', axis=1)

A B C
0 1.0 2.0 3.0
1 2.0 4.0 6.0
2 3.0 6.0 9.0
3 4.0 6.0 9.0
4 4.0 8.0 9.0
5 5.0 10.0 30.0
d.interpolate(method='ffill')

 A     B     C

0 1.0 2.0 3.0
1 2.0 4.0 6.0
2 3.0 6.0 9.0
3 4.0 4.0 4.0
4 NaN 8.0 8.0
5 5.0 10.0 30.0
d.interpolate()

 A     B     C

0 1.0 2.0 3.0
1 2.0 4.0 6.0
2 3.0 6.0 9.0
3 4.0 7.0 16.0
4 4.5 8.0 23.0
5 5.0 10.0 30.0

[CloseChoice]

Hi @saarahrasheed and thanks for taking the time to report your issue.
From the docstring in the ffill-method (the functionality actually lies in the fillna function) ffill/fillna:

ffill: propagate last valid observation forward to next valid

I see exactly that when looking at your examples. The given axis is iterated over to find the last valid example.

Or did I miss something?

[saarahrasheed]

Shouldn't all methods of interpolate function the same way? I mean, when you set the axis=0 (default), it should seek to interpolate along the indices and when it is zero, it should along the columns. If fillna() works differently, that would seem a bit confusing no?

In my example, column A for instance, the values are 1,2,3,4,nan,5. By interpolate along axis= 0 (linear default method), it should set the value to 4.5. Now if I use method as ffill, on default axis, it should set the value of nan to 4. But it doesn't do so. Whereas, if I set the axis to one, it does so. So to iterate, shouldn't all methods of interpolation treat axis in the same manner for sake of consistency? Please correct me if I am understanding this wrong.

[CloseChoice]

For explanation: ffill takes the last valid entry along the given axis and fills NaNs.

   A   B   C
0 1.0 2.0 3.0
1 2.0 4.0 6.0
2 3.0 6.0 9.0
3 4.0 NaN NaN
4 NaN 8.0 NaN
5 5.0 10.0 30.0

Given axis=0 means going along the indices and take the last valid entry and therefore the expected result is:

df.interpolate(method='ffill', axis=0)
   A   B   C
0 1.0 2.0 3.0
1 2.0 4.0 6.0
2 3.0 6.0 9.0
3 4.0 4.0 4.0
4 NaN 8.0 8.0
5 5.0 10.0 30.0

Given axis=1 means going along the columns:

df.interpolate(method='ffill', axis=1)
   A   B   C
0 1.0 2.0 3.0
1 2.0 4.0 6.0
2 3.0 6.0 9.0
3 4.0 6.0 9.0
4 4.0 8.0 9.0
5 5.0 10.0 30.0

But for example

df
     A     B     C
3  4.0   NaN   NaN
4  NaN   8.0   NaN
5  5.0  10.0  30.0

df.interpolate(method='ffill', axis=1)
     A     B    C
3  4.0   NaN   NaN
4  4.0   8.0   NaN
5  5.0  10.0  30.0

[CloseChoice]

@saarahrasheed can we close this ticket?

[saarahrasheed]

Hi. Thanks for the prompt responses and explanation. I believe you're right. Sorry the confusion was at my end! Thanks again.