Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BUG/ER: Stricter testing of 'monotocity' when reindexing with ffill or bfill. #4483

Closed
wants to merge 2 commits into from
Closed

Conversation

prossahl
Copy link
Contributor

@prossahl prossahl commented Aug 6, 2013

closes #4484

Reversing an index and forward filling succeeds but produces misleading data:

>>> dr = pd.date_range('2013-08-01', periods=6, freq='B')
>>> df = pd.DataFrame(np.random.randn(6,1), index=dr, columns=list('A'))
>>> df['A'][3] = np.nan
>>> df.reindex(df.index[::-1], method='ffill')
                   A
2013-08-08  2.127302
2013-08-07       NaN
2013-08-06       NaN
2013-08-05       NaN
2013-08-02       NaN
2013-08-01       NaN

This change enforces "monotocity" (well actually weakly increasing) and causes a ValueError to be raised.

@jreback
Copy link
Contributor

jreback commented Aug 6, 2013

Can you also add an example of how to do this in the missing_data.rst ? (and maybe a pointer from the cookbook)?

also need release notes (put in the API section), reference the issue #4484

This is the solution (but yours is nice on error reporting!)

In [11]: >>> df.reindex(df.index[::-1]).reset_index().ffill().set_index('index')
Out[11]: 
                   A
index               
2013-08-08  1.112043
2013-08-07  0.250710
2013-08-06  0.250710
2013-08-05 -0.499677
2013-08-02 -0.327536
2013-08-01 -1.774846

@jreback
Copy link
Contributor

jreback commented Aug 6, 2013

also pls hook up travis to this (see contributing.md)

@jreback
Copy link
Contributor

jreback commented Aug 8, 2013

@prossahl ok...this looks good (ignore the failure on travis)...that's a weird error that is fixed in master

merge?

@jreback
Copy link
Contributor

jreback commented Aug 8, 2013

and I added the example I put above to the cookbook (will be there later today)

@prossahl
Copy link
Contributor Author

I have moved this to a PR #4642.

@jreback
Copy link
Contributor

jreback commented Aug 22, 2013

ok...close this one then?

@prossahl
Copy link
Contributor Author

Will do. Over to GH4642.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

BUG/ER: misleading results when filling with a reversed index
2 participants