Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BUG/ER: Stricter testing of 'monotocity' when reindexing with ffill or bfill. #4483

Closed
wants to merge 2 commits into from

Conversation

@prossahl
Copy link
Contributor

commented Aug 6, 2013

closes #4484

Reversing an index and forward filling succeeds but produces misleading data:

>>> dr = pd.date_range('2013-08-01', periods=6, freq='B')
>>> df = pd.DataFrame(np.random.randn(6,1), index=dr, columns=list('A'))
>>> df['A'][3] = np.nan
>>> df.reindex(df.index[::-1], method='ffill')
                   A
2013-08-08  2.127302
2013-08-07       NaN
2013-08-06       NaN
2013-08-05       NaN
2013-08-02       NaN
2013-08-01       NaN

This change enforces "monotocity" (well actually weakly increasing) and causes a ValueError to be raised.

@jreback

This comment has been minimized.

Copy link
Contributor

commented Aug 6, 2013

Can you also add an example of how to do this in the missing_data.rst ? (and maybe a pointer from the cookbook)?

also need release notes (put in the API section), reference the issue #4484

This is the solution (but yours is nice on error reporting!)

In [11]: >>> df.reindex(df.index[::-1]).reset_index().ffill().set_index('index')
Out[11]: 
                   A
index               
2013-08-08  1.112043
2013-08-07  0.250710
2013-08-06  0.250710
2013-08-05 -0.499677
2013-08-02 -0.327536
2013-08-01 -1.774846
@jreback

This comment has been minimized.

Copy link
Contributor

commented Aug 6, 2013

also pls hook up travis to this (see contributing.md)

@jreback

This comment has been minimized.

Copy link
Contributor

commented Aug 8, 2013

@prossahl ok...this looks good (ignore the failure on travis)...that's a weird error that is fixed in master

merge?

@jreback

This comment has been minimized.

Copy link
Contributor

commented Aug 8, 2013

and I added the example I put above to the cookbook (will be there later today)

@prossahl

This comment has been minimized.

Copy link
Contributor Author

commented Aug 22, 2013

I have moved this to a PR #4642.

@jreback

This comment has been minimized.

Copy link
Contributor

commented Aug 22, 2013

ok...close this one then?

@prossahl

This comment has been minimized.

Copy link
Contributor Author

commented Aug 22, 2013

Will do. Over to GH4642.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants
You can’t perform that action at this time.