Add example usage to DataFrame.filter

[cswarth]

Updates doc comments for DataFrame.filter and adds usage examples.
Fixed errors identified by flake8 and correctly rebase my branch before issuing PR.

DataFrame.filter(items=None, like=None, regex=None, axis=None)¶

Subset rows or columns of dataframe according to labels in the index.

Note that this routine does not filter a dataframe on its contents. The filter is applied to the labels of the index. This method is a thin veneer on top of DateFrame Select

Parameters:	items : list-like List of info axis to restrict to (must not all be present) like : string Keep info axis where “arg in col == True” regex : string (regular expression) Keep info axis with re.search(regex, col) == True axis : int or None The axis to filter on.
Returns:	same type as input object with filtered info axis

Notes

The items, like, and regex parameters should be mutually exclusive, but this is not checked.

axis defaults to the info axis that is used when indexing with [].

Examples

>>> df
        one  two  three
mouse     1    2      3
rabbit    4    5      6

>>> # select columns by name
>>> df.filter(items=['one', 'three'])  
        one  three
mouse     1      3
rabbit    4      6

>>> # select columns by regular expression
>>> df.filter(regex='e$', axis=1)
        one  three
mouse     1      3
rabbit    4      6

>>> # select rows containing 'bbi'
>>> df.filter(like='bbi', axis=0)
        one  two  three
rabbit    4    5      6

[jreback]

this last sentence is not necessary.

[jreback]

this axis args should be generated like we do for many other methods, see .reindex or .fillna

[jreback]

can you make this change

[jreback]

can you update the axis parameters as well (can be a string)

[jreback]

this is not checked? are you sure? if so then that's a much bigger problem. defering to documentation is not a good idea.

[cswarth]

Yes, mutual exclusion of parameters really is not checked. That line was already in the docs, I think I just reworded it a bit. As @MaximilianR said of this routine, "it seems to do a lot of stuff not very well"

[jreback]

can you put in a check for this and a couple of tests?

[jreback]

you can add a see also section to direct to .select

[cswarth]

Removed the "thin veneer" comment and added a See Also referring to pandas.DataFrame.select.
The point is to demonstrate that this routine does not filter on the contents of the dataframe, but on the index. Hopefully this will help those coming from R who might be expecting DataFrame.filter to act like dplyr::filter

[jreback]

can you update

[cswarth]

Sorry for being a newbie, but update in what way? Do you want me to rebase my doc/df_filter development branch to upstream/master and push that to github? It looks like that will be push 201 commits so I want to make sure that is the right thing to do.

[jreback]

you need to rebase on master first

[cswarth]

I updated my branch, but 24hrs later the CI builds are still not complete. Is that unusual or expected?

Also the CI build for python 3.5 failed but I don't see how it has anything to do with changes I created.
https://travis-ci.org/pydata/pandas/jobs/124031500

[jreback]

Travis was having some issues ok now

ithe 3.5 build on. numpy master is currently failing but that is ok

[jreback]

can you rebase / update according to comments

[jreback]

can you rebase / update

[jreback]

can you update

[codecov-io]

Current coverage is 83.76%

Merging #12399 into master will decrease coverage by 0.47%

@@             master   #12399   diff @@
========================================
  Files           138      135      -3   
  Lines         50721    49640   -1081   
  Methods           0        0           
  Branches          0        0           
========================================
- Hits          42723    41583   -1140   
- Misses         7998     8057     +59   
  Partials          0        0           

Powered by Codecov. Last updated by 2061e9e...5e492cd

[cswarth]

As requested, added test for mutually exclusive arguments in DataFrame.filter and added unit tests for same.

[jorisvandenbossche]

This seems a bit complicated for what we want to achieve (the locals is certainly not needed I think).
Using a similar construct: sum(map(lambda x: x is not None, [item, like, regex])) (but using a list comprehension is maybe easier to read: sum([x is not None for x in [item, like, regex]]))

[cswarth]

thanks, changing to,

nkw = sum([operator.getitem(args,x) is not None for x in ['items', 'like', 'regex']])
if nkw > 1:
        raise TypeError("filter(): keyword arguments are mutually exclusive")

[cswarth]

opps, I didn't understand...now using,

        nkw = sum([x is not None for x in [items, like, regex]])

[jorisvandenbossche]

This one is already raised below

[cswarth]

deleted this exception and updated the tests to expect the latter one.

[jorisvandenbossche]

There is already a test_filter in https://github.com/pydata/pandas/blob/c2ea8fb20611370c7db7e11506727b0b67a76662/pandas/tests/frame/test_axis_select_reindex.py#L621, maybe good to add this there?

[cswarth]

Thanks, I'll move the tests there.

[cswarth]

Enforcing mutually exclusive arguments in frame.filter() is an incompatible change that is going to break some code, somewhere. Should this be called out in the release notes?

I would like to make a case that this API should be deprecated at this time. It adds little value over reindex and select at the expense of an API that is different than just about anything else.

[jreback]

e.g. checked here

[jreback]

@cswarth yes pls add a whatsnew note about the mutually exclusive args.

[jreback]

already slated to deprecate in 0.19.0: pydata#12401

[jreback]

thanks @cswarth