Compatibility issues with numpy's fromnumeric.py

[gfyoung]

A recent spate of issues/PR's stemming from calling functions defined in numpy's fromnumeric.py module here that have identically-named but differently implemented methods/functions in pandas is indicative of a much larger compatibility issue between the two libraries with this module. A thorough overview of all of the functions from the fromnumeric.py module and cross-referencing them to implementations in pandas is needed to avoid similar issues.

Relevant PRs:
#12413 (issue: #12238)
#12603 (issue: #12600)
#12638

#7325 (from numpy)

[gfyoung]

As mentioned in #12600, I'll tackle this as a follow-up once these PR's are landed.

[jreback]

as discussed we basically have 2 classes of issues:

• like sorter a seemingly innocuous argument that numpy needs, but pandas does not. So soln is now to pass thru, with no checks (and its still a named argument, currently passed via position from numpy). we should note in the doc-string this behavior.
• like .round,.idxmax,stat functions. mainly the out argument which is not needed (and confusing to pandas). soln is to allow **kwargs, but check them for invalid args (to avoid misspellings and such). and raise if this particular arg is not None (in this case out).

[jreback]

cc @mortada @jorisvandenbossche @shoyer @wesm

[wesm]

IMHO we should not be striving to make pandas API compatible with NumPy (except offering an __array__ API, of course), but we should avoid unnecessary / common conflicts if possible

[gfyoung]

@wesm: Agreed. I think in this case though trying to "align" the API with numpy's makes sense because it should be perfectly legal for example to call either np.searchsorted or Series.searchsorted without Python blowing up on the user.

[jreback]

@gfyoung numpy's behavior is a bug really, in that it shouldn't just call _wrap_it and assume everything is a sub-class (like it does). I know you are fixing that, so this is really for compat.

[gfyoung]

@jreback : Right, I guess "align" gives the connotation that pandas is doing something wrong, when we're really just trying to "accommodate" numpy's buggy API.

[gfyoung]

Well, now pandas is not alone. numpy's close cousin scipy has these exact same compatibility issues too as I filed just now.

[gfyoung]

My numpy PR has been merged. So now (hopefully) we can just worry about backwards compatibility.

[jreback]

maybe add order arg from np.argsort as well (IOW we could remove from pandas)

[jreback]

prob could start with seeing which functions call _wrap_it on the numpy side here.

[gfyoung]

A massive PR (#12810) addressing this issue is finally up. There were a lot more incompatibilities than I had expected. Hopefully this PR should address almost if not all of them.