Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BUG: sort_values with ascending=None and key!=None raises exception #41318

Open
2 of 3 tasks
gshimansky opened this issue May 4, 2021 · 4 comments
Open
2 of 3 tasks
Labels
Enhancement Error Reporting Incorrect or improved errors from pandas Needs Discussion Requires discussion from core team before further action

Comments

@gshimansky
Copy link

  • I have checked that this issue has not already been reported.

  • I have confirmed this bug exists on the latest version of pandas.

  • (optional) I have confirmed this bug exists on the master branch of pandas.


Note: Please read this guide detailing how to provide the necessary information for us to reproduce your bug.

Code Sample, a copy-pastable example

import pandas as pd

def do_nothing(value):
    return value

df = pd.DataFrame(
    {
        "col1": [2, 51, 17],
        "col2": [5.5, 3.3, 4.4],
        "col3": ["a", "b", "c"],
    },
)
print(df)
df1 = df.sort_values(by=["col1"], ascending=None, key=do_nothing)
print(df1)

Problem description

Output is

   col1  col2 col3
0     2   5.5    a
1    51   3.3    b
2    17   4.4    c
Traceback (most recent call last):
  File "sort_values_test9.py", line 14, in <module>
    df1 = df.sort_values(by=["col1"], ascending=None, key=do_nothing)
  File "/home/gregory/work/miniconda3/envs/modin/lib/python3.8/site-packages/pandas/core/frame.py", line 5464, in sort_values
    indexer = nargsort(
  File "/home/gregory/work/miniconda3/envs/modin/lib/python3.8/site-packages/pandas/core/sorting.py", line 354, in nargsort
    return nargsort(
  File "/home/gregory/work/miniconda3/envs/modin/lib/python3.8/site-packages/pandas/core/sorting.py", line 368, in nargsort
    return items.argsort(ascending=ascending, kind=kind, na_position=na_position)
  File "/home/gregory/work/miniconda3/envs/modin/lib/python3.8/site-packages/pandas/core/arrays/base.py", line 583, in argsort
    ascending = nv.validate_argsort_with_ascending(ascending, args, kwargs)
  File "/home/gregory/work/miniconda3/envs/modin/lib/python3.8/site-packages/pandas/compat/numpy/function.py", line 150, in validate_argsort_with_ascending
    validate_argsort_kind(args, kwargs, max_fname_arg_count=3)
  File "/home/gregory/work/miniconda3/envs/modin/lib/python3.8/site-packages/pandas/compat/numpy/function.py", line 67, in __call__
    validate_args_and_kwargs(
  File "/home/gregory/work/miniconda3/envs/modin/lib/python3.8/site-packages/pandas/util/_validators.py", line 205, in validate_args_and_kwargs
    validate_kwargs(fname, kwargs, compat_args)
  File "/home/gregory/work/miniconda3/envs/modin/lib/python3.8/site-packages/pandas/util/_validators.py", line 149, in validate_kwargs
    _check_for_default_values(fname, kwds, compat_args)
  File "/home/gregory/work/miniconda3/envs/modin/lib/python3.8/site-packages/pandas/util/_validators.py", line 65, in _check_for_default_values
    raise ValueError(
ValueError: the 'axis' parameter is not supported in the pandas implementation of argsort()

Function validate_argsort_with_ascending inserts a value for ascending argument in case it is None, but later this value is considered as a value of axis parameter for some reason. This leads to an exception instead of a normal operation.

Expected Output

Output of pd.show_versions()

[paste the output of pd.show_versions() here leaving a blank line after the details tag]

INSTALLED VERSIONS

commit : 2cb9652
python : 3.8.6.final.0
python-bits : 64
OS : Linux
OS-release : 5.4.0-72-generic
Version : #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021
machine : x86_64
processor : x86_64
byteorder : little
LC_ALL : None
LANG : en_US.UTF-8
LOCALE : en_US.UTF-8

pandas : 1.2.4
numpy : 1.19.5
pytz : 2021.1
dateutil : 2.8.1
pip : 21.0.1
setuptools : 49.6.0.post20210108
Cython : None
pytest : 6.2.3
hypothesis : None
sphinx : 3.5.4
blosc : None
feather : 0.4.1
xlsxwriter : None
lxml.etree : 4.6.3
html5lib : None
pymysql : None
psycopg2 : None
jinja2 : 2.11.3
IPython : 7.20.0
pandas_datareader: None
bs4 : 4.9.3
bottleneck : None
fsspec : 0.8.5
fastparquet : None
gcsfs : None
matplotlib : 3.2.2
numexpr : 2.7.2
odfpy : None
openpyxl : 3.0.7
pandas_gbq : 0.15.0
pyarrow : 1.0.0
pyxlsb : None
s3fs : None
scipy : 1.6.2
sqlalchemy : 1.4.8
tables : 3.6.1
tabulate : None
xarray : 0.17.0
xlrd : 2.0.1
xlwt : None
numba : None

@gshimansky gshimansky added Bug Needs Triage Issue that has not been reviewed by a pandas team member labels May 4, 2021
@phofl
Copy link
Member

phofl commented May 4, 2021

Hi, thanks for your report.

ascending should either be a bool or a list of bool, not None. This conversion is done to ensure compatibility with numpy.

@jreback
Copy link
Contributor

jreback commented May 4, 2021

does this raise the same on master? I think we merged a change to make this more informative

@phofl
Copy link
Member

phofl commented May 4, 2021

No same error

@phofl
Copy link
Member

phofl commented May 4, 2021

We could validate if ascending is not None maybe, but this would break some peoples code probably

@mzeitlin11 mzeitlin11 added Error Reporting Incorrect or improved errors from pandas Needs Discussion Requires discussion from core team before further action and removed Bug Needs Triage Issue that has not been reviewed by a pandas team member labels Jul 1, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Enhancement Error Reporting Incorrect or improved errors from pandas Needs Discussion Requires discussion from core team before further action
Projects
None yet
Development

No branches or pull requests

5 participants