CLN: Centralised _check_percentile #27584

hedonhermdev · 2019-07-25T10:23:11Z

Fixes CLN: centrailize _check_percentile #27559
Moved the _check_percentile method on NDFrame to algorithms as
check_percentile.
Changed the references to _check_percentile in pandas/core/series.py
and pandas/core/frame.py
closes CLN: centrailize _check_percentile #27559
tests added / passed
passes black pandas
passes git diff upstream/master -u -- "*.py" | flake8 --diff
whatsnew entry

- Fixes GH27559. - Moved the _check_percentile method on NDFrame to algorithms as check_percentile. - Changed the references to _check_percentile in pandas/core/series.py and pandas/core/frame.py

WillAyd · 2019-07-25T14:28:34Z

Can you also update the groupby reference from the OP?

WillAyd

See above comment

hedonhermdev · 2019-07-25T16:23:43Z

See above comment

Sorry I couldn't figure out what it was referring to. I didnt find any references to _check_percentile in groupby. Can you help me figure it out?

WillAyd · 2019-07-25T16:26:08Z

Oh sorry misread that on my end - not there yet for groupby so I think OK for now

WillAyd · 2019-07-25T16:27:50Z

pandas/core/algorithms.py

@@ -1105,6 +1105,22 @@ def _get_score(at):
        return result


+def check_percentile(q):


Can you annotate this function? I think q should just be float, Iterable[float]

Return should be an ndarray

WillAyd · 2019-07-25T17:55:54Z

pandas/core/algorithms.py

+    """
+
+    msg = "percentiles should all be in the interval [0, 1]. " "Try {0} instead."
+    q = np.asarray(q)


You can just create a new variable here q_arr instead of reusing the argument; should help with some of the typing errors

pandas/core/algorithms.py

Co-Authored-By: William Ayd <william.ayd@icloud.com>

jreback · 2019-07-25T22:16:29Z

ok will have to backport this as #27473 depends

pandas/core/algorithms.py

pep8speaks · 2019-07-26T14:46:25Z

Hello @hedonhermdev! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

There are currently no PEP 8 issues detected in this Pull Request. Cheers! 🍻

Comment last updated at 2019-10-03 15:22:43 UTC

TomAugspurger · 2019-07-29T18:42:01Z

@hedonhermdev seems like there's still a linting issue, and maybe some CI failures.

TomAugspurger · 2019-08-19T17:14:27Z

@hedonhermdev can you update?

TomAugspurger · 2019-08-20T19:19:10Z

Pushing to 1.0

jbrockmendel · 2019-08-28T16:48:57Z

@hedonhermdev can you rebase

WillAyd · 2019-09-13T01:35:48Z

Nice idea but I think this has gone stale. @hedonhermdev please ping if you'd like to pick this back up

hedonhermdev · 2019-09-14T05:33:15Z

Hey sorry for the disappearance, if possible I would like to work on it again. Can you reopen the pull request?

WillAyd · 2019-09-14T05:52:20Z

Sure - thanks for the contribution

- Fixes GH27559. - Moved the _check_percentile method on NDFrame to algorithms as check_percentile. - Changed the references to _check_percentile in pandas/core/series.py and pandas/core/frame.py Annotated check_percentile function. Update pandas/core/algorithms.py Co-Authored-By: William Ayd <william.ayd@icloud.com> Fixed typing error in check_percentile. Refactored docstring of check_percentile function. Fixed PEP8 issues.

jbrockmendel · 2019-09-16T16:37:57Z

pandas/core/algorithms.py

@@ -1102,6 +1102,37 @@ def _get_score(at):
        return result


+def check_percentile(q: Union[float, Iterable[float]]) -> np.ndarray:


would this make more sense in validators?

I think that's reasonable

Should I shift it to utilts/_validators.py ?

Yea let's do that

Would renaming it to validate_percentile be better?

jreback · 2019-09-18T12:14:29Z

pandas/core/algorithms.py

@@ -1864,7 +1860,7 @@ def searchsorted(arr, value, side="left", sorter=None):
        else:
            value = array(value, dtype=dtype)
    elif not (
-        is_object_dtype(arr) or is_numeric_dtype(arr) or is_categorical_dtype(arr)
+            is_object_dtype(arr) or is_numeric_dtype(arr) or is_categorical_dtype(arr)


can you not add unrelated changes (all of this whitespace)

Sorry for that.

pandas/util/_validators.py

TomAugspurger · 2019-09-19T20:15:36Z

pandas/util/_validators.py

+    otherwise raises a ValueError.
+
+    Parameters
+    -------


Needs to be the length of Parameters.

TomAugspurger · 2019-09-19T20:15:49Z

pandas/core/algorithms.py

-        or is_period_dtype(dtype)
-        or is_datetime64_any_dtype(dtype)
-        or is_timedelta64_dtype(dtype)
+            needs_i8_conversion(values)


Why these changes?

Sorry for these

TomAugspurger · 2019-09-19T22:24:20Z

Looks like a linting failure: https://dev.azure.com/pandas-dev/pandas/_build/results?buildId=17752

Try running black pandas. Instructions are in the contributing guide.

TomAugspurger · 2019-09-23T16:37:11Z

@hedonhermdev linting issue: https://dev.azure.com/pandas-dev/pandas/_build/results?buildId=17804

In the future, I'd recommend using pre-commit as described in https://dev.pandas.io/docs/development/contributing.html#python-pep8-black

jreback

lgtm (small comments). pls merge master and ping on green

jreback · 2019-10-01T13:12:51Z

pandas/core/algorithms.py

@@ -1165,7 +1163,6 @@ def compute(self, method):

        # slow method
        if n >= len(self.obj):
-


can you revert these files as they are not actually changed

isort fails if I don't commit them

ok that's odd

jbrockmendel · 2019-10-01T14:38:33Z

pandas/util/_validators.py

+    ------
+    ValueError if percentiles are not in given interval([0, 1]).
+    """
+    msg = "percentiles should all be in the interval [0, 1]. " "Try {0} instead."


Extra quotation marks before 'Try'

hedonhermdev · 2019-10-01T21:40:26Z

@jbrockmendel should I modify excel.py too? As I'm getting a linting issue because of it.

jbrockmendel · 2019-10-02T14:24:20Z

It looks like it just wants you to revert the whitespace change in the excel.py file

jreback · 2019-10-03T17:11:49Z

thanks @hedonhermdev

* master: (22 commits) DOC: fix PR09,PR08 errors for pandas.Timestamp (pandas-dev#28739) WEB: Add diversity note to team.md (pandas-dev#28630) DOC: Minor fixes in pandas/testing.py docstring. (pandas-dev#28752) TST: port maybe_promote tests from pandas-dev#23982 (pandas-dev#28764) Bugfix/groupby datetime issue (pandas-dev#28569) reenable codecov (pandas-dev#28750) CLN: Centralised _check_percentile (pandas-dev#27584) DEPR: Deprecate Index.set_value (pandas-dev#28621) CLN: Fix typo in contributing.rst (pandas-dev#28761) Fixed docstring errors in pandas.period range and pandas.PeriodIndex (pandas-dev#28756) BUG: Fix TypeError raised in libreduction (pandas-dev#28643) DOC: Pandas.Series.drop docstring PR02 (pandas-dev#27976) (pandas-dev#28742) DOC: Fixed doctring errors PR08, PR09 in pandas.io (pandas-dev#28748) TST: Fix broken test cases where Timedelta/Timestamp raise (pandas-dev#28729) REF: Consolidate alignment calls in DataFrame ops (pandas-dev#28638) BUG: Fix dep generation (pandas-dev#28734) Added doctstring to fixture (pandas-dev#28727) DOC: Fixed PR06 docstrings errors in pandas.timedelta_range (pandas-dev#28719) replaced safe_import with a corresponding test decorator (pandas-dev#28731) BUG: Fix RangeIndex.get_indexer for decreasing RangeIndex (pandas-dev#28680) ...

CLN: Centralised _check_percentile

69ef619

- Fixes GH27559. - Moved the _check_percentile method on NDFrame to algorithms as check_percentile. - Changed the references to _check_percentile in pandas/core/series.py and pandas/core/frame.py

WillAyd added the Refactor Internal refactoring of code label Jul 25, 2019

WillAyd requested changes Jul 25, 2019

View reviewed changes

Annotated check_percentile function.

715ac7d

WillAyd requested changes Jul 25, 2019

View reviewed changes

Update pandas/core/algorithms.py

de8a4ab

Co-Authored-By: William Ayd <william.ayd@icloud.com>

ghost mentioned this pull request Jul 25, 2019

BUG: quantile segfaults on invalid quantile values #27473

Closed

4 tasks

jreback added this to the 0.25.1 milestone Jul 25, 2019

Fixed typing error in check_percentile.

7db44a4

jreback requested changes Jul 26, 2019

View reviewed changes

pandas/core/algorithms.py Outdated Show resolved Hide resolved

Refactored docstring of check_percentile function.

350a624

Fixed PEP8 issues.

4b4ca39

TomAugspurger modified the milestones: 0.25.1, 1.0 Aug 20, 2019

WillAyd closed this Sep 13, 2019

WillAyd reopened this Sep 14, 2019

jbrockmendel reviewed Sep 16, 2019

View reviewed changes

Moved check_percentile to utils/_validators.py as validate_percentile.

93a7970

hedonhermdev requested a review from jreback September 18, 2019 08:16

jreback reviewed Sep 18, 2019

View reviewed changes

jreback requested changes Sep 18, 2019

View reviewed changes

pandas/util/_validators.py Outdated Show resolved Hide resolved

pandas/util/_validators.py Outdated Show resolved Hide resolved

Cleanup

5b0122f

TomAugspurger reviewed Sep 19, 2019

View reviewed changes

hedonhermdev added 2 commits September 20, 2019 02:00

Fixed cleanup.

946ee3f

Fixed import in algorithms.

3c56c6b

hedonhermdev added 2 commits September 20, 2019 13:57

Whitespace issues.

786e172

Linting issues

d81a08d

hedonhermdev requested a review from jreback September 20, 2019 18:11

Fixed linting issues.

631a049

jreback requested changes Oct 1, 2019

View reviewed changes

jbrockmendel reviewed Oct 1, 2019

View reviewed changes

Extra quotation.

f66f314

Linting issues.

4e399c6

jreback approved these changes Oct 3, 2019

View reviewed changes

jreback merged commit 5686e9a into pandas-dev:master Oct 3, 2019

jbrockmendel pushed a commit to jbrockmendel/pandas that referenced this pull request Oct 8, 2019

CLN: Centralised _check_percentile (pandas-dev#27584)

624dc21

proost pushed a commit to proost/pandas that referenced this pull request Dec 19, 2019

CLN: Centralised _check_percentile (pandas-dev#27584)

9e2b893

proost pushed a commit to proost/pandas that referenced this pull request Dec 19, 2019

CLN: Centralised _check_percentile (pandas-dev#27584)

8d71dc0

bongolegend pushed a commit to bongolegend/pandas that referenced this pull request Jan 1, 2020

CLN: Centralised _check_percentile (pandas-dev#27584)

10410d3

hedonhermdev deleted the small-refactor branch September 7, 2020 11:32

		@@ -1105,6 +1105,22 @@ def _get_score(at):
		return result


		def check_percentile(q):

		@@ -1102,6 +1102,37 @@ def _get_score(at):
		return result


		def check_percentile(q: Union[float, Iterable[float]]) -> np.ndarray:

		@@ -1165,7 +1163,6 @@ def compute(self, method):

		# slow method
		if n >= len(self.obj):

CLN: Centralised _check_percentile #27584

CLN: Centralised _check_percentile #27584

Conversation

hedonhermdev commented Jul 25, 2019 • edited

WillAyd commented Jul 25, 2019

WillAyd left a comment

Choose a reason for hiding this comment

hedonhermdev commented Jul 25, 2019

WillAyd commented Jul 25, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jreback commented Jul 25, 2019

pep8speaks commented Jul 26, 2019 • edited

Comment last updated at 2019-10-03 15:22:43 UTC

TomAugspurger commented Jul 29, 2019

TomAugspurger commented Aug 19, 2019

TomAugspurger commented Aug 20, 2019

jbrockmendel commented Aug 28, 2019

WillAyd commented Sep 13, 2019

hedonhermdev commented Sep 14, 2019

WillAyd commented Sep 14, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

TomAugspurger commented Sep 19, 2019

TomAugspurger commented Sep 23, 2019

jreback left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

hedonhermdev Oct 1, 2019 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

hedonhermdev commented Oct 1, 2019

jbrockmendel commented Oct 2, 2019

jreback commented Oct 3, 2019

hedonhermdev commented Jul 25, 2019 •

edited

pep8speaks commented Jul 26, 2019 •

edited

hedonhermdev Oct 1, 2019 •

edited