Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DOC: update the pandas.DataFrame.clip_lower docstring #20289

76 changes: 69 additions & 7 deletions pandas/core/generic.py
Original file line number Diff line number Diff line change
Expand Up @@ -5716,24 +5716,86 @@ def clip_upper(self, threshold, axis=None, inplace=False):

def clip_lower(self, threshold, axis=None, inplace=False):
"""
Return copy of the input with values below given value(s) truncated.
Return copy of the input with values below given value(s) trimmed.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's use "caller" instead of "input". FWIW I guess it's not technically true that it returns a copy of the caller because inplace would modify directly. Any idea on how to make this statement more accurate?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Something along the line of

Return a copy of the caller (or the caller itself if inplace == True) with values below given threshold trimmed.

?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm well I don't think you need to get in the discussion of return values since that's documented later in the docstring. First line is supposed to be very simple so I would mirror what's there for clip and say something like Trim the caller's values below a given threshold

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok. Since Series.clip_lower uses the same docstring (the function is defined in generic.py) I thought that I should change the "See Also" description for Series.clip_lower to be the same. But when I tested the generation of the HTML I realized that if we use the "See Also" as it is right now, for Series.clip_lower we will not have a reference to DataFrame.clip_lower and it will have a reference to itself.

Does this make sense? Is there an elegant way to solve it?

Copy link
Member

@WillAyd WillAyd Mar 12, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes that makes sense. Typically in this case you could use the @Substitution decorator - you'll see some other functions using in that same module.

That said, since your method is currently just inherited by Series and DataFrame and not necessarily overriden, I don't think there's a good way without overriding those implementation (if even just to call super) to implement Substitution. Unless you can think of a better way, I'd suggest you remove the clip_lower reference in See Also and open up a separate issue to try and make substitution work

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nope, I can also only think of overriding in order for it to work. But is it worth the effort just to make the "See Also" complete? Would a reference to just clip_lower (without DataFrame. or Series. prefixes) be a good enough option?

I'll sleep through it to see if I can think of something better.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would say just remove but if you come up with a better idea then by all means


Elements below the `threshold` will be changed to match the
`threshold` value(s).

Parameters
----------
threshold : float or array_like
axis : int or string axis name, optional
threshold : float or array-like
Lower value(s) to which the input value(s) will be trimmed.
axis : {0 or 'index', 1 or 'columns', None}, default None
Align object with threshold along the given axis.
inplace : boolean, default False
Whether to perform the operation in place on the data
.. versionadded:: 0.21.0
Whether to perform the operation in place on the data.

.. versionadded:: 0.21.0

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry if I forgot some discussion about it, but don't we want a See Also section? I think clip and clip_upper should be listed.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes good spot we definitely still want this - @adatasetaday can you add back in?

I know we had some back and forth about Series.clip_lower being self-referential with the shared doc system. If you find that confusing just remove that one entry, but all the rest should remain

See Also
--------
clip
DataFrame.clip : General purpose method to trim `DataFrame` values to
given threshold(s)
DataFrame.clip_upper : Trim `DataFrame` values above given
threshold(s)
Series.clip : General purpose method to trim `Series` values to given
threshold(s)
Series.clip_lower : Trim `Series` values below given threshold(s)
Series.clip_upper : Trim `Series` values above given threshold(s)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ideally, for DataFrame.clip_lower we would like to have in See Also:

  • Series.clip_lower
  • DataFrame.clip
  • DataFrame.clip_upper

So, the same in Series and nothing else from Series. And the similar from the same class (DataFrame). As this is being reused by both Series and DataFrame, I'd probably list all clip, clip_lower and clip_upper, but I'm not sure. @jreback ?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only caveat here is that the user is working on a doc that is shared by Series and DataFrame. Mentioned earlier that without further updates (i.e. implementing substitution) that the clip_lower reference would be self-referential half of the time.

I advised just removing it to not get hung up on it for now, the other option would be to explicitly list both DataFrame.clip_lower and Series.clip_lower and live with the fact that one will be self referencing. No strong preference on my end.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, I was waiting for a little clarification, since I'm not sure how to proceed. I can go either way, but I haven't seen a conclusive definition. Was a decision made on how to proceed with these?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I'm not wrong, in another ticket the final implementation just listed everything. As @WillAyd says in some cases the same method being documented will be self-referenced, but it's not a big issue.


Returns
-------
clipped : same type as input
Series or DataFrame
Same type as caller.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For this method may be it's a bit obvious, but for consistency I'd say in the description what this method returns (e.g. Original data with values trimmed). The note about the time is all right (not sure if other methods have it as part of the type or the description).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would any of these work?

Series or DataFrame (same type as caller)
Original data with values trimmed.

Series or DataFrame
Original data (in same type as caller) with values trimmed .

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They look great. If you want to take a look at the latest merged pull requests, I think this happens in few, and may be there is a way more standard than another. If you don't find any, from my side you can choose whatever you want.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Based on what I see in the code, this seems to be the most common format

type of caller
Original data with values trimmed.

I was thinking that there would be some value in creating PRs for standarizing these kind of things. For example, do one that fixes the axis parameters in ALL methods to match the format agreed in the discussions in this sprint. Another one could be done to make sure there is no period after the versionadded (which a lot of them have it, probably due to the validating script complaining if it was not there). Would this be a good idea? Who should I contact if I want to take care of some of these?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Feel free to do it. I'd probably wait until most of the PRs from the sprint are closed, to avoid conflicts. And more than contacting someone, you can create an issue (or one per change) with what you're planning to do, you can get some discussion and feedback before doing all the work.


Examples
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice job here - I think the examples are very clear

--------
>>> df = pd.DataFrame({'a': [0.740518, 0.450228, 0.710404, -0.771225],
... 'b': [0.040507, -0.45121, 0.760925, 0.010624]})
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Personally I find the data used in #20212 easier to understand. It requires a decent amount of concentration to see what the examples are doing with so many decimal numbers.


>>> df
a b
0 0.740518 0.040507
1 0.450228 -0.451210
2 0.710404 0.760925
3 -0.771225 0.010624

Clip to a scalar value

>>> df.clip_lower(0.2)
a b
0 0.740518 0.200000
1 0.450228 0.200000
2 0.710404 0.760925
3 0.200000 0.200000

Clip to an array along the index axis

>>> df.clip_lower([0.2, 0.4, 0.6, 0.8], axis=0)
a b
0 0.740518 0.200000
1 0.450228 0.400000
2 0.710404 0.760925
3 0.800000 0.800000

Clip to an array column the index axis
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe for consistency here you meant to say "Clip to an array along the column axis"


>>> df.clip_lower([0.5, 0.0], axis=1)
a b
0 0.740518 0.040507
1 0.500000 0.000000
2 0.710404 0.760925
3 0.500000 0.010624

Clip in place

>>> df.clip_lower(0.2, inplace=True)
>>> df
a b
0 0.740518 0.200000
1 0.450228 0.200000
2 0.710404 0.760925
3 0.200000 0.200000
"""
return self._clip_with_one_bound(threshold, method=self.ge,
axis=axis, inplace=inplace)
Expand Down