ENH: Implement groupby.sample #34069
Conversation
pandas/core/groupby/groupby.py
Outdated
| else: | ||
| ws = [None] * self.ngroups | ||
|
|
||
| if random_state: |
There was a problem hiding this comment.
i don't think this is enough, you need to always have a random_state here that is consistent across the entire groupby.
There was a problem hiding this comment.
I think either is fine. Either we get a random state from NumPy's global random state initially and re-use it, or we have each group draw from the global random state pool. It's similar to these two calls
.sample(random_state=0)# each call uses the seed 0.sample(random_state=np.random.RandomState(0))# each call makes an independent draw
There was a problem hiding this comment.
I actually meant to make this random_state is not None (didn't consider other "falsey" values)
doc/source/whatsnew/v1.1.0.rst
Outdated
| @@ -275,6 +275,7 @@ Other enhancements | |||
| such as ``dict`` and ``list``, mirroring the behavior of :meth:`DataFrame.update` (:issue:`33215`) | |||
| - :meth:`~pandas.core.groupby.GroupBy.transform` and :meth:`~pandas.core.groupby.GroupBy.aggregate` has gained ``engine`` and ``engine_kwargs`` arguments that supports executing functions with ``Numba`` (:issue:`32854`, :issue:`33388`) | |||
| - :meth:`~pandas.core.resample.Resampler.interpolate` now supports SciPy interpolation method :class:`scipy.interpolate.CubicSpline` as method ``cubicspline`` (:issue:`33670`) | |||
| - :class:`DataFrameGroupBy` and :class:`SeriesGroupBy` now implement the ``sample`` method for doing random sampling within groups (:issue:`31775`) | |||
There was a problem hiding this comment.
Need the full path to these classes in the docs.
pandas/core/groupby/groupby.py
Outdated
| else: | ||
| ws = [None] * self.ngroups | ||
|
|
||
| if random_state: |
There was a problem hiding this comment.
I think either is fine. Either we get a random state from NumPy's global random state initially and re-use it, or we have each group draw from the global random state pool. It's similar to these two calls
.sample(random_state=0)# each call uses the seed 0.sample(random_state=np.random.RandomState(0))# each call makes an independent draw
| the underlying object and will be used as sampling probabilities | ||
| after normalization within each group. | ||
| random_state : int, array-like, BitGenerator, np.random.RandomState, optional | ||
| If int, array-like, or BitGenerator (NumPy>=1.17), seed for |
There was a problem hiding this comment.
It it is a BitGenerator, do you use a Generator to produce the random samples or a RandomState. Best practice is to use a Generator since RandomState is effectively frozen in time. If an int, it is used as a seed for np.random.default_rng() or RandomState if NumPy >= 1.17?
There was a problem hiding this comment.
This is following a pattern similar to the one used in pandas.core.generic.sample of processing the random_state according to pandas.core.common.random_state:
Line 394 in c71bfc3
|
thanks @dsaxton very nice! |
black pandasgit diff upstream/master -u -- "*.py" | flake8 --diff