/pandas

ENH: Allow exponentially weighted functions to specify alpha directly #12492

Closed
wants to merge 1 commit into
from

Conversation

Reviewers
No reviews
Assignees
No one assigned
Labels
API Design Numeric
Projects
None yet
Milestone
  0.18.0
2 participants
@evectant @jreback
@evectant
Contributor

evectant commented Feb 29, 2016

  • Closes #10789
  • Passes Travis CI, nosetests, and flake8
  • Covered by new tests in test_window.py:TestMoments
  • Documented in computation.rst and v0.18.0.txt
@jreback
Contributor

jreback commented Feb 29, 2016

I am not sure we do any validity checks on any com parameters

eg is alpha=0 valid?
@evectant
Contributor

evectant commented Feb 29, 2016

That's actually a question I had. Zero alpha is not okay, but zero half-life isn't either, and the code is not checking for that. Should I add the checks for 0 < alpha <= 1 and span >= 1, com >= 0, half-life > 0 following from that?
@jreback
Contributor

jreback commented Feb 29, 2016

yep I think we need some sanity checks as well here - raising a helpful ValueError would be nice (rather than giving weird/odd results)

@jreback jreback commented on an outdated diff Mar 1, 2016

pandas/tests/test_window.py
@@ -1066,6 +1066,67 @@ def test_ewma_halflife_arg(self):
halflife=50)
self.assertRaises(Exception, mom.ewma, self.arr)
+ def test_ewma_alpha_old_api(self):
+ with tm.assert_produces_warning(FutureWarning, check_stacklevel=False):
@jreback

jreback Mar 1, 2016

Contributor

add the issue number as a comment here

jreback added this to the 0.18.0 milestone Mar 1, 2016

@jreback jreback added API Design Numeric labels Mar 1, 2016

@jreback
Contributor

jreback commented Mar 1, 2016

thanks @evectant just minor comments.
ping when pushed and green.
@evectant
Contributor

evectant commented Mar 2, 2016

@jreback: Added the comments, slightly reworded the description in computation.rst. Pushed and green.

@jreback jreback commented on an outdated diff Mar 2, 2016

pandas/core/window.py
:math:`\alpha = 1 - exp(log(0.5) / halflife)`
+ alpha : float, optional
+ Specify smoothing factor :math:`\alpha` directly
@jreback

jreback Mar 2, 2016

Contributor

can you add a versionadded tag here

@jreback jreback commented on an outdated diff Mar 2, 2016

doc/source/computation.rst
\end{cases}
-One must specify precisely one of the three to the EW functions. **Span**
-corresponds to what is commonly called a "20-day EW moving average" for
-example. **Center of mass** has a more physical interpretation. For example,
-**span** = 20 corresponds to **com** = 9.5. **Halflife** is the period of
-time for the exponential weight to reduce to one half.
+One must specify precisely one of **span**, **center of mass**, **half-life**
+and **alpha** to the EW functions. **Span** corresponds to what is commonly
@jreback

jreback Mar 2, 2016

Contributor

add somewhere that alpha can be specified starting in 0.18.0

@jreback jreback commented on an outdated diff Mar 2, 2016

doc/source/computation.rst
\end{cases}
-One must specify precisely one of the three to the EW functions. **Span**
-corresponds to what is commonly called a "20-day EW moving average" for
-example. **Center of mass** has a more physical interpretation. For example,
-**span** = 20 corresponds to **com** = 9.5. **Halflife** is the period of
-time for the exponential weight to reduce to one half.
+One must specify precisely one of **span**, **center of mass**, **half-life**
+and **alpha** to the EW functions. **Span** corresponds to what is commonly
+called an "N-day EW moving average". **Center of mass** has a more physical
+interpretation and can be thought of in terms of span: :math:`c = (s - 1) / 2`.
+**Half-life** is the period of time for the exponential weight to reduce to
@jreback

jreback Mar 2, 2016

Contributor

can you add a mini chart showing the allowed domains (use csv-table to show it in markdown)

@jreback jreback commented on an outdated diff Mar 2, 2016

pandas/core/window.py
@@ -1037,7 +1041,7 @@ class EWM(_Rolling):
Notes
-----
- Either center of mass, span or halflife must be specified
+ Either center of mass, span, half-life or alpha must be specified
EWMA is sometimes specified using a "span" parameter `s`, we have that the
decay parameter :math:`\alpha` is related to the span as
@jreback

jreback Mar 2, 2016

Contributor

can you add the domain requirements here as well (after the formula is good)

@jreback jreback commented on an outdated diff Mar 2, 2016

pandas/stats/moments.py
@@ -68,12 +68,16 @@
_ewm_kw = r"""com : float. optional
- Center of mass: :math:`\alpha = 1 / (1 + com)`,
+ Specify decay in terms of center of mass,
+ :math:`\alpha = 1 / (1 + com)`
span : float, optional
@jreback

jreback Mar 2, 2016

Contributor

add the domain here as well
@jreback
Contributor

jreback commented Mar 2, 2016

looks good. just some minor documentation comments.
@jreback
Contributor

jreback commented Mar 5, 2016

pls rebase and push again, master has been updated

@jreback jreback commented on an outdated diff Mar 6, 2016

pandas/core/window.py
if valid_count > 1:
- raise Exception("com, span, and halflife are mutually exclusive")
-
- if span is not None:
- # convert span to center of mass
+ raise Exception("com, span, halflife, and alpha "
@jreback

jreback Mar 6, 2016

Contributor

can you make this a ValueError

@jreback jreback commented on an outdated diff Mar 6, 2016

pandas/core/window.py
decay = 1 - np.exp(np.log(0.5) / halflife)
com = 1 / decay - 1
- elif com is None:
- raise Exception("Must pass one of com, span, or halflife")
+ elif alpha is not None:
+ if alpha <= 0 or alpha > 1:
+ raise ValueError("alpha must satisfy: 0 < alpha <= 1")
+ com = (1.0 - alpha) / alpha
+ else:
+ raise Exception("Must pass one of com, span, halflife, or alpha")
@jreback

jreback Mar 6, 2016

Contributor

same here
@evectant evectant ENH: Allow exponentially weighted functions to specify alpha directly 
Closes #10789. Adds domain checks for exponentially weighted functions.
a8c2753
@evectant
Contributor

evectant commented Mar 6, 2016

@jreback: Pushed and green.

jreback closed this in 547c784 Mar 6, 2016

@jreback
Contributor

jreback commented Mar 6, 2016

thanks @evectant great PR!
@jreback
Contributor

jreback commented Mar 6, 2016

give a look at the built documentation (should be done shortly), http://pandas-docs.github.io/pandas-docs-travis/ and make sure looks ok (obviously where you changed things!). If not pls issue a follow.

evectant referenced this pull request Mar 6, 2016

Closed

BLD: Install texlive packages in Travis CI #12544

@evectant
Contributor

evectant commented Mar 7, 2016

@jreback The docs look good to me. Thanks for your help with the commit.
@jreback
Contributor

jreback commented Mar 7, 2016

looks good

thanks for the PR!

evectant deleted the evectant:issue-10789 branch Mar 21, 2016

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment