Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SPARK-44349][R] Add math functions to SparkR #41914

Closed
wants to merge 6 commits into from

Conversation

zhengruifeng
Copy link
Contributor

@zhengruifeng zhengruifeng commented Jul 10, 2023

What changes were proposed in this pull request?

Add following math functions to SparkR:

  • e (discard together with pi, they are both math constants)
  • pi (conflict with R's built-in pi constant)
  • std
  • ln
  • negative
  • positive
  • pow (discard together with power)
  • power (conflict with R's built-in power function)
  • width_bucket

Why are the changes needed?

for parity

Does this PR introduce any user-facing change?

yes

How was this patch tested?

updated UT

@zhengruifeng zhengruifeng marked this pull request as draft July 10, 2023 05:18
@zhengruifeng zhengruifeng force-pushed the math_r branch 4 times, most recently from 3688761 to 24dcf31 Compare July 11, 2023 04:50
@zhengruifeng zhengruifeng marked this pull request as ready for review July 11, 2023 04:50
@zhengruifeng
Copy link
Contributor Author

zhengruifeng commented Jul 11, 2023

pi and power are built-in R functions:

> pi
[1] 3.141593
> power
function (lambda = 1)
{
    if (!is.numeric(lambda) || is.na(lambda))
        stop("invalid argument 'lambda'")
    if (lambda <= 0)
        return(make.link("log"))
    if (lambda == 1)
        return(make.link("identity"))
    linkfun <- function(mu) mu^lambda
    linkinv <- function(eta) pmax(eta^(1/lambda), .Machine$double.eps)
    mu.eta <- function(eta) pmax((1/lambda) * eta^(1/lambda -
        1), .Machine$double.eps)
    valideta <- function(eta) all(is.finite(eta)) && all(eta >
        0)
    link <- paste0("mu^", round(lambda, 3))
    structure(list(linkfun = linkfun, linkinv = linkinv, mu.eta = mu.eta,
        valideta = valideta, name = link), class = "link-glm")
}
<bytecode: 0x1444e5aa8>
<environment: namespace:stats>

so remove pi, e (it will be weird to add e alone), pow, power

@zhengruifeng
Copy link
Contributor Author

merged to master

ragnarok56 pushed a commit to ragnarok56/spark that referenced this pull request Mar 2, 2024
### What changes were proposed in this pull request?
Add following math functions to `SparkR`:

- ~~e~~ (discard together with `pi`, they are both math constants)
- ~~pi~~ (conflict with R's built-in `pi` constant)
- std
- ln
- negative
- positive
- ~~pow~~ (discard together with `power`)
- ~~power~~ (conflict with R's built-in `power` function)
- width_bucket

### Why are the changes needed?
for parity

### Does this PR introduce _any_ user-facing change?
yes

### How was this patch tested?
updated UT

Closes apache#41914 from zhengruifeng/math_r.

Authored-by: Ruifeng Zheng <ruifengz@apache.org>
Signed-off-by: Ruifeng Zheng <ruifengz@apache.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
2 participants