Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GForce should be able to work with `:=` as well. #1414

Open
arunsrinivasan opened this Issue Oct 29, 2015 · 2 comments

Comments

Projects
None yet
5 participants
@arunsrinivasan
Copy link
Member

arunsrinivasan commented Oct 29, 2015

No description provided.

@arunsrinivasan arunsrinivasan added this to the v1.9.8 milestone Oct 29, 2015

@arunsrinivasan arunsrinivasan self-assigned this Nov 12, 2015

@arunsrinivasan arunsrinivasan modified the milestones: v2.0.0, v1.9.8 Apr 10, 2016

@franknarf1

This comment has been minimized.

Copy link

franknarf1 commented May 18, 2016

Just ran into this today looking at a question on SO:

actions = data.table(User_id = c("Carl","Carl","Carl","Lisa","Moe"),
                     category = c(1,1,2,2,1),
                     value= c(10,20,30,40,50))
users = actions[, other_var := 1, by=User_id]

# verbose says: the following is not optimized
users[, value_one := 0 ]
users[actions[category==1], value_one := sum(value), on="User_id", by=.EACHI, verbose=TRUE]

# verbose says: the following is optimized
rbind( 
    actions[category==1], 
    unique(actions[,"User_id", with=FALSE])[, value := 0 ],
fill=TRUE)[, sum(value), by=User_id, verbose=TRUE]

To me, the first way looks idiomatic, considering the variable needs to end up in users in the end.

Another: https://stackoverflow.com/a/47338118/ (gtail)

Another https://stackoverflow.com/a/51569126/ should do DT[, mx := max(pt), by=Subject][, diff := mx - pt][] I guess

Another, specifically interested in memory performance: https://stackoverflow.com/q/52189712 "data.table reference semantics: memory usage of iterating through all columns"

Another, wants to scale/demean multiple variables: https://stackoverflow.com/q/52528123

Another taking max by group with a subsetting condition and adding with := (see akrun's answer) https://stackoverflow.com/a/54911855/ also related to the already-completed part of #971

@mattdowle mattdowle removed this from the Candidate milestone May 10, 2018

@brodieG

This comment has been minimized.

Copy link

brodieG commented Mar 11, 2019

Just wanted to emphasize that enabling this can allow using GForce effectively for complex expressions, albeit with some work. For example I show in this post how to enable it for:

slope <- function(x, y) {
  x_ux <- x - mean(x)
  uy <- mean(y)
  sum(x_ux * (y - uy)) / sum(x_ux ^ 2)
}

By doing:

DT <- data.table(grp, x, y)
setkey(DT, grp)
DTsum <- DT[, .(ux=mean(x), uy=mean(y)), keyby=grp]
DT[DTsum, `:=`(x_ux=x - ux, y_uy=y - uy)]
DT[, `:=`(x_ux.y_uy=x_ux * y_uy, x_ux2=x_ux^2)]
DTsum <- DT[, .(x_ux.y_uy=sum(x_ux.y_uy), x_ux2=sum(x_ux2)), keyby=grp]
res.slope.dt2 <- DTsum[, .(grp, V1=x_ux.y_uy / x_ux2)]

Whereas if GForce was supported in := we could do:

DT <- data.table(grp, x, y)
DT[, `:=`(ux=mean(x), uy=mean(y)), keyby=grp]
DT[, `:=`(x_ux=x - ux, y_uy=y - uy)]
DT[, `:=`(x_ux.y_uy=x_ux * y_uy, x_ux2=x_ux^2)]
DTsum <- DT[, .(x_ux.y_uy=sum(x_ux.y_uy), x_ux2=sum(x_ux2)), keyby=grp]
res.slope.dt3 <- DTsum[, .(grp, x_ux.y_uy/x_ux2)]

Which looks cleaner and should be faster.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.