Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

data.table j=list(var) slower than j=var #34

Open
tdhock opened this issue Mar 14, 2024 · 4 comments
Open

data.table j=list(var) slower than j=var #34

tdhock opened this issue Mar 14, 2024 · 4 comments

Comments

@tdhock
Copy link
Owner

tdhock commented Mar 14, 2024

@Anirban166 @DorisAmoakohene would be good to translate Rdatatable/data.table#712 to atime

@tdhock tdhock changed the title j=list(var) slower than j=var data.table j=list(var) slower than j=var Mar 14, 2024
@DorisAmoakohene
Copy link
Collaborator

@tdhock I tried this issue and I keep on getting this error

Error: object 'DT' not found

I have my DT defined, but looks like is not able to run in the setup

This is my atime code

result.withby <- atime::atime(
  N = 1:1e7,
  setup = {
    DT <- data.table(x=rep(N, 2L), y=1L)
    dd <- data.table(x=N,key="x")
  },
  expr.list = list(
    ans1 <- DT[, .N, by=x],
    ans2 <- DT[, list(N = .N), by=x],
    de <- dd[dd,1,by=.EACHI],
    df <- dd[dd,list(1),by=.EACHI],
    dg <- dd[dd,x,by=.EACHI],
    dh <- dd[dd,list(x),by=.EACHI]
  )
)

@tdhock
Copy link
Owner Author

tdhock commented Mar 16, 2024

It is normal to get that error about DT not found.
expr.list should be a list of un-evaluated expressions, see ?atime man page and examples.
I'm not sure you need to use expr.list at all here though.

@DorisAmoakohene
Copy link
Collaborator

@tdhock this is the result for this with this code

result.withby <- atime::atime(
  N =unique(as.integer(1:1e9)),
  setup = {
    DT <- data.table(x=rep(N, 2L), y=1L)
    dd <- data.table(x=N,key="x")
  },
  seconds.limit = 0.001,
  
    ans1 = DT[, .N, by=x],
    ans2 = DT[, list(N = .N), by=x],
    de = dd[dd,1,by=.EACHI],
    df =  dd[dd,list(1),by=.EACHI],
    dg = dd[dd,x,by=.EACHI],
    dh = dd[dd,list(x),by=.EACHI]
  
)

result withby

@tdhock
Copy link
Owner Author

tdhock commented Mar 26, 2024

can you please increase seconds.limit?
and also maybe only include the three examples here Rdatatable/data.table#712 (comment)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants