Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

When using set() "Internal error, please report" #2996

Closed
iamamutt opened this issue Aug 10, 2018 · 6 comments
Closed

When using set() "Internal error, please report" #2996

iamamutt opened this issue Aug 10, 2018 · 6 comments
Assignees
Milestone

Comments

@iamamutt
Copy link

iamamutt commented Aug 10, 2018

Get an internal error with set() when loading an object to an environment and getting then using object from same environment.

Example:

library(data.table)
print(sessionInfo())

load_tmp_rda <- function(v, folder, env) {
  obj_name <- paste0("tmp", v)
  filename <- file.path(folder, paste0(obj_name, ".rda"))
  load(filename, env)
  obj <- get(obj_name, env)
  message(obj_name, " has ", nrow(obj), " rows")
  set(obj, NULL, "version", v)
}

# doesn't work
local({
  tmp1 <- data.table(runif(10000))
  tmp_dir <- tempdir()
  tmp_file <- file.path(tmp_dir, "tmp1.rda")
  save(tmp1, file=tmp_file)
  this <- environment()
  load_tmp_rda(1L, tmp_dir, this)
})


# works
local({
  tmp1 <- data.table(runif(10000))
  obj_name <- paste0("tmp", 1L)
  this <- environment()
  obj <- get(obj_name, this)
  set(obj, NULL, "version", 1L)
})

Output:

R version 3.5.1 (2018-07-02)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows >= 8 x64 (build 9200)

Matrix products: default

locale:
[1] LC_COLLATE=English_United States.1252  LC_CTYPE=English_United States.1252    LC_MONETARY=English_United States.1252 LC_NUMERIC=C                          
[5] LC_TIME=English_United States.1252    

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] data.table_1.11.4

loaded via a namespace (and not attached):
 [1] Rcpp_0.12.18     pillar_1.3.0     compiler_3.5.1   plyr_1.8.4       bindr_0.1.1      tools_3.5.1      digest_0.6.15    packrat_0.4.9-3  evaluate_0.11   
[10] tibble_1.4.2     gtable_0.2.0     pkgconfig_2.0.1  rlang_0.2.1      rstudioapi_0.7   bindrcpp_0.2.2   gridExtra_2.3    stringr_1.3.1    dplyr_0.7.6     
[19] styler_1.0.2     knitr_1.20       rprojroot_1.3-2  grid_3.5.1       tidyselect_0.2.4 glue_1.3.0       R6_2.2.2         rmarkdown_1.10   ggplot2_3.0.0   
[28] purrr_0.2.5      mejr_1.2.0.9000  magrittr_1.5     rematch2_2.0.1   backports_1.1.2  scales_0.5.0     htmltools_0.3.6  rfmt_1.0         assertthat_0.2.0
[37] colorspace_1.3-2 stringi_1.1.7    lazyeval_0.2.1   munsell_0.5.0    crayon_1.3.4    
tmp1 has 10000 rows
Error in set(obj, NULL, "version", v) : 
  Internal error, please report (including result of sessionInfo()) to data.table issue tracker: oldtncol (0) < oldncol (1) but tl of class is marked.
@MichaelChirico
Copy link
Member

MichaelChirico commented Aug 10, 2018

@jangorecki
Copy link
Member

jangorecki commented Mar 27, 2019

using setDT resolve the problem, this is documented that after reading from binary format you should use setDT.

library(data.table)

load_tmp_rda <- function(v, folder, env) {
  obj_name <- paste0("tmp", v)
  filename <- file.path(folder, paste0(obj_name, ".rda"))
  load(filename, env)
  obj <- get(obj_name, env)
  setDT(obj)     ## added this line
  message(obj_name, " has ", nrow(obj), " rows")
  set(obj, NULL, "version", v)
}

local({
  tmp1 <- data.table(runif(10000))
  tmp_dir <- tempdir()
  tmp_file <- file.path(tmp_dir, "tmp1.rda")
  save(tmp1, file=tmp_file)
  this <- environment()
  load_tmp_rda(1L, tmp_dir, this)
})
#tmp1 has 10000 rows

error is gone.
@iamamutt please report back if you are able to reproduce issue using setDT after reading from binary format.

@MichaelChirico
Copy link
Member

MichaelChirico commented Mar 27, 2019

Internal error, please report (including result of sessionInfo()) to data.table issue tracker: oldtncol (0) < oldncol (1) but tl of class is marked.

Probably we can improve the error message, it sounds like you did something really scary to see Internal error.

Can have a branch if oldtncol is 0 to recommend setDT & Internal error otherwise.

@MichaelChirico
Copy link
Member

MichaelChirico commented Mar 28, 2019

Is there not an FAQ about this issue? Or something in the Wiki? Would like to refer to something in the error message...

@franknarf1
Copy link
Contributor

franknarf1 commented Mar 28, 2019

@MichaelChirico

Is there not an FAQ about this issue? Or something in the Wiki? Would like to refer to something in the error message...

In the FAQ, under "Reading data.table from RDS or RData file" , it says to use alloc.col.

I always use setDT. If that's recommended or just as good as alloc.col, maybe the FAQ can be updated to match.

@jangorecki
Copy link
Member

jangorecki commented Mar 28, 2019

error message has to point to setDT, any internal errors should not be exposed to users.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants