Skip to content

potentially reg destructive bug: syncing in case of errors, doJobCollection #135

Description

@berndbischl
  1. I have 0 chance to post a reproducible example here

  2. What happened:

  • In a large benchmark, for some reason, loading a dependency failed somewhere.

  • When we call getStatus() (or anything else), bt fails with

Syncing 3677 files ...
Error in rbindlist(updates) :
  Item 22 has 5 columns, inconsistent with item 1 which has 6 columns. If instead you need to fill missing columns, use set argument 'fill' to TRUE.
  • this destroys all measures to further use results or status

  • I have the registry locally on my machine and can debug, and reproduce there

  • it failed with bt version 0.9.3, but I guess this is still present in the current GH version

  1. where do we fail:
> traceback()
7: rbindlist(updates) at syncRegistry.R#32
6: syncRegistry(reg = reg) at loadRegistry.R#78
5: loadRegistry("thresholdTuning-benchmark-files", work.dir = "~/cos/tune_threshold_benchmark/src",
       update.paths = TRUE) at testbt.R#3
4: eval(ei, envir)
3: eval(ei, envir)
2: withVisible(eval(ei, envir))
1: source("testbt.R")

the code is

  updates = lapply(fns, function(fn) {
    x = try(readRDS(fn), silent = TRUE)
    if (is.error(x)) NULL else x
  })

  failed = vlapply(updates, is.null)
  updates = rbindlist(updates)

here is what is in "updates"

> uu[[1]]
   job.id   started      done error memory written
1:   2316 1.502e+09 1.502e+09    NA     NA   FALSE

> uu[[22]]
   job.id   started      done
1:   1827 1.502e+09 1.502e+09
                                                                                                                                 error
1: Error loading registry dependencies: Error in loadRegistryDependencies(jc, switch.wd = FALSE) : \n  Failed to load packages: OpenML
   memory
1:     NA

you can see the problem: the element 22 lacks column "written"

  1. what might be wrong - I am reading the code and changing bt src code, but I need to guess here to some extent
#' @export
doJobCollection.JobCollection = function(jc, output = NULL) {
  now = function() strftime(Sys.time())

  error = function(msg, ...) {
    now = ustamp()
    updates = data.table(job.id = jc$jobs$job.id, started = now, done = now,
      error = stri_trunc(stri_trim_both(sprintf(msg, ...)), 500L, " [truncated]"),
      memory = NA_real_, key = "job.id")
    writeRDS(updates, file = file.path(jc$file.dir, "updates", sprintf("%s.rds", jc$job.hash)))
    invisible(NULL)
  }

in the above code, we never set "written" to anything.
later we seem to exit here

  # load registry dependencies: packages, source files, ...
  # note that this should happen _before_ parallelMap is initialized
  ok = try(loadRegistryDependencies(jc, must.work = TRUE), silent = TRUE)
  if (is.error(ok))
    return(error("Error loading registry dependencies: %s", as.character(ok)))

if this only happens in some jobs, but not all --> we are fucked

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions