Get both stdout and stderr

[mllg]

I'm wondering if it is possible to get both output streams together, with lines in the right (chronological) order. One way to achieve this with system2 is to redirect stdout and stderr to the same connection, but this does not seem to work with callr, c.f. the following small snippet:

library(callr)

f = function() {
  print("print")
  message("message")
  return("return")
}
out = tempfile()

# with callr
r(f, stdout = out, stderr = out)
readLines(out)
# -> "print" missing

# with system2
rscript = tempfile()
fp = file(rscript, "w")
dump("f", file = fp)
writeLines("f()", fp)
close(fp)

system2("Rscript", rscript, stdout = out, stderr = out)
readLines(out)

[gaborcsardi]

Sorry for the slow response.

Writing both stdout and stderr to the same file is tricky, because of non-atomic IO, possible race conditions, the buffering of stdout, possible more problems.

Making this to work across platforms is not an easy task imo.

But if you have some ideas, I would be happy to hear them.

[gaborcsardi]

Maybe if we just redirect with 2&>1 >file that works well enough? Let me give it a try....

[mllg]

Sorry for not responding in time.
Another possibility would be to just wrap the expression in a R function which calls sink twice (for stdout and stderr):

wrapper = function(expr, dst) {
  fp = file(dst, open = "wt")
  sink(file = fp)
  sink(file = fp, type = "message")
  on.exit({ sink(type = "message"); sink(type = "output"); close(fp) })
  expr
}

This is basically what I use in batchtools and it works quite okay most of the time. Your solution might be even better, because you could also record output from C which does not use Rprintf etc. However, the wrapper might be a nice fallback for windows where you do not have pipes and redirections (or do you?).

[gaborcsardi]

sink would work for r, but not for rcmd, because that just runs R CMD. So I would prefer an approach that works for both, because they both use the same safe_system internally.

Redirection works fine on windows, even 2>&1 is fine, in fact R itself uses redirection in system:

...
    if (ignore.stdout)
        command <- paste(command, ">/dev/null")
    if (ignore.stderr)
        command <- paste(command, "2>/dev/null")
...

So I think this will be fine. r (and rcmd) both return stdout and stderr separately, and I would keep this. But if you specify the same file for stdout and stderr, then I would do 2>&1.

[gaborcsardi]

Oh, wait. If I redirect both to the same file, then I cannot return them separately.....

[gaborcsardi]

OK, looks like there is no way to collect stdout and stderr in the right order...... sink() would be the only way, i.e. within R, but that does not work for rcmd().

I can "fix" the current behavior, in that I can make sure that if you redirect output and error to the same file, then they will be in the file, although in the wrong order....

[gaborcsardi]

Hmmm, looking at this again, r does not return the output, actually....
So I guess we could use sink() for it.