Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rJava support #59

Merged
merged 8 commits into from
Nov 5, 2018
Merged

rJava support #59

merged 8 commits into from
Nov 5, 2018

Conversation

weshinsley
Copy link
Contributor

@weshinsley weshinsley commented Jul 25, 2018

This seems to work!

options(didehpc.use_java = TRUE)
options(didehpc.java_home = "\\\\fi--san03\\homes\\wrh1\\jre")

(java_home here points to a JVM I made; it can be left blank to take the cluster's favourite)

options(didehpc.cluster = 'fi--didemrchnb')
packages <- c("rJava", "bartMachine")
ctx <- context::context_save("contexts", packages = packages, sources = "test.R")
obj <- didehpc::queue_didehpc(ctx)
t <- obj$enqueue(testj())

where test.R says:

testj <- function() {
  library(rJava)
  options(java.parameters = "-Xmx5g")
  library(bartMachine)
  paste0(getOption("java.parameters"), " : ",Sys.getenv("JAVA_HOME"))
}

And the result is:-

Welcome to bartMachine v1.2.3! You have 0.51GB memory available.

If you run out of memory, restart R, and use e.g.
'options(java.parameters = "-Xmx5g")' for 5GB of RAM before you call
'library(bartMachine)'.

This is fine - except that bartMachine only says we've got 0.51GB, not the 5Gb we asked for. This is because the two library commands in testj() are not actually doing anything - the libraries are already loaded by didehpc, hence the options are set too late.

The work-around for now is: after running the above once to provision the packages, set packages <- c() and recreate the context. Longer term, we should pass java.parameters formally as a configuration parameter.

@weshinsley weshinsley requested a review from richfitz July 25, 2018 14:31
Copy link
Member

@richfitz richfitz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you please run devtools::document() to regenerate the docs?

@weshinsley
Copy link
Contributor Author

Done - I've run devtools:document(), which has updated man\didehpc_config.Rd

@richfitz richfitz merged commit 544ee57 into master Nov 5, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants