Error and output log paths cause scripts to fail

[mmehan]

Not sure if this is unique to our cluster but I found that the error and output log paths in script.sh cause the job array to fail. When I initially ran the example in the README the individual jobs in the array didn't generate any output and the final job didn't complete, which caused my R session to hang. I got the same behavior when I submitted the script.sh job my self on the cluster. There were no logs in the log folder so I added the full path to the log paths in script.sh and it completed successfully. Here's the change I made to lines 209-210 of qsub_run.sh to get it to work:

    sprintf("#$ -e %s/log/log.$TASK_ID.e.txt\n", qs$remote_dir),
    sprintf("#$ -o %s/log/log.$TASK_ID.o.txt\n", qs$remote_dir),

Thanks for developing this package, extremely useful!

[rcannood]

Thanks @mmehan for your comments! I'm not sure what the underlying solution was, but I modified qsub_run.R based on your suggestions.

Are you able to run the example code in the readme now?

[mmehan]

Yeah it's working now - thanks! I had to make another minor modification to get the rsync from the remote to work. I think the rsync command failed because I defined my remote server in the config with the ssh port at the end like the README example. This led to two colons in the remote destination argument for rsync. When I removed the port from the remote_src in rsync_remote it transferred successfully. This didn't come up before because the master branch is using scp.

[rcannood]

Thanks, this was an oversight on my part :) I updated the rsync function such that it should now work with both notations.

[rcannood]

qsub 1.1.0 includes the fixes for this issue and is on its way to CRAN. Thanks for the feedback!