BDS LSF Support

[jhooge]

Hello Pablo,

I'm working as a Bioinformatician and Bayer Healthcare and I'm responsible for the development of our NGS pipelines. Currently I'm using bpipe, however due to its code intransparency I'm a bit unhappy with it and I'm currently looking into alternatives. We're using an LSF job scheduler on our HPC and BDS seems a viable alternative to our current pipeline solution, that is, if it did support LSF. However, skimming through your documentation, this doesn't seem to be the case. Are you planning to provide LSF support in the future?

Kind regards,
Jens

[pcingola]

Hi! I think I can add support for LSF (it shouldn't be hard).
Would you be willing to beta test it?

[jhooge]

Hello Pablo,

unfortunately I wouldn't be able to convince my department of BDS, when
I'll use the word "beta", and due to my time constraints wouldn't be of
much help in testing.
I will keep an eye on your project though.

Best,
Jens

2014-09-29 17:22 GMT+02:00 Pablo Cingolani notifications@github.com:

Hi! I think I can add support for LSF (it shouldn't be hard).
Would you be willing to beta test it?

—
Reply to this email directly or view it on GitHub
#33 (comment)
.

[pcingola]

Sorry, but I'm a bit confused.
You asked me if I was willing to add support for LSF, and when I tell you that "yes, I'm willing to add support", your answer is "then I don't want it"? Am I misunderstanding you answer?

[jhooge]

Hello Pablo,

LSF support would of course be appreciated, but I won't be able to beta
test it myself.

Best,
Jens

2014-09-29 17:41 GMT+02:00 Pablo Cingolani notifications@github.com:

Sorry, but I'm a bit confused.
You asked me if I was willing to add support for LSF, and when I tell you
that "yes, I'm willing to add support", your answer is "then I don't want
it"? Am I misunderstanding you answer?

—
Reply to this email directly or view it on GitHub
#33 (comment)
.

[ryanabo]

I would be interested in LSF support as well. I would be willing to beta test its functionality. Is this something that is in development?

[pcingola]

Awsome.
It is not in development right now, but I'll try to work on this (hopefully in the next couple of weeks).

[johnlees]

I would also be interested in using this on LSF, and would be happy to help beta test

[mxqian]

I am interested in using this on LSF. What's going on about this now? Has LSF been supported yet? Thanks.

[pcingola]

I don't have access to an LSF cluster, but you should be able to easily adapt bds for any cluster type using the "Generic cluster" hooks.

[mxqian]

I'm trying to understand and do it.
So, there were 4 perl scripts/settings:
clusterGenericRun = ~/.bds/clusterGeneric/run.pl
clusterGenericKill = ~/.bds/clusterGeneric/kill.pl
clusterGenericStat = ~/.bds/clusterGeneric/stat.pl
clusterGenericPostMortemInfo = ~/.bds/clusterGeneric/postMortemInfo.pl

One way is to replace these files according to the cluster (e.g. LFS/bsub). And then call "bds -s generic". Right?

Is it possible to rename the variable and folder name? For example:
clusterLFSRun =~/.bds/clusterLFS/run.pl
And then how let the "-s" to recognize the option?

Many thanks in advance.

[pcingola]

Exactly.
You can other scripts, just change the bds.config file accordingly. Look for
the entries: clusterGenericRun, clusterGenericKill, clusterGenericStat and clusterGenericPostMortemInfo. In the example bds.config there is a brief
description for each. I'm assuming it should be straightforward to
implement, although I don't use LSF.

It is not possible to rename at the moment. But if you send me your
implementation, I'll add them using that name and add the "-s lsf" option
for it.

[rfarouni]

Hi,
I have modified the files to run on LSF. Here is what I have:

For kill.pl

#!/usr/bin/perl
#---
# Parse command line arguments
#---
die "Error: Missing arguments.\nUsage: kill.pl jobId\n" if $#ARGV < 0 ;
#$jobId = shift @ARGV;
$jobId = join(' ', @ARGV);

# Execute cluster command to kill task
#---
$exitCode = system "bkill $jobId";

# OK
exit($exitCode);

for run.pl

#!/usr/bin/perl

use POSIX;

die "Error: Missing arguments.\nUsage: run.pl timeout cpus mem queue saveStdout saveStderr cmd arg1 ... argN\n" if $#ARGV < 6 ;

$timeout = shift @ARGV;
$cpus = shift @ARGV;
$mem = shift @ARGV;
$queue = shift @ARGV;
$saveStdout = shift @ARGV;
$saveStderr = shift @ARGV;
$cmd = join(' ', @ARGV);

$qsub = "bsub ";

if( $cpus > 0 ) {
	$qsub .= "-n $cpus";
}


if( $mem > 0 ) {
	$mem = ceil($mem/1000000); # MB
	$qsub .= "-M $mem ";
}
if( $timeout > 0 ) {
	$timeout = ceil($timeout/60); # minute
	$qsub .= "-W $timeout ";
}
if ( $queue ne "" ) {
        $qsub .= "-q $queue "
}

$pid = open QSUB, " | $qsub";
die "Cannot run command '$qsub'\n" if ! kill(0, $pid); # Check that process exists
print QSUB "$cmd\n";		# Send cluster's task via qsub's STDIN
close QSUB;

exit(0);

for stat.pl

#!/usr/bin/perl

$exitCode = system "bjobs";

# OK
exit($exitCode);

for postMortemInfo.pl

#!/usr/bin/perl

die "Error: Missing arguments.\nUsage: postMortemInfo.pl jobId\n" if $#ARGV < 0 ;
$jobId = shift @ARGV;

#---
# Execute cluster command to show task details
#---
$exitCode = system "bjobs $jobId";

# OK
exit($exitCode);

[pcingola]

Great! I'll add them to the repository.
Thank you!

[ekernf01]

I ran into a couple errors while getting this working. Here they are, in case any of the info is useful to others.

In run.pl:

1. A Perl error about how the ceil subroutine is not defined. Solved by adding use POSIX; to the top.
2. An LSF error Bad argument for option -n. Job not submitted. Solved by filling in a missing space: $qsub .= "-n $cpus"; becomes $qsub .= "-n $cpus ";.
3. A sporadic error with incorrect or insufficient memory requests. Solved by correcting LSF syntax: $qsub .= "-M $mem "; becomes $qsub .= "-R rusage[mem=$mem] ";.

[ekernf01]

Here's another one! I got an error about BDS being unable to find the .exitCode file. Solved by replacing the end of run.pl with this. The problem is that bsub does not conform to the criteria for a clusterGeneric run script: it prints too much junk aside from the job ID.

use IPC::Open2;
$pid = open2(QSUB_IN, QSUB_OUT, " $qsub $cmd ", );
@result_split = split('<|>', <QSUB_IN>); # creates an @answer array
print @result_split[1] . "\n" ;
close QSUB_OUT;
close QSUB_IN;

# OK
exit(0);

[pcingola]

Added changes to scripts. I don't have an LSF cluster to test, so I didn't try them.

[ekernf01]

Well, these are working fine for me so I think you can leave this issue closed until or unless it comes up again.