Join GitHub today
GitHub is home to over 40 million developers working together to host and review code, manage projects, and build software together.Sign up
LSF: Accept use_stdin in the constructor #360
Right now, all LSF options can be specified either in the config OR in the constructor arguments, except the new
I don't see why we'd want
cluster = LSFCluster(cores=15, memory='25GB', use_stdin=True) # <-- now allowed
Side note: I suspect this new setting will be needed by many, if not most, LSF users, so I added some verbose documentation for it.
FWIW, I tested these changes on my LSF cluster, and they work as expected.
OK, I got the tests passing, but I'm seeing intermittent failures from Travis (unrelated to this PR).
FWIW, here's the error:
At first this error appeared in the
I'll force-push one more time to see if I can get lucky with a successful build.
@stuarteberg do you think we should default to
Also for further reference could you answer these two questions:
Disclaimer: I am not an LSF expert, and I only have experience with one LSF cluster, of which I am merely a user, not an administrator.
Yes, I think we should. If
Again, I'm not an expert, but if
Yes, both of those work for me, as long as
None of us are LSF experts even less LSF administrators ... as someone who has access to a LSF cluster and from your earlier comments I think you qualify as the dask-jobqueue LSF expert ;-).
Thanks a lot your feed-back, it is extremely useful! Also it aligns very much with my understanding of the problem so I think we should switch to use_stdin=True by default.
If you don't mind, a separate PR would be preferrable. This PR is adding the use_stdin parameter, which was an oversight of #307.