Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improvements for parameterize/queues #680

Closed
1 of 2 tasks
stefdoerr opened this issue May 10, 2018 · 5 comments
Closed
1 of 2 tasks

Improvements for parameterize/queues #680

stefdoerr opened this issue May 10, 2018 · 5 comments

Comments

@stefdoerr
Copy link
Contributor

stefdoerr commented May 10, 2018

  • Make names more sensible to distinguish them on queueing systems (i.e. for parameterize add prm at the beginning, or anything the user specifies?)
  • Queue.wait should have an option for reporting current inprogress every x minutes. Dihedral jobs with thousands of angles/dihedrals look quite dead after a few days.

@raimis you think you could do number 1?

@raimis
Copy link
Contributor

raimis commented May 10, 2018

Ideally, the job name should identify molecule, dihedral, and rotomer index, but in practice queue systems limit job names to a few characters. I'll think about a naming scheme.

@j3mdamas
Copy link
Contributor

The current name is automatically generated through _autoJobName. Example for LSF:

def _autoJobName(self, path):
return os.path.basename(os.path.abspath(path)) + '_' + ''.join([random.choice(string.digits) for _ in range(5)])

If we draw the parallelism with adaptive, where a single jobname is used for all epochs and replicas, we could do the same in parameterize cli:

SimQueue.jobname = args.filename.split('.')[0]

If one wants, we can also add -min, -esp, and -dihe to the end, if necessary to distinguish between all.

@j3mdamas j3mdamas added this to the 1.14.0 (stable) milestone May 10, 2018
@j3mdamas
Copy link
Contributor

For 2, here:

def wait(self, sentinel=False):

we can add a report=None argument, which throws a logging message every report seconds, which we can set in the parameterize cli.

@stefdoerr
Copy link
Contributor Author

stefdoerr commented May 17, 2018

For 2: #687

@j3mdamas j3mdamas assigned j3mdamas and unassigned raimis Jul 9, 2018
@j3mdamas j3mdamas removed their assignment Feb 20, 2019
@giadefa giadefa removed this from the 1.14.0 (stable) milestone Mar 1, 2019
@stefdoerr
Copy link
Contributor Author

#1 is very queue specific. Considering we have PM now we will just solve this there

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants