implemented passing memory requirements, removed mandatary queue spec #33
Conversation
| @@ -16,6 +16,8 @@ class RubinTask: | |||
| executable: str = None | |||
| maxwalltime: int = None | |||
| maxattempt: int = None | |||
| maxrss: int = None | |||
| cloud: str = None | |||
| lfns: list = None | |||
| local_pfns: list = None | |||
| dependencies: list = None | |||
There was a problem hiding this comment.
Could you please add a brief description to each field of this data class? (It would be nice if the descriptions for maxwalltime and maxrss include information about the units they use, e.g., if the walltime needs to be specified in seconds, minutes, etc.)
There was a problem hiding this comment.
Thanks, I've added the suggested comment.
| task.queue = self.computing_queue_himem if self.tasks_steps[task_name] \ | ||
| in self.himem_tasks else self.computing_queue | ||
| bps_node = next(filter(lambda x: x['job'].label == self.tasks_steps[task_name], | ||
| self.bps_workflow.nodes.values()))['job'] |
There was a problem hiding this comment.
If I got it right it looks like, for each task, you iterate over all nodes of the generic workflow to find a node with a job having a specific label. Is there a reason why you can't create and use some kind of a "lookup table" (for example, a dictionary mapping task_steps keys onto these nodes)?
There was a problem hiding this comment.
Filter provides the iterator and the values are computed lazy. This is why the next function will lead to finding only a single bps workflow node per task. The lookup table could do the same, I agree, but will require an extra variable. I would leave as it is now, if it is OK with you. Thank you!
…ification Added changes required by the code review Fixed E261 alerts
SergeyPod
force-pushed
the
tickets/DM-30801
branch
from
43875e4
to
d1fa959
Compare
No description provided.