New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[RFE] Add optional argument to --add-schedule for bulk scheduling #187

Open
grafuls opened this Issue Sep 13, 2018 · 5 comments

Comments

Projects
None yet
4 participants
@grafuls

grafuls commented Sep 13, 2018

Is your feature request related to a problem? Please describe.
As a user I would like to have the ability to schedule multiple hosts at once.

Describe the solution you'd like
Adding an optional argument --from-file, to be mutually exclusive with --host argument, that takes as input the path to a plain text file with the hostnames to be scheduled on new lines.
Additionally, provide a summary at the end of the scheduling with number of scheduled/broken hosts.

@grafuls grafuls added this to the 1.1 milestone Sep 13, 2018

@grafuls grafuls self-assigned this Sep 13, 2018

@sadsfae

This comment has been minimized.

Show comment
Hide comment
@sadsfae

sadsfae Sep 13, 2018

Member

This seems like a solid improvement. In our documentation examples we tend to prefer shell for loops for running multiple --add-schedule commands across a set of disparate hosts. I think by itself this can lend for easier scheduling usage from a user perspective when doing singular scheduling.

Member

sadsfae commented Sep 13, 2018

This seems like a solid improvement. In our documentation examples we tend to prefer shell for loops for running multiple --add-schedule commands across a set of disparate hosts. I think by itself this can lend for easier scheduling usage from a user perspective when doing singular scheduling.

@kambiz-aghaiepour

This comment has been minimized.

Show comment
Hide comment
@kambiz-aghaiepour

kambiz-aghaiepour Sep 13, 2018

Contributor

I like this idea. Currently what happens when a single host schedule request cannot be fulfilled due to scheduling conflict (or some other logic error, eg. start date after end date), is that quads reports the conflict or error condition. If we allow the --from-file option do we want to schedule what we can, while reporting on the ones we can't? Or do we optionally want to allow both possibilities? For example:

quads --add-schedule --from-file --schedule-start --schedule-end --schedule-cloud

or:

quads --add-schedule --multi-host host1,host2,host3,host4,.... --schedule-start --schedule-end --schedule-cloud

the above would schedule as many of the hosts that it can, and report on the ones that it cannot schedule. However, we may also want something like:

quads --add-schedule --from-file --schedule-start --schedule-end --schedule-cloud --atomic

or:

quads --add-schedule --multi-host host1,host2,host3,host4,.... --schedule-start --schedule-end --schedule-cloud --atomic

the above would ONLY schedule the hosts if ALL of the hosts can be scheduled for the requested schedule.

Thoughts?

Contributor

kambiz-aghaiepour commented Sep 13, 2018

I like this idea. Currently what happens when a single host schedule request cannot be fulfilled due to scheduling conflict (or some other logic error, eg. start date after end date), is that quads reports the conflict or error condition. If we allow the --from-file option do we want to schedule what we can, while reporting on the ones we can't? Or do we optionally want to allow both possibilities? For example:

quads --add-schedule --from-file --schedule-start --schedule-end --schedule-cloud

or:

quads --add-schedule --multi-host host1,host2,host3,host4,.... --schedule-start --schedule-end --schedule-cloud

the above would schedule as many of the hosts that it can, and report on the ones that it cannot schedule. However, we may also want something like:

quads --add-schedule --from-file --schedule-start --schedule-end --schedule-cloud --atomic

or:

quads --add-schedule --multi-host host1,host2,host3,host4,.... --schedule-start --schedule-end --schedule-cloud --atomic

the above would ONLY schedule the hosts if ALL of the hosts can be scheduled for the requested schedule.

Thoughts?

@abondvt89

This comment has been minimized.

Show comment
Hide comment
@abondvt89

abondvt89 Sep 13, 2018

abondvt89 commented Sep 13, 2018

@sadsfae

This comment has been minimized.

Show comment
Hide comment
@sadsfae

sadsfae Sep 13, 2018

Member

On Thu, Sep 13, 2018 at 9:42 AM Kambiz Aghaiepour ***@***.***> wrote: I like this idea. Currently what happens when a single host schedule request cannot be fulfilled due to scheduling conflict (or some other logic error, eg. start date after end date), is that quads reports the conflict or error condition. If we allow the --from-file option do we want to schedule what we can, while reporting on the ones we can't? Or do we optionally want to allow both possibilities? For example: quads --add-schedule --from-file --schedule-start --schedule-end --schedule-cloud or: quads --add-schedule --multi-host host1,host2,host3,host4,.... --schedule-start --schedule-end --schedule-cloud the above would schedule as many of the hosts that it can, and report on the ones that it cannot schedule. However, we may also want something like: quads --add-schedule --from-file --schedule-start --schedule-end --schedule-cloud --atomic or: quads --add-schedule --multi-host host1,host2,host3,host4,.... --schedule-start --schedule-end --schedule-cloud --atomic the above would ONLY schedule the hosts if ALL of the hosts can be scheduled for the requested schedule. Thoughts?
I think an actual bulk scheduling command should be an all or nothing event. By the time the scheduling command is issued we should have a good idea on whether those machines are free so the times where it fails because of conflicts should be the exception not the norm. We will be building other automation to help with finding the available machines and the best fit schedule as a pre-cursor to actually doing the scheduling. So eventually the bulk input would be something built by the best fit automation similar to what Sahil was working on. I would be afraid that if we allow a partial schedule to be put in that we would end up not catching some of those cases where not all machines were assigned and someone would get a partial allocation that we would then have to deal with when they try to use their allocation.

I think this can be accomplished with an addition to --from-file if we can bake in a check that the host is not marked with broken_state: true in Foreman where we record this. It can be a pre-check to passing ---from-file prior to executing adding schedules. What do you guys think?

e.g.

/opt/quads/bin/quads-cli --add-schedule --from-file /tmp/newcloud.txt --schedule-start --schedule-end --schedule-cloud cloud04
Checking health of resources in /tmp/newcloud.txt .. OK

Gonza already has a patch that adds warning / error if a host has broken_state: true for singular schedule commands so this would be a nice addition to have.

In both cases we should print the host(s) in question that have issues before failing.

Member

sadsfae commented Sep 13, 2018

On Thu, Sep 13, 2018 at 9:42 AM Kambiz Aghaiepour ***@***.***> wrote: I like this idea. Currently what happens when a single host schedule request cannot be fulfilled due to scheduling conflict (or some other logic error, eg. start date after end date), is that quads reports the conflict or error condition. If we allow the --from-file option do we want to schedule what we can, while reporting on the ones we can't? Or do we optionally want to allow both possibilities? For example: quads --add-schedule --from-file --schedule-start --schedule-end --schedule-cloud or: quads --add-schedule --multi-host host1,host2,host3,host4,.... --schedule-start --schedule-end --schedule-cloud the above would schedule as many of the hosts that it can, and report on the ones that it cannot schedule. However, we may also want something like: quads --add-schedule --from-file --schedule-start --schedule-end --schedule-cloud --atomic or: quads --add-schedule --multi-host host1,host2,host3,host4,.... --schedule-start --schedule-end --schedule-cloud --atomic the above would ONLY schedule the hosts if ALL of the hosts can be scheduled for the requested schedule. Thoughts?
I think an actual bulk scheduling command should be an all or nothing event. By the time the scheduling command is issued we should have a good idea on whether those machines are free so the times where it fails because of conflicts should be the exception not the norm. We will be building other automation to help with finding the available machines and the best fit schedule as a pre-cursor to actually doing the scheduling. So eventually the bulk input would be something built by the best fit automation similar to what Sahil was working on. I would be afraid that if we allow a partial schedule to be put in that we would end up not catching some of those cases where not all machines were assigned and someone would get a partial allocation that we would then have to deal with when they try to use their allocation.

I think this can be accomplished with an addition to --from-file if we can bake in a check that the host is not marked with broken_state: true in Foreman where we record this. It can be a pre-check to passing ---from-file prior to executing adding schedules. What do you guys think?

e.g.

/opt/quads/bin/quads-cli --add-schedule --from-file /tmp/newcloud.txt --schedule-start --schedule-end --schedule-cloud cloud04
Checking health of resources in /tmp/newcloud.txt .. OK

Gonza already has a patch that adds warning / error if a host has broken_state: true for singular schedule commands so this would be a nice addition to have.

In both cases we should print the host(s) in question that have issues before failing.

@grafuls

This comment has been minimized.

Show comment
Hide comment
@grafuls

grafuls Sep 14, 2018

On Thu, Sep 13, 2018 at 9:42 AM Kambiz Aghaiepour ***@***.***> wrote: I like this idea. Currently what happens when a single host schedule request cannot be fulfilled due to scheduling conflict (or some other logic error, eg. start date after end date), is that quads reports the conflict or error condition. If we allow the --from-file option do we want to schedule what we can, while reporting on the ones we can't? Or do we optionally want to allow both possibilities? For example: quads --add-schedule --from-file --schedule-start --schedule-end --schedule-cloud or: quads --add-schedule --multi-host host1,host2,host3,host4,.... --schedule-start --schedule-end --schedule-cloud the above would schedule as many of the hosts that it can, and report on the ones that it cannot schedule. However, we may also want something like: quads --add-schedule --from-file --schedule-start --schedule-end --schedule-cloud --atomic or: quads --add-schedule --multi-host host1,host2,host3,host4,.... --schedule-start --schedule-end --schedule-cloud --atomic the above would ONLY schedule the hosts if ALL of the hosts can be scheduled for the requested schedule. Thoughts?
I think an actual bulk scheduling command should be an all or nothing event. By the time the scheduling command is issued we should have a good idea on whether those machines are free so the times where it fails because of conflicts should be the exception not the norm. We will be building other automation to help with finding the available machines and the best fit schedule as a pre-cursor to actually doing the scheduling. So eventually the bulk input would be something built by the best fit automation similar to what Sahil was working on. I would be afraid that if we allow a partial schedule to be put in that we would end up not catching some of those cases where not all machines were assigned and someone would get a partial allocation that we would then have to deal with when they try to use their allocation.

I think this can be accomplished with an addition to --from-file if we can bake in a check that the host is not marked with broken_state: true in Foreman where we record this. It can be a pre-check to passing ---from-file prior to executing adding schedules. What do you guys think?

e.g.

/opt/quads/bin/quads-cli --add-schedule --from-file /tmp/newcloud.txt --schedule-start --schedule-end --schedule-cloud cloud04
Checking health of resources in /tmp/newcloud.txt .. OK

Gonza already has a patch that adds warning / error if a host has broken_state: true for singular schedule commands so this would be a nice addition to have.

In both cases we should print the host(s) in question that have issues before failing.

I agree with @abondvt89 on the fact that we should have Quads look for the most suitable and available hosts for the user, which would make for the bulk add-schedule not to have any reasons for failure.
I would though include that on a separate RFE.

Regarding @kambiz-aghaiepour suggestion, we thought of having a new property on quads configuration file for either allowing a schedule of a broken host, showing a warning message, or not allowing it at all and showing the proper error message but having an optional argument like --atomic seems much more intuitive from a user's perspective. +1 on this.

+1 as well on @sadsfae suggestion for doing a pre-check of all hosts when bulk scheduling.

grafuls commented Sep 14, 2018

On Thu, Sep 13, 2018 at 9:42 AM Kambiz Aghaiepour ***@***.***> wrote: I like this idea. Currently what happens when a single host schedule request cannot be fulfilled due to scheduling conflict (or some other logic error, eg. start date after end date), is that quads reports the conflict or error condition. If we allow the --from-file option do we want to schedule what we can, while reporting on the ones we can't? Or do we optionally want to allow both possibilities? For example: quads --add-schedule --from-file --schedule-start --schedule-end --schedule-cloud or: quads --add-schedule --multi-host host1,host2,host3,host4,.... --schedule-start --schedule-end --schedule-cloud the above would schedule as many of the hosts that it can, and report on the ones that it cannot schedule. However, we may also want something like: quads --add-schedule --from-file --schedule-start --schedule-end --schedule-cloud --atomic or: quads --add-schedule --multi-host host1,host2,host3,host4,.... --schedule-start --schedule-end --schedule-cloud --atomic the above would ONLY schedule the hosts if ALL of the hosts can be scheduled for the requested schedule. Thoughts?
I think an actual bulk scheduling command should be an all or nothing event. By the time the scheduling command is issued we should have a good idea on whether those machines are free so the times where it fails because of conflicts should be the exception not the norm. We will be building other automation to help with finding the available machines and the best fit schedule as a pre-cursor to actually doing the scheduling. So eventually the bulk input would be something built by the best fit automation similar to what Sahil was working on. I would be afraid that if we allow a partial schedule to be put in that we would end up not catching some of those cases where not all machines were assigned and someone would get a partial allocation that we would then have to deal with when they try to use their allocation.

I think this can be accomplished with an addition to --from-file if we can bake in a check that the host is not marked with broken_state: true in Foreman where we record this. It can be a pre-check to passing ---from-file prior to executing adding schedules. What do you guys think?

e.g.

/opt/quads/bin/quads-cli --add-schedule --from-file /tmp/newcloud.txt --schedule-start --schedule-end --schedule-cloud cloud04
Checking health of resources in /tmp/newcloud.txt .. OK

Gonza already has a patch that adds warning / error if a host has broken_state: true for singular schedule commands so this would be a nice addition to have.

In both cases we should print the host(s) in question that have issues before failing.

I agree with @abondvt89 on the fact that we should have Quads look for the most suitable and available hosts for the user, which would make for the bulk add-schedule not to have any reasons for failure.
I would though include that on a separate RFE.

Regarding @kambiz-aghaiepour suggestion, we thought of having a new property on quads configuration file for either allowing a schedule of a broken host, showing a warning message, or not allowing it at all and showing the proper error message but having an optional argument like --atomic seems much more intuitive from a user's perspective. +1 on this.

+1 as well on @sadsfae suggestion for doing a pre-check of all hosts when bulk scheduling.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment