Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add ports to ResumeProgram ports and add ResumeFailProgram #16

Open
wants to merge 17 commits into
base: master
Choose a base branch
from

Conversation

sjpb
Copy link
Collaborator

@sjpb sjpb commented Mar 8, 2022

Improves support for autoscaling:

  • Modifies ResumeProgram: If a port is found on the specified network with the same name as the node, the instance is attached to that port. Allows for using predefined ports for e.g. non-default binding configurations or no DNS,
  • Adds ResumeFailProgram: Handles cases for failed Resume (nodes -> DOWN where:
    • Instance not created: Node set to resume state to allow Slurm to retry that node (as required by scheduler)
    • Instance created but in error state due to "not enough hosts available": instance deleted, node set to resume state.

This also does a big refactor to remove duplicated code and allow use of suspend() and resume() functions from other scripts.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant