Skip to content
This repository has been archived by the owner on Nov 1, 2023. It is now read-only.

Add the setup state to the unavailable list #1731

Merged

Conversation

tevoinea
Copy link
Member

@tevoinea tevoinea commented Mar 31, 2022

Summary of the Pull Request

What is this about?

There is a race condition when creating a scale set. The following sequence of events causes an error:

  1. Scale set is created. It can be either in init or setup state.
  2. 1 or more nodes spawn and pick up work. The nodes receive scale-in protection.
  3. We attempt to configure auto scale for the scale set with a minimum instance count less than the number of nodes that picked up work.

Step 3 will fail because auto scaling does not allow the minimum number of instances in the configuration to be smaller than the number of nodes with scale-in protection.

To fix this race condition, nodes cannot pick up work until auto scaling is configured. Since auto scaling is added while the scale set is still in the 'setup' state, nodes cannot pick up work while the scale set is in 'init' or 'setup'.

PR Checklist

  • Applies to work item: #xxx
  • CLA signed. If not, go over here and sign the CLI.
  • Tests added/passed
  • Requires documentation to be updated
  • I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #xxx

Info on Pull Request

What does this include?

Prevent nodes from picking up work until the scale set is at least 'running'.

Validation Steps Performed

How does someone test & validate?

@tevoinea tevoinea merged commit 9cbbf86 into microsoft:main Apr 1, 2022
@ghost ghost locked as resolved and limited conversation to collaborators May 1, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants