Mark committed_capacity field for removal #12041
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
SUMMARY
Explaining why this field exists:
This app used to use celery and RabbitMQ as the messaging system.
In that old design, the instance groups corresponded to literal queues in RabbitMQ. When the task manager dispatched a task, it submitted it to its instance group queue. This meant that we had no control over which node (inside of the group) received the job, because all nodes in the group would be pulling from that queue.
This was redesigned entirely, we don't use RabbitMQ, replacing it with the postgres message bus. Instead of having queues for instance groups, every instance had a queue. Instead of the task manager selecting an instance group alone, it now selects the
instance_group
,controller_node
, andexecution_node
, and sends the message to the controller_node queue.The construct of "committed" capacity is for jobs with "waiting" status which have already been submitted to the instance group queue (the type of queue that no longer exists).
In the modern system, all capacity use is committed to a particular instance. The consumed capacity for an instance group is just the consumed capacity of the instances in that group, added up.
This field is archaic at this point.
OPTIONS with this change
ISSUE TYPE
COMPONENT NAME