Repository navigation
Release 30.0.1-patch2
Improvements
Includes a way to limit on the number of tasks by type, running at the same time.
This limit controls the percentage or absolute number of task slots that can be allocated to specific task types.
When the limit is reached, tasks will wait for resources instead of blocking other tasks and failing due to timeouts.
New Property: taskLimits
Introduced new property taskLimits in the select strategy, configurable via the Overlord dynamic configuration.
TaskLimits properties:
- maxSlotCountByType - A map where each key is a task type (
String), and the corresponding value represents the absolute limit on the number of task slots that tasks of this type can occupy. The value is anIntegerthat is greater than or equal to 0. For example, a value of 5 means that tasks of this type can occupy up to 5 task slots in total. If both absolute and ratio limits are specified for the same task type, the effective limit will be the smaller of the absolute limit and the limit derived from the corresponding ratio.maxSlotCountByType = {"index_parallel": 3, "query_controller": 5}. In this example, parallel indexing tasks can occupy up to 3 task slots, and query controllers can occupy up to 5 task slots. - maxSlotRatioByType - A map where each key is a task type (
String), and the corresponding value is aDoublewhich should be in the range [0, 1], representing the ratio of task slots that tasks of this type can occupy. This ratio defines the proportion of total task slots a task type can use, calculated asratio * totalSlots. If both absolute and ratio limits are specified for the same task type, the effective limit will be the smaller of the absolute limit and the limit derived from the corresponding ratio.maxSlotRatioByType = {"index_parallel": 0.5, "query_controller": 0.25}. In this example, parallel indexing tasks can occupy up to 50% of the total task slots, and query controllers can occupy up to 25% of the total task slots.
Example Usage
{
"type": "default",
"selectStrategy": {
"type": "fillCapacity",
"affinityConfig": {
"affinity": {
"datasource1": [
"host2:port",
"host1:port"
],
"datasource2": [
"host3:port"
]
},
"strong": false
},
"taskLimits": {
"maxSlotCountByType": {},
"maxSlotRatioByType": {
"query_controller": 0.1
}
}
},
"autoScaler": null
}
- This configuration ensures that query controller tasks cannot occupy more than 10% of the total available task slots.
- Other task types remain unrestricted unless specified in taskLimits.
Note: This release includes all fixes and improvements from previous 30.0.1-patchX releases.