Skip to content

Commit

Permalink
dcache-qos: set default task thread pool sizes all to max concurrent …
Browse files Browse the repository at this point in the history
…running

Motivation:

Lessons learned from Bulk throughput optimizations.

It makes more sense to allocate each of the task
queues to be max running size (this is the purpose of
the BoundedCachedExecutor class: idle threads
will be released and decommissioned).

Modification:

Set them all to be controlled by default
by the `${qos.limits.verifier.max-running-operations}`
value.

Result:

Much faster throughput.

>IMPORTANT:  Also updated the default
>size of the QoS Engine's modify thread pool
>to match the 500Hz Bulk default.
>Currently the value (32) is an
>order of magnitude too little.

Target: master
Request: 9.2
Patch: https://rb.dcache.org/r/14139/
Requires-notes: yes
Acked-by: Tigran
  • Loading branch information
alrossi committed Oct 17, 2023
1 parent d466c36 commit 1a8bb02
Show file tree
Hide file tree
Showing 2 changed files with 5 additions and 5 deletions.
2 changes: 1 addition & 1 deletion skel/share/defaults/qos-engine.properties
Expand Up @@ -61,7 +61,7 @@ qos.limits.engine.namespace-request-threads=32
# ---- Thread queue used to process requests to modify the qos. These can
# be from the bulk service or the frontend.
#
qos.limits.engine.qos-modify-request-threads=32
qos.limits.engine.qos-modify-request-threads=500

# ---- Thread queue used to process policy state updates and queries. These
# can be from the engine's manager thread or from admin or frontend interfaces.
Expand Down
8 changes: 4 additions & 4 deletions skel/share/defaults/qos-verifier.properties
Expand Up @@ -87,7 +87,7 @@ qos.limits.verifier.bulk-threads=8
#
# This queue is used for requests originating from the namespace.
#
qos.limits.verifier.task-threads=24
qos.limits.verifier.task-threads=${qos.limits.verifier.max-running-operations}

# ---- Thread queue used when an operation becomes active (to verify
# the requirements and send a message to the adjuster). Each thread makes
Expand All @@ -96,7 +96,7 @@ qos.limits.verifier.task-threads=24
#
# This queue is used for requests originating from the frontend, bulk service or qos engine.
#
qos.limits.verifier.modify-task-threads=24
qos.limits.verifier.modify-task-threads=${qos.limits.verifier.max-running-operations}

# ---- Thread queue used when an operation becomes active (to verify
# the requirements and send a message to the adjuster). Each thread makes
Expand All @@ -105,11 +105,11 @@ qos.limits.verifier.modify-task-threads=24
#
# This queue is used for requests originating from the scanner.
#
qos.limits.verifier.scanner-task-threads=12
qos.limits.verifier.scanner-task-threads=${qos.limits.verifier.max-running-operations}

# ---- Thread queue used to execute postprocess and eventual removal of operations.
#
qos.limits.verifier.post-process-threads=24
qos.limits.verifier.post-process-threads=${qos.limits.verifier.max-running-operations}

# ---- This number is a maximum per operation queue.
#
Expand Down

0 comments on commit 1a8bb02

Please sign in to comment.