Allow config.ReplicationLowPriorityTaskParallelism to take effect.#10051
Merged
robholland merged 5 commits intotemporalio:mainfrom Apr 24, 2026
Merged
Allow config.ReplicationLowPriorityTaskParallelism to take effect.#10051robholland merged 5 commits intotemporalio:mainfrom
robholland merged 5 commits intotemporalio:mainfrom
Conversation
yux0
reviewed
Apr 24, 2026
| return farm.Fingerprint32(idBytes) | ||
| // 0..parallelism-1, stable for a given RunID. Different runs of the same workflow can share | ||
| // a slot, so up to P sequential queues (and workers) can progress them concurrently. | ||
| slot := int(farm.Fingerprint32([]byte(workflowKey.RunID)) % uint32(parallelism)) |
yux0
approved these changes
Apr 24, 2026
Allows concurrent tx map to optimise queue operations.
yux0
approved these changes
Apr 24, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed?
Use
WorkflowKeyfor the low priority replication scheduler queue ID rather than a string. Deterministically map RunID to a slot number usingconfig.ReplicationLowPriorityTaskParallelismslots.Why?
Using strings for the queue ID was preventing concurrent tx map from optimising queue operations.
While we had code to allow parallelising tasks for different executions of the same workflow ID, the code was not reachable. Switch the parallelisation to the queue ID level which is subjectively more intuitive.
How did you test it?
Potential risks
The
config.ReplicationLowPriorityTaskParallelismcould be set to a non-default number on clusters but would not currently be taking effect. After this change the parallelism would kick in, which may cause surprising behaviour.Note
Medium Risk
Changes low-priority replication task queueing and hashing, which can increase concurrency and alter task ordering/throughput when
ReplicationLowPriorityTaskParallelismis >1. Risk is moderate due to potential load/behavior shifts in replication processing under real traffic.Overview
Low-priority replication scheduling now honors
config.ReplicationLowPriorityTaskParallelism. Instead of using a random slot per task and string queue IDs, the scheduler deterministically maps eachRunIDto a stable slot (0..P-1) and encodes that slot into adefinition.WorkflowKeyqueue ID, enabling up to P concurrent sequential queues per (namespace, workflow).Separately, the non-batching
sequentialTaskQueueFactoryProvidernow uses adefinition.WorkflowKey(with empty third field) as the queue ID rather than a concatenated string, aligning queue identity with the hash function and reducing reliance on string IDs.Reviewed by Cursor Bugbot for commit aa9f9bf. Bugbot is set up for automated code reviews on this repo. Configure here.