Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Guarantee history task execution #2864

Merged
merged 6 commits into from
May 20, 2022
Merged

Conversation

yycptt
Copy link
Member

@yycptt yycptt commented May 18, 2022

What changed?

  • New Single cursor mode for transfer and timer processor.
    • When enabled, only one queue processor (instead of one for active and n for standby clusters) will be started and handling tasks for both active and standby namespaces.
    • When enabled, queue processor won't register cluster metadata callback for (re-)creating standby processors.
    • When enabled, history engine will still register namespace change callback since the callback contains some logic for namespace migration. However, lock/unlock task processing and failover namespace call inside the callback will be a no-op (so no more failover queue).
    • When enabled, standby task execution logic will use current cluster time.
  • History task executor now also returns a metrics scope that should be used for emitting metrics for task processing. The reason is that active & standby tasks need to use different metric scope, but whether task is processed as active or passive is unknown before calling the executor.
  • Remove unused time source in timer sequence.
  • This PR depends on some code refactoring on xdc history resender so that resender is no longer bind to a certain remote cluster, which is still WIP.

Why?

  • Short term implementation for guarantee task execution and ensure no task loss during namespace failover. In long term, queue processor will be re-written using new multi-cursor queue components.

How did you test it?

  • Tested locally using go canary running in a global namespace with both single cursor flags enabled

Potential risks

Is hotfix candidate?

service/history/queues/executor_wrapper.go Outdated Show resolved Hide resolved
service/history/queues/executor_wrapper.go Show resolved Hide resolved
service/history/shard/context_impl.go Outdated Show resolved Hide resolved
service/history/shard/context_impl.go Outdated Show resolved Hide resolved
service/history/transferQueueProcessor.go Outdated Show resolved Hide resolved
@yycptt yycptt marked this pull request as ready for review May 20, 2022 00:27
@yycptt yycptt requested a review from a team as a code owner May 20, 2022 00:27
@yycptt yycptt enabled auto-merge (squash) May 20, 2022 07:32
@yycptt yycptt merged commit a40214d into temporalio:master May 20, 2022
@yycptt yycptt deleted the guarantee-execution branch May 20, 2022 07:58
Sushisource pushed a commit to Sushisource/temporal that referenced this pull request Jun 7, 2022
New Single cursor mode for transfer and timer processor
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants