Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update backfill history in sync workflow state #3042

Merged
merged 4 commits into from
Jul 1, 2022

Conversation

yux0
Copy link
Contributor

@yux0 yux0 commented Jun 30, 2022

What changed?

  1. Handle sync workflow state task in dlq.
  2. Backfill history when history check and handle ancestor branch information correctly.

Why?
Correctness issue

How did you test it?
New unit tests and functional tests.

Potential risks

Is hotfix candidate?
Yes.

@yux0 yux0 marked this pull request as ready for review June 30, 2022 06:39
@yux0 yux0 requested a review from a team as a code owner June 30, 2022 06:39
@yux0 yux0 requested review from alexshtin and yycptt June 30, 2022 16:04
return &persistence.PutReplicationTaskToDLQRequest{
ShardID: p.shard.GetShardID(),
SourceClusterName: p.sourceCluster,
TaskInfo: &persistencespb.ReplicationTaskInfo{
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I kinda wonder if we should generate a tasks.SyncWorkflowStateTask here and let persistence layer do the conversion to proto representation. Similar to how normal replication tasks are generated & persisted.

If we were to have DLQ for other queues in the future, that should help unify the code path.

service/history/nDCHistoryReplicator.go Outdated Show resolved Hide resolved
service/history/nDCHistoryReplicator.go Outdated Show resolved Hide resolved
service/history/nDCHistoryReplicator.go Outdated Show resolved Hide resolved
latestBranchID := historyBranch.GetBranchId()
var prevBranchID string

sortedAncestors := copyAncestors(historyBranch.GetAncestors())
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do you copy and then substitute original historyBranch instead of just sorting ancestors in place and then create new filteredHistoryBranch and fill in the loop bellow?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is easier to achieve. For example, there are 3 branches. B1 has no ancestor. B2 has B1 ancestor, B3 has B1,B2 ancestors.
B1

B2
/
B3

If create new filteredHistoryBranch in the loop, then I need to fill in the previous ancestors.

service/history/nDCHistoryReplicator.go Outdated Show resolved Hide resolved
@yux0 yux0 merged commit 2b31d0d into temporalio:master Jul 1, 2022
@yux0 yux0 deleted the fix-sync-wf-task branch July 1, 2022 00:27
yux0 added a commit that referenced this pull request Jul 1, 2022
* Update backfill history in sync workflow state

* append node with cleanup info

* get last batch node id
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants