Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Hopefully avoid panic in DescribeWorkflowExecution #5057

Conversation

bergundy
Copy link
Member

What changed?

Clone suspected mutating fields after a returning from DescribeWorkflowExecution.

Why?

Noticed panics in serialization of the DescribeWorkflowExecutionResponse object.

How did you test it?

No tests added, assuming that normal test coverage is enough.
Haven't reproduced the issue, this is based on a hunch.

Potential risks

None that I can foresee.

Is hotfix candidate?

Yes.

@bergundy bergundy requested a review from a team as a code owner October 31, 2023 21:45
Status: executionState.Status,
HistoryLength: mutableState.GetNextEventID() - common.FirstEventID,
ExecutionTime: executionInfo.ExecutionTime,
// Memo and SearchAttributes are set below
AutoResetPoints: executionInfo.AutoResetPoints,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you also make a deep copy here?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes

@bergundy bergundy force-pushed the hopefully-avoid-panic-in-describe-workflow-execution branch from d345aef to a55eed2 Compare October 31, 2023 21:53
for k, v := range source {
metadata := make(map[string][]byte, len(v.GetMetadata()))
for mk, mv := range v.GetMetadata() {
metadata[mk] = mv
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we also want to copy byte slices because, even though another thread can't append to this or delete from it, they can modify its elements.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This won't happen in practice, I don't think we mutate any payload metadata or data directly anywhere, only replace the entire slice.
In any case it's safe in terms of serialization and would mitigate the panic since the slice's length won't change.

}
target[k] = &commonpb.Payload{
Metadata: metadata,
Data: v.GetData(),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto here

defer func() { weCtx.GetReleaseFn()(retError) }()

mutableState := weCtx.GetMutableState()
executionInfo := mutableState.GetExecutionInfo()
executionState := mutableState.GetExecutionState()

resetPoints := &workflowpb.ResetPoints{
Points: make([]*workflowpb.ResetPointInfo, 0, len(executionInfo.AutoResetPoints.GetPoints())),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will cause an OOB exception. You need to remove the second argument.

@bergundy bergundy force-pushed the hopefully-avoid-panic-in-describe-workflow-execution branch from a55eed2 to 122f052 Compare October 31, 2023 22:15
@bergundy bergundy enabled auto-merge (squash) October 31, 2023 22:27
@bergundy bergundy merged commit a553296 into temporalio:main Oct 31, 2023
10 checks passed
@bergundy bergundy deleted the hopefully-avoid-panic-in-describe-workflow-execution branch October 31, 2023 22:42
bergundy added a commit that referenced this pull request Nov 1, 2023
**What changed?**

Clone suspected mutating fields after a returning from
DescribeWorkflowExecution.

**Why?**

Noticed panics in serialization of the
`DescribeWorkflowExecutionResponse` object.

**How did you test it?**

No tests added, assuming that normal test coverage is enough.
Haven't reproduced the issue, this is based on a hunch.

**Potential risks**

None that I can foresee.

**Is hotfix candidate?**

Yes.
rodrigozhou pushed a commit that referenced this pull request Nov 1, 2023
**What changed?**

Clone suspected mutating fields after a returning from
DescribeWorkflowExecution.

**Why?**

Noticed panics in serialization of the
`DescribeWorkflowExecutionResponse` object.

**How did you test it?**

No tests added, assuming that normal test coverage is enough.
Haven't reproduced the issue, this is based on a hunch.

**Potential risks**

None that I can foresee.

**Is hotfix candidate?**

Yes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants