[Bug] Unexpected "History is out of order" error during nondeterministic code change

[Sushisource]

What are you really trying to do?

Describe the bug

User reported that a nondeterministic code change (adding a timer) they made triggered this error:

sdk-core/core/src/worker/workflow/machines/workflow_machines.rs

Line 608 in 7b3776c

return Err(WFMachinesError::Fatal(format!(

However, that error should really only happen in the event we receive malformed history from server.

Specifically, their workflow looks like it runs a local activity first, then they added this timer (and seemingly, another invocation of the same local activity) between that and a normal activity.

Minimal Reproduction

Need more info from user. Workflow history + code sample would help.