Log branchToken on potential data loss error

[yycptt]

What changed?

• Log branchToken on potential data loss error. The logged branchToken is in base64 format and can be decoded with the command below. The treeID is usually the workflow runID. Also the branchToken is constructed by persistence layer, so there could be additional info re. the workflow in the branchToken.

./tdbg decode proto --type temporal.server.api.persistence.v1.HistoryBranch --hex-data "$(echo "CiQwMTkzZTY1OC1lZjc0LTc1ZDAtYTU4NS1hZWI0MzZlYThmNDESJDAxOTNlNjU4LWVmNzQtNzYyNi05ZTlmLWI1ODAwMGI2MjNkMQ==" | base64 --decode | hexdump -ve '/1 "%02x"')"
{
 "treeId": "0193e658-ef74-75d0-a585-aeb436ea8f41",
 "branchId": "0193e658-ef74-7626-9e9f-b58000b623d1"
}

Why?

• We recently introduced retry for data loss errors in persistence client, but that also means application logic may not see those errors and won't emit any logs with workflowInfo. Inside persistence layer, there's no workflow key info (e.g. workflowID, runID), the best we can do is log branchToken, which gives us some hint re which workflow is running into the error.

How did you test it?

• Tested locally.

Potential risks

Documentation

Is hotfix candidate?