Remove namespace max retention #3148
Conversation
|
Pending on #3159 |
| // This scanner will face racing condition with archiver because it relys on describe mutable state returning entityNotExist error. | ||
| // That's why we need to keep MaxWorkflowRetentionPeriod stable and not decreasing all the time. | ||
| cleanUpThreshold = common.MaxWorkflowRetentionPeriod * 2 | ||
| cleanUpThreshold = 60 * 24 * time.Hour |
There was a problem hiding this comment.
I think we should use a larger duration. The reason is, for each history branch, the scavenger will check if they are older than this threshold, if so it will load mutable state, otherwise it will skip. For majority case, this will be a no-op, so loading mutable state is quite costly if we have to load every single mutable state. Make the duration longer will reduce the chance because by the time when it is over the threshold duration, most of history will already be deleted. This also depends on the duration of the life time of a workflow. So I think we may even need to consider make this configurable via dynamic config. The cost of keeping small percent of garbage history is probably less than the cost of scanning the whole history and mutable states.
There was a problem hiding this comment.
Is it worth to make it configurable and give an arbitrary number?
There was a problem hiding this comment.
Updated to be configurable and default set to 90 days.
* Remove namespace max retention
What changed?
Why?
Support retention > 30 days.
How did you test it?
Potential risks
Is hotfix candidate?