Set global checkpoint before open engine from store #27972
Conversation
In PR elastic#27965, we set the global checkpoint from the translog in a store recovery. However, we set after an engine is opened. This causes the global checkpoint assertion in TranslogWriter violated as if we are forced to close the engine before we set the global checkpoint. A closing engine will close translog which in turn read the current global checkpoint; however it is still unassigned and smaller than the initial global checkpoint from translog. Closes elastic#27970
dnhatn
added
:Distributed/Recovery
There was a problem hiding this comment.
Good that the assertion caught this. This PR essentially reverts a small change that's part of #27965, which is the simplest thing to do right now to get CI back to green (I can't think of any other simple fix). It further shows that InternalEngine needs some refactoring, the information flow is difficult to reason about right now.
There was a problem hiding this comment.
Thanks @dnhatn for picking this up. I intentionally changed this to rely on the validation in engine that the checkpoint actually means something and that the translog is the translog we want to read from. Sadly this has unexpected consequences as we expose the engine before we fully bootstrap from it. I don't like it but I think it's the easiest for now to do what you did.
In PR #27965, we set the global checkpoint from the translog in a store recovery. However, we set after an engine is opened. This causes the global checkpoint assertion in TranslogWriter violated as if we are forced to close the engine before we set the global checkpoint. A closing engine will close translog which in turn read the current global checkpoint; however it is still unassigned and smaller than the initial global checkpoint from translog. Closes #27970
In PR #27965, we set the global checkpoint from the translog in a store
recovery. However, we set after an engine is opened. This causes the
global checkpoint assertion in TranslogWriter violated as if we are
forced to close the engine before we set the global checkpoint. A
closing engine will close translog which in turn read the current global
checkpoint; however it is still unassigned and smaller than the initial
global checkpoint from translog.
Closes #27970