DBZ-3630 Mining session stopped due to the ORA-01281 ( SCN range spec…

[shiawu]

…ified is invalid)

https://issues.redhat.com/browse/DBZ-3630

[Naros]

I'm going to review this a bit more in depth. I had another user who reported a similar problem submit a TRACE log so perhaps that'll give some insight as well. Basically when you've done here is reverted the mining loop's SCN logic similar to what we had before introducing BLOB / CLOB data type support.

But looking at the code a bit more, I'm starting to suspect that perhaps the miningStartScn can be removed entirely since I think that the call to startScn = transactionalBuffer.updateOffsetContext(offsetContext); is sufficient enough to manage the state of startScn such that we don't need the miningStartScn at all.

[shiawu]

I agree, 2 start SCN is confusing and cause issue. With our current PR, we use only miningStartScn, and it is running a week without problem. We have not tried use the startSCN only. We will wait for your response. Thanks John From: Chris Cranford ***@***.***> Reply-To: debezium/debezium ***@***.***> Date: Tuesday, June 15, 2021 at 3:40 PM To: debezium/debezium ***@***.***> Cc: John Wu ***@***.***>, Author ***@***.***> Subject: Re: [debezium/debezium] DBZ-3630 Mining session stopped due to the ORA-01281 ( SCN range spec… (#2474) I'm going to review this a bit more in depth. I had another user who reported a similar problem submit a TRACE log so perhaps that'll give some insight as well. Basically when you've done here is reverted the mining loop's SCN logic similar to what we had before introducing BLOB / CLOB data type support. But looking at the code a bit more, I'm starting to suspect that perhaps the miningStartScn can be removed entirely since I think that the call to startScn = transactionalBuffer.updateOffsetContext(offsetContext); is sufficient enough to manage the state of startScn such that we don't need the miningStartScn at all. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub<#2474 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AFRYSCOBELTT6WJI4CP2IIDTS7JF3ANCNFSM46XZIICQ>.

[Naros]

So as I commented on the other related Jira DBZ-3627, I definitely see where the failure comes in the call to LogMinerHelper#getEndScn when passing startScn instead of the miningStartScn. I'm now going to begin looking at the blob/clob stuff now that I understand the root-cause more to see whether we can consolidate these two SCN variables or not.

[Naros]

Hi @shiawu, I just want to follow-up here and let you know that I think this PR can be closed and will be superseded by the changes in #2483. This new PR goes back to the 1.5 SCN algorithm as the baseline again but we're handling this "remine" approach slightly different.

The goal in this new PR is to remine when necessary. This means that unless we see a commit for a transaction that is prior to the current mining window, we'll continue to advance forward using the adaptive SCN algorithm from 1.5. When we detect a commit for a transaction that is prior to the current mining window's start point, we're going to do a couple things:

First we'll check to see if the starting point for this transaction began prior to the oldest log in the mining session. If it does, we terminate the current mining session, start a new one with the right set of logs from the transaction's starting scn to its commit, and capture that transaction's history. If the starting point for that transaction is in the logs for the current mining session, we simply issue a new query to get the transaction's history from its starting point to commit point.

Once that concludes, we unwind from that special remine step and we proceed forward by mining from the aforementioned transaction's commit point onward.

Because we remine only when necessary, the hope is that this keeps the connector advancing forward more regularly and that it only takes the performance hit to remine old transactions only when necessary.

In conjunction with these changes, I do intend to add a sanity check to #getEndScn() anyway to guarantee that under no circumstance does it ever return a value that comes before the starting SCN value too for good measure. I don't think it's going to be needed with this change we're making, but I'd rather be safe.