Skip to content

Commit

Permalink
Merge pull request debezium#62 in N4FRA/debezium from DSCON-487_NPE_o…
Browse files Browse the repository at this point in the history 
…bserved_after_snapshot to master

Squashed commit of the following:

commit 79855e89144c46f890dafdf47cf3e6f941184662
Author: AndreyIg <gnyiny@gmail.com>
Date:   Thu Jul 2 08:23:27 2020 -0700

    DSCON-487
    NPE observed one time after snapshot completed in DBC

commit c76688c
Merge: 9dc2116 52dfe6a
Author: AndreyIg <gnyiny@gmail.com>
Date:   Sat Jun 27 08:51:29 2020 -0700

    Merge branch 'master' into DBZ-137_candidate_merge_branch

commit 52dfe6a
Merge: 26a08a8 1c76971
Author: AndreyIg <gnyiny@gmail.com>
Date:   Mon Jun 22 14:06:42 2020 -0700

    Merge branch 'master' of http://git.navis.lan/scm/n4fra/debezium

commit 9dc2116
Merge: 0d01cf5 26a08a8
Author: AndreyIg <gnyiny@gmail.com>
Date:   Tue Jun 16 12:20:07 2020 -0700

    Merge branch 'master' into DBZ-137_candidate_merge_branch

commit 26a08a8
Merge: 1e7ee39 b28747a
Author: AndreyIg <gnyiny@gmail.com>
Date:   Tue Jun 16 12:19:39 2020 -0700

    Merge branch 'master' of http://git.navis.lan/scm/n4fra/debezium

commit 0d01cf5
Merge: dbe96fe 1e7ee39
Author: AndreyIg <gnyiny@gmail.com>
Date:   Tue Jun 9 15:24:40 2020 -0700

    Merge branch 'master' into DBZ-137_candidate_merge_branch

commit 1e7ee39
Merge: cadd6c1 7ce9f6e
Author: AndreyIg <gnyiny@gmail.com>
Date:   Thu Jun 4 11:04:03 2020 -0700

    Merge branch 'master' of http://git.navis.lan/scm/n4fra/debezium

commit cadd6c1
Merge: b14ea98 5f80774
Author: AndreyIg <gnyiny@gmail.com>
Date:   Thu Jun 4 10:59:12 2020 -0700

    Merge branch 'master' of http://git.navis.lan/scm/n4fra/debezium

commit b14ea98
Merge: fc66328 c2d8ec5
Author: AndreyIg <gnyiny@gmail.com>
Date:   Mon Jun 1 15:58:27 2020 -0700

    Merge branch 'master' of http://git.navis.lan/scm/n4fra/debezium

commit fc66328
Merge: 539c214 88315df
Author: AndreyIg <gnyiny@gmail.com>
Date:   Thu May 28 14:09:07 2020 -0700

    Merge branch 'master' of http://git.navis.lan/scm/n4fra/debezium

commit 539c214
Merge: a54f7a3 57a0b1c
Author: AndreyIg <gnyiny@gmail.com>
Date:   Thu May 28 13:43:27 2020 -0700

    Merge branch 'master' of http://git.navis.lan/scm/n4fra/debezium

commit a54f7a3
Merge: 796bf4b 7c49efd
Author: AndreyIg <gnyiny@gmail.com>
Date:   Fri May 22 14:31:05 2020 -0700

    Merge branch 'master' of http://git.navis.lan/scm/n4fra/debezium

commit 796bf4b
Merge: 8b36431 2c1fc9e
Author: AndreyIg <gnyiny@gmail.com>
Date:   Fri May 15 12:19:16 2020 -0700

    Merge branch 'master' of http://git.navis.lan/scm/n4fra/debezium

commit 8b36431
Merge: 66c207f 70ad303
Author: AndreyIg <gnyiny@gmail.com>
Date:   Wed May 13 16:37:01 2020 -0700

    Merge branch 'master' of http://git.navis.lan/scm/n4fra/debezium

commit 66c207f
Merge: f1810b9 d5de8d8
Author: AndreyIg <gnyiny@gmail.com>
Date:   Tue May 12 11:17:49 2020 -0700

    Merge branch 'master' of http://git.navis.lan/scm/n4fra/debezium

commit f1810b9
Merge: 3f0c9de a043ebd
Author: AndreyIg <gnyiny@gmail.com>
Date:   Thu May 7 14:59:46 2020 -0700

    Merge branch 'master' of http://git.navis.lan/scm/n4fra/debezium

commit 3f0c9de
Merge: 2de0428 5481902
Author: AndreyIg <gnyiny@gmail.com>
Date:   Thu May 7 14:52:06 2020 -0700

    Merge branch 'master' of http://git.navis.lan/scm/n4fra/debezium

commit 2de0428
Merge: b514c5e 274e969
Author: AndreyIg <gnyiny@gmail.com>
Date:   Mon May 4 11:46:06 2020 -0700

    Merge branch 'master' of http://git.navis.lan/scm/n4fra/debezium

commit b514c5e
Merge: babd47d 2b016d7
Author: AndreyIg <gnyiny@gmail.com>
Date:   Mon May 4 11:41:54 2020 -0700

    Merge branch 'master' of http://git.navis.lan/scm/n4fra/debezium

commit babd47d
Merge: e97206f a991266
Author: AndreyIg <gnyiny@gmail.com>
Date:   Mon Apr 20 12:49:30 2020 -0700

    Merge branch 'master' of http://git.navis.lan/scm/n4fra/debezium

... and 49 more commits
  • Loading branch information
Ignatenko Andrey committed Jul 2, 2020
1 parent 1c76971 commit ba72767
Show file tree
Hide file tree
Showing 3 changed files with 13 additions and 10 deletions.
1 change: 1 addition & 0 deletions ...-connector-oracle/src/main/java/io/debezium/connector/oracle/logminer/LogMinerHelper.java
View file
Expand Up @@ -89,6 +89,7 @@ static void createAuditTable(Connection connection) throws SQLException {
/**
* This method returns next SCN for mining and also updates MBean metrics
* We use a configurable limit, because the larger mining range, the slower query from Log Miner content view.
* In addition capturing unlimited number of changes can blow up Java heap.
* Gradual querying helps to catch up faster after long delays in mining.
*
* @param connection container level database connection
Expand Down
20 changes: 11 additions & 9 deletions ...cle/src/main/java/io/debezium/connector/oracle/logminer/LogMinerQueryResultProcessor.java
View file
Expand Up @@ -220,16 +220,18 @@ int processResult(ResultSet resultSet) {
* The criteria is the offset SCN remains the same in five mining cycles
*/
private void warnStuckScn() {
if (currentOffsetScn == offsetContext.getScn() && currentOffsetCommitScn != offsetContext.getCommitScn()) {
stuckScnCounter++;
// logWarn only once
if (stuckScnCounter == 5) {
LogMinerHelper.logWarn(transactionalBufferMetrics,
"Offset SCN {} did not change in five mining cycles, hence the oldest transaction was not committed. Offset commit SCN: {}", currentOffsetScn, offsetContext.getCommitScn());
transactionalBufferMetrics.incrementScnFreezeCounter();
if (offsetContext != null && offsetContext.getCommitScn() != null) {
if (currentOffsetScn == offsetContext.getScn() && currentOffsetCommitScn != offsetContext.getCommitScn()) {
stuckScnCounter++;
// logWarn only once
if (stuckScnCounter == 5) {
LogMinerHelper.logWarn(transactionalBufferMetrics,
"Offset SCN {} did not change in five mining cycles, hence the oldest transaction was not committed. Offset commit SCN: {}", currentOffsetScn, offsetContext.getCommitScn());
transactionalBufferMetrics.incrementScnFreezeCounter();
}
} else {
stuckScnCounter = 0;
}
} else {
stuckScnCounter = 0;
}
}
}
2 changes: 1 addition & 1 deletion ...racle/src/main/java/io/debezium/connector/oracle/logminer/TransactionalBufferMetrics.java
View file
Expand Up @@ -68,7 +68,7 @@ public void setTimeDifference(AtomicLong timeDifference) {
void calculateLagMetrics(Instant changeTime){
if (changeTime != null) {
Instant correctedChangeTime = changeTime.plus(Duration.ofMillis(timeDifference.longValue()));
lagFromTheSource.set(Duration.between(correctedChangeTime, Instant.now()));
lagFromTheSource.set(Duration.between(correctedChangeTime, Instant.now()).abs());

if (maxLagFromTheSource.get().toMillis() < lagFromTheSource.get().toMillis()) {
maxLagFromTheSource.set(lagFromTheSource.get());
Expand Down

0 comments on commit ba72767

Please sign in to comment.