Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(db): add checkpoint v2 for db inconsistency #4614

Conversation

tomatoishealthy
Copy link
Contributor

@tomatoishealthy tomatoishealthy commented Aug 29, 2022

What does this PR do?

Add checkpoint v2 in case of ungraceful shutdown.

Why are these changes required?

This PR has been tested by:

  • Unit Tests
  • Manual Testing

@tomatoishealthy tomatoishealthy force-pushed the feature/db-checkpoint-v2 branch 9 times, most recently from 3209498 to c1edb8f Compare August 31, 2022 03:29
@codecov-commenter
Copy link

codecov-commenter commented Aug 31, 2022

Codecov Report

Merging #4614 (c295061) into release_v4.6.0 (2b3783d) will decrease coverage by 0.25%.
The diff coverage is 76.41%.

@@                 Coverage Diff                  @@
##             release_v4.6.0    #4614      +/-   ##
====================================================
- Coverage             54.40%   54.15%   -0.26%     
+ Complexity             7980     7976       -4     
====================================================
  Files                   831      832       +1     
  Lines                 47154    47333     +179     
  Branches               5106     5136      +30     
====================================================
- Hits                  25655    25634      -21     
- Misses                19472    19661     +189     
- Partials               2027     2038      +11     
Impacted Files Coverage Δ
...c/main/java/org/tron/core/store/CheckTmpStore.java 22.22% <0.00%> (-2.78%) ⬇️
...in/java/org/tron/common/prometheus/MetricKeys.java 0.00% <ø> (ø)
...in/java/org/tron/core/store/CheckPointV2Store.java 22.22% <22.22%> (ø)
...a/org/tron/tool/litefullnode/LiteFullNodeTool.java 72.75% <64.44%> (-1.91%) ⬇️
...n/java/org/tron/core/db2/core/SnapshotManager.java 71.42% <80.74%> (-2.04%) ⬇️
.../common/storage/leveldb/LevelDbDataSourceImpl.java 64.83% <100.00%> (+0.14%) ⬆️
...e/src/main/java/org/tron/core/db/TronDatabase.java 82.92% <100.00%> (+0.87%) ⬆️
...a/org/tron/common/prometheus/MetricsHistogram.java 81.81% <100.00%> (+0.42%) ⬆️
...c/main/java/org/tron/core/config/args/Storage.java 76.05% <100.00%> (+1.42%) ⬆️
...a/org/tron/common/application/ApplicationImpl.java 92.30% <100.00%> (+0.20%) ⬆️
... and 20 more

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@tomatoishealthy tomatoishealthy force-pushed the feature/db-checkpoint-v2 branch 4 times, most recently from 70769fa to 2f8f8d0 Compare September 1, 2022 07:20
@@ -55,7 +52,7 @@ protected TronDatabase(String dbName) {
CommonParameter.getInstance().getStorage().getDbDirectory()).toString();
dbSource =
new RocksDbDataSourceImpl(parentName, dbName, CommonParameter.getInstance()
.getRocksDBCustomSettings());
.getRocksDBCustomSettings(), getDirectComparator());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why this?

}
}
}
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

extract method .

retreat();
checkPointV2Store.close();
logger.info("checkpoint recover success, block number{}", blockNumber);
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

extract method .

if ("block".equals(db.getDbName())) {
currentBlockNum = new BlockCapsule(v.getBytes()).getNum();
}
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

extract method .

if (cpList.size() < 3) {
return;
}
for (String cp: cpList.subList(0,3)) {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

3 ---> length - 3

} else {
Histogram.Timer createV2 = Metrics.histogramStartTimer(
MetricKeys.Histogram.DB_FLUSH, "create2");
createCheckpointV2();
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

createv2 merge to create


dbs.forEach(db -> db.getHead().getRoot().merge(db.getHead()));
retreat();
private void recover(TronDatabase<byte[]> tronDatabase, Map<String, Chainbase> dbMap) {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

change dbMap to dbs

@tomatoishealthy tomatoishealthy force-pushed the feature/db-checkpoint-v2 branch 4 times, most recently from 3c89a02 to 6838cd2 Compare September 15, 2022 03:05
@tomatoishealthy tomatoishealthy changed the title feat(db): add checkpoint v2 in case db inconsistent when ungraceful shutdown feat(db): add checkpoint v2 for db inconsistency Sep 15, 2022
@tomatoishealthy tomatoishealthy merged commit 5cfd21d into tronprotocol:release_v4.6.0 Sep 15, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants