Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

9479: Add more logs to debug virtual map reconnect issues #9481

Merged
merged 5 commits into from
Nov 1, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,7 @@
import com.swirlds.common.merkle.synchronization.views.CustomReconnectRoot;
import com.swirlds.common.merkle.synchronization.views.LearnerTreeView;
import com.swirlds.common.merkle.synchronization.views.StandardLearnerTreeView;
import com.swirlds.common.merkle.utility.MerkleTreeVisualizer;
import com.swirlds.common.threading.manager.ThreadManager;
import com.swirlds.common.threading.pool.StandardWorkGroup;
import com.swirlds.logging.payloads.SynchronizationCompletePayload;
Expand Down Expand Up @@ -149,7 +150,14 @@
* Attempt to free any and all resources that were acquired during the reconnect attempt.
*/
private void abort() {
logger.warn(RECONNECT.getMarker(), "deleting partially constructed tree");
logger.warn(
RECONNECT.getMarker(),

Check warning on line 154 in platform-sdk/swirlds-common/src/main/java/com/swirlds/common/merkle/synchronization/LearningSynchronizer.java

View check run for this annotation

Codecov / codecov/patch

platform-sdk/swirlds-common/src/main/java/com/swirlds/common/merkle/synchronization/LearningSynchronizer.java#L153-L154

Added lines #L153 - L154 were not covered by tests
"Deleting partially constructed tree:\n{}",
new MerkleTreeVisualizer(newRoot)
.setDepth(5)
.setUseHashes(false)
.setUseMnemonics(false)
.render());

Check warning on line 160 in platform-sdk/swirlds-common/src/main/java/com/swirlds/common/merkle/synchronization/LearningSynchronizer.java

View check run for this annotation

Codecov / codecov/patch

platform-sdk/swirlds-common/src/main/java/com/swirlds/common/merkle/synchronization/LearningSynchronizer.java#L157-L160

Added lines #L157 - L160 were not covered by tests
try {
if (newRoot != null) {
newRoot.release();
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -196,18 +196,8 @@
indexString = Integer.toString(route.getStep(-1));
}

final Hash hash = node == null ? CryptographyHolder.get().getNullHash() : node.getHash();
final String className = node == null ? "null" : node.getClass().getSimpleName();

final String hashString;
if (hash == null) {
hashString = "null";
} else if (hashLength == -1) {
hashString = hash.toString();
} else {
hashString = hash.toString().substring(0, hashLength);
}

final String formattedIndexString = useColors ? BRIGHT_CYAN.apply(indexString) : indexString;
final String formattedClassName = useColors ? BRIGHT_YELLOW.apply(className) : className;

Expand All @@ -229,15 +219,27 @@
table.addToRow(formattedRouteString);
}

if (useMnemonics) {
final String mnemonic = hash == null ? "" : hash.toMnemonic();
final String formattedMnemonic = useColors ? WHITE.apply(mnemonic) : mnemonic;
table.addToRow(formattedMnemonic);
}
if (useHashes || useMnemonics) {
final Hash hash = node == null ? CryptographyHolder.get().getNullHash() : node.getHash();
final String hashString;
if (hash == null) {
hashString = "null";

Check warning on line 226 in platform-sdk/swirlds-common/src/main/java/com/swirlds/common/merkle/utility/MerkleTreeVisualizer.java

View check run for this annotation

Codecov / codecov/patch

platform-sdk/swirlds-common/src/main/java/com/swirlds/common/merkle/utility/MerkleTreeVisualizer.java#L226

Added line #L226 was not covered by tests
} else if (hashLength == -1) {
hashString = hash.toString();

Check warning on line 228 in platform-sdk/swirlds-common/src/main/java/com/swirlds/common/merkle/utility/MerkleTreeVisualizer.java

View check run for this annotation

Codecov / codecov/patch

platform-sdk/swirlds-common/src/main/java/com/swirlds/common/merkle/utility/MerkleTreeVisualizer.java#L228

Added line #L228 was not covered by tests
} else {
hashString = hash.toString().substring(0, hashLength);

Check warning on line 230 in platform-sdk/swirlds-common/src/main/java/com/swirlds/common/merkle/utility/MerkleTreeVisualizer.java

View check run for this annotation

Codecov / codecov/patch

platform-sdk/swirlds-common/src/main/java/com/swirlds/common/merkle/utility/MerkleTreeVisualizer.java#L230

Added line #L230 was not covered by tests
}

if (useMnemonics) {
final String mnemonic = hash == null ? "" : hash.toMnemonic();
final String formattedMnemonic = useColors ? WHITE.apply(mnemonic) : mnemonic;
table.addToRow(formattedMnemonic);

Check warning on line 236 in platform-sdk/swirlds-common/src/main/java/com/swirlds/common/merkle/utility/MerkleTreeVisualizer.java

View check run for this annotation

Codecov / codecov/patch

platform-sdk/swirlds-common/src/main/java/com/swirlds/common/merkle/utility/MerkleTreeVisualizer.java#L236

Added line #L236 was not covered by tests
}

if (useHashes) {
final String formattedHashString = useColors ? GRAY.apply(hashString) : hashString;
table.addToRow(formattedHashString);
if (useHashes) {
final String formattedHashString = useColors ? GRAY.apply(hashString) : hashString;
table.addToRow(formattedHashString);

Check warning on line 241 in platform-sdk/swirlds-common/src/main/java/com/swirlds/common/merkle/utility/MerkleTreeVisualizer.java

View check run for this annotation

Codecov / codecov/patch

platform-sdk/swirlds-common/src/main/java/com/swirlds/common/merkle/utility/MerkleTreeVisualizer.java#L241

Added line #L241 was not covered by tests
}
}
});

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -736,29 +736,37 @@
// shut down all executors
shutdownThreadsAndWait(storeInternalExecutor, storeKeyToPathExecutor, snapshotExecutor);
} finally {
// close all closable data stores
logger.info(MERKLE_DB.getMarker(), "Closing Data Source [{}]", tableName);
if (hashStoreRam != null) {
hashStoreRam.close();
}
if (hashStoreDisk != null) {
hashStoreDisk.close();
}
pathToDiskLocationInternalNodes.close();
pathToDiskLocationLeafNodes.close();
if (longKeyToPath != null) {
longKeyToPath.close();
}
if (objectKeyToPath != null) {
objectKeyToPath.close();
try {
// close all closable data stores

Check warning on line 740 in platform-sdk/swirlds-jasperdb/src/main/java/com/swirlds/merkledb/MerkleDbDataSource.java

View check run for this annotation

Codecov / codecov/patch

platform-sdk/swirlds-jasperdb/src/main/java/com/swirlds/merkledb/MerkleDbDataSource.java#L740

Added line #L740 was not covered by tests
logger.info(MERKLE_DB.getMarker(), "Closing Data Source [{}]", tableName);
if (hashStoreRam != null) {
hashStoreRam.close();

Check warning on line 743 in platform-sdk/swirlds-jasperdb/src/main/java/com/swirlds/merkledb/MerkleDbDataSource.java

View check run for this annotation

Codecov / codecov/patch

platform-sdk/swirlds-jasperdb/src/main/java/com/swirlds/merkledb/MerkleDbDataSource.java#L743

Added line #L743 was not covered by tests
}
if (hashStoreDisk != null) {
hashStoreDisk.close();

Check warning on line 746 in platform-sdk/swirlds-jasperdb/src/main/java/com/swirlds/merkledb/MerkleDbDataSource.java

View check run for this annotation

Codecov / codecov/patch

platform-sdk/swirlds-jasperdb/src/main/java/com/swirlds/merkledb/MerkleDbDataSource.java#L746

Added line #L746 was not covered by tests
}
pathToDiskLocationInternalNodes.close();
pathToDiskLocationLeafNodes.close();

Check warning on line 749 in platform-sdk/swirlds-jasperdb/src/main/java/com/swirlds/merkledb/MerkleDbDataSource.java

View check run for this annotation

Codecov / codecov/patch

platform-sdk/swirlds-jasperdb/src/main/java/com/swirlds/merkledb/MerkleDbDataSource.java#L748-L749

Added lines #L748 - L749 were not covered by tests
if (longKeyToPath != null) {
longKeyToPath.close();
}

Check warning on line 752 in platform-sdk/swirlds-jasperdb/src/main/java/com/swirlds/merkledb/MerkleDbDataSource.java

View check run for this annotation

Codecov / codecov/patch

platform-sdk/swirlds-jasperdb/src/main/java/com/swirlds/merkledb/MerkleDbDataSource.java#L751-L752

Added lines #L751 - L752 were not covered by tests
if (objectKeyToPath != null) {
objectKeyToPath.close();
}

Check warning on line 755 in platform-sdk/swirlds-jasperdb/src/main/java/com/swirlds/merkledb/MerkleDbDataSource.java

View check run for this annotation

Codecov / codecov/patch

platform-sdk/swirlds-jasperdb/src/main/java/com/swirlds/merkledb/MerkleDbDataSource.java#L754-L755

Added lines #L754 - L755 were not covered by tests
pathToKeyValue.close();
// Store metadata

Check warning on line 757 in platform-sdk/swirlds-jasperdb/src/main/java/com/swirlds/merkledb/MerkleDbDataSource.java

View check run for this annotation

Codecov / codecov/patch

platform-sdk/swirlds-jasperdb/src/main/java/com/swirlds/merkledb/MerkleDbDataSource.java#L757

Added line #L757 was not covered by tests
saveMetadata(dbPaths.metadataFile);
} catch (final Exception e) {
logger.warn(EXCEPTION.getMarker(), "Exception while closing Data Source [{}]", tableName);
} catch (final Error t) {
logger.error(EXCEPTION.getMarker(), "Error while closing Data Source [{}]", tableName);
throw t;
} finally {
// updated count of open databases
COUNT_OF_OPEN_DATABASES.decrement();
// Notify the database
database.closeDataSource(this);
}
pathToKeyValue.close();
// updated count of open databases
COUNT_OF_OPEN_DATABASES.decrement();
// Store metadata
saveMetadata(dbPaths.metadataFile);
// Notify the database
database.closeDataSource(this);
}
}
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -965,7 +965,7 @@
if (dataSource != null) {
try {
dataSource.close();
} catch (final IOException e) {
} catch (final Exception e) {

Check warning on line 968 in platform-sdk/swirlds-virtualmap/src/main/java/com/swirlds/virtualmap/internal/merkle/VirtualRootNode.java

View check run for this annotation

Codecov / codecov/patch

platform-sdk/swirlds-virtualmap/src/main/java/com/swirlds/virtualmap/internal/merkle/VirtualRootNode.java#L968

Added line #L968 was not covered by tests
logger.error(
EXCEPTION.getMarker(), "Could not close the dataSource after all copies were destroyed", e);
}
Expand Down
Loading