Skip to content

HDDS-8251. RocksIterator is not closed properly in snapshot#4461

Merged
duongkame merged 2 commits into
apache:masterfrom
duongkame:HDDS-8251
Mar 23, 2023
Merged

HDDS-8251. RocksIterator is not closed properly in snapshot#4461
duongkame merged 2 commits into
apache:masterfrom
duongkame:HDDS-8251

Conversation

@duongkame
Copy link
Copy Markdown
Contributor

What changes were proposed in this pull request?

RocksObject leaks in snapshot.

2023-03-23 09:44:20,288 [Finalizer] WARN  managed.ManagedRocksObjectUtils (ManagedRocksObjectUtils.java:assertClosed(54)) - RocksIterator is not closed properly
 StackTrace for unclosed instance: org.apache.hadoop.hdds.utils.db.managed.ManagedObject.<init>(ManagedObject.java:35)
org.apache.hadoop.hdds.utils.db.managed.ManagedRocksIterator.<init>(ManagedRocksIterator.java:28)
org.apache.hadoop.hdds.utils.db.managed.ManagedRocksIterator.managed(ManagedRocksIterator.java:32)
org.apache.hadoop.hdds.utils.db.RocksDatabase.newIterator(RocksDatabase.java:757)
org.apache.hadoop.hdds.utils.db.RDBTable.iterator(RDBTable.java:167)
org.apache.hadoop.hdds.utils.db.TypedTable.iterator(TypedTable.java:286)
org.apache.hadoop.ozone.om.SnapshotChainManager.loadFromSnapshotInfoTable(SnapshotChainManager.java:266)
org.apache.hadoop.ozone.om.SnapshotChainManager.<init>(SnapshotChainManager.java:63)
org.apache.hadoop.ozone.om.OmMetadataManagerImpl.start(OmMetadataManagerImpl.java:480)
org.apache.hadoop.ozone.om.OmMetadataManagerImpl.<init>(OmMetadataManagerImpl.java:319)
org.apache.hadoop.ozone.om.OzoneManager.instantiateServices(OzoneManager.java:747)
org.apache.hadoop.ozone.om.OzoneManager.<init>(OzoneManager.java:628)
org.apache.hadoop.ozone.om.OzoneManager.createOm(OzoneManager.java:713)
org.apache.hadoop.ozone.MiniOzoneHAClusterImpl$Builder.createOMService(MiniOzoneHAClusterImpl.java:518)
org.apache.hadoop.ozone.MiniOzoneHAClusterImpl$Builder.build(MiniOzoneHAClusterImpl.java:426)
org.apache.hadoop.ozone.om.TestOmSnapshot.init(TestOmSnapshot.java:175)
org.apache.hadoop.ozone.om.TestOmSnapshot.<init>(TestOmSnapshot.java:144) 
2023-03-23 10:10:20,868 [Finalizer] WARN  managed.ManagedRocksObjectUtils (ManagedRocksObjectUtils.java:assertClosed(54)) - RocksIterator is not closed properly
 StackTrace for unclosed instance: org.apache.hadoop.hdds.utils.db.managed.ManagedObject.<init>(ManagedObject.java:35)
org.apache.hadoop.hdds.utils.db.managed.ManagedRocksIterator.<init>(ManagedRocksIterator.java:28)
org.apache.hadoop.ozone.om.snapshot.RocksDbPersistentList.iterator(RocksDbPersistentList.java:85)
org.apache.hadoop.ozone.om.snapshot.SnapshotDiffManager.addToReport(SnapshotDiffManager.java:658)
org.apache.hadoop.ozone.om.snapshot.SnapshotDiffManager.generateDiffReport(SnapshotDiffManager.java:612)
org.apache.hadoop.ozone.om.snapshot.SnapshotDiffManager.generateSnapshotDiffReport(SnapshotDiffManager.java:343)
org.apache.hadoop.ozone.om.snapshot.SnapshotDiffManager.getSnapshotDiffReport(SnapshotDiffManager.java:196)
org.apache.hadoop.ozone.om.OmSnapshotManager.getSnapshotDiffReport(OmSnapshotManager.java:455)
org.apache.hadoop.ozone.om.OzoneManager.snapshotDiff(OzoneManager.java:4493)
org.apache.hadoop.ozone.protocolPB.OzoneManagerRequestHandler.snapshotDiff(OzoneManagerRequestHandler.java:1223)
org.apache.hadoop.ozone.protocolPB.OzoneManagerRequestHandler.handleReadRequest(OzoneManagerRequestHandler.java:300)
org.apache.hadoop.ozone.protocolPB.OzoneManagerProtocolServerSideTranslatorPB.submitReadRequestToOM(OzoneManagerProtocolServerSideTranslatorPB.java:223)
org.apache.hadoop.ozone.protocolPB.OzoneManagerProtocolServerSideTranslatorPB.processRequest(OzoneManagerProtocolServerSideTranslatorPB.java:177)
org.apache.hadoop.hdds.server.OzoneProtocolMessageDispatcher.processRequest(OzoneProtocolMessageDispatcher.java:87) 

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-8251

How was this patch tested?

No more leak warnings when running TestOmSnapshot integration test.

@duongkame duongkame requested a review from symious March 23, 2023 18:31
@duongkame duongkame requested a review from adoroszlai March 23, 2023 18:45
@duongkame duongkame marked this pull request as ready for review March 23, 2023 18:45
@duongkame
Copy link
Copy Markdown
Contributor Author

@hemantk-12 Can you please review this?

Copy link
Copy Markdown
Contributor

@hemantk-12 hemantk-12 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

Thanks for the quick fix @duongkame.

Sorry for all these memory leaks. I tired to fix this in #4438

Copy link
Copy Markdown
Contributor

@adoroszlai adoroszlai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @duongkame for the fix.

@smengcl
Copy link
Copy Markdown
Contributor

smengcl commented Mar 23, 2023

Thanks @duongkame for the quick fix!

@smengcl
Copy link
Copy Markdown
Contributor

smengcl commented Mar 24, 2023

I'm wondering if there is a findbugs spotbugs warning for leaking resource check so that we can hopefully avoid similar issues before a commit is even merged. Then I realize spotbugs does have some. Filed HDDS-8268

@duongkame duongkame deleted the HDDS-8251 branch April 12, 2025 00:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants