HDDS-9702. Improve logging when Recon gets a full update from OM

[devmadhuu]

What changes were proposed in this pull request?

Since Recon will fallback to full OM DB snapshot update in case failed to get incremental updates, but we don't need this below full trace rather a brief message with some important details is enough.

2023-10-18 23:09:01,152 WARN [pool-28-thread-1]-org.apache.hadoop.ozone.recon.spi.impl.OzoneManagerServiceProviderImpl: Unable to get and apply delta updates from OM.
INTERNAL_ERROR org.apache.hadoop.ozone.om.exceptions.OMException: Unable to read full data from RocksDB wal to get delta updates. It may have partially been flushed to SSTs. Requested sequence number is 69523 and first available sequence number is 224239 in wal.
        at org.apache.hadoop.ozone.om.protocolPB.OzoneManagerProtocolClientSideTranslatorPB.handleError(OzoneManagerProtocolClientSideTranslatorPB.java:728)
        at org.apache.hadoop.ozone.om.protocolPB.OzoneManagerProtocolClientSideTranslatorPB.getDBUpdates(OzoneManagerProtocolClientSideTranslatorPB.java:2091)
        at org.apache.hadoop.ozone.recon.spi.impl.OzoneManagerServiceProviderImpl.innerGetAndApplyDeltaUpdatesFromOM(OzoneManagerServiceProviderImpl.java:450)
        at org.apache.hadoop.ozone.recon.spi.impl.OzoneManagerServiceProviderImpl.getAndApplyDeltaUpdatesFromOM(OzoneManagerServiceProviderImpl.java:415)
        at org.apache.hadoop.ozone.recon.spi.impl.OzoneManagerServiceProviderImpl.syncDataFromOM(OzoneManagerServiceProviderImpl.java:502)
        at org.apache.hadoop.ozone.recon.spi.impl.OzoneManagerServiceProviderImpl.lambda$startSyncDataFromOM$0(OzoneManagerServiceProviderImpl.java:258)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305)
        at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:834)

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-9702

How was this patch tested?

This patch was tested using manual Junit test case as well as by forcing full OM DB snaphsot update from OM.

[devmadhuu]

@dombizita @sumitagrawl Pls review.

[ArafatKhan2198]

Can we add the same changes here as well, printing out the error message rather than the whole stack trace.

ozone/hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/spi/impl/OzoneManagerServiceProviderImpl.java

Line 555 in c0968b1

LOG.error("Unable to update Recon's metadata with new OM DB. ", e);

[ArafatKhan2198]

I have a question!
Would it be a good idea to mention the this.getClass().getName() to over here, since the stack trace would not be printed?

[devmadhuu]

I have a question! Would it be a good idea to mention the this.getClass().getName() to over here, since the stack trace would not be printed?

Whenever there is a log printed in logs, log4j always prints the class name from where this log has appeared, so I believe that is not needed.

[devmadhuu]

Can we add the same changes here as well, printing out the error message rather than the whole stack trace.

ozone/hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/spi/impl/OzoneManagerServiceProviderImpl.java

Line 555 in c0968b1

LOG.error("Unable to update Recon's metadata with new OM DB. ", e);

No, we don't want to hide details for full snapshot error, so left as it is.

[sumitagrawl]

@devmadhuu LGTM

[dombizita]

Thanks for improving this @devmadhuu, it looks good to me!