No ledger found while trying to read last entry: 306

[ru-guo]

03:29:34.194 [BookieReadThreadPool-OrderedExecutor-0-0] ERROR org.apache.bookkeeper.proto.ReadLacProcessorV3 - No ledger found while performing readLac from ledger: 306
org.apache.bookkeeper.bookie.Bookie$NoLedgerException: Ledger 306 not found
at org.apache.bookkeeper.bookie.LedgerDescriptor.createReadOnly(LedgerDescriptor.java:52) ~[org.apache.bookkeeper-bookkeeper-server-4.12.0.jar:4.12.0]
at org.apache.bookkeeper.bookie.HandleFactoryImpl.getReadOnlyHandle(HandleFactoryImpl.java:61) ~[org.apache.bookkeeper-bookkeeper-server-4.12.0.jar:4.12.0]
at org.apache.bookkeeper.bookie.Bookie.getExplicitLac(Bookie.java:1364) ~[org.apache.bookkeeper-bookkeeper-server-4.12.0.jar:4.12.0]
at org.apache.bookkeeper.proto.ReadLacProcessorV3.getReadLacResponse(ReadLacProcessorV3.java:71) [org.apache.bookkeeper-bookkeeper-server-4.12.0.jar:4.12.0]
at org.apache.bookkeeper.proto.ReadLacProcessorV3.safeRun(ReadLacProcessorV3.java:118) [org.apache.bookkeeper-bookkeeper-server-4.12.0.jar:4.12.0]
at org.apache.bookkeeper.common.util.SafeRunnable.run(SafeRunnable.java:36) [org.apache.bookkeeper-bookkeeper-common-4.12.0.jar:4.12.0]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_282]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_282]
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [io.netty-netty-common-4.1.60.Final.jar:4.1.60.Final]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_282]
03:29:34.195 [BookieReadThreadPool-OrderedExecutor-0-0] ERROR org.apache.bookkeeper.proto.ReadLacProcessorV3 - No ledger found while trying to read last entry: 306
org.apache.bookkeeper.bookie.Bookie$NoLedgerException: Ledger 306 not found

[Jason918]

@ru-guo It would be easier for others to understand if you could provide the steps to reproduce this.

[codelipenghui]

@ru-guo Could you please try to use bin/bookeeper shell ledgermetadata -ledgerid to check if the ledger metadata exists and also can use bin/bookeeper shell readledger to try to read data from the Ledger.

[hangc0276]

When we create a ledger and not entry writes to the ledger, and then call readLastAddConfirmedAsync, it will just check whether the request ledger has been exist in ledgerStorage. Due to we have not write any entry into the ledger, the check will return false, and it will throw NoLedgerException. The following test will reproduce this issue.

@Test
    public void testReadLacOnEmptyLedger() {
        try {
            CompletableFuture<Void> future = new CompletableFuture<>();
            LedgerHandle lh = bkc.createLedger(1, 1, 1, digestType, "test".getBytes());
            lh.readLastAddConfirmedAsync().thenAccept(lastAddConfirmed -> {
                logger.info("lastAddConfirmed: {}", lastAddConfirmed);
                future.complete(null);
            }).exceptionally(ex -> {
                logger.error("e: ", ex);
                future.completeExceptionally(null);
                return null;
            });
            future.get();
        } catch (Exception e) {
            logger.error("Read last add confirmed failed: ", e);
            fail();
        }

However, the response builder will wrap the exception and just return -1 as the lastAddConfirmed to represent Invalid entry id. And in pulsar client, it won't throw exception and just use -1 to generate ledger info

pulsar/managed-ledger/src/main/java/org/apache/bookkeeper/mledger/impl/ReadOnlyManagedLedgerImpl.java

Lines 68 to 91 in 2c2922d

	bookKeeper.newOpenLedgerOp().withRecovery(false).withLedgerId(lastLedgerId)
	.withDigestType(config.getDigestType()).withPassword(config.getPassword()).execute()
	.thenAccept(readHandle -> {
	readHandle.readLastAddConfirmedAsync().thenAccept(lastAddConfirmed -> {
	LedgerInfo info = LedgerInfo.newBuilder().setLedgerId(lastLedgerId)
	.setEntries(lastAddConfirmed + 1).setSize(readHandle.getLength())
	.setTimestamp(clock.millis()).build();
	ledgers.put(lastLedgerId, info);
	future.complete(createReadOnlyCursor(startPosition));
	}).exceptionally(ex -> {
	if (ex instanceof CompletionException
	&& ex.getCause() instanceof IllegalArgumentException) {
	// The last ledger was empty, so we cannot read the last add confirmed.
	LedgerInfo info = LedgerInfo.newBuilder().setLedgerId(lastLedgerId)
	.setEntries(0).setSize(0).setTimestamp(clock.millis()).build();
	ledgers.put(lastLedgerId, info);
	future.complete(createReadOnlyCursor(startPosition));
	} else {
	future.completeExceptionally(new ManagedLedgerException(ex));
	}
	return null;
	});
	}).exceptionally(ex -> {

In the end, the exception is normal and it is no harm for pulsar to process messages.
We have an PR to change the log level from error to warn. apache/bookkeeper#2757

[github-actions]

The issue had no activity for 30 days, mark with Stale label.