make ledger metadata immutable

[sijie]

FEATURE REQUEST

1. Please describe the feature you are requesting.

Address the TODO item in LedgerHandle

            if (isClosed()) {
                // TODO: make ledger metadata immutable
                // Although the metadata is already closed, we don't need to proceed zookeeper metadata update, but
                // we still need to error out the pending add ops.
                //
                // There is a race condition a pending add op is enqueued, after a close op reset ledger metadata state
                // to unclosed to resolve metadata conflicts. If we don't error out these pending add ops, they would be
                // leak and never callback.
                //
                // The race condition happen in following sequence:
                // a) ledger L is fenced
                // b) write entry E encountered LedgerFencedException, trigger ledger close procedure
                // c) ledger close encountered metadata version exception and set ledger metadata back to open
                // d) writer tries to write entry E+1, since ledger metadata is still open (reset by c))
                // e) the close procedure in c) resolved the metadata conflicts and set ledger metadata to closed
                // f) writing entry E+1 encountered LedgerFencedException which will enter ledger close procedure
                // g) it would find that ledger metadata is closed, then it callbacks immediately without erroring out any pendings
                synchronized (LedgerHandle.this) {
                    pendingAdds = drainPendingAddsToErrorOut();
                }
                errorOutPendingAdds(rc, pendingAdds);
                cb.closeComplete(BKException.Code.OK, LedgerHandle.this, ctx);
                return;
            }

2. Indicate the importance of this issue to you (blocker, must-have, should-have, nice-to-have). Are you currently using any workarounds to address this issue?

nice-to-have

3. Provide any additional detail on your proposed use case for this feature.

[sijie]

@ivankelly @jiazhai I marked this for 4.7.0 now. there have been discussions and concerns around the mutability of ledger metadata. let's pick this up in 4.7.0.

[sijie]

I don't think we are able to do this in any time soon. dropping the release version at this moment.

[ivankelly]

ya, not a hope for 4.7.0. This needs yahoo and saleforce merge to be 100% complete. We're close, but better to postpone til 4.8.0.

I remember when I did it last time, the change only took about a day.

[ivankelly]

@reddycharan @jvrao Are salesforce planning to make any changes to the client metadata handling?

[ivankelly]

The metadata is immutable now. See the long list of changes for details.