[dlog] Issue 1544: ConcurrentModificationException with nonblocking logReader.readNext(true)

[sijie]

Descriptions of the changes in this PR:

Motivation

Fixes #1544. ConcurrentModificationException is thrown when trying to access a non-thread-safe hashmap from different threads.

Changes

Make sure accessing to this non-thread-safe hashmap is under synchronized block.

Tests

It is a bit tricky to reproduce this race condition in a unit test or an integration test. so not going to attempt adding any tests.

Master Issue: #1544

---

In order to uphold a high standard for quality for code contributions, Apache BookKeeper runs various precommit
checks for pull requests. A pull request can only be merged when it passes precommit checks. However running all
the precommit checks can take a long time, some trivial changes don't need to run all the precommit checks. You
can check following list to skip the tests that don't need to run for your pull request. Leave them unchecked if
you are not sure, committers will help you:

• [skip bookkeeper-server bookie tests]: skip testing org.apache.bookkeeper.bookie in bookkeeper-server module.
• [skip bookkeeper-server client tests]: skip testing org.apache.bookkeeper.client in bookkeeper-server module.
• [skip bookkeeper-server replication tests]: skip testing org.apache.bookkeeper.replication in bookkeeper-server module.
• [skip bookkeeper-server tls tests]: skip testing org.apache.bookkeeper.tls in bookkeeper-server module.
• [skip bookkeeper-server remaining tests]: skip testing all other tests in bookkeeper-server module.
• [skip integration tests]: skip docker based integration tests. if you make java code changes, you shouldn't skip integration tests.
• [skip build java8]: skip build on java8. ONLY skip this when ONLY changing files under documentation under site.
• [skip build java9]: skip build on java9. ONLY skip this when ONLY changing files under documentation under site.

---

---

Be sure to do all of the following to help us incorporate your contribution
quickly and easily:

If this PR is a BookKeeper Proposal (BP):

• Make sure the PR title is formatted like:
  <BP-#>: Description of bookkeeper proposal
e.g. BP-1: 64 bits ledger is support
• Attach the master issue link in the description of this PR.
• Attach the google doc link if the BP is written in Google Doc.

Otherwise:

• Make sure the PR title is formatted like:
  <Issue #>: Description of pull request
e.g. Issue 123: Description ...
• Make sure tests pass via mvn clean apache-rat:check install spotbugs:check.
• Replace <Issue #> in the title with the actual Issue number.

---

[sijie]

@infodog this should fix the issue you reported in #1544

[eolivelli]

I wonder why spotbugs is okay with this change. We are not always accessing listeners using a synchronized block.

By the way, I don't this is it good to synchronize over a ConcurrentMap, maybe an alternative solution is to clone the keySet and iterate over the clone.

Maybe I am missing something

[eolivelli]

Sorry now I see, listeners is not the main listeners field.

+1

[eolivelli]

retest this please

[sijie]

rebuild java8
rebuild java9
rerun integration tests
rerun pr validation

[eolivelli]

rebuild java9