GEODE-8706: getConnection should get readLock to sync with destroyCon…

[gesterzhou]

…nection

Thank you for submitting a contribution to Apache Geode.

In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:

For all changes:

• Is there a JIRA ticket associated with this PR? Is it referenced in the commit message?

• Has your PR been rebased against the latest commit within the target branch (typically develop)?

• Is your initial contribution a single, squashed commit?

• Does gradlew build run cleanly?

• Have you written or updated unit tests to verify your changes?

• If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?

Note:

Please ensure that once the PR is submitted, check Concourse for build issues and
submit an update to your PR as soon as possible. If you need help, please send an
email to dev@geode.apache.org.

[kohlmu-pivotal]

Whilst the change LGTM... I cannot help but feel that the whole logic of (un)/locking of the connectionLifeCycleLock should really be a little more hidden. This locking logic proliferates throughout the whole GatewaySenderEventRemoteDispatcher. Which makes it way too easy to actually miss logic where the connection lock needs to be used.
@boglesby @gesterzhou maybe we should be locking at moving all this logic to another location.
There is also no indication of WHY this lock is required? Is it protecting the accessing of connection properties or is it avoiding the accidental setting of the connection field with another instance? It is not clear.

[boglesby]

The lock is controlling access to the connection field. The destroyConnection and initializeConnection methods set it so they use the writeLock, and the readAcknowledgement and _dispatchBatch methods access it so they use the readLock. The getConnection method should also use the readLock, but it hasn't up until now with this change.

[boglesby]

Does this debug statement need to be moved inside the readLock too since it calls this.connection.getServer()?

[gesterzhou]

No need, you see the original author used:
(this.connection == null) ? "null" : ...

That means he realized this connection could be null here