binder: A Channel might check its SecurityPolicy even after declaring termination

[jdcormie]

What version of gRPC-Java are you using?

master/HEAD on any version of Android

What did you expect to see?

Some reasoning/guarantee that a Channel will no longer call into its SecurityPolicy after it declares termination.

Steps to reproduce the bug

One of many ways is to call ManagedChannel.shutdownNow() while it's establishing a new connection.

Analysis

When using AsyncSecurityPolicy, BinderClientTransport calls checkServerAuthorizationAsync() while holding its this lock and only after carefully checking that it's not in state SHUTDOWN or SHUTDOWN_TERMINATED. This is safe because clientTransportListener.transportTerminated() is only called after transitioning into those states while holding the same lock. (We do this on the assumption that AsyncSecurityPolicy checks don't block)

However, when using an ordinary blocking SecurityPolicy, checkServerAuthorizationAsync() is implemented by submit()ing this presumably blocking work to offloadExecutor. We first do the same state checks, but because we release the this lock, the transport could shutdown and declare termination after this submission but before the offload executor actually calls checkAuthorization().