[BEAM-14181] Make sure to evict connections from cache after closing them #17187
Conversation
|
@youngoli - Could we consider this for cherry picking into the release branch? |
reuvenlax
changed the title
| @@ -173,6 +173,12 @@ public DestinationState( | |||
| this.useDefaultStream = useDefaultStream; | |||
| } | |||
|
|
|||
| void teardown() { | |||
| if (streamAppendClient != null) { | |||
| runAsyncIgnoreFailure(closeWriterExecutor, streamAppendClient::unpin); | |||
There was a problem hiding this comment.
do we need to invalidate it from the cache as well here?
There was a problem hiding this comment.
No - unpin will only close the connection if close() has already been called. Since the client is still healthy, we can leave it in the cache
|
LGTM otherwise |
reuvenlax
changed the title
|
A cherry-pick shouldn't be a problem. I'm currently waiting on some other release-blocking issues so I can cherry-pick this in. Can someone add more information to the Jira to make it explicit why this is a release-blocking issue (especially identifying when the issue was introduced and describing what problems it's causing)? |
|
@youngoli Yes - roughly this causes the BigQuery sink to sometimes get full stuck and never recover, and the pipeline grinds to a halt. I believe the regression was introduced in the last Beam release. |
|
Run Java Precommit |
|
Run Java PreCommit |
…ections from cache after closing them (cherry picked from commit e7d7525)
No description provided.