Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix transaction buffer lookup #10257

Conversation

congbobo184
Copy link
Contributor

Motivation

now transaction buffer client handle transaction coordinator command by find topic address and create connect. it can't init PersistentTopic in broker, so the command will always fail.

implement

  1. transaction buffer client handle transaction coordinator command should lookup topic once, the lookup command will init PersistentTopic.
  2. add a cache for the transaction buffer client.

Verifying this change

Add the tests for it

Does this pull request potentially affect one of the following parts:
If yes was chosen, please highlight the changes

Dependencies (does it add or upgrade a dependency): (no)
The public API: (no)
The schema: (no)
The default values of configurations: (no)
The wire protocol: (no)
The rest endpoints: (no)
The admin cli options: (no)
Anything that affects deployment: (no)

@eolivelli eolivelli changed the title Congbobo184 fix transaction buffer lookup Fix transaction buffer lookup Apr 18, 2021
CompletableFuture<ClientCnx> siFuture = getClientCnx(topic);
siFuture.whenComplete((si, cause) -> {
if (null != cause) {
cache.asMap().remove(topic, siFuture);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure I understand this "remove"
probably it is not allowed to modify the cache inside the same loader, especially for the same key

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it may happen that when we execute "whenComplete" are still inside the execution of this loader, for instance if getClientCnx
returns a completed CompletableFuture and so probably we will encounter some bad error.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good suggestion.

Copy link
Contributor

@eolivelli eolivelli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall is good.

I left a question, please take a look before merging

op.recycle();
}
} else {
cache.refresh(topic);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is it better to use "refresh" or "invalidate" here ?

refresh tries to load a new value, asynchronously, if we received an error, it is probable that we will see an error during refresh as well
https://guava.dev/releases/19.0/api/docs/com/google/common/cache/LoadingCache.html#refresh(K)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"invalidate" is better, i will change. good check thanks.

@@ -90,6 +90,8 @@
import org.apache.pulsar.broker.admin.AdminResource;
import org.apache.pulsar.broker.cache.ConfigurationCacheService;
import org.apache.pulsar.broker.cache.LocalZooKeeperCacheService;
import org.apache.pulsar.broker.intercept.BrokerInterceptor;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

these two lines seems unrelated

congbo added 2 commits April 22, 2021 22:47
# Conflicts:
#	pulsar-broker/src/main/java/org/apache/pulsar/broker/transaction/buffer/impl/TransactionBufferHandlerImpl.java
#	pulsar-broker/src/test/java/org/apache/pulsar/broker/transaction/buffer/TransactionBufferClientTest.java
@congbobo184
Copy link
Contributor Author

@eolivelli please review again. thanks.

Copy link
Contributor

@eolivelli eolivelli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

@codelipenghui codelipenghui merged commit e970c29 into apache:master Apr 23, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants