Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[pulsar-discovery] Replace MetadataStore with ZooKeeper in discoveryservice #9967

Merged
merged 2 commits into from
Mar 19, 2021

Conversation

rdhabalia
Copy link
Contributor

Motivation

one of the Implementations of #9646 . Remove direct ZooKeeper dependency from discovery-service and replace with metadata-store. Rename ZookeeperCacheLoader to MetadataStoreCacheLoader and use MetadataStore access zk.

Copy link
Contributor

@eolivelli eolivelli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

great work

I left a little comment about a test, there is a static variable that is not clearer.
I believe we should fix it before merging


// DiscoveryServiceServlet gets initialized by a server and this map will help to retrieve ZK while mocking
// DiscoveryServiceServlet
private static final Map<String, MetadataStoreExtended> metadataStoreInstanceCache = Maps.newConcurrentMap();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it looks like we are not clearing this map at the end of the test.
we will leak MetadataStoreExtended instances

Copy link
Contributor Author

@rdhabalia rdhabalia Mar 19, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not leaking. static map is intentional because of the DiscoveryServiceServlet which gets initialized by Webserver and we need to inject MetadataStore into dynamically initialized servelet. so, DiscoveryServiceServletTest extends the discovery-servlet class and access metadata store from this static map.
It's not leaking because everytime when test starts it puts the metadatastore into the map and then it closes the metadata store at the end of test so, it will not leak.
So, static map is the way to access metadata-store from dynamically initialized servlet which we can't be mocked or controlled.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am fine with having the static map. it is a good idea.

I would only add a

@AlfterClass
public void clearMetadataStoreInstanceCache() { 
    metadataStoreInstanceCache.clear();
}

if we do not clear the map then we are keeping a reference to the (closed) MetadataStore into a static variable.

@@ -56,6 +56,9 @@
public class DiscoveryServiceWebTest extends ProducerConsumerBase {

private final Client client = ClientBuilder.newClient(new ClientConfig().register(LoggingFeature.class));
// DiscoveryServiceServlet gets initialized by a server and this map will help to retrieve ZK while mocking
// DiscoveryServiceServlet
private static final Map<String, MetadataStoreExtended> metadataStoreInstanceCache = Maps.newConcurrentMap();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it looks like we are not clearing this map at the end of the test.
we will leak MetadataStoreExtended instances

Copy link
Contributor

@eolivelli eolivelli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@rdhabalia rdhabalia merged commit dd32435 into apache:master Mar 19, 2021
@rdhabalia rdhabalia deleted the zkloader_dep branch March 19, 2021 21:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
No open projects
Development

Successfully merging this pull request may close these issues.

None yet

4 participants