Skip to content

Adding multi-node ZKCluster test util class#1753

Merged
sijie merged 2 commits into
apache:masterfrom
reddycharan:multinodezkcluster
Oct 25, 2018
Merged

Adding multi-node ZKCluster test util class#1753
sijie merged 2 commits into
apache:masterfrom
reddycharan:multinodezkcluster

Conversation

@reddycharan
Copy link
Copy Markdown
Contributor

Descriptions of the changes in this PR:

  • adding multi-node ZKCluster util class
  • adding new interface for ZKCluster util and having separate implementaions for
    single node ZKCluster and multi-node ZKCluster

Motivation

This helps in adding more test coverage to Zookeeper aspects.

- adding multi-node ZKCluster util class
- adding new interface for ZKCluster util and having separate implementaions for
single node ZKCluster and multi-node ZKCluster
* reasons
*/
System.setProperty("zookeeper.4lw.commands.whitelist", "*");
System.setProperty("zookeeper.admin.enableServer", "false");
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about adding zookeeper.forceSync=no ?
We do not care about fsync in tests

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

from doc -

"Unsafe Options
The following options can be useful, but be careful when you use them. The risk of each is explained along with the explanation of what the variable does.
forceSync
(Java system property: zookeeper.forceSync)
Requires updates to be synced to media of the transaction log before finishing processing the update. If this option is set to no, ZooKeeper will not require updates to be synced to the media."

maybe worth considering, but this must not be specific to multinode zk cluster. So I'll leave it now for this change. We can revisit if it is needed for both kind of zkclusters (singlenode and multinode).

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This code is only for test cases. You don't really need to fsync in test cases, as processes do not crash.
Waiting/forcing an fsync at every zk write is useless in our tests corpus, isn't it?
I mean, this is not a production zk cluster

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

couple of things,

  1. people may take snapshot of ZK for analysis of test failure, so not fsyncing might be an issue for them.
  2. even if we decide not to do fsync, i think it should be in different commit with full details, since it affects the behavior of single noded ZKCluster utility class also. So I'm not inclined to make that change in this PR.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay.
I usually just add the sys prop on the command line.
We can think about changing the command line on CI.
Thank you for your explanations


String getMetadataServiceUri(String zkLedgersRootPath, String type);

void startServer() throws Exception;
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about renaming to startServers() or startCluster() ?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would suggest startCluster. and also change stopServer, restartServer, killServer to stopCluster, restartCluster and killCluster.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah, but the issue is I've to change in 100's of places where these methods are called in existing testcode..will check

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I hope your IDE will help....

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done.

Copy link
Copy Markdown
Member

@sijie sijie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

overall looks good to me.

@sijie
Copy link
Copy Markdown
Member

sijie commented Oct 25, 2018

IGNORE IT CI

@sijie sijie added this to the 4.9.0 milestone Oct 25, 2018
@sijie sijie merged commit afb6232 into apache:master Oct 25, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants