Add TestKit

[cmccabe]

This is a proof of concept of some ideas for making a public Kafka cluster testing interface. The main idea here is that you have a public, stable interface where you can easily spin up a Kafka cluster for testing with code like this:

try (MiniKafkaCluster cluster = new MiniKafkaClusterBuilder()
    .addZookeeperNode(new MiniZookeeperNodeBuilder())
    .addNode(new MiniKafkaNodeBuilder())
    .build()) {
  // do stuff with the brokers, zookeepers, etc.
}

Because the MiniKafkaCluster extends AutoCloseable, it can be used in a Java7 try-with-resources block. It will be automatically closed when it goes out of scope.

Because we use builders everywhere, it is possible to easily add new parameters to any of the classes without breaking backwards compatibility. For example, we could add a way to configure the port or address that each Kafka node binds to when it starts up, and so forth.

When parameters are not supplied, they should default to something reasonable. For example, if no Kafka log directories are specified, we create a single Kafka log directory for each broker. When broker ids are not specified, we default to automatic ID assignment through ZooKeeper.

Tests can shut down brokers or zookepeer nodes as part of the test by invoking shutdown on the relevant Java object. They services can be restarted by calling start.

During cluster startup and shutdown, we parallelize operations that take a long time, like starting a broker or shutting down a broker. This cuts test times.

This is a rough sketch that doesn't include clients or SSL / SASL.

[cmccabe]

This has been implemented, although in a slightly different form. :)