Skip to content
CI/CD pipeline for Apache Zookeeper
Ruby Python Shell HTML
Branch: master
Clone or download
Pull request Compare This branch is 2 commits ahead, 2 commits behind ali1dc:master.
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
cookbooks/zookeeper-config
packer
pipeline
provisioning
.gitignore
.kitchen.yml
Berksfile
Gemfile
Gemfile.lock
Jenkinsfile
LICENSE.md
README.md
Rakefile
docker-compose.yaml

README.md

Apache Zookeeper

This repository contains pipeline and infrastructure code for deploying Zookeeper services for Excella Stream Processing Platform.

What is Zookeeper

Zookeeper is a centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. All of these kinds of services are used in some form or another by distributed applications. Each time they are implemented there is a lot of work that goes into fixing the bugs and race conditions that are inevitable. Because of the difficulty of implementing these kinds of services, applications initially usually skimp on them, which make them brittle in the presence of change and difficult to manage. Even when done correctly, different implementations of these services lead to management complexity when the applications are deployed.

Zookeeper aims at distilling the essence of these different services into a very simple interface to a centralized coordination service. The service itself is distributed and highly reliable. Consensus, group management, and presence protocols will be implemented by the service so that the applications do not need to implement them on their own. Application specific uses of these will consist of a mixture of specific components of Zookeeper and application specific conventions.

Connection Url:

10.100.1.100:2181,10.100.2.100:2181,10.100.3.100:2181

How to use it

  1. Download and install Confluent Platform
  2. Some useful command:
    export zk=10.100.1.100:2181,10.100.2.100:2181,10.100.3.100:2181
    
    # get list of items
    $ zookeeper-shell $zk <<< "ls /"
    
    # get list of kafka brokers
    $ zookeeper-shell $zk <<< "ls /brokers/ids"
    
    # get a broker detail
    $ zookeeper-shell $zk <<< "get /brokers/ids/{id}"

Run in locally

First, create a docker network named xsp-network

$ docker network create xsp-network

Now run the docker compose:

$ docker-compose up

Tech Stack

  • AWS
    • EC2 / ASG
    • EBS
    • ENI - Elastic Network Interface
    • Cloudformation
    • AMI
  • Chef
  • Packer
  • Jenkins
You can’t perform that action at this time.