zookeeper

What is it

ZooKeeper is a centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. All of these kinds of services are used in some form or another by distributed applications. (from http://zookeeper.apache.org/)

It is written in Java and from the Apache foundation. It is used in Apache Hadoop context.

Functionality

• enforce an order of procedures
• works replicated, if one node fails another takes action
  • for simplicity we can start with only one instance
• can be used for discover services (like new docear webservice instances)
• cab be used to shard resources or data (which docear instance posesses which mind map)
• provides a namespace like unix file paths
• works in memory (fast) but uses hard disk for snapshots (less data loss)
• Barriers
• Locking
• Producer-Consumer Queues
• Leader election

Limitations

• ZooKeeper instances must now each other
• client must now ZooKeeper instances (can be simplified with DNS)
• fast on read requests, throughput scales with number of servers (only one server reads)
• slower on write requests, throughput decreases with number servers (everyone must write)

Used by projects/companies

• Apache Hadoop MapReduce
• Apache Solr
• Yahoo!
• Rackspace
• Zynga
• LinkedIn
• Facebook

Links

• Home page
• Wiki
• Documentation for 3.4.5
  • Api Doc
  • Receipes
• Wikipedia entry

Internals

• nodes are called znodes
• znodes can have data (default 1MB) and subnodes, so there are a mix of folders and files

Possible Libraries

• https://github.com/Netflix/curator/wiki