Admins who manage the compute cluster with a specific number of nodes and high availability (HA) requests will at some point need a central management tool that, for example, takes care of the naming, grouping, or configurations of the menagerie. Thanks to ZooKeeper [1], which is available under the Apache 2.0 license, not every cluster has to provide a synchronization service itself. The software can be mounted in existing systems – for example, in a Hadoop cluster.

Server and Clients

A ZooKeeper server keeps track of the status of all system nodes. Larger decentralized systems and multiple replicating servers can be used (Figure  1). They then synchronize node status information among themselves, making sure that system tasks run in a fixed order and that no inconsistencies occur.

Figure 1: ZooKeeper takes care of communication between nodes in a cluster.

You can imagine ZooKeeper as a distributed filesystem, because it organizes its information analogously to a filesystem. It is headed by a root directory (/). ZooKeeper nodes, or znodes , are maintained below this; the name is intended to distinguish them from computer nodes.

A znode acts both as a binary file and a directory for more znodes, which serve as subnodes. Like most filesystems, every znode comes with metadata which, in addition to version information, regulates read and write permissions.

Order in the Cluster

You can run a ZooKeeper server in standalone mode or with replication; you can see a sample configuration in the online manual [2] [3]. The second case seems more favorable for distributed filesystems, whereas the first is more suitable for testing and development. A group of replicated servers, which the admin assigns to an application and which use the same configuration, form a quorum (see the "Quorum" box).

If three or more independent servers are used in the ZooKeeper setup, they form a ZooKeeper cluster and select a master. The master processes all writes and informs the other servers about the order of the changes. These servers, in turn, offer redundancy if the master fails and offload read requests and notifications from the clients.

It's important to understand the concept of order, on which ZooKeeper's service quality is based. ZooKeeper defines the order of all operations as they arrive. This information is spread across the ZooKeeper cluster to the other clients – even if the master node fails. Although two clients might not see their environment in an exactly synchronous state at any time, they do observe changes in same order.

Available Clients

The ZooKeeper project maintains two clients that are written in Java and C; additionally, wrappers are available for other programming languages. The client expects ZooKeeper to be running on the same server and automatically connects to port 2181 on localhost. The -server 127.0.0.1:2181 line in Listing 1 can thus typically be omitted.

beiske-retina:~ beiske$ bin/zkCli -server 127.0.0.1:2181
[...]
Welcome to ZooKeeper!
JLine support is enabled
[zk: localhost:2181(CONNECTING) 0]
[zk: localhost:2181(CONNECTED) 0]