What is ZooKeeper?

Apache ZooKeeper is an open-source service that lets distributed applications coordinate with each other through znodes. As a result, you can read, write and observe updates to data.

Too much? 😵 Let's break it down by understanding what are distributed systems and znodes.

What are distributed systems?

Distributed systems have several software components that run independently and at the same time across several physical machines (or servers).

While distributed systems are great for processing large amounts of data, they come with several challenges. One such challenge is when a server or a process fails. While other servers will continue running, they must detect which server or process failed and then probably take over the faulty process. When this happens, messages can get lost and communication can fail.

That's where ZooKeeper comes in.

How does ZooKeeper work?

Think of ZooKeeper as a central repository for distributed applications to read and store data.

ZooKeeper runs on an odd number of servers, which is called the ZooKeeper cluster or ensemble. All servers in an ensemble know about all the other servers.

Whenever a change happens, it's considered successful if it's written to a quorum (at least half) of the servers in the ensemble.

ZooKeeper architecture. Image courtesy: DevStacks

Since the ensemble requires a majority, it always contains an odd number of servers.

If the ensemble has 5 servers, then ZooKeeper will consider a write (storing data) successful if it's written on at least 3 of those 5 servers. If 2 servers fail, the rest of the ensemble continues functioning.

When the 2 servers come back online, they're synchronized with the ensemble and can resume their functions. This helps reduce communication failure.

What is a znode?

znode is the basic building block of the ZooKeeper file system. It stores data and has a storage limit of 1 MB.

Applications can synchronize their tasks across an ensemble by updating their status in a znode, which then informs the rest of the cluster regarding the status change of a specific node.

This makes it easier to manage tasks across a distributed servers.

Why is it called ZooKeeper?

According to the creators of ZooKeeper (Benjamin Reed and Flavio Junqueira), here's the story behind its name. Distributed systems are a zoo—chaotic and hard to manage. ZooKeeper is meant to keep them under control.🐻🦁

Yet another Yahoo! article compared distributed systems management to herding cats. (ZooKeeper is a much better name than CatHerder though, isn't it? 😺)

Who uses ZooKeeper?

ZooKeeper was initially developed at Yahoo to fix the bugs that cropped up while deploying big data applications.

Several organizations worldwide such as RackSpace, Facebook, Yahoo!, Twitter and eBay use ZooKeeper.

Think we're missing something? 🧐 Help us update this article by sending us your suggestions here. 🙏

See also

Articles you might be interested in

  1. Explaining Apache ZooKeeper
  2. ZooKeeper by Benjamin Reed and Flavio Junqueira
  3. How-to: Use Apache ZooKeeper to build distributed apps (and why)