zNode in Apache ZooKeeper

As already told, zNodes are a key part in Apache ZooKeeper. They store information shared among different servers directly (as binary data) or indirectly (as parent directories).

Data Engineering Design Patterns

Looking for a book that defines and solves most common data engineering problems? I'm currently writing one on that topic and the first chapters are already available in πŸ‘‰ Early Release on the O'Reilly platform

I also help solve your data engineering problems πŸ‘‰ contact@waitingforcode.com πŸ“©

In this article we'll describe zNodes more in details. The first part is about theoretical aspects. They will be presented concretely in the next part as unit tests.

zNodes in Apache ZooKeeper

zNodes aren't complicated to understand. As already told, they can act either as files or directories. And because they're working in distributed file system, it means that they can be replicated between servers. But there are not only one point describing them. Below list should help to describe zNodes better:

Working with zNodes in Apache ZooKeeper Java API

After discovering a little theory about zNodes, it's time to play with it through Java API. Strictly speaking, ZooKeeper Java API doesn't have a class to represent zNode. Instead, it has some of classes to get zNodes information. The first one is org.apache.zookeeper.data.Stat which, as the name tells, represents data structure holding information about zNode (creation, change time, versions, data length or children count). It can be get through a call of ZooKeeper.exists(String, boolean) method which can return null if given zNode doesn't exist.

To be able to get zNode stats, we must create it before. Intuitively, the creation can be done through ZooKeeper.create(String, byte[], List<ACL>, CreateMode) method which takes in signature: path, content, permissions and type (ephemeral, persistent, ephemeral-sequential or persistent-sequential). Once created, it returns a path of created zNode - particularly useful when working with sequential zNodes.

As repeated previously, zNodes can hold other zNodes (acts like directory), binary data (acts like files) or both. To get zNode content we can also use special Java method: ZooKeeper.getData(String, boolean, Stat). To find children of given zNode, we can use another ZooKeeper's method: getChildren(). It returns only paths of associated children.

Example of zNode manipulation in Apache ZooKeeper Java API

After the second part describing a little how to use zNode operations in Apache ZooKeeper, we can pass to JUnit tests illustrating the practical use of presented theory:

@Test
public void should_correctly_create_persistent_new_znode() throws KeeperException, InterruptedException {
   /**
    * Created node is not ephemeral. It means that without clean() method call, the next run of this test
    * would fail. Persistent zNodes, unlike ephemeral, are stored in persistent way, event after ZooKeeper
    * server restart.
    */
  String path = 
    zooKeeper.create("/home", "Home directory".getBytes(), ZooDefs.Ids.OPEN_ACL_UNSAFE, CreateMode.PERSISTENT);

  assertThat(path).isEqualTo("/home");
  Stat nodeStat = zooKeeper.exists("/home", false);
  assertThat(nodeStat).isNotNull();
  assertThat(nodeStat.getEphemeralOwner()).isEqualTo(0L);
}

@Test
public void should_detect_node_as_not_existent() throws KeeperException, InterruptedException {
  assertThat(zooKeeper.exists("/fake_node", false)).isNull();
}

@Test
public void should_correctly_create_ephemeral_node() throws KeeperException, InterruptedException, IOException {
   /**
    * Ephemeral zNode doesn't need manual clean up in clean() method. It's because it's automatically
    * deleted by ZooKeeper at the end of session.
    */
  String path = zooKeeper.create("/home_ephemeral", "Home ephemeral directory".getBytes(),
    ZooDefs.Ids.OPEN_ACL_UNSAFE, CreateMode.EPHEMERAL);

  assertThat(path).isEqualTo("/home_ephemeral");
  Stat nodeStat = zooKeeper.exists("/home_ephemeral", false);
  assertThat(nodeStat).isNotNull();
  assertThat(nodeStat.getEphemeralOwner()).isNotEqualTo(0L);
  zooKeeper.close();

   /**
    * Ephemeral zNodes live only within session which created them. After closing this session, thing what
    * it's done one line before, these zNodes aren't anymore reachable.
    */
  zooKeeper = new ZooKeeper("127.0.0.1:2181", CONNECTION_TIMEOUT, null);
  while (zooKeeper.getState() == ZooKeeper.States.CONNECTING) {
  }
  assertThat(zooKeeper.exists("/home_ephemeral", false)).isNull();
}

@Test
public void should_correctly_create_children() throws KeeperException, InterruptedException {
  zooKeeper.create("/home_with_children", "Root".getBytes(), 
    ZooDefs.Ids.OPEN_ACL_UNSAFE, CreateMode.PERSISTENT);
  zooKeeper.create("/home_with_children/1", "1".getBytes(), 
    ZooDefs.Ids.OPEN_ACL_UNSAFE, CreateMode.PERSISTENT);
  zooKeeper.create("/home_with_children/2", "2".getBytes(), 
    ZooDefs.Ids.OPEN_ACL_UNSAFE, CreateMode.PERSISTENT);

  List<String> children = zooKeeper.getChildren("/home_with_children", false);

  assertThat(children).hasSize(2);
  assertThat(children).containsOnly("1", "2");
}

@Test(expected = KeeperException.NoChildrenForEphemeralsException.class)
public void should_fail_on_creating_children_on_ephemeral_node() throws KeeperException, InterruptedException {
  zooKeeper.create("/home_ephemeral_with_children", "Root".getBytes(), 
    ZooDefs.Ids.OPEN_ACL_UNSAFE, CreateMode.EPHEMERAL);
  zooKeeper.create("/home_ephemeral_with_children/1", "1".getBytes(), 
    ZooDefs.Ids.OPEN_ACL_UNSAFE, CreateMode.EPHEMERAL);
}

@Test(expected = KeeperException.NodeExistsException.class)
public void should_fail_on_creating_the_same_znode_twice() throws KeeperException, InterruptedException {
  zooKeeper.create("/duplicated_node", "duplicated content".getBytes(), 
    ZooDefs.Ids.OPEN_ACL_UNSAFE, CreateMode.PERSISTENT);
  zooKeeper.create("/duplicated_node", "duplicated content".getBytes(), 
    ZooDefs.Ids.OPEN_ACL_UNSAFE, CreateMode.PERSISTENT);
}

@Test
public void should_get_correct_data_of_created_znode() throws KeeperException, InterruptedException, IOException {
  String path = "/home_data_ephemeral";
  zooKeeper.create(path, "1".getBytes(), 
    ZooDefs.Ids.OPEN_ACL_UNSAFE, CreateMode.EPHEMERAL);

  Stat stat = new Stat();
  byte[] data = zooKeeper.getData(path, false, stat);

  assertThat(new String(data)).isEqualTo("1");
  assertThat(stat.getDataLength()).isEqualTo(1);
}

@Test
public void should_get_correct_data_of_created_directory_znode() throws KeeperException, InterruptedException, IOException {
  zooKeeper.create(MIXED_PARENT_FILE_NODE, "1".getBytes(), 
    ZooDefs.Ids.OPEN_ACL_UNSAFE, CreateMode.PERSISTENT);
  String childrenPath =
    zooKeeper.create(MIXED_PARENT_FILE_NODE+"/1", "2".getBytes(), ZooDefs.Ids.OPEN_ACL_UNSAFE, CreateMode.EPHEMERAL);

  Stat stat = new Stat();
  byte[] data = zooKeeper.getData(MIXED_PARENT_FILE_NODE, false, stat);
  List<String> children = zooKeeper.getChildren(MIXED_PARENT_FILE_NODE, false);
  Stat childStat = zooKeeper.exists(childrenPath, false);


  assertThat(new String(data)).isEqualTo("1");
  assertThat(stat).isNotNull();
  assertThat(stat.getDataLength()).isEqualTo(1);
  assertThat(children).hasSize(1);
  assertThat(children.get(0)).isEqualTo("1");
  assertThat(childStat).isNotNull();
  assertThat(childStat.getDataLength()).isEqualTo(1);
}

@Test(expected = KeeperException.ConnectionLossException.class)
public void should_fail_on_saving_znode_bigger_than_1_mb() throws KeeperException, InterruptedException {
  zooKeeper.create("/home_big_file", makeBigFile(), ZooDefs.Ids.OPEN_ACL_UNSAFE, CreateMode.EPHEMERAL);
}

The article shows how to work with zNodes. The first part describes theory hidden behind them. We can learn from it that zNodes can be of 1 from 4 categories (ephemeral, persistent, ephemeral-sequential, persistent-sequential). We can also see that they're not represented as a kind of zNode objects, but more like stored content or historical stats. The second part describes main Java API methods used in the last part to show zNode features.


If you liked it, you should read:

πŸ“š Newsletter Get new posts, recommended reading and other exclusive information every week. SPAM free - no 3rd party ads, only the information about waitingforcode!