Manipulate zNodes in Apache ZooKeeper

Until now we've seen how to create zNodes. But creation is not the single thing that Apache ZooKeeper does.

Looking for a better data engineering position and skills?

You have been working as a data engineer but feel stuck? You don't have any new challenges and are still writing the same jobs all over again? You have now different options. You can try to look for a new job, now or later, or learn from the others! "Become a Better Data Engineer" initiative is one of these places where you can find online learning resources where the theory meets the practice. They will help you prepare maybe for the next job, or at least, improve your current skillset without looking for something else.

👉 I'm interested in improving my data engineering skillset

See you there, Bartosz

In this article we cover 3 other available operations: edit, removal and bulk. Each of them is presented in separated part inside which we can find quick theoretical presentation and JUnit test cases.

Edit in Apache ZooKeeper

Edit of created zNodes groups permissions and date changes. These operations can be made with setACL(String, List<ACL>, int) and setData(String, byte[], int ). As you can see, the last parameter of these methods is an integer. It represents the version of zNode which we want to change. This parameter is a little bit confusing. We could think that we are allowed to modify old versions of zNodes. However, it's not the case, according to the ZooKeeper Javadoc and one of further presented test cases. To avoid memorizing current version of given node, we can pass -1 which matches any node's versions.

The version value is incremented on every modification. It can be observed through class, storing information about given zNode. It defines 2 types of versions: one representing data changes and another one representing ACL changes. In the case of zNode having some children, the version of parent is not impacted by changes made on children nodes.

We can see now corresponding test cases:

public void should_correctly_update_znode_content() throws KeeperException, InterruptedException {
  zooKeeper.create(ZNODE_CONTENT_UPDATE, "Home directory".getBytes(),   
    ZooDefs.Ids.OPEN_ACL_UNSAFE, CreateMode.PERSISTENT);
  Stat initialStat = zooKeeper.exists(ZNODE_CONTENT_UPDATE, false);
  zooKeeper.setData(ZNODE_CONTENT_UPDATE, "New home directory".getBytes(), ALL_VERSIONS);

  Stat newStat = zooKeeper.exists(ZNODE_CONTENT_UPDATE, false);

  assertChanges(initialStat, newStat);
  // Children and ACL weren't changed in this test case, so they should be equal

@Test(expected = KeeperException.BadVersionException.class)
public void should_fail_on_updating_only_old_version() throws KeeperException, InterruptedException {
  zooKeeper.create(ZNOODE_OLD_VERSION, "1".getBytes(), 
    ZooDefs.Ids.OPEN_ACL_UNSAFE, CreateMode.EPHEMERAL);
  Stat initialStat = zooKeeper.exists(ZNOODE_OLD_VERSION, false);
  // This update increases zNode version
  zooKeeper.setData(ZNOODE_OLD_VERSION, "3".getBytes(), ALL_VERSIONS);
  // And this one should fail since we try to update a version which is not current
  zooKeeper.setData(ZNOODE_OLD_VERSION, "2".getBytes(), initialStat.getVersion());

public void should_correctly_update_children_node() throws KeeperException, InterruptedException {
  // create parent
  zooKeeper.create(ZNODE_CHILDREN_UPDATE, "Parent".getBytes(), 
    ZooDefs.Ids.OPEN_ACL_UNSAFE, CreateMode.PERSISTENT);
  Stat initialParentStat = zooKeeper.exists(ZNODE_CHILDREN_UPDATE, false);
  // create and update child
  zooKeeper.create(ZNODE_CHILDREN_UPDATE_CHILD, "Child".getBytes(), 
    ZooDefs.Ids.OPEN_ACL_UNSAFE, CreateMode.PERSISTENT);
  Stat initialChildStat = zooKeeper.exists(ZNODE_CHILDREN_UPDATE_CHILD, false);
  zooKeeper.setData(ZNODE_CHILDREN_UPDATE_CHILD, "New child".getBytes(), ALL_VERSIONS);

  // get new states after changes
  Stat newParentStat = zooKeeper.exists(ZNODE_CHILDREN_UPDATE, false);
  Stat newChildStat = zooKeeper.exists(ZNODE_CHILDREN_UPDATE_CHILD, false);

  assertChanges(initialChildStat, newChildStat);
  // Parent was not changed, so the version should be still the same
  // Unlike in previous test, children were changed, so the children version should be incremented
  // Children and ACL weren't changed in this test case, so they should be equal

public void should_correctly_detect_acl_changes() throws KeeperException, InterruptedException {
  zooKeeper.create(ZNODE_ACL_UPDATE, "ACL".getBytes(), 
    ZooDefs.Ids.OPEN_ACL_UNSAFE, CreateMode.PERSISTENT);
  Stat initialStat = zooKeeper.exists(ZNODE_ACL_UPDATE, false);

  Stat newStat = zooKeeper.exists(ZNODE_ACL_UPDATE, false);

  // This time ACL was modified, so it should be reflected on this test


private static void assertChanges(Stat oldStat, Stat newStat) {
  // According to doc, version is bumped
  // New data should be defined too
  // Modification stats (last time and last id) should be different
  // But creation date should remain the same

Removal in Apache ZooKeeper

zNodes removal is quite easy operation. It needs a single call to delete(String, int) ZooKeeper's method. Here also, the last parameter represents version to delete. As in the case of edition, we can't remove other version than the current one.

The removal is a little bit complicated when zNode has children nodes. The delete method is not recursive, so we should first delete all children and only at the end their parent. However, there is simpler solution to do it. Instead of using delete method, we can pass by ZKUtil.deleteRecursive(ZooKeeper, String) utilitary method where the second parameter represents parent directory to delete.

We can see these assumptions in following test cases:

public void should_correctly_remove_znode_by_specific_version() throws KeeperException, InterruptedException {
  zooKeeper.create(HOME_TO_DELETE, "Home directory".getBytes(), 
    ZooDefs.Ids.OPEN_ACL_UNSAFE, CreateMode.PERSISTENT);
  int version = zooKeeper.exists(HOME_TO_DELETE, false).getVersion();

  zooKeeper.delete(HOME_TO_DELETE, version);

  assertThat(zooKeeper.exists(HOME_TO_DELETE, false)).isNull();

@Test(expected = KeeperException.BadVersionException.class)
public void should_correctly_remove_znode_by_specific_version_which_is_not_current() throws KeeperException, InterruptedException {
  String path = "/home_multi_versions";
  zooKeeper.create(path, "Home directory".getBytes(), 
    ZooDefs.Ids.OPEN_ACL_UNSAFE, CreateMode.EPHEMERAL);
  int version = zooKeeper.exists(path, false).getVersion();
  zooKeeper.setData(path, new byte[0], ALL_VERSIONS);

  zooKeeper.delete(path, version);

@Test(expected = KeeperException.NotEmptyException.class)
public void should_fail_on_removing_znode_with_children_when_children_are_not_deleted_first() throws KeeperException, InterruptedException {


public void should_correctly_remove_znode_when_children_are_removed_before() throws KeeperException, InterruptedException {

  assertThat(zooKeeper.exists(HOME_WITH_CHILDREN, false)).isNull();
  assertThat(zooKeeper.exists(HOME_WITH_CHILDREN_1, false)).isNull();
  assertThat(zooKeeper.exists(HOME_WITH_CHILDREN_2, false)).isNull();

public void should_delete_recursively_with_zkutils() throws KeeperException, InterruptedException {

  ZKUtil.deleteRecursive(zooKeeper, HOME_WITH_CHILDREN);

  assertThat(zooKeeper.exists(HOME_WITH_CHILDREN, false)).isNull();
  assertThat(zooKeeper.exists(HOME_WITH_CHILDREN_1, false)).isNull();
  assertThat(zooKeeper.exists(HOME_WITH_CHILDREN_2, false)).isNull();

private void createTree() throws KeeperException, InterruptedException {
  zooKeeper.create(HOME_WITH_CHILDREN, "Root".getBytes(), 
    ZooDefs.Ids.OPEN_ACL_UNSAFE, CreateMode.PERSISTENT);
  zooKeeper.create(HOME_WITH_CHILDREN_1, "1".getBytes(), 
    ZooDefs.Ids.OPEN_ACL_UNSAFE, CreateMode.PERSISTENT);
  zooKeeper.create(HOME_WITH_CHILDREN_2, "2".getBytes(), 
    ZooDefs.Ids.OPEN_ACL_UNSAFE, CreateMode.PERSISTENT);

Bulk operations in Apache ZooKeeper

ZooKeeper class has also some methods to make multiple operations in a single call. This specific method is multi(Iterable<Op>). It uses an abstraction of operation represented by org.apache.zookeeper.Op class. This class contains some of factory methods which can be used to create Op instances corresponding to executed operation: create for zNode creation, delete for removal, setData for data definition and check for zNode check operation.

Bulk operations is all-or-nothing ones, ie. either all operations succeed or all fail. Operations results are represented by classes corresponding to operations categories: CreateResult for create, SetDataResult for data definiot, DeleteResult for delete and CheckResult for check. A special kind of object is ErrorResult which represents operation failure. The results are returned by multi() method.

Below you can find two cases showing bulk operations in Apache ZooKeeper:

public void should_correctly_execute_bulk_znodes_creation() throws KeeperException, InterruptedException {
  zooKeeper.create(NODE_2, "Content".getBytes(), 
    ZooDefs.Ids.OPEN_ACL_UNSAFE, CreateMode.EPHEMERAL);
  Op createNode1 = Op.create(NODE_1, "content".getBytes(), 
    ZooDefs.Ids.OPEN_ACL_UNSAFE, CreateMode.EPHEMERAL);
  Op removeNode2 = Op.delete(NODE_2, ALL_VERSIONS);

    * {@code OpResult} is only an abstract class. It contains 
    * specific implementations for creation, delete,
    * data setting or version checking operations.
  List<OpResult> results = 
    zooKeeper.multi(Lists.newArrayList(createNode1, removeNode2));

  assertThat(results).extracting("type").containsOnly(ZooDefs.OpCode.create, ZooDefs.OpCode.delete);
  assertThat( -> result.getClass().getCanonicalName()).collect(Collectors.toList())).containsOnly(
    "org.apache.zookeeper.OpResult.CreateResult", "org.apache.zookeeper.OpResult.DeleteResult"
  assertThat(zooKeeper.exists(NODE_1, false)).isNotNull();
  assertThat(zooKeeper.exists(NODE_2, false)).isNull();

public void should_execute_only_some_valid_operations() throws KeeperException, InterruptedException {
  Op createNode1 = Op.create(NODE_1, "content".getBytes(), 
    ZooDefs.Ids.OPEN_ACL_UNSAFE, CreateMode.EPHEMERAL);
  Op createNode2 = Op.create(NODE_1, "content".getBytes(), 
    ZooDefs.Ids.OPEN_ACL_UNSAFE, CreateMode.EPHEMERAL);
  Op removeNode3 = Op.delete(NODE_2, ALL_VERSIONS);

  try {
    zooKeeper.multi(Lists.newArrayList(createNode1, createNode2, removeNode3));
    fail("Should fail on executing bulk operation with 1 failure");
  } catch (KeeperException.NodeExistsException exception) {
    List<OpResult> results = exception.getResults();

    assertThat( -> result.getClass().getCanonicalName()) 

  assertThat(zooKeeper.exists(NODE_1, false)).isNull();

Edit, removal and bulk operations are 3 topics presented in this article oriented more in practice. Its first part shows two methods useful in zNode modification. The modification can apply on stored data or associated permission. In the part about delete we saw that delete is easy for file-looking zNode. In the case of parent-directory role, it's better to use utilitary deleteRecursive method. The last part illustrated the use of bulk operations through multi() method.

If you liked it, you should read:

📚 Newsletter Get new posts, recommended reading and other exclusive information every week. SPAM free - no 3rd party ads, only the information about waitingforcode!