ZNodes & The Data Model

ZooKeeper Namespace (typical production layout):

/
├── kafka
│   ├── brokers
│   │   ├── ids
│   │   │   ├── 0          data: {"host":"broker0","port":9092}
│   │   │   ├── 1          data: {"host":"broker1","port":9092}
│   │   │   └── 2          data: {"host":"broker2","port":9092}
│   │   └── topics
│   │       └── orders
│   │           └── partitions
│   │               ├── 0   data: {"leader":1,"isr":[1,2]}
│   │               └── 1   data: {"leader":2,"isr":[2,0]}
│   └── controller          data: {"brokerid":0}
├── hbase
│   ├── master              data: <master server address>
│   └── rs                  (region servers registered here)
└── myapp
    ├── config
    │   ├── database        data: {"host":"db.prod","port":5432}
    │   └── feature-flags   data: {"dark_mode":true,"beta":false}
    ├── leader              data: "service-instance-3"
    └── locks
        └── payment-processing
            ├── _c_...0001  (lock holder)
            └── _c_...0002  (waiting)

Key rules:
  - Paths are absolute (always start with /)
  - No relative paths (no . or ..)
  - Path components cannot be empty (/a//b is invalid)
  - "zookeeper" is reserved (/zookeeper is system-internal)

What to store in znode data:

✅ Good — small coordination metadata:
  /myapp/leader         → "instance-id-42"              (15 bytes)
  /myapp/config/db      → {"host":"db.prod","port":5432} (35 bytes)
  /myapp/locks/job-1    → ""                            (0 bytes — existence is enough)
  /kafka/brokers/ids/0  → {"host":"10.0.1.1","port":9092,"rack":"us-east-1a"}

❌ Bad — too large or wrong use case:
  /myapp/user-sessions  → <50KB JSON blob>    (too large, too many updates)
  /myapp/logs           → <application logs>  (wrong tool entirely)
  /myapp/images/avatar  → <binary image>      (use S3/object storage)

Why the 1MB limit exists:
  - All znode data lives in memory on EVERY server
  - Every write replicates to ALL servers via Zab
  - Large znodes = more memory, more network, slower snapshots
  - If you need more than 1MB, you're using ZK wrong

// Creating persistent znodes (Java client)
zk.create("/myapp/config", "v1".getBytes(), 
          ZooDefs.Ids.OPEN_ACL_UNSAFE, CreateMode.PERSISTENT);

// This node survives:
//   ✅ Client disconnects
//   ✅ Client session expires
//   ✅ ZooKeeper server restarts
//   ✅ Leader election
//   ❌ Only removed by explicit delete() call

// Use cases:
//   /myapp/config/database    → connection string (rarely changes)
//   /myapp/config/features    → feature flags
//   /kafka/brokers/topics/X   → topic metadata
//   /myapp/locks/             → parent node for lock children

// Service registration with ephemeral nodes
zk.create("/services/payment/instance-1", 
          "10.0.1.5:8080".getBytes(),
          ZooDefs.Ids.OPEN_ACL_UNSAFE, 
          CreateMode.EPHEMERAL);

// What happens when the service crashes:
// 1. TCP connection to ZooKeeper drops
// 2. ZooKeeper waits for session timeout (e.g., 30 seconds)
// 3. No heartbeat received → session expired
// 4. /services/payment/instance-1 is AUTOMATICALLY deleted
// 5. Watches on /services/payment fire (children changed)
// 6. Other services discover instance-1 is gone

// Why no children?
// If ephemeral nodes could have children, what happens to the
// children when the parent is auto-deleted? Cascading deletes
// would be surprising and dangerous. So ZK forbids it.

// The stat structure reveals the owner:
// ephemeralOwner = session ID of the creating client
// (0 for persistent nodes)

// Creating sequential nodes
zk.create("/locks/job-lock-", data, acl, CreateMode.EPHEMERAL_SEQUENTIAL);

// ZooKeeper appends a 10-digit sequence number:
// First call:  /locks/job-lock-0000000001
// Second call: /locks/job-lock-0000000002
// Third call:  /locks/job-lock-0000000003

// The counter is:
//   - Monotonically increasing (never goes backward)
//   - Unique within the parent (/locks/)
//   - Never reused (even after deletion)
//   - 10 digits, zero-padded
//   - Maintained per-parent (different parents have independent counters)

// Four combinations of node types:
// ┌─────────────────────────────────────────────────────────┐
// │ PERSISTENT              → /config/db                    │
// │ PERSISTENT_SEQUENTIAL   → /queue/task-0000000001        │
// │ EPHEMERAL               → /services/payment/instance-1  │
// │ EPHEMERAL_SEQUENTIAL    → /locks/write-0000000003       │
// └─────────────────────────────────────────────────────────┘

// EPHEMERAL_SEQUENTIAL is the most powerful combination:
//   - Auto-deleted on session end (ephemeral)
//   - Globally ordered (sequential)
//   - Enables: fair locks, leader election, ordered queues

// Container nodes — auto-cleanup of empty parents
zk.create("/locks/job-123", null, acl, CreateMode.CONTAINER);

// Children (lock holders) come and go:
// /locks/job-123/lock-0000000001  (created, then deleted)
// /locks/job-123/lock-0000000002  (created, then deleted)
// When last child is deleted → /locks/job-123 is auto-deleted

// Without container nodes, you'd accumulate thousands of empty
// /locks/job-XXX parent nodes that need manual cleanup.

// TTL nodes — time-based expiration
// Requires: zookeeper.extendedTypesEnabled=true in zoo.cfg
zk.create("/leases/client-42", data, acl, 
          CreateMode.PERSISTENT_WITH_TTL, ttl=30000); // 30 seconds

// If not modified and has no children after 30s → auto-deleted
// Useful for:
//   - Lease tokens that should expire
//   - Temporary coordination state
//   - Cleanup of abandoned resources

// Note: TTL is checked lazily (not exactly at expiry time)
// Don't rely on precise timing — it's approximate cleanup

The Stat Structure:
═══════════════════════════════════════════════════════════════════
Field            │ Type   │ Description
═══════════════════════════════════════════════════════════════════
czxid            │ long   │ Transaction ID that created this znode
mzxid            │ long   │ Transaction ID of last modification
pzxid            │ long   │ Transaction ID of last child change
ctime            │ long   │ Time created (ms since epoch)
mtime            │ long   │ Time last modified (ms since epoch)
version          │ int    │ Data version (incremented on setData)
cversion         │ int    │ Child version (incremented on child add/remove)
aversion         │ int    │ ACL version (incremented on setACL)
ephemeralOwner   │ long   │ Session ID of owner (0 if persistent)
dataLength       │ int    │ Length of data in bytes
numChildren      │ int    │ Number of children
═══════════════════════════════════════════════════════════════════

Example stat output (zkCli):
  [zk: localhost:2181(CONNECTED)] stat /myapp/leader
  cZxid = 0x300000012
  ctime = Wed Mar 15 10:23:45 UTC 2024
  mZxid = 0x300000015
  mtime = Wed Mar 15 10:24:01 UTC 2024
  pZxid = 0x300000012
  cversion = 0
  dataVersion = 3
  aclVersion = 0
  ephemeralOwner = 0x18e3a4c2b010001  ← this is an ephemeral node!
  dataLength = 24
  numChildren = 0