Redis Replication & High Availability

Single Redis instance:
  Write throughput:  100,000 ops/sec     ✅
  Read throughput:   100,000 ops/sec     ✅
  Availability:      ONE node failure = TOTAL outage  ❌
  Data durability:   Lost if node dies (unless AOF/RDB)  ❌
  Recovery time:     Minutes (restart + reload from disk)  ❌

Redis with 2 replicas:
  Write throughput:  100,000 ops/sec (primary only)  ✅
  Read throughput:   300,000 ops/sec (primary + 2 replicas)  ✅
  Availability:      Survives 1-2 node failures  ✅
  Data durability:   Data exists on 3 nodes  ✅
  Recovery time:     Seconds (promote replica)  ✅

# On the replica node
replicaof 192.168.1.100 6379

# Authentication (if primary requires a password)
masterauth your-secret-password

# Allow reads on the replica (default: yes)
replica-read-only yes

# Replication backlog size (for partial resync)
# Larger = survives longer disconnects without full resync
repl-backlog-size 64mb

# Diskless replication (primary sends RDB over socket, no disk I/O)
repl-diskless-sync yes
repl-diskless-sync-delay 5

Direct replication (primary sends to all):
  Primary ──→ Replica 1
         ──→ Replica 2
         ──→ Replica 3
  Primary handles 3 replication streams (high CPU/bandwidth)

Chained replication (reduces primary load):
  Primary ──→ Replica 1 ──→ Replica 3
                         ──→ Replica 4
         ──→ Replica 2 ──→ Replica 5
  Primary handles only 2 streams
  Trade-off: Replica 3-5 have higher replication lag

# On the primary — shows connected replicas and their lag
redis-cli INFO replication

# Key fields:
#   connected_slaves: 2
#   slave0: ip=10.0.0.2,port=6379,state=online,offset=1234567,lag=0
#   slave1: ip=10.0.0.3,port=6379,state=online,offset=1234560,lag=1
#
# "lag" = seconds since last ACK from replica
# "offset" difference = bytes of replication stream behind

# Alert if lag > 1 second for latency-sensitive reads
# Alert if offset difference > 10MB for data safety

# Each Sentinel node runs this configuration
port 26379

# Monitor the primary named "mymaster" at 192.168.1.100:6379
# Quorum of 2 — at least 2 Sentinels must agree primary is down
sentinel monitor mymaster 192.168.1.100 6379 2

# Primary is considered down after 5 seconds of no response
sentinel down-after-milliseconds mymaster 5000

# Only 1 replica syncs from new primary at a time during failover
# (prevents all replicas from being unavailable simultaneously)
sentinel parallel-syncs mymaster 1

# Failover times out after 60 seconds
sentinel failover-timeout mymaster 60000

# Authentication
sentinel auth-pass mymaster your-secret-password

// Clients connect to Sentinel, NOT directly to Redis
const redis = new Redis({
  sentinels: [
    { host: "sentinel-1", port: 26379 },
    { host: "sentinel-2", port: 26379 },
    { host: "sentinel-3", port: 26379 },
  ],
  name: "mymaster",  // The monitored master name
});

// ioredis automatically:
// 1. Asks Sentinel for the current primary address
// 2. Connects to the primary
// 3. Subscribes to failover events
// 4. Reconnects to the new primary after failover

Total hash slots: 16,384

Node A (primary): slots 0 – 5460      (5,461 slots)
Node B (primary): slots 5461 – 10922  (5,462 slots)
Node C (primary): slots 10923 – 16383 (5,461 slots)

Key routing examples:
  SET user:1000 "Alice"
  → CRC16("user:1000") = 7142
  → 7142 % 16384 = 7142
  → Slot 7142 → owned by Node B

  SET session:abc "data"
  → CRC16("session:abc") = 13104
  → 13104 % 16384 = 13104
  → Slot 13104 → owned by Node C

Each node only stores keys that map to its assigned slots.

Without hash tags (keys land on different nodes):
  SET user:1000:profile "..."   → CRC16("user:1000:profile") → Slot 3921
  SET user:1000:settings "..."  → CRC16("user:1000:settings") → Slot 11842
  SET user:1000:cart "..."      → CRC16("user:1000:cart") → Slot 7291
  ❌ MGET user:1000:profile user:1000:settings → CROSSSLOT error

With hash tags (all hash to the same substring):
  SET {user:1000}:profile "..."   → CRC16("user:1000") → Slot 7142
  SET {user:1000}:settings "..."  → CRC16("user:1000") → Slot 7142
  SET {user:1000}:cart "..."      → CRC16("user:1000") → Slot 7142
  ✅ MGET {user:1000}:profile {user:1000}:settings → works!

Rule: Redis hashes only the content between the first { and first }.
Use this to co-locate all data for a single entity on one node.

Shard 1:  Primary A (slots 0-5460)      ←→  Replica A'
Shard 2:  Primary B (slots 5461-10922)  ←→  Replica B'
Shard 3:  Primary C (slots 10923-16383) ←→  Replica C'

Total: 6 nodes (3 primaries + 3 replicas)
  - Write capacity: 3x single node (writes distributed across primaries)
  - Read capacity: 6x single node (if reading from replicas)
  - Fault tolerance: survives 1 primary failure per shard

Minimum recommended: 6 nodes (3 primaries + 3 replicas)
Production: often 6-12+ nodes depending on data size and throughput

MOVED — permanent redirection (slot ownership changed):
  Client → Node A: GET user:1000
  Node A → Client: MOVED 7142 10.0.0.2:6379
  Meaning: "Slot 7142 permanently lives on 10.0.0.2. Update your routing table."
  Client updates its slot map and retries on Node B.

ASK — temporary redirection (slot is being migrated):
  Client → Node A: GET user:1000
  Node A → Client: ASK 7142 10.0.0.2:6379
  Meaning: "Slot 7142 is being moved to 10.0.0.2. Try there THIS TIME ONLY."
  Client sends ASKING + GET to Node B, but doesn't update its slot map.

Smart clients (ioredis, Jedis, Lettuce):
  - Cache the full slot map on startup (CLUSTER SLOTS command)
  - Route commands directly to the correct node
  - Handle MOVED by updating the cached map
  - Handle ASK with a one-time redirect

# Create a 6-node cluster (3 primaries + 3 replicas)
redis-cli --cluster create   10.0.0.1:6379 10.0.0.2:6379 10.0.0.3:6379   10.0.0.4:6379 10.0.0.5:6379 10.0.0.6:6379   --cluster-replicas 1

# Add a new node to an existing cluster
redis-cli --cluster add-node 10.0.0.7:6379 10.0.0.1:6379

# Reshard: move 1000 slots to the new node
redis-cli --cluster reshard 10.0.0.1:6379   --cluster-from all   --cluster-to <new-node-id>   --cluster-slots 1000   --cluster-yes

# Rebalance slots evenly across all primaries
redis-cli --cluster rebalance 10.0.0.1:6379