Compaction & Operations

SizeTiered Compaction Strategy (STCS):

Tier 0 (small):   [4MB] [4MB] [5MB] [4MB] → compact → [17MB]
Tier 1 (medium):  [15MB] [17MB] [16MB] [18MB] → compact → [66MB]
Tier 2 (large):   [60MB] [66MB] [55MB] [70MB] → compact → [251MB]

Trigger: min_threshold (default 4) SSTables of similar size
         "Similar" = within bucket_low (0.5) to bucket_high (1.5) ratio

Characteristics:
  ✅ Low write amplification (each byte written ~4-5 times total)
  ✅ Good for write-heavy workloads
  ❌ High space amplification (50% overhead for temporary files)
  ❌ High read amplification (many SSTables per partition)
  ❌ Unpredictable compaction timing (large compactions are bursty)

Leveled Compaction Strategy (LCS):

L0: [memtable flushes land here — may overlap]
    [5MB] [5MB] [5MB] [5MB]  → promote to L1 when 4 accumulate

L1: [50MB total, non-overlapping ranges]
    [A-F: 5MB] [G-M: 5MB] [N-S: 5MB] [T-Z: 5MB] ...

L2: [500MB total, non-overlapping ranges]  (10x L1)
    [A-B: 5MB] [C-D: 5MB] [E-F: 5MB] ...

L3: [5GB total, non-overlapping ranges]    (10x L2)
    ...

Compaction: When L(n) exceeds its size limit, one SSTable from L(n)
is merged with all overlapping SSTables in L(n+1).

Key guarantee: Within each level, no two SSTables contain the
same partition. Read amplification = at most (number of levels + 1).

Characteristics:
  ✅ Low read amplification (predictable, bounded)
  ✅ Low space amplification (~10% overhead)
  ✅ Predictable performance (no large bursty compactions)
  ❌ High write amplification (each byte written ~10-30 times)
  ❌ More I/O bandwidth consumed by compaction

TimeWindow Compaction Strategy (TWCS):

Time windows (1-day example):
  ┌─────────────────────────────────────────────────────┐
  │ Jan 15 (closed): [SSTable: all Jan 15 data, 2GB]    │ ← never compacted again
  │ Jan 16 (closed): [SSTable: all Jan 16 data, 2GB]    │ ← never compacted again
  │ Jan 17 (active): [5MB] [5MB] [5MB] [5MB]            │ ← STCS within window
  └─────────────────────────────────────────────────────┘

When TTL expires (e.g., 30 days):
  Jan 15 SSTable: all data expired → DROP entire file (instant!)
  No tombstones needed, no compaction needed.

Characteristics:
  ✅ Minimal write amplification (compact once per window)
  ✅ Efficient TTL expiration (drop whole SSTables, no tombstones)
  ✅ Predictable disk usage (old windows drop off)
  ❌ Only works for time-series / append-only data
  ❌ Out-of-order writes break the model (data in wrong window)
  ❌ Updates/deletes to old windows cause problems

-- TWCS configuration for time-series with 30-day TTL
CREATE TABLE sensor_data (
  sensor_id TEXT,
  day DATE,
  event_time TIMESTAMP,
  value DOUBLE,
  PRIMARY KEY ((sensor_id, day), event_time)
) WITH compaction = {
  'class': 'TimeWindowCompactionStrategy',
  'compaction_window_unit': 'DAYS',
  'compaction_window_size': 1
}
AND default_time_to_live = 2592000;  -- 30 days in seconds

-- Key rule: TTL should be a multiple of the window size
-- Window = 1 day, TTL = 30 days → each window SSTable lives 30 days then drops

Tombstone Lifecycle:

1. DELETE FROM users WHERE user_id = 'alice';
   → Writes tombstone: {user_id: 'alice', deleted_at: 2024-01-15T10:00:00}

2. Tombstone propagates via:
   - Normal replication (write to all replicas)
   - Read repair (if stale replica is read)
   - Anti-entropy repair (nodetool repair)

3. After gc_grace_seconds (default: 864000 = 10 days):
   - Tombstone is eligible for garbage collection
   - Next compaction purges the tombstone AND the original data

4. If repair hasn't run within gc_grace_seconds:
   ⚠️ DANGER: Tombstone is purged, but a stale replica still has
   the original data. Read repair will "resurrect" the deleted data!

Types of tombstones:
  - Cell tombstone: single column deleted
  - Row tombstone: entire row deleted (DELETE FROM t WHERE pk=x AND ck=y)
  - Range tombstone: range of clustering keys deleted
  - Partition tombstone: entire partition deleted (DELETE FROM t WHERE pk=x)
  - TTL tombstone: auto-generated when TTL expires

# ═══════════════════════════════════════════════════════════
# CLUSTER STATUS & HEALTH
# ═══════════════════════════════════════════════════════════

# Show cluster topology and node status
nodetool status
# UN = Up/Normal, DN = Down/Normal, UJ = Up/Joining, UL = Up/Leaving

# Show node info (heap, load, uptime, key cache hit rate)
nodetool info

# Show gossip state for all nodes
nodetool gossipinfo

# ═══════════════════════════════════════════════════════════
# TABLE STATISTICS & DIAGNOSTICS
# ═══════════════════════════════════════════════════════════

# Table-level stats (SSTable count, read/write latency, partition size)
nodetool tablestats my_keyspace.my_table

# Histogram of partition sizes and cell counts
nodetool tablehistograms my_keyspace.my_table

# Show compaction activity
nodetool compactionstats

# Show thread pool stats (pending/blocked tasks)
nodetool tpstats

# ═══════════════════════════════════════════════════════════
# MAINTENANCE OPERATIONS
# ═══════════════════════════════════════════════════════════

# Flush memtables to SSTables
nodetool flush my_keyspace

# Trigger compaction manually
nodetool compact my_keyspace my_table

# Run anti-entropy repair
nodetool repair my_keyspace

# Take a snapshot (backup)
nodetool snapshot my_keyspace -t my_snapshot_name

# Clear a snapshot
nodetool clearsnapshot -t my_snapshot_name

# ═══════════════════════════════════════════════════════════
# NODE LIFECYCLE
# ═══════════════════════════════════════════════════════════

# Gracefully remove a node (streams data to other nodes)
nodetool decommission

# Remove a dead node from the cluster
nodetool removenode <host_id>

# Drain (flush + stop accepting writes, for graceful shutdown)
nodetool drain