Replication & Caching

Incoming: 100,000 reads/sec

Layer 1 — CDN (images, CSS, JS, static HTML):
  Absorbs: 60% of all requests
  Remaining: 40,000 reads/sec → next layer

Layer 2 — Application Cache (Redis):
  Absorbs: 90% of remaining API requests
  Remaining: 4,000 reads/sec → next layer

Layer 3 — Read Replicas (3 replicas):
  Absorbs: 75% of remaining DB queries
  Remaining: 1,000 reads/sec → primary DB

Layer 4 — Primary Database:
  Handles: 1,000 reads/sec (+ all writes)
  Original load: 100,000 → Actual load: 1,000
  → 99% reduction in primary DB load

                    
                    ┌──────────────┐
  Writes ──────────→│  Primary DB  │
                    └──────┬───────┘
                           │ WAL stream
              ┌────────────┼────────────┐
              ▼            ▼            ▼
        ┌──────────┐ ┌──────────┐ ┌──────────┐
        │ Replica 1│ │ Replica 2│ │ Replica 3│
        └──────────┘ └──────────┘ └──────────┘
              ▲            ▲            ▲
              └────────────┼────────────┘
                           │
  Reads ───────────────────┘ (load balanced)

Capacity:
  Primary alone: 5,000 reads/sec
  + 3 replicas:  20,000 reads/sec (4x improvement)
  + 10 replicas: 55,000 reads/sec (11x improvement)

Problem:
  User updates profile → writes to Primary
  User refreshes page → reads from Replica (stale!)
  User sees old name → "My update didn't save!"

Solutions:
  1. Read-your-own-writes:
     After a write, route that user's reads to the Primary
     for the next N seconds. Other users read from replicas.

  2. Causal consistency:
     Track the write's WAL position. Route reads to a replica
     that has caught up to at least that position.

  3. Sticky sessions:
     Route a user to the same replica consistently.
     If they wrote to Primary, route reads to Primary briefly.

Request: GET /api/products/42

L1 — Browser Cache (client-side):
  Browser checks: "Do I have this cached?"
  Cache-Control: max-age=60 → cached for 60 seconds
  HIT → return instantly (0ms, no network request)
  MISS → proceed to L2

L2 — CDN Edge Cache (Cloudflare, CloudFront):
  CDN edge in user's region checks its cache
  HIT → return from edge (10-30ms)
  MISS → proceed to L3

L3 — Application Cache (Redis / Memcached):
  App server checks Redis: GET product:42
  HIT → return from Redis (1-5ms)
  MISS → proceed to L4

L4 — Database (PostgreSQL / MySQL):
  Query: SELECT * FROM products WHERE id = 42
  Return result (20-100ms)
  Store in Redis for next request (cache-aside)

Typical hit rates:
  L1 (browser):  30-50% of requests never leave the browser
  L2 (CDN):      60-80% of remaining requests served from edge
  L3 (Redis):    90-95% of remaining requests served from cache
  L4 (database): handles only 1-5% of original traffic

Without CDN:
  User (Mumbai) → Origin Server (Virginia)
  Round trip: ~200ms
  Every image, every CSS file, every JS bundle: 200ms each

With CDN:
  User (Mumbai) → CDN Edge (Mumbai)
  Round trip: ~15ms
  Images, CSS, JS served from local edge

  First request (cache miss):
    CDN Edge → Origin → CDN caches it → serves to user
    Latency: ~200ms (one-time cost)

  Subsequent requests (cache hit):
    CDN Edge serves directly
    Latency: ~15ms (93% faster)

For a page with 30 assets:
  Without CDN: 30 × 200ms = 6 seconds (sequential)
  With CDN:    30 × 15ms  = 450ms
  → 13x faster page load

Original traffic: 200K API reads/sec + 5M media reads/sec

Layer 1 — CDN:
  5M media reads → 4.75M served from edge (95%)
  250K cache misses → origin

Layer 2 — Redis Cache:
  200K API reads → 180K served from Redis (90%)
  20K cache misses → read replicas

Layer 3 — Read Replicas (5x):
  20K reads → distributed across 5 replicas
  4K reads/sec per replica

Layer 4 — Primary DB:
  0 reads (all absorbed above)
  2K writes/sec only

Summary:
  Total incoming: 5.2M requests/sec
  Primary DB handles: 2K writes/sec
  → 99.96% of traffic never touches the primary database