Design a Video Streaming Platform (YouTube / Netflix)

Given:
  500 hours/min uploaded
  Average video length: ~5 min (short-form dominates)
  → 500 × 60 / 5 = 6,000 videos uploaded per minute
  → 100 videos/sec (average)
  → Peak: ~300 videos/sec (3× burst)

Average raw file size (1080p, H.264, 8 Mbps):
  5 min × 60s × 8 Mbps / 8 = 300 MB per video

Daily raw ingestion:
  6,000/min × 60 × 24 = 8.64M videos/day
  8.64M × 300 MB = 2.6 PB/day raw upload

After transcoding (4 resolutions × 2 codecs = 8 renditions):
  Each rendition averages ~40% of original (lower res = smaller)
  Storage multiplier: ~3.2× original
  2.6 PB × 3.2 = ~8.3 PB/day total storage (all renditions + original)

Implication:
  → Object storage (S3) is the only option. No filesystem handles this.
  → Storage cost is THE dominant cost. Hot/warm/cold tiering is mandatory.
  → Transcoding 100 videos/sec requires massive parallel compute.

Given:
  1B video views/day
  Average watch duration: 7 min (not all videos watched fully)
  Average bitrate served: 4 Mbps (mix of 480p mobile + 1080p desktop)

Concurrent viewers (peak hour = 2× average):
  1B / 86,400 = ~11,600 views starting per second (average)
  Peak: ~23,000 views/sec
  Average concurrent streams: ~4.8M (7 min avg × 11,600/sec × 60)
  Peak concurrent: ~10M streams

Egress bandwidth:
  4.8M concurrent × 4 Mbps = 19.2 Tbps average
  Peak: ~40 Tbps

Implication:
  → No single origin can serve this. CDN is the product.
  → CDN cache hit rate must be >95% to keep origin load manageable.
  → Origin bandwidth: 40 Tbps × 5% miss = 2 Tbps from origin.

Daily storage growth: ~8.3 PB/day (all renditions)
Monthly: ~250 PB
Yearly: ~3 EB (exabytes)

With compression + deduplication + cold-tier archival:
  Active hot storage (last 30 days): ~250 PB
  Warm storage (30-365 days): ~2 EB
  Cold storage (>1 year, rarely accessed): grows unbounded

Cost at S3 pricing ($0.023/GB/month for standard):
  250 PB hot = $5.75M/month
  With Glacier for cold: ~$1/TB/month → massive savings

Implication:
  → Storage tiering is a P0 cost optimization, not a nice-to-have.
  → Videos not watched in 90 days → move to warm/cold tier.
  → Delete lowest-resolution renditions for unwatched videos.

Input: 100 videos/sec average, 300 videos/sec peak
Each video → 8 renditions (4 resolutions × 2 codecs)
Each rendition takes ~1× real-time on a GPU (5-min video = 5 min to transcode)
  → With parallelized segment-level transcoding: ~30s per rendition

Workers needed (peak):
  300 videos/sec × 8 renditions × 30s per rendition = 72,000 worker-seconds/sec
  → 72,000 concurrent GPU workers at peak

With segment-level parallelism (split into 4s segments):
  5 min video = 75 segments
  Each segment transcodes in ~2s on GPU
  → 300 videos × 75 segments × 8 renditions = 180,000 segment-jobs/sec
  → But each finishes in 2s, so steady-state: ~360,000 concurrent segment jobs

Implication:
  → Transcoding is the most expensive compute workload.
  → Must be async (queue-based). Blocking upload on transcode = terrible UX.
  → Spot/preemptible instances save 60-70% on GPU cost.
  → Custom ASICs (YouTube VCU, Netflix ASIC) at hyperscale.

Per-video metadata:
  video_id         : 8 bytes (UUID or Snowflake)
  title            : 100 bytes avg
  description      : 500 bytes avg
  uploader_id      : 8 bytes
  duration_ms      : 4 bytes
  upload_status    : 1 byte
  created_at       : 8 bytes
  thumbnail_url    : 100 bytes
  tags (JSONB)     : 200 bytes
  manifest_urls    : 300 bytes
  view_count       : 8 bytes
  ─────────────────────────────
  ≈ 1.2 KB per video

Total metadata (5 years):
  8.64M videos/day × 365 × 5 = ~15.8B videos
  15.8B × 1.2 KB = ~19 TB

Implication:
  → Metadata fits comfortably in a sharded relational DB.
  → The hard problem is video bytes, not metadata bytes.
  → Shard by hash(video_id) — point lookups dominate.

POST /api/v1/videos/upload-init HTTP/1.1
Host: api.vidstream.io
Authorization: Bearer <jwt>
Content-Type: application/json

{
  "title": "System Design Interview Tips",
  "description": "How to ace the YouTube design question",
  "file_name": "interview-tips.mp4",
  "file_size_bytes": 524288000,        // 500 MB
  "content_type": "video/mp4",
  "chunk_size_bytes": 10485760,        // 10 MB per chunk
  "tags": ["system-design", "interview"]
}

--- 201 Created ---
{
  "video_id": "vid_8f4b3c2a",
  "upload_id": "upl_9x7k2m",          // S3 multipart upload ID
  "chunk_count": 50,
  "presigned_urls": [                  // One per chunk
    { "chunk_index": 0, "url": "https://s3.../part0?X-Amz-Signature=...", "expires_at": "..." },
    { "chunk_index": 1, "url": "https://s3.../part1?X-Amz-Signature=...", "expires_at": "..." },
    // ... 48 more
  ],
  "status": "uploading"
}

--- 400 Bad Request ---
{ "error": "FILE_TOO_LARGE", "message": "Maximum file size is 256 GB" }

--- 429 Too Many Requests ---
{ "error": "UPLOAD_LIMIT", "message": "Max 10 concurrent uploads per user" }

POST /api/v1/videos/vid_8f4b3c2a/complete HTTP/1.1
Authorization: Bearer <jwt>
Content-Type: application/json

{
  "upload_id": "upl_9x7k2m",
  "chunks": [
    { "chunk_index": 0, "etag": ""abc123..."" },
    { "chunk_index": 1, "etag": ""def456..."" },
    // ... all 50 ETags from S3 responses
  ]
}

--- 202 Accepted ---
{
  "video_id": "vid_8f4b3c2a",
  "status": "processing",
  "estimated_ready_at": "2026-05-17T10:15:00Z",
  "webhook_url": "/api/v1/videos/vid_8f4b3c2a/status"
}

--- 409 Conflict ---
{ "error": "CHUNKS_MISSING", "missing": [12, 37], "message": "2 chunks not yet uploaded" }

GET /api/v1/videos/vid_8f4b3c2a/manifest HTTP/1.1
Host: cdn.vidstream.io
Accept: application/vnd.apple.mpegurl

--- 200 OK ---
Content-Type: application/vnd.apple.mpegurl
Cache-Control: public, max-age=86400
X-CDN-Cache: HIT

#EXTM3U
#EXT-X-VERSION:4
#EXT-X-STREAM-INF:BANDWIDTH=800000,RESOLUTION=640x360,CODECS="avc1.4d401e,mp4a.40.2"
/segments/vid_8f4b3c2a/360p/playlist.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=1400000,RESOLUTION=854x480,CODECS="avc1.4d401f,mp4a.40.2"
/segments/vid_8f4b3c2a/480p/playlist.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=2800000,RESOLUTION=1280x720,CODECS="avc1.4d401f,mp4a.40.2"
/segments/vid_8f4b3c2a/720p/playlist.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=5000000,RESOLUTION=1920x1080,CODECS="avc1.640028,mp4a.40.2"
/segments/vid_8f4b3c2a/1080p/playlist.m3u8

GET /api/v1/videos/vid_8f4b3c2a/upload-status HTTP/1.1
Authorization: Bearer <jwt>

--- 200 OK ---
{
  "video_id": "vid_8f4b3c2a",
  "upload_id": "upl_9x7k2m",
  "status": "uploading",
  "total_chunks": 50,
  "uploaded_chunks": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11],
  "missing_chunks": [12, 13, 14, ..., 49],
  "presigned_urls": [
    { "chunk_index": 12, "url": "https://s3.../part12?...", "expires_at": "..." },
    // fresh presigned URLs for remaining chunks
  ]
}

CREATE TABLE videos (
  id              UUID         PRIMARY KEY DEFAULT gen_random_uuid(),
  uploader_id     UUID         NOT NULL REFERENCES users(id),
  title           VARCHAR(500) NOT NULL,
  description     TEXT,
  duration_ms     INT,                          -- populated after transcoding
  status          VARCHAR(20)  NOT NULL DEFAULT 'uploading',
                  -- uploading | processing | ready | failed | deleted
  visibility      VARCHAR(10)  NOT NULL DEFAULT 'public',
                  -- public | unlisted | private
  thumbnail_url   TEXT,
  manifest_url    TEXT,                         -- set after transcoding completes
  original_url    TEXT         NOT NULL,        -- S3 path to raw upload
  file_size_bytes BIGINT       NOT NULL,
  tags            JSONB        DEFAULT '[]',
  view_count      BIGINT       NOT NULL DEFAULT 0,  -- denormalized, async-updated
  like_count      INT          NOT NULL DEFAULT 0,
  created_at      TIMESTAMPTZ  NOT NULL DEFAULT now(),
  updated_at      TIMESTAMPTZ,
  published_at    TIMESTAMPTZ                   -- when status → ready
);

-- Primary access pattern: fetch by ID (player page)
-- Already covered by PK

-- Secondary: user's videos (channel page)
CREATE INDEX idx_videos_uploader ON videos (uploader_id, created_at DESC)
  WHERE status != 'deleted';

-- Secondary: trending/search (read from search index, not this)
-- Secondary: processing queue (internal)
CREATE INDEX idx_videos_status ON videos (status, created_at)
  WHERE status IN ('uploading', 'processing');

CREATE TABLE transcoding_jobs (
  id              UUID         PRIMARY KEY DEFAULT gen_random_uuid(),
  video_id        UUID         NOT NULL REFERENCES videos(id),
  resolution      VARCHAR(10)  NOT NULL,  -- 360p, 480p, 720p, 1080p, 4k
  codec           VARCHAR(20)  NOT NULL,  -- h264, h265, vp9, av1
  status          VARCHAR(20)  NOT NULL DEFAULT 'pending',
                  -- pending | segmenting | transcoding | assembling | done | failed
  segment_count   INT,                    -- total segments for this rendition
  segments_done   INT          DEFAULT 0, -- progress tracking
  worker_id       VARCHAR(50),            -- which worker claimed this job
  output_url      TEXT,                   -- S3 path to transcoded segments
  started_at      TIMESTAMPTZ,
  completed_at    TIMESTAMPTZ,
  error_message   TEXT,
  retry_count     INT          DEFAULT 0,
  created_at      TIMESTAMPTZ  NOT NULL DEFAULT now(),
  updated_at      TIMESTAMPTZ
);

-- Workers poll for pending jobs
CREATE INDEX idx_jobs_pending ON transcoding_jobs (status, created_at)
  WHERE status = 'pending';

-- Dashboard: all jobs for a video
CREATE INDEX idx_jobs_video ON transcoding_jobs (video_id);

Bucket: vidstream-segments-{region}

Path structure:
  /raw/{video_id}/original.mp4                    -- raw upload
  /segments/{video_id}/{resolution}/{codec}/
    ├── playlist.m3u8                             -- media manifest for this rendition
    ├── segment_000.ts                            -- 4-second segment
    ├── segment_001.ts
    ├── segment_002.ts
    └── ...
  /manifests/{video_id}/master.m3u8              -- master manifest (all renditions)
  /thumbnails/{video_id}/thumb_{timestamp}.jpg   -- auto-generated thumbnails

Example:
  /segments/vid_8f4b3c2a/1080p/h264/playlist.m3u8
  /segments/vid_8f4b3c2a/1080p/h264/segment_042.ts
  /segments/vid_8f4b3c2a/720p/vp9/segment_042.ts

CREATE TABLE view_events (
  event_id       UUID,
  video_id       UUID,
  viewer_id      UUID,              -- nullable for anonymous viewers
  watched_ms     UInt32,            -- how much they actually watched
  total_ms       UInt32,            -- video duration
  device_type    LowCardinality(String),  -- mobile, desktop, tv, tablet
  country_code   FixedString(2),
  quality        LowCardinality(String),  -- 360p, 480p, 720p, 1080p
  referrer       String,            -- search, recommendation, direct, external
  session_id     UUID,
  created_at     DateTime64(3)
)
ENGINE = MergeTree()
PARTITION BY toYYYYMM(created_at)
ORDER BY (video_id, created_at)
TTL created_at + INTERVAL 2 YEAR;

Total budget: <2s to first frame

0ms     ─  DNS resolution (cached after first visit)
50ms    ─  TLS handshake (HTTP/2 keep-alive eliminates on subsequent)
100ms   ─  Fetch master manifest from CDN edge (cache hit: ~20ms)
150ms   ─  Player parses manifest, selects initial quality (bandwidth probe)
200ms   ─  Fetch first segment from CDN edge (~800KB for 4s at 1.6 Mbps)
            ├─ CDN hit (95%): ~100ms (edge is <50ms RTT away)
            └─ CDN miss (5%): ~300ms (origin shield → S3)
1200ms  ─  First segment decoded and rendered
1500ms  ─  Playback begins, next segment prefetched in parallel
            ───────────────────────────────────────────────
            Total on CDN hit:  ~1.2s  ✅
            Total on CDN miss: ~1.8s  ✅ (still under 2s budget)

Seek operation:
  Player requests segment at new timestamp from CDN
  Budget: <500ms (CDN hit) / <800ms (CDN miss)

async function initUpload(req: InitUploadRequest): Promise<InitUploadResponse> {
  // 1. Validate
  if (req.fileSizeBytes > MAX_FILE_SIZE) throw new FileTooLargeError();
  if (!ALLOWED_CONTENT_TYPES.includes(req.contentType)) throw new InvalidContentTypeError();

  // 2. Check user upload limits (max 10 concurrent)
  const activeUploads = await db.countActiveUploads(req.userId);
  if (activeUploads >= MAX_CONCURRENT_UPLOADS) throw new UploadLimitError();

  // 3. Create video record
  const video = await db.createVideo({
    uploaderId: req.userId,
    title: req.title,
    description: req.description,
    fileSizeBytes: req.fileSizeBytes,
    status: 'uploading',
  });

  // 4. Initiate S3 multipart upload
  const chunkCount = Math.ceil(req.fileSizeBytes / req.chunkSizeBytes);
  const { uploadId } = await s3.createMultipartUpload({
    bucket: RAW_UPLOADS_BUCKET,
    key: `raw/${video.id}/original.${getExtension(req.fileName)}`,
    contentType: req.contentType,
  });

  // 5. Generate presigned URLs for each chunk
  const presignedUrls = await Promise.all(
    Array.from({ length: chunkCount }, (_, chunkIndex) =>
      s3.getSignedUrl('uploadPart', {
        bucket: RAW_UPLOADS_BUCKET,
        key: `raw/${video.id}/original.${getExtension(req.fileName)}`,
        uploadId,
        partNumber: chunkIndex + 1,
        expiresIn: 3600, // 1 hour
      }).then(url => ({ chunkIndex, url, expiresAt: Date.now() + 3600_000 }))
    )
  );

  // 6. Store upload state
  await db.createUploadSession({
    videoId: video.id,
    uploadId,
    chunkCount,
    chunkSizeBytes: req.chunkSizeBytes,
  });

  return { videoId: video.id, uploadId, chunkCount, presignedUrls, status: 'uploading' };
}

Chunk size options:
  5 MB   — Best for mobile / flaky connections. 100 chunks for 500 MB file.
            More S3 API calls ($0.005 per 1000 PUTs), but minimal re-upload on failure.
  10 MB  — Default. Good balance. 50 chunks for 500 MB.
  50 MB  — Best for high-bandwidth desktop uploads. 10 chunks for 500 MB.
            Fewer API calls, but a failure wastes up to 50 MB of progress.

Adaptive strategy (what YouTube does):
  Start with 10 MB chunks.
  If 3 consecutive chunks succeed in <2s each → increase to 25 MB.
  If a chunk fails or takes >30s → decrease to 5 MB.
  This adapts to the user's actual bandwidth without configuration.

S3 constraints:
  Minimum part size: 5 MB (except last part)
  Maximum part size: 5 GB
  Maximum parts per upload: 10,000
  → Maximum file size: 10,000 × 5 GB = 50 TB (well above our 256 GB limit)

Step 1: PROBE (1 worker, ~2s)
  ─ Read file headers with ffprobe
  ─ Extract: codec, resolution, bitrate, duration, frame rate, audio tracks
  ─ Decide which output renditions to generate based on source quality
    (don't upscale — if source is 720p, skip 1080p and 4K renditions)

Step 2: SEGMENT (1 worker, ~30s for 10-min video)
  ─ Split the raw video into 4-second segments at keyframe boundaries
  ─ Output: segment_000.ts through segment_149.ts (for 10-min video)
  ─ Write segments to S3 temp path: /temp/{video_id}/raw_segments/
  ─ This step is sequential — must complete before transcoding begins

Step 3: TRANSCODE (N×M workers in parallel, ~2-5s per segment per rendition)
  ─ For each segment × each rendition: spawn a worker
  ─ Example: 150 segments × 8 renditions = 1,200 parallel tasks
  ─ Each worker:
    1. Downloads one raw segment from S3 (~4s of video = ~2MB)
    2. Transcodes to target resolution + codec (ffmpeg)
    3. Uploads output segment to S3: /segments/{video_id}/{res}/{codec}/segment_042.ts
    4. Reports completion to orchestrator
  ─ Workers are stateless — any worker can process any segment
  ─ Failed segments are retried on a different worker (idempotent)

Step 4: AUDIO PROCESSING (parallel with video transcoding)
  ─ Extract audio track(s) from original
  ─ Normalize loudness (EBU R128 standard)
  ─ Encode to AAC at multiple bitrates (64kbps, 128kbps, 256kbps)
  ─ Generate subtitle tracks if speech-to-text is enabled

Step 5: MANIFEST GENERATION (1 worker, ~5s, after all transcoding done)
  ─ Generate media playlists for each rendition (lists all segments)
  ─ Generate master playlist (lists all renditions with bandwidth/resolution)
  ─ Write to S3: /manifests/{video_id}/master.m3u8
  ─ This is the "assembly" step — must wait for ALL segments to complete

Step 6: THUMBNAIL GENERATION (parallel with everything after probe)
  ─ Extract frames at 25%, 50%, 75% of duration
  ─ Run scene-detection to find visually interesting frames
  ─ Generate multiple sizes (120px, 320px, 640px) for different contexts
  ─ Upload to S3 + CDN pre-warm

Step 7: FINALIZE
  ─ Update video record: status='ready', manifest_url, duration_ms, thumbnail_url
  ─ Publish 'video-ready' event to Kafka
  ─ Notify creator via push notification / email
  ─ Clean up temp segments from S3

// Simplified Temporal workflow for video transcoding
async function transcodeVideoWorkflow(videoId: string, s3Key: string): Promise<void> {
  // Step 1: Probe the source file
  const probeResult = await activities.probeVideo(videoId, s3Key);
  // { codec: 'h264', resolution: '1920x1080', duration: 600000, fps: 30 }

  // Step 2: Determine renditions based on source quality
  const renditions = selectRenditions(probeResult);
  // e.g., [360p/h264, 480p/h264, 720p/h264, 1080p/h264, 720p/vp9, 1080p/vp9]

  // Step 3: Segment the source video
  const segments = await activities.segmentVideo(videoId, s3Key, {
    segmentDuration: 4, // seconds
    splitAtKeyframes: true,
  });
  // Returns: { count: 150, tempPath: '/temp/{videoId}/raw_segments/' }

  // Step 4: Fan-out — transcode all segments × all renditions in parallel
  const transcodePromises = renditions.flatMap((rendition) =>
    Array.from({ length: segments.count }, (_, segmentIndex) =>
      activities.transcodeSegment(videoId, segmentIndex, rendition)
    )
  );

  // Temporal handles parallelism limits, retries, and timeouts
  await Promise.all(transcodePromises);

  // Step 5: Audio processing (ran in parallel with step 4)
  // (Temporal supports parallel branches in the DAG)
  await activities.processAudio(videoId, s3Key);

  // Step 6: Generate manifests
  const manifestUrl = await activities.generateManifests(videoId, renditions, segments.count);

  // Step 7: Generate thumbnails
  const thumbnailUrl = await activities.generateThumbnails(videoId, s3Key, probeResult.duration);

  // Step 8: Finalize
  await activities.finalizeVideo(videoId, {
    status: 'ready',
    manifestUrl,
    thumbnailUrl,
    durationMs: probeResult.duration,
  });

  // Step 9: Cleanup temp files
  await activities.cleanupTempSegments(videoId);
}

1. Spot/Preemptible instances (60-70% savings)
   ─ Transcoding is stateless and idempotent — perfect for spot
   ─ If a spot instance is reclaimed, the segment job is retried elsewhere
   ─ Mix: 80% spot + 20% on-demand for guaranteed baseline capacity

2. Tiered transcoding priority
   ─ Channels with >10K subscribers: transcode immediately (all renditions)
   ─ New uploads from small channels: transcode 720p first (playable fast)
   ─ Generate remaining renditions in background (lower priority queue)
   ─ If video gets 0 views in 7 days: skip remaining renditions entirely

3. Codec selection based on popularity
   ─ All videos: H.264 (universal compatibility)
   ─ Videos with >1K views: add VP9/H.265 (bandwidth savings justify cost)
   ─ Videos with >100K views: add AV1 (best compression, slowest to encode)

4. Resolution capping
   ─ Never upscale — if source is 720p, don't generate 1080p or 4K
   ─ For audio-only content (podcasts, music): skip video transcoding entirely

5. GPU instance right-sizing
   ─ Short videos (<1 min): small GPU instances (T4)
   ─ Long videos (>30 min): large GPU instances (A100) for parallelism
   ─ Batch small videos together on one instance to amortize startup cost

#EXTM3U
#EXT-X-VERSION:4

# Audio-only track for very low bandwidth
#EXT-X-STREAM-INF:BANDWIDTH=128000,CODECS="mp4a.40.2"
audio_only/playlist.m3u8

# 360p — mobile on 3G
#EXT-X-STREAM-INF:BANDWIDTH=800000,RESOLUTION=640x360,CODECS="avc1.4d401e,mp4a.40.2"
360p/h264/playlist.m3u8

# 480p — standard mobile
#EXT-X-STREAM-INF:BANDWIDTH=1400000,RESOLUTION=854x480,CODECS="avc1.4d401f,mp4a.40.2"
480p/h264/playlist.m3u8

# 720p — desktop / good mobile
#EXT-X-STREAM-INF:BANDWIDTH=2800000,RESOLUTION=1280x720,CODECS="avc1.4d401f,mp4a.40.2"
720p/h264/playlist.m3u8

# 1080p — high bandwidth
#EXT-X-STREAM-INF:BANDWIDTH=5000000,RESOLUTION=1920x1080,CODECS="avc1.640028,mp4a.40.2"
1080p/h264/playlist.m3u8

# 1080p VP9 — better compression for supported devices
#EXT-X-STREAM-INF:BANDWIDTH=3200000,RESOLUTION=1920x1080,CODECS="vp09.00.40.08,opus"
1080p/vp9/playlist.m3u8

#EXTM3U
#EXT-X-VERSION:4
#EXT-X-TARGETDURATION:4
#EXT-X-MEDIA-SEQUENCE:0
#EXT-X-PLAYLIST-TYPE:VOD

#EXTINF:4.000,
segment_000.ts
#EXTINF:4.000,
segment_001.ts
#EXTINF:4.000,
segment_002.ts
#EXTINF:4.000,
segment_003.ts
...
#EXTINF:3.240,
segment_149.ts

#EXT-X-ENDLIST

After downloading each segment, the player measures:
  1. throughput = segment_size_bytes / download_time_seconds
  2. buffer_level = seconds of video already buffered ahead of playhead

Decision rules:
  IF buffer_level < 2s (danger zone — about to stall):
    → Switch DOWN immediately to lowest quality
    → Priority: avoid buffering at all costs

  ELSE IF buffer_level < 5s (low buffer):
    → Select quality where bitrate ≤ 70% of measured throughput
    → Conservative — build buffer before upgrading

  ELSE IF buffer_level > 15s (healthy buffer):
    → Select quality where bitrate ≤ 90% of measured throughput
    → Aggressive — can afford a brief stall if estimate is wrong

  ELSE (normal operation, 5-15s buffer):
    → Select quality where bitrate ≤ 80% of measured throughput
    → Balanced — standard operation

  STABILITY RULE:
    → Don't switch quality unless the new level has been "better" for
      3 consecutive segments (prevents oscillation on variable networks)

Real implementations (BOLA, MPC, Pensieve):
  → Use buffer-based + throughput-based hybrid algorithms
  → Some use ML models trained on millions of sessions (Pensieve/Netflix)
  → YouTube uses a proprietary algorithm tuned for their CDN topology

User seeks to 2:34 (154 seconds):
  1. target_segment = floor(154 / 4) = segment_038
  2. Flush current buffer
  3. Fetch segment_038.ts from CDN (likely cached — popular videos)
  4. Decode from nearest keyframe (segment starts are always keyframes)
  5. Render frame at 154s
  6. Prefetch segment_039, segment_040 in parallel
  
  Total seek time:
    CDN hit: ~100-200ms (fetch + decode)
    CDN miss: ~300-500ms (origin fetch + decode)
    
  Why it's fast:
    → Segments start at keyframes (no need to decode from GOP start)
    → CDN caches segments individually (seek doesn't invalidate cache)
    → Player prefetches adjacent segments speculatively

Tier 1: Edge PoP (200+ locations globally)
  ─ Closest to the user (same city or ISP)
  ─ Storage: 10-50 TB per PoP (SSD)
  ─ Serves: ~80% of requests (hot content)
  ─ TTL: 24h for segments, 1h for manifests
  ─ Eviction: LRU — popular content stays, long-tail evicts

Tier 2: Regional Shield (10-20 locations)
  ─ Aggregates cache misses from multiple edge PoPs in a region
  ─ Storage: 200-500 TB per shield (HDD + SSD)
  ─ Serves: ~15% of requests (warm content)
  ─ Purpose: Prevents "thundering herd" to origin when content goes viral
  ─ One regional shield serves 10-20 edge PoPs

Tier 3: Origin Shield (2-3 locations)
  ─ Single point of contact with S3 origin
  ─ Storage: 1-5 PB (the "last cache" before origin)
  ─ Serves: ~4% of requests (cold content being accessed)
  ─ Purpose: Collapses all origin requests to a single fetch per object
  ─ Request coalescing: 1000 concurrent misses for same segment → 1 S3 GET

Tier 4: S3 Origin
  ─ Source of truth — all segments and manifests
  ─ Serves: <1% of total requests (truly cold content)
  ─ Latency: 50-200ms depending on region
  ─ Cost: $0.09/GB egress — minimizing origin hits saves millions

Cache hit rates (production targets):
  Edge:           80% hit rate
  Edge + Shield:  95% hit rate
  Total (all tiers): 99%+ hit rate
  Origin sees:    <1% of total streaming traffic

1. Predictive warming (for known-popular content)
   ─ Trigger: Video from channel with >1M subscribers finishes transcoding
   ─ Action: Push first 30 segments (2 minutes of video) to all edge PoPs
   ─ Why: First 2 minutes covers 90% of viewers who click but don't finish
   ─ Cost: 30 segments × 200 PoPs × 2MB = 12 GB of proactive distribution

2. Reactive warming (for viral content)
   ─ Trigger: Cache miss rate for a video exceeds threshold (>100 misses/min at one PoP)
   ─ Action: Pre-fetch all segments of that video to the regional shield
   ─ Why: Viral content goes from 0 to millions of views in minutes
   ─ The shield absorbs the thundering herd while edge caches fill organically

3. Geographic warming (for region-specific content)
   ─ Trigger: Video metadata indicates language/region (e.g., Hindi video)
   ─ Action: Warm edge PoPs in India, skip PoPs in South America
   ─ Why: A Hindi video won't be popular in Brazil — don't waste cache space

4. Time-based warming (for scheduled content)
   ─ Trigger: Creator schedules a premiere for 8 PM IST
   ─ Action: Warm Indian PoPs 30 minutes before premiere
   ─ Why: Thousands of viewers will click simultaneously at premiere time

Cache key = path only (no query params, no cookies, no headers)

Example:
  /segments/vid_8f4b3c2a/720p/h264/segment_042.ts

Why path-only:
  ─ Same segment is identical for all viewers (no personalization)
  ─ Query params (auth tokens, tracking IDs) would create unique cache entries
    for the same content → 0% hit rate
  ─ Auth is handled at the edge via signed URLs with short TTL, not query params

Signed URL approach:
  ─ CDN validates the signature at the edge (CloudFront signed URLs)
  ─ After validation, strips the signature and uses path as cache key
  ─ Result: authenticated access + shared cache entries

Manifest cache key (different strategy):
  /manifests/vid_8f4b3c2a/master.m3u8
  ─ Shorter TTL (1 hour) because manifests can be updated
    (e.g., new rendition added, segment URLs rotated)
  ─ Cache-Control: public, max-age=3600, stale-while-revalidate=60

Scenario 1: Copyright takedown (DMCA)
  ─ Legal requirement: remove within 24 hours (often faster)
  ─ Action: CDN purge API → invalidate all paths matching /segments/{video_id}/*
  ─ Propagation: 5-30 seconds across all PoPs (CloudFront) / 2-5 min (Akamai)
  ─ Fallback: Origin returns 403 for purged video_id (CDN miss → blocked)

Scenario 2: Creator deletes their video
  ─ Soft-delete in DB (status='deleted')
  ─ CDN purge for manifest (segments will naturally evict via LRU)
  ─ S3 objects moved to "deleted" prefix (retained 30 days for appeals)

Scenario 3: Re-transcoding (quality improvement)
  ─ New segments get a version suffix: /segments/{video_id}/v2/720p/...
  ─ New manifest points to v2 segments
  ─ Old segments evict naturally — no explicit purge needed
  ─ This is why versioned paths > cache purging for planned updates

Scenario 4: Manifest update (new rendition added)
  ─ Manifest has short TTL (1 hour) — natural refresh
  ─ For urgent updates: CDN purge on manifest path only (lightweight)
  ─ Segments are additive — old segments remain valid

A view is counted when ALL of these are true:
  1. watched_ms >= 30,000 (30 seconds of actual playback)
  2. NOT a duplicate from same session (dedupe by session_id + video_id)
  3. NOT from a known bot User-Agent
  4. Rate limit: max 1 view per video per user per 4 hours
  5. Player was visible (not a background tab — Page Visibility API)

Where validation happens:
  ─ Rule 1, 5: Client-side (player SDK only fires event after 30s of visible play)
  ─ Rule 2: Ingestion API (Redis SET with TTL for session dedup)
  ─ Rule 3: Ingestion API (User-Agent blocklist)
  ─ Rule 4: Flink (stateful dedup window per user+video, 4h TTL)

Why not validate everything at ingestion?
  ─ Rule 4 requires stateful processing across millions of user+video pairs
  ─ Too expensive to do synchronously at 23K events/sec
  ─ Flink handles stateful dedup efficiently with RocksDB state backend

// Flink writes (every 1-minute window):
// After aggregating valid views in a tumbling window
await redis.incrBy(`views:${videoId}`, windowCount);
// e.g., INCRBY views:vid_8f4b3c2a 847
// (847 valid views in the last minute for this video)

// Metadata Service reads (on every video page load):
const viewCount = await redis.get(`views:${videoId}`);
// Returns the real-time count — at most 1 minute stale

// Periodic DB flush (every 5 minutes, batch job):
// Reads Redis counters, batch-updates Postgres
// UPDATE videos SET view_count = $1, updated_at = now() WHERE id = $2
// This makes the count durable — Redis is volatile

// Why not just use Postgres directly?
// At 23K views/sec, UPDATE ... SET view_count = view_count + 1
// creates a hot row with row-level locking → ~500 updates/sec max per row
// Redis INCRBY is lock-free and handles millions of ops/sec

Trending score formula:
  score = views_last_1h × w1 + views_last_6h × w2 + views_last_24h × w3
  where w1 = 1.0, w2 = 0.3, w3 = 0.1 (recency-weighted)

  Normalization: divide by channel_subscriber_count^0.3
  (prevents large channels from always dominating trending)

Implementation with Count-Min Sketch + Heap:

Flink streaming job:

1. Input: view-events Kafka topic (partitioned by video_id)

2. Sliding window: 1-hour window, sliding every 1 minute
   ─ For each video_id, count views in the current window

3. Count-Min Sketch (per window):
   ─ Space: ~100 KB for tracking millions of videos
   ─ Error: <1% over-count (acceptable for trending)
   ─ Update: for each view event, increment CMS counters

4. Min-Heap of size K (K=100 for "Top 100 Trending"):
   ─ Maintains the K videos with highest trending scores
   ─ When a video's score exceeds the heap minimum, swap it in
   ─ O(log K) per update — fast even at 23K events/sec

5. Output: Every minute, emit the current Top-100 to Redis
   ─ Key: trending:global (sorted set with scores)
   ─ Key: trending:{country_code} (per-country trending)
   ─ The feed service reads these sorted sets directly

6. Decay: Scores naturally decay as old events leave the sliding window
   ─ No explicit "decay factor" needed — the window handles it

Source 1: Collaborative filtering (ANN search)
  ─ User embedding → find nearest video embeddings in vector space
  ─ Uses Approximate Nearest Neighbor search (HNSW index in Pinecone/Milvus)
  ─ Returns: ~200 candidates in <10ms
  ─ Signal: "users like you watched these videos"

Source 2: Content-based (same topic/channel)
  ─ Given user's recent watch history, find videos with similar tags/topics
  ─ Elasticsearch query: match on tags, category, channel
  ─ Returns: ~200 candidates in <20ms
  ─ Signal: "you watched X, here's more like X"

Source 3: Subscription feed
  ─ Recent uploads from channels the user subscribes to
  ─ Simple DB query: videos WHERE channel_id IN (subscriptions) AND created_at > 7d
  ─ Returns: ~100 candidates in <10ms
  ─ Signal: "your subscriptions have new content"

Source 4: Trending / popular
  ─ Read from the trending sorted set in Redis (computed by Flink)
  ─ Returns: ~100 candidates in <5ms
  ─ Signal: "everyone is watching this right now"

Source 5: Exploration (serendipity)
  ─ Random sample from recent uploads outside user's usual interests
  ─ Returns: ~50 candidates
  ─ Signal: "break the filter bubble, discover new content"

Total candidates after dedup: ~500-1000 unique videos
Latency budget: <50ms (all sources run in parallel)