Design a Video Streaming Platform (Netflix/YouTube)
Video upload pipeline, transcoding, adaptive bitrate streaming, CDN distribution, recommendation engine integration, and handling peak traffic.
Problem Statement
A video streaming platform lets users upload, discover, and watch video content. The two hard problems are: (1) ingestion — processing raw uploads (potentially 4K/8K) into multiple quality tiers quickly, and (2) delivery — serving billions of video segments to hundreds of millions of concurrent viewers with minimal buffering.
Requirements
| Functional | Non-Functional |
|---|---|
| Upload videos up to 10 GB | Start playback within 2 seconds |
| Stream at multiple quality levels (360p–4K) | 99.99% streaming availability |
| Adaptive bitrate (ABR) based on bandwidth | Support 1 B+ daily stream starts |
| Search and recommendations | Video processing complete within 30 min of upload |
| Resume playback, chapters, subtitles | Globally low latency via CDN |
Capacity Estimation
| Metric | Estimate |
|---|---|
| Videos uploaded / minute | 500 hours of video (YouTube-scale) |
| Avg raw video size | 1 GB per 10 min at 1080p |
| Transcoded outputs per video | 6 quality tiers × 3 codecs = ~18 renditions |
| Storage per original video (10 min) | ~1 GB raw + ~3 GB transcoded |
| Daily stream starts | 1 B (avg 20 min per session = ~1 Pbps bandwidth) |
| CDN egress (at 5 Mbps avg bitrate) | ~833 Tbps peak |
High-Level Architecture
Video Upload Pipeline
Transcoding and Adaptive Bitrate
Raw video is transcoded into multiple renditions using HLS (HTTP Live Streaming) or DASH (Dynamic Adaptive Streaming over HTTP). Each rendition is segmented into 2–10 second chunks. The player selects which quality segment to request based on current network conditions, switching mid-stream without interruption.
| Quality | Resolution | Bitrate | Segment Size (4s) |
|---|---|---|---|
| 360p | 640×360 | 400 Kbps | 200 KB |
| 480p | 854×480 | 800 Kbps | 400 KB |
| 720p | 1280×720 | 2.5 Mbps | 1.25 MB |
| 1080p | 1920×1080 | 5 Mbps | 2.5 MB |
| 4K | 3840×2160 | 15 Mbps | 7.5 MB |
Parallel Transcoding
Split the raw video into 1-minute chunks and transcode each chunk in parallel across a GPU farm. A 2-hour movie becomes 120 independent 1-minute jobs. This reduces time-to-availability from hours to minutes. Stitch the completed segments back together into master playlists.
CDN Strategy
Video segments are immutable files — perfect for aggressive CDN caching. Netflix's Open Connect deploys custom CDN appliances directly inside ISP networks, serving video from servers that may be physically inside the same building as the viewer. Key CDN strategies:
- Long cache TTL: video segments never change — cache forever (`Cache-Control: max-age=31536000, immutable`).
- Manifest short TTL: the `.m3u8` playlist may update for live streams — short TTL (5–30 seconds).
- Pre-fetching popular content: predict trending content and pre-warm CDN PoPs before demand hits.
- Geo-routing: route viewers to the nearest CDN PoP based on IP geolocation and BGP anycast.
Metadata Service
Video metadata (title, description, tags, thumbnails, like counts) is stored in MySQL for structured queries and indexed in Elasticsearch for full-text search. The recommendation engine reads from a data warehouse fed by Kafka events (watches, likes, completions) and writes recommended `videoId` lists back to Redis per user.
Scaling Considerations
- Transcoding farm auto-scaling: use a job queue (SQS) + spot GPU instances that auto-scale. Job priority queue ensures new uploads are processed before re-encoding old content.
- Resumable uploads: for large files, use S3 multipart upload with client-side chunk retry. Store upload state in Redis so creators can resume after connection drop.
- DRM: encrypt segments with AES-128 or Widevine/FairPlay. Store encryption keys in a Key Management Service separate from the CDN.
- Analytics pipeline: stream watch events (start, pause, seek, end) to Kafka → data warehouse for content analytics and ML training.
- Cold storage: move videos with < 1 play/month to Glacier or equivalent cold storage to save costs.
Interview Tip
Interviewers care most about two things here: the upload pipeline (do you know about direct S3 uploads via pre-signed URLs rather than routing through your servers?) and the CDN/ABR strategy. Explicitly walk through how HLS adaptive bitrate works — it shows you understand actual video engineering, not just generic distributed systems concepts.
Practice this pattern
Design a video streaming platform like Netflix