Design Instagram Reels / YouTube Shorts

Instagram Reels, YouTube Shorts, TikTok, Moj, Josh, and ShareChat's short video products all solve the same problem: an infinite vertical feed of 15 to 90 second videos where each swipe must start playing instantly, the content is overwhelmingly user-generated and fresh, and the feed has to learn what you like within a handful of swipes.

If you have already prepared YouTube/Netflix video streaming, much of the pipeline carries over. This post focuses on what changes for short video, which is precisely what interviewers probe when they ask this as a separate question.

Why Short Video Is a Different Design Problem

Requirements

Functional

• Creators record or upload a vertical video (up to ~90 s), add music, captions, effects, and publish.
• Viewers scroll a personalised, infinite feed; videos autoplay and loop.
• Like, comment, share, save, follow; view and like counts displayed.
• Creator analytics (views, watch time, reach).

Non-functional

• Video start time after a swipe: target under ~200 ms perceived (effectively instant) for prefetched videos.
• Rebuffering rate near zero; graceful adaptation on 3G/4G networks common outside metros.
• Upload to "visible to followers" in under a minute for typical clips.
• High availability for playback; feed can degrade (less personalised) rather than fail.
• Cost efficiency: egress bandwidth is the dominant cost.

Capacity Estimates

Illustrative numbers for a platform at Reels-like scale (assumptions, not published figures):

• 300M DAU, ~100 reels viewed per user per day → 30B views/day ≈ 350k views/s, peaks around 1M/s in the evening.
• Average reel 30 s; average delivered bitrate ~1.5 Mbps (720p H.264 on mobile) → ~5.6 MB for a full view. Users skip a lot, so assume ~4 MB delivered per view on average, including prefetched-but-skipped bytes.
• Egress: 30B × 4 MB = 120 PB/day ≈ 1.4 TB/s ≈ 11 Tbps average, much more at peak. This number alone justifies a deep CDN discussion and codec efficiency work.
• Uploads: 1% of DAU uploads one reel/day → 3M uploads/day ≈ 35/s average, spiky (evenings, festivals, trends).
• Storage: originals ~30 MB each → 90 TB/day; renditions across the ladder and codecs maybe ~20 MB → 60 TB/day. ~150 TB/day, ~55 PB/year. Originals can move to cold storage after transcoding.
• Transcoding compute: if one reel needs ~2 CPU-minutes per rendition-codec pair and we produce ~10 outputs, that is 20 CPU-minutes × 3M = 60M CPU-minutes/day ≈ 42,000 cores busy around the clock (lower with hardware encoders). Not trivial.
• Engagement events: views, likes, watch-time pings: several million events/s into the analytics log.

API and Data Model

Core tables

• reels (sharded by reel_id): author_id, caption, audio_id, status (PROCESSING/LIVE/BLOCKED), renditions (manifest paths), duration, created_at.
• reels_by_author: (author_id, created_at desc) → reel_id, for profile grids.
• likes: primary key (reel_id, user_id) for idempotency; plus (user_id, reel_id) for "liked by you".
• reel_counters: reel_id → views, likes, comments, shares (eventually consistent, aggregated).
• watch_history: per user recent reel ids (for "don't show again" and ranking features).

Ids should be time-sortable so "newest reels by author" is an index range scan; see Snowflake-style ID generation.

High-Level Architecture

Upload and Transcoding Pipeline

Upload

1. Client-side preparation: the app trims, applies effects and often re-encodes on device to a reasonable bitrate before upload. This saves upload time on slow networks and server compute.
2. Resumable chunked upload directly to object storage via presigned URLs (multipart under the hood; see object storage design). A dropped connection resumes from the last acknowledged chunk.
3. Publish request with metadata creates the reel record in PROCESSING. The creator sees "Your reel is being processed".
4. Object-created event enqueues a processing job, deduplicated by upload_id.

The processing DAG

Processing is a workflow of tasks, some sequential and some parallel, run by an orchestrator (a job queue with a DAG engine, or something like Temporal/Airflow-style workers specialised for media):

Encoding ladder for vertical video

These are typical ranges, not any platform's actual settings. Many platforms use per-title (content-aware) encoding: a static talking-head clip needs far fewer bits than a dance video with confetti, so bitrates are chosen per video.

ffmpeg -i input.mp4 \
  -vf "scale=720:-2" -c:v libx264 -profile:v main -preset veryfast \
  -crf 23 -maxrate 2000k -bufsize 4000k \
  -g 60 -keyint_min 60 -sc_threshold 0 \
  -c:a aac -b:a 96k \
  -f hls -hls_time 2 -hls_playlist_type vod \
  -hls_segment_filename "720p_%03d.ts" 720p.m3u8

Two details worth saying in an interview: scale=720:-2 sets the width to 720 for a vertical video (a common bug is -2:720, which yields a tiny 405-pixel-wide output), and fixed GOP length (-g 60 at 30 fps = 2 s, scene-cut keyframes disabled) keeps keyframes aligned across renditions so the player can switch bitrate at segment boundaries.

Priorities and backlog

Uploads spike during trends and festivals. Use priority queues: fast-path transcodes first; creators with large followings ahead (their reels will be viewed immediately); expensive AV1 encodes deferred and done only for reels that cross a view threshold. Autoscale transcoding workers on queue depth; they are stateless and can run on spot/preemptible capacity since tasks are retryable.

Feed Generation and Ranking

The Reels feed is mostly unconnected content: videos from accounts you do not follow, chosen by a recommender. The full design of that recommender (candidate generation, ranking, re-ranking, cold start) is in design a recommendation system; here is what matters specifically for short video.

• Strong implicit signals: watch time relative to duration, completion, loops (rewatching), quick skips (negative), shares, saves, follows after watching. Each reel produces several labels within seconds, so models learn fast.
• Session-aware ranking: the next page should react to what you did in the last 20 swipes. The client sends recent events with the feed request, and nearline features update within seconds.
• Fresh content cold start: new reels get a small exploration audience chosen by content embeddings and creator history; if early engagement is strong, distribution widens in stages.
• Page size and lookahead: return ~10 reels per page and request the next page when the user is ~3 reels from the end, so ranking latency (~100 to 200 ms) is hidden.
• Diversity rules: avoid the same creator or audio back-to-back; mix in followed creators' new reels.
• Dedup: do not re-show watched reels; a per-user seen set (recent list plus a Bloom filter) filters candidates.

Each feed item carries signed URLs (short expiry) for its manifest and poster, plus a small JSON with the URLs of the first segment of each rendition so the client can prefetch without fetching the manifest first. That saves one round-trip per video, which at 50 to 100 ms each matters when you swipe every few seconds.

Prefetching: The Secret to Instant Playback

Without prefetching, every swipe costs: DNS/TLS (if cold) + manifest fetch + first segment fetch + decoder start. On a 4G connection with 80 ms RTT and 5 Mbps of real throughput, a 2-second 1.5 Mbps segment (~375 KB) alone takes ~600 ms, plus round-trips: roughly a second. Users perceive that as broken.

With prefetching, the first segment of the next reels is already on the device when the swipe happens, so start time is just decoder startup.

A practical prefetch policy

def plan_prefetch(queue, bandwidth_kbps, on_wifi, data_saver, swipe_rate_per_min):
    """Return (reel_id, rendition, bytes) prefetch tasks for upcoming reels.
    queue: upcoming reels, each a dict with 'id' and 'first_segment' sizes per rendition."""
    if data_saver:
        depth, rendition, seconds = 1, "low", 1
    elif on_wifi:
        depth, rendition, seconds = 5, "high", 4
    else:
        depth = 3 if bandwidth_kbps > 3000 else 2
        rendition = "medium" if bandwidth_kbps > 1500 else "low"
        seconds = 2
    if swipe_rate_per_min > 20:          # user is skimming: fetch less of each
        seconds = 1
    tasks = []
    for reel in queue[:depth]:
        per_second = reel["first_segment"][rendition] // 2   # segments are ~2 s
        tasks.append((reel["id"], rendition, per_second * seconds))
    return tasks

Design principles

• Prefetch the beginning, not the whole video. Most reels are skipped within seconds; full downloads waste the user's data and your egress.
• Start low, upgrade fast. Prefetch a lower rendition for instant start; the adaptive bitrate (ABR) logic switches up at the next segment if bandwidth allows.
• Cancel aggressively. When the user scrolls past several reels quickly, cancel in-flight prefetches for reels they have skipped.
• Prioritise the current video. Prefetch traffic must never starve the segment the user is watching; use lower request priority or pause prefetch when the playback buffer is low.
• Respect data costs. In India, many users are on prepaid data packs; a data saver mode with shallow, low-bitrate prefetch is not optional.
• Warm everything else too: keep HTTP/2 or HTTP/3 connections to the CDN open, pre-resolve DNS, preload posters, and keep a small pool of initialised video decoders/players so swapping is instant.

Segments vs progressive MP4

For very short clips some players skip manifests and use a single MP4 with the moov atom at the front (faststart) and HTTP range requests: fetch the first few hundred KB to prefetch, continue with range requests on play. It is simpler and saves a manifest round-trip, but adapting bitrate mid-clip is harder. Adaptive segmented streaming (HLS/DASH with CMAF) is better for longer reels and poor networks. Both are valid; say why you pick one.

CDN Strategy for Short Video

At ~11 Tbps average, the CDN is the system. Short video stresses CDNs differently from long-form: a much larger set of distinct objects, a long tail of rarely watched reels, and sudden virality.

Tiered caching

• Edge caches inside ISPs: large platforms place cache appliances inside ISP networks (Netflix's Open Connect and Meta's network appliances are publicly documented examples) to keep traffic off the backbone. Jio, Airtel and Vi carry most mobile traffic in India, so these placements matter.
• Origin shield + request coalescing: when a reel goes viral, thousands of edges miss at once. The shield collapses concurrent misses for the same segment into one origin fetch.
• Popularity-aware admission: do not let the long tail evict hot content. Edges can admit an object only on its second request within a window, while regional tiers keep a broader set.
• Proactive warming: when a creator with millions of followers publishes, or when the ranking system starts widening a reel's distribution, push its first segments to regional caches before the views arrive.
• Cache keys without user-specific query strings: signed URLs must be validated at the edge with the signature excluded from the cache key, or every user gets a cache miss.
• Multi-CDN: route traffic across your own and third-party CDNs by real-user measurements of latency and error rates per ISP and city.

Engagement Counters at Scale

A viral reel can receive tens of thousands of views and likes per second. UPDATE reels SET likes = likes + 1 WHERE id = ? on one row serialises all of them on a single lock: the classic hot-row problem.

Separate the record from the count

• The like record (reel_id, user_id) is written with an idempotent upsert; it answers "did I like this?" and prevents double counting when the user taps twice or the client retries.
• The count is derived asynchronously. The like service emits a LIKE +1 or UNLIKE −1 event only when the record actually changes state.

Aggregate before writing

import time
from collections import defaultdict

class CounterAggregator:
    """Consumes like/view events, sums deltas per reel, flushes in batches."""
    def __init__(self, store, flush_every_s=2.0, max_keys=50_000):
        self.store, self.flush_every_s, self.max_keys = store, flush_every_s, max_keys
        self.pending = defaultdict(lambda: defaultdict(int))
        self.last_flush = time.monotonic()

    def on_event(self, reel_id, metric, delta):
        self.pending[reel_id][metric] += delta
        if (len(self.pending) >= self.max_keys or
                time.monotonic() - self.last_flush >= self.flush_every_s):
            self.flush()

    def flush(self):
        if self.pending:
            # one write per reel per flush, e.g. Redis HINCRBY in a pipeline or a
            # Cassandra counter update; 10,000 likes/s on a reel become 1 write per 2 s
            self.store.increment_many({r: dict(m) for r, m in self.pending.items()})
        self.pending.clear()
        self.last_flush = time.monotonic()
        # commit Kafka offsets only after a successful flush (at-least-once);
        # make increments idempotent per (partition, offset range) if exactness matters

Partition the event topic by reel_id so one aggregator owns each reel's deltas. For a single reel so hot that one partition struggles, add a second level: split the key into N sub-keys (reel:42#0…#7), aggregate each, and sum on read.

Views are trickier than likes

• Define a view: e.g. played for at least a few seconds or a fraction of the duration, counted client-side and validated server-side for plausibility.
• Dedupe and fraud: cap views per user per reel per time window; filter bot traffic before counts become ranking features or creator payouts.
• Display vs truth: the displayed count can lag by seconds and be rounded ("1.2M"). Creator analytics and any monetisation use a separate, exact batch pipeline over the event warehouse, reconciled daily.

Failure Modes and Cost Levers

Cost levers (egress dominates)

• Codec efficiency: HEVC and AV1 can deliver similar quality at noticeably lower bitrates than H.264, at higher encode cost. Encode them for the popular minority of reels that drive most views.
• Prefetch discipline: bytes prefetched for reels never watched are pure waste.
• Cache hit ratio: every percentage point at the edge saves backbone and origin bandwidth.
• Storage tiering: originals and unpopular renditions to cold storage; regenerate rare renditions on demand.

Interview Guide: What to Emphasise

1. State the difference from long-form video up front: startup per swipe, UGC volume, freshness.
2. Estimate egress and let it drive the CDN and codec discussion.
3. Walk the upload DAG with a fast path and moderation gating.
4. Explain prefetching as a policy with trade-offs (depth, bitrate, cancellation, data saver).
5. Show the counters design: idempotent records plus asynchronous aggregated counts.
6. Close with failure modes and what degrades gracefully.

Follow-ups you should expect

• "How do you make a new reel visible in under a minute?" Client-side encode, resumable upload, fast-path single rendition, moderation on the fast path for wide distribution, rest of the ladder later.
• "How would you build 'remix' or 'use this audio'?" Audio is a first-class entity with an id; reels reference it; an audio page lists reels by audio id with its own ranking; trending audios come from a heavy-hitters pipeline like trending topics.
• "Live streams?" A different ingest and low-latency delivery path; see Hotstar live streaming.
• "How do followers' new reels get mixed in?" A followed-creators candidate source, using the fan-out ideas from news feed design.

Common mistakes

• Copy-pasting a YouTube design without discussing prefetching or per-swipe startup.
• Uploading videos through application servers instead of directly to object storage.
• Transcoding the full ladder synchronously before publishing.
• Incrementing like counts directly on the reel row.
• Ignoring mobile data costs and network variability in Indian markets.
• Forgetting moderation, which is a regulatory and brand-safety necessity for UGC platforms.

Frequently Asked Questions

Pranjal Jain

Ex-Microsoft SDE · IIT Kanpur · Founder of Prepflix. Helps engineers crack startup and product company interviews across India.