Design Google Drive / Dropbox: File Sync System Design

"Design Dropbox" or "Design Google Drive" is a favourite for SDE-2 and senior rounds because the naive answer (upload the file to S3, store a row in MySQL) works for a demo and fails at everything that makes the product good: syncing a 2 GB video edit over a hotel Wi-Fi, keeping a laptop, phone and office desktop consistent, never losing a file when two people edit it offline, and storing exabytes without paying for the same PDF a million times.

This guide designs a sync-first system (Dropbox's core), then notes where Google Drive differs. Real-time co-editing inside a document is a separate problem, covered in Design Google Docs.

Requirements

Functional

• Upload, download, update, delete, move and rename files and folders from web, desktop and mobile.
• Automatic sync across a user's devices; offline edits sync when back online.
• Version history (restore any version from the last 30+ days) and trash.
• Share files and folders with users or via links, with viewer/editor roles.
• Files up to, say, 50 GB.

Non-functional

• Durability above all: never lose a committed file (eleven nines on content).
• Consistency of metadata: every device converges to the same tree; no silent overwrites.
• Bandwidth efficiency: transfer only what changed; resume interrupted transfers.
• Sync latency: a change on one device appears on others within seconds.
• Scale: hundreds of millions of users, exabytes of data.

Capacity Estimation

Two conclusions to state out loud: content storage is enormous and cold (most files are rarely read again), so it belongs in cheap, durable object storage with erasure coding and tiering. Metadata is much smaller but hot and transactional, so it needs a sharded database with strong consistency per user or team. The dedup percentage is an assumption; real savings depend heavily on workload (shared team folders and common installers dedup far better than personal photos).

The Core Idea: Blocks + Metadata

A file is not stored as a file. It is stored as:

• A set of blocks: chunks of up to ~4 MB, each named by the SHA-256 of its contents. Blocks are immutable: the same hash always means the same bytes, so they can be cached and deduplicated anywhere.
• A file version: metadata saying "/Projects/report.pdf, version 7, is blocks [h1, h2, h9, h4] in that order, 14.2 MB, modified by Asha at 10:42".

Every interesting property follows from this split. Updating one page of a PDF changes one block, so version 8 is [h1, h2, h12, h4]: one new block uploaded. Two users storing the same installer share all blocks. Restoring version 7 is a metadata operation. Resuming an interrupted upload means uploading the blocks not yet acknowledged. Dropbox has publicly described its architecture in these terms (a block server and a metadata server), with blocks historically sized at 4 MB.

API Design

# 1. Propose a commit: "this file should now be these blocks"
POST /v1/files/commit
  { namespace_id, path: "/Projects/report.pdf",
    base_version: 7,                  # what the client last synced
    block_hashes: ["h1","h2","h12","h4"], size: 14893211 }
  → 200 { status: "need_blocks", missing: ["h12"] }        # dedup check
  → 200 { status: "committed", version: 8, journal_seq: 99812 }
  → 409 { status: "conflict", server_version: 8 }

# 2. Upload missing blocks (directly to block storage via a pre-signed URL)
PUT  /v1/blocks/{sha256}            body: raw bytes (≤ 4 MB), verified on receipt

# 3. Pull changes
GET  /v1/namespaces/{ns}/changes?cursor=99790&limit=500
  → { entries:[{seq, path, op:"upsert"|"delete"|"move", version, block_hashes}],
      cursor: 99812, has_more: false }

# 4. Wait for changes (long poll)
GET  /v1/notify?cursors=ns1:99812,ns2:5531&timeout=60
  → { changes: true, namespaces:["ns1"] }  |  { changes: false } after timeout

GET  /v1/blocks/{sha256}            # download a block (CDN-cacheable, signed URL)

Notice the commit is idempotent: re-sending the same commit after a timeout returns the same version, because the server can detect that the current version already has exactly these blocks and the same client operation ID.

Data Model

namespaces      (namespace_id PK, owner_id, type: 'user_root'|'shared_folder'|'team')
files           (namespace_id, file_id, path, parent_id, is_dir, latest_version,
                 deleted, PRIMARY KEY (namespace_id, file_id),
                 UNIQUE (namespace_id, path))
file_versions   (namespace_id, file_id, version, size, block_hashes BLOB,
                 modified_by, device_id, created_at,
                 PRIMARY KEY (namespace_id, file_id, version))
journal         (namespace_id, seq BIGINT, file_id, op, version, ts,
                 PRIMARY KEY (namespace_id, seq))      -- append-only, per-namespace order
acl             (namespace_id, principal_id, role: 'owner'|'editor'|'viewer')
mounts          (user_id, namespace_id, mount_path)    -- where a shared folder appears

-- Block index (separate store, sharded by hash)
blocks          (sha256 PK, size, storage_key, refcount, created_at)

The namespace is the unit of sharding and consistency. A user's personal files form one namespace; each shared folder is its own namespace mounted into members' trees. All metadata for a namespace sits on one shard, so a commit (update files, insert file_versions, append to journal) is a single-shard transaction. The journal is the heart of sync: a gap-free, ordered log of every change in the namespace, which devices replay from their cursor. For sharding strategies, see database sharding explained.

High-Level Architecture

Deep Dive 1: Chunking Files

Fixed-size chunking

Split every 4 MB. Simple, fast, and perfect for files modified in place (databases, VM images, many binary formats) where edits overwrite bytes without shifting them. The weakness is insertions: add one byte at the start of a 100 MB text file and every subsequent block boundary shifts by one byte, so every block hash changes and the whole file re-uploads.

Content-defined chunking (CDC)

Choose boundaries based on content. Slide a small window (say 48 bytes) over the data computing a rolling hash; declare a boundary wherever hash & mask == 0. With a 20-bit mask, boundaries occur on average every 2²⁰ bytes = 1 MiB. After an insertion, the chunks after the edit re-synchronise at the same content-determined boundaries, so only the chunk(s) around the edit change. Enforce minimum and maximum chunk sizes to avoid tiny or huge chunks.

import hashlib
import random

WINDOW = 48
MASK = (1 << 13) - 1             # average chunk ~8 KiB for the demo (use ~2^20 in prod)
MIN_CHUNK, MAX_CHUNK = 2048, 65536
BASE, MOD = 257, (1 << 61) - 1
POW = pow(BASE, WINDOW, MOD)      # to remove the byte leaving the window

def cdc_chunks(data: bytes):
    chunks, start, h = [], 0, 0
    for i, byte in enumerate(data):
        h = (h * BASE + byte) % MOD
        if i - start >= WINDOW:                     # slide: drop oldest byte in window
            h = (h - data[i - WINDOW] * POW) % MOD
        size = i - start + 1
        if (size >= MIN_CHUNK and (h & MASK) == 0) or size >= MAX_CHUNK:
            chunks.append(data[start:i + 1])
            start, h = i + 1, 0
    if start < len(data):
        chunks.append(data[start:])
    return chunks

def block_hashes(data):
    return [hashlib.sha256(c).hexdigest() for c in cdc_chunks(data)]

random.seed(7)
original = bytes(random.getrandbits(8) for _ in range(500_000))
edited = original[:1000] + b"INSERTED A NEW PARAGRAPH" + original[1000:]

a, b = block_hashes(original), block_hashes(edited)
print(len(a), len(b), "new blocks:", len(set(b) - set(a)))   # only ~1 new block

Run this and you will see the edited file differs from the original in about one block, although every byte after offset 1,000 moved. With fixed 8 KiB blocks, nearly all of the ~60 blocks would be new. Production systems use faster rolling hashes (Rabin fingerprints, Gear/FastCDC), but the principle is identical. The rsync algorithm achieves a similar effect differently: it compares rolling checksums of the old file's blocks at every offset of the new file.

Picking a chunk size

Smaller chunks give finer-grained dedup and smaller delta uploads but multiply metadata (a 50 GB file at 4 MB is 12,800 hashes; at 64 KB it is 800,000) and object storage requests. A few MB is a common compromise for file sync; backup systems that prize dedup often use smaller average chunks.

Deep Dive 2: Block-Level Deduplication

Dedup is the commit protocol's "need_blocks" step: the server checks each proposed hash against the block index and replies with only the missing ones. Three scopes:

To use global dedup safely, require the client to actually upload the block the first time that user references it (proof of possession), or restrict dedup to within an account or team. Encryption interacts too: if each user's blocks are encrypted with a user key, identical plaintext produces different ciphertext and cross-user dedup disappears; convergent encryption (key derived from content hash) restores dedup but reintroduces the probing risk.

Garbage collection: when versions expire from history and files are purged from trash, block refcounts drop. Deleting a block with refcount 0 races with a concurrent commit that references it, so use a grace period (mark, wait days, re-check, then delete) instead of immediate deletion. The block index lookup can be fronted by a Bloom filter so most "is this new?" checks avoid a database read.

Deep Dive 3: Upload and Sync Flow

1. Detect. The desktop client's file watcher (FSEvents, inotify, ReadDirectoryChangesW) notices report.pdf changed. It waits briefly for writes to settle so it does not sync a half-saved file.
2. Chunk and hash locally; compare with the local index to find changed blocks.
3. Commit attempt with base_version and the full block list. Server checks ACL, then the block index.
4. Upload missing blocks in parallel, directly to object storage via pre-signed URLs. The block service verifies each block's SHA-256 before acknowledging. Interrupted uploads resume block by block.
5. Re-commit. Now all blocks exist; the metadata service, in one transaction, verifies base_version == latest_version, inserts the new version, updates the file row, and appends a journal entry with the next sequence number.
6. Notify. A commit event goes to Kafka; the notification service wakes long-poll connections of every device mounted on that namespace.
7. Other devices pull. Each calls changes?cursor=…, receives the new version's block list, downloads only blocks it does not already have locally, assembles the file in a temp location and atomically renames it into place.

Mobile clients differ: they usually do not sync everything. They browse metadata, download on demand, and upload camera photos, so the same APIs serve both modes. Large downloads are served via a CDN with signed URLs, since blocks are immutable and cache perfectly.

Deep Dive 4: Change Notifications with Long Polling

A hundred million devices must hear about changes within seconds, but each device sees a change rarely. Options:

Dropbox has publicly described using long polling for its notification service. The client sends GET /notify with its cursors and the server holds the request open for up to ~60 seconds. If a namespace changes, the server responds immediately with "changed"; the client then fetches changes through the normal metadata API and opens a new long poll. The notification payload stays tiny and carries no file data: it is only a doorbell. This keeps the notification tier simple and stateless apart from the connection map.

Scaling: an event-driven server can hold hundreds of thousands of idle connections. 100M connected clients ÷ ~500K per server ≈ 200+ servers. A subscription map (namespace → connections) lives in each server; commit events are routed to the servers holding interested connections via a pub/sub layer. Add jitter to timeouts so millions of polls do not expire in lockstep. More on the protocol trade-offs in WebSockets vs SSE vs long polling.

Deep Dive 5: Sync Conflicts

Asha edits budget.xlsx on her laptop on a flight. Rahul edits the same shared file from the office. Both are based on version 7.

1. Rahul's commit arrives first: base_version 7 == latest 7 → committed as version 8.
2. Asha lands and her client commits with base_version 7, but latest is now 8 → 409 conflict.
3. The client downloads version 8 into budget.xlsx and saves Asha's edits as budget (Asha's conflicted copy 2026-10-10).xlsx, which syncs as a new file.
4. Both versions exist; nobody's work is lost; a human merges.

Why not merge automatically? For arbitrary binary files there is no safe generic merge; an automatic merge that corrupts a spreadsheet is far worse than an extra file. "First committer wins, others get a copy" is a deliberate choice. (Last-writer-wins would silently discard Rahul's work, which is unacceptable for a storage product.)

Other conflict cases

• Edit vs delete: one device deletes while another edits; prefer keeping the edited file (resurrect it) since deletion is recoverable from trash anyway.
• Rename/move races: two devices move the same folder to different places; the journal order decides and the loser's client replays the server state.
• Case-insensitive file systems: Report.pdf and report.pdf can coexist on Linux but collide on Windows and macOS; the client must rename one locally.

The server-side check is classic optimistic concurrency control: a conditional update keyed on version, like UPDATE files SET latest_version = 8 WHERE file_id = ? AND latest_version = 7. The same compare-and-set pattern appears in booking and payment designs.

Deep Dive 6: Sharing and Access Control

Sharing a folder in a Dropbox-style design makes it its own namespace with an ACL: (namespace, principal, role). Each member gets a mount placing that namespace somewhere in their tree. Since all sync is per namespace, a member's device simply follows one more journal. Unsharing removes the ACL row and mount; devices delete the local copy on next sync.

Google Drive's model is more granular: any item can have its own permissions, inherited down the folder tree, and files can live in shared drives owned by an organisation. Checking "can Asha view this file?" then means walking ancestors and group memberships. Google described its global authorization system, Zanzibar, in a 2019 paper: relationship tuples like doc:123#viewer@group:eng#member evaluated with caching and consistency tokens. For an interview, you can say: store relationship tuples, evaluate with a cached graph check, and invalidate on changes.

• Shared links: an unguessable token (128+ bits of randomness) mapped to (item, role, expiry, password hash optional). Anyone with the link is a principal.
• Enforcement everywhere: every metadata read and every block download must be authorised. Block URLs are short-lived signed URLs issued only after an ACL check, because the hash alone must never be a capability.
• Audit: log shares, unshares and downloads for enterprise compliance.
• Identity: users, groups and SSO for organisations; see authentication and SSO design.

Interview Tips, Bottlenecks and Common Mistakes

Bottlenecks to raise yourself

• Huge shared namespaces (a company-wide folder with millions of files and thousands of members): one shard becomes hot. Split very large team spaces into multiple namespaces or shard the journal by subtree.
• First sync of a new device downloading 200 GB: throttle, prioritise recently used files, and allow selective sync.
• Thundering notifications: one commit to a folder shared with 50,000 employees wakes 50,000+ devices, all calling changes at once. Jitter the client fetch and cache the journal tail.
• Small files: millions of 2 KB files make per-block object storage overhead dominate; pack small blocks into larger storage objects.
• Cold storage: move rarely accessed blocks to cheaper tiers; see object storage design for erasure coding and tiering.

What interviewers look for

• The blocks vs metadata split and content addressing, stated early.
• A precise commit protocol: dedup check, upload, conditional commit, journal append.
• Cursor-based sync from an ordered journal rather than "diff the whole tree".
• A clear conflict policy with reasons.
• Awareness of security: ACL on every access, signed block URLs, dedup side channels.

If the interviewer pivots to "now let two people type in the same document at once", that is the cue to move from block-level sync to operation-level merging: read real-time collaborative editing with OT and CRDTs.

Frequently Asked Questions

Pranjal Jain

Ex-Microsoft SDE · IIT Kanpur · Founder of Prepflix. Helps engineers crack startup and product company interviews across India.