System Design Fundamentals
Top 30 Concepts

What Is System Design?

Imagine you are asked to build a house. You would not just start stacking bricks randomly. You would first think about how many rooms you need, where the kitchen should go, how the plumbing connects, and how the electrical wiring runs through the walls. System Design is exactly like that, but for software. It is the process of defining the architecture, components, modules, interfaces, and data flow of a system to satisfy specified requirements.

In the world of software engineering, System Design refers to the art and science of designing the backbone of applications that can handle millions (or even billions) of users, process enormous amounts of data, and remain available around the clock. Think about the applications you use daily: Instagram, YouTube, Amazon, WhatsApp. Under the hood, these are not simple programs running on a single computer. They are massive, distributed systems made up of hundreds or thousands of servers, databases, caches, and queues, all working together seamlessly.

Why Should You Learn System Design?

1. It Is Critical for Technical Interviews
Almost every major technology company, including Google, Amazon, Meta, Netflix, Microsoft, and Apple, includes System Design as a key round in their interview process. Unlike coding interviews where there is often a single correct answer, System Design interviews test your ability to think broadly, make trade-off decisions, and communicate your architectural reasoning clearly.

2. It Makes You a Better Engineer
Understanding how large-scale systems work gives you a mental model for debugging production issues, evaluating new technologies, and making architectural decisions at work.

3. It Helps You Build Real Products
If you ever want to build your own product or startup, you need to know how to architect a system that can scale. You cannot redesign your entire architecture every time your user base doubles.

4. It Bridges the Gap Between Theory and Practice
Computer Science teaches you algorithms and data structures. System Design teaches you how to put those building blocks together to create real, working, large-scale applications.

How to Approach System Design

There Is No Single Right Answer. Unlike a coding problem where you either pass or fail test cases, System Design is open-ended. Two excellent engineers might design the same system differently, and both designs could be perfectly valid.

Always Start with Requirements. Before drawing any boxes or arrows, ask: What exactly are we building? System Design questions are intentionally vague. Your first job is to clarify:

• Functional Requirements: What should the system do? For example, users should be able to upload photos, follow other users, and see a news feed.
• Non-Functional Requirements: How well should the system perform? For example, 100 million daily active users, less than 200ms latency, 99.99% uptime.

Think in Terms of Trade-Offs. Want faster reads? You might sacrifice write speed. Want higher availability? You might have to accept occasional stale data. The best engineers understand the constraints and make informed compromises.

Almost every web application you use today is built on a fundamental pattern called Client-Server Architecture. This is the starting point for understanding how the internet works.

In this model, there are two main players. The Client is any device or application that makes a request — your web browser, a mobile app, or a command-line tool. The Server is a powerful computer that receives these requests, processes them, and sends back a response.

The beauty of this model is the separation of concerns. The client handles the user interface and experience. The server handles business logic, data storage, and heavy processing. This separation means you can update the server without changing the client, and vice versa.

Every device connected to the internet has a unique identifier called an IP (Internet Protocol) Address. Think of it like a postal address for your computer. When you visit a website, your computer needs to know the exact IP address of the server hosting that website.

But IP addresses are numbers like 142.250.190.46. Nobody wants to type numbers every time they visit a website. This is where DNS comes in.

Domain Name System (DNS) is essentially the phonebook of the internet. It translates human-readable domain names (like google.com) into the numerical IP addresses that computers use.

When you type www.google.com into your browser, a complex but lightning-fast process happens:

1. Your browser checks its local cache to see if it already knows the IP address.
2. If not found, it asks your operating system, which checks its own cache.
3. The request goes to a DNS Resolver (usually provided by your ISP or a service like Google DNS at 8.8.8.8).
4. The resolver queries the Root DNS Servers, which direct it to the appropriate TLD server (for .com, .org, etc.).
5. The TLD server points to the Authoritative DNS Server for google.com.
6. The authoritative server returns the actual IP address.
7. Your browser can now connect to that IP and load the website.

An API is a contract between two pieces of software that defines how they communicate. If Client-Server Architecture tells you who is talking, APIs define what they are saying and how they are saying it.

Think of an API like a waiter in a restaurant. You (the client) do not go into the kitchen to cook your food. Instead, you give your order to the waiter (the API), who takes it to the kitchen (the server), and brings back your food (the response).

REST APIs are the most common API design pattern on the web. They use standard HTTP methods:

GraphQL is a newer alternative to REST, developed by Facebook. Instead of multiple endpoints, GraphQL provides a single endpoint where the client specifies exactly what data it needs — solving over-fetching and under-fetching.

gRPC is a high-performance API framework by Google. It uses Protocol Buffers (a compact binary format) and HTTP/2, making it significantly faster. Widely used for internal communication between microservices.

Every application needs to store data. The two major categories are SQL (relational) and NoSQL (non-relational), and choosing between them is one of the most important design decisions you will make.

SQL databases store data in structured tables with predefined schemas. Each table has rows and columns, and tables can be linked together through relationships. Examples: MySQL, PostgreSQL, Oracle. They follow ACID properties and guarantee strong consistency.

NoSQL databases were born from the need for greater flexibility and scalability. They come in several flavors:

• Key-Value Stores (Redis, DynamoDB): The simplest model. Blazing fast for lookups.
• Document Stores (MongoDB, CouchDB): Store data as flexible JSON-like documents.
• Column-Family Stores (Cassandra, HBase): Organize data by columns. Excellent for massive distributed datasets. Used by Netflix and Uber.
• Graph Databases (Neo4j, Amazon Neptune): Designed for data with complex relationships, like social networks.

When your application grows and starts getting more traffic than a single server can handle, you need to scale. There are two fundamental approaches.

Vertical Scaling (Scaling Up) means making your existing server more powerful by adding more CPU, RAM, or storage. It is like replacing your sedan with a sports car. Simple — no code changes needed — but there is a hard limit and it creates a single point of failure.

Horizontal Scaling (Scaling Out) means adding more servers to handle the load. Instead of one powerful server, you have ten (or a hundred) smaller servers working together. Virtually unlimited scaling and no single point of failure, but much more complex to manage.

Once you have multiple servers (horizontal scaling), you need a way to distribute incoming requests across them. This is exactly what a Load Balancer does. It sits between the clients and your pool of servers, receiving all incoming requests and forwarding each one to an appropriate server.

Common Load Balancing Algorithms:

• Round Robin: Requests distributed sequentially — Server 1, Server 2, Server 3, repeat. Like dealing cards.
• Weighted Round Robin: Powerful servers get more requests based on their capacity.
• Least Connections: Sends requests to the server with the fewest active connections. Better for varying processing times.
• IP Hash: Client's IP determines the server, ensuring session persistence.

Caching is one of the most powerful performance optimization techniques in System Design. The core idea: store frequently accessed data in a fast, temporary storage layer so you do not have to fetch it from the slower original source every time. Think of it like writing a formula on a sticky note instead of flipping through the textbook each time.

Types of Caches:

• Browser Cache: Your browser stores images, CSS, and JavaScript locally for fast revisits.
• Distributed Cache (Redis, Memcached): A shared cache accessible by multiple servers. Redis can serve millions of read operations per second.
• CDN Cache: Caches static content at servers geographically close to users.

Caching Strategies:

• Cache-Aside (Lazy Loading): Check cache first. On miss, fetch from DB, store in cache, return. Most common strategy.
• Write-Through: Every write goes to both cache and database simultaneously. Always up-to-date but adds write latency.
• Write-Back: Writes go to cache first, then async to database. Very fast writes, but risk of data loss on cache crash.

Cache Eviction Policies:

• LRU (Least Recently Used): Removes data not accessed for the longest time. Most commonly used.
• LFU (Least Frequently Used): Removes data accessed the fewest times.
• TTL (Time to Live): Data automatically expires after a set duration.

A CDN is a network of servers distributed across the globe that delivers content to users from the server geographically closest to them. The primary goal is to reduce latency.

What CDNs Cache:

• Static content: images, videos, CSS files, JavaScript files, fonts
• Dynamic content: some CDNs can also cache API responses and dynamic HTML
• Streaming content: video chunks for services like Netflix and YouTube

A proxy is an intermediary server that sits between a client and a destination server, forwarding requests and responses between the two.

Forward Proxy sits between clients and the internet. The destination server sees the proxy's IP, not the client's. Used for anonymity, content filtering, and bypassing geographic restrictions.

Reverse Proxy sits in front of web servers. The client never knows about backend servers. Used for load balancing, SSL termination, caching, security, and compression. When you visit a website, you are almost certainly talking to a reverse proxy (like NGINX or Cloudflare).

The CAP Theorem is one of the most important theoretical foundations in distributed systems. It states that in any distributed data store, you can only guarantee two out of three properties simultaneously:

• Consistency (C): Every read receives the most recent write. All nodes see the same data at the same time.
• Availability (A): Every request receives a response, even if the data might not be the most recent.
• Partition Tolerance (P): The system continues to function even when there is a network failure between nodes.

Since network failures are inevitable in real-world distributed systems, Partition Tolerance is always required. The real choice is between Consistency and Availability:

ACID is a set of properties that guarantee reliable processing of database transactions. It is the gold standard for data integrity in SQL databases:

• Atomicity: All-or-nothing. Either all operations succeed, or none do. A bank transfer either moves money completely, or not at all.
• Consistency: A transaction brings the database from one valid state to another. All rules are maintained.
• Isolation: Concurrent transactions do not interfere with each other.
• Durability: Once committed, data is permanently saved, even if the system crashes immediately after.

Data Replication is the practice of storing copies of the same data on multiple servers (replicas). It improves availability (if one server dies, others have the data) and performance (read requests can be distributed across replicas).

Leader-Follower (Master-Slave) Replication: One server is the Leader — all writes go there. Changes replicate to Followers. Read operations can be served by any follower, distributing the read load. Simple but has replication lag and requires failover if the leader goes down.

Leader-Leader (Multi-Master) Replication: Multiple servers accept writes, each replicating changes to the others. Higher write availability but complex conflict resolution when two leaders receive conflicting writes simultaneously.

When your database grows so large that a single server cannot handle it, you split the data across multiple servers. This is Sharding. Think of a library splitting across multiple buildings by subject.

Sharding Strategies:

• Range-Based Sharding: Users A-M go to Shard 1, N-Z to Shard 2. Simple but can lead to uneven distribution (hotspots).
• Hash-Based Sharding: A hash function determines the shard. More uniform distribution but makes range queries difficult.
• Directory-Based Sharding: A lookup table maps each piece of data to its shard. Flexible but introduces a single point of failure.

Consistent Hashing solves a critical problem: how to distribute data across multiple servers while minimizing disruption when servers are added or removed. With regular hashing (hash(key) % N), adding a server changes N and remaps almost all keys. Consistent hashing avoids this.

How It Works: Imagine a circular number line (a ring). Both servers and data keys are hashed onto positions on this ring. To find which server stores a key, walk clockwise from the key's position until you find a server. When you add a new server, only keys between it and the previous server are remapped. The rest stay put.

Virtual Nodes: Each physical server maps to multiple positions on the ring (virtual nodes), ensuring a more balanced distribution of keys across servers.

A database index dramatically speeds up data retrieval at the cost of additional storage and slightly slower writes. Without an index, the database must scan every row (full table scan). With an index, it jumps directly to the relevant rows.

How Indexes Work: Most databases use B-Trees for indexes. This tree structure keeps data sorted and allows searches in logarithmic time. Instead of scanning millions of rows, the database traverses a tree just a few levels deep. Hash Indexes are extremely fast for exact-match lookups but do not support range queries.

A Message Queue allows different parts of a system to communicate asynchronously. Instead of Service A directly calling Service B, Service A places a message in a queue, and Service B picks it up when ready. Think of it like a mailbox.

Key Benefits:

• Decoupling: Services do not need to know about each other. Either side can be updated or scaled independently.
• Buffering: If the consumer is slow, messages pile up instead of overwhelming it.
• Resilience: If the consumer crashes, messages remain safely in the queue until recovery.

Popular message queue systems include RabbitMQ, Amazon SQS, Apache Kafka, and Redis.

Pub/Sub is a messaging pattern where publishers send messages to a topic, and any number of subscribers receive them. Unlike a traditional queue where each message is consumed by one consumer, in Pub/Sub a single message can be delivered to many subscribers simultaneously.

Popular Pub/Sub systems include Apache Kafka, Google Cloud Pub/Sub, Amazon SNS, and Redis Pub/Sub.

An API Gateway is a server that acts as the single entry point for all client requests in a multi-service system. Instead of clients communicating with dozens of microservices directly, they send all requests to the gateway, which routes them appropriately.

What an API Gateway Does:

• Request Routing: Routes requests to the correct backend service based on URL path, headers, or other criteria.
• Authentication & Authorization: Verifies identity and permissions once at the gateway level.
• Rate Limiting: Prevents any single client from overwhelming the system.
• Response Aggregation: Fans out to multiple services, collects responses, and returns a unified response.
• SSL Termination: Handles HTTPS encryption so backend services can communicate over plain HTTP internally.

Microservices Architecture builds a large application as a collection of small, independent services, each responsible for a specific business function, running in its own process, and communicating over the network via APIs.

Rate Limiting controls how many requests a client can make within a given time period. It prevents abuse, ensures fair usage, and protects servers from being overwhelmed.

Common Algorithms:

• Token Bucket: A bucket holds tokens; each request consumes one. Tokens are added at a fixed rate. Allows bursts up to bucket capacity. Most widely used.
• Leaky Bucket: Requests enter a queue and are processed at a fixed rate, smoothing burst traffic into a steady flow.
• Fixed Window: Divides time into fixed intervals and counts requests per window. Simple but allows bursts at window boundaries.
• Sliding Window: A rolling window that smooths out the boundary issue. More accurate but slightly more complex.

WebSockets provide full-duplex, bidirectional communication over a single, long-lived TCP connection. Unlike HTTP (where the client must initiate every interaction), WebSockets let both sides send data at any time. HTTP is like sending letters; WebSockets are like a phone call.

When to Use WebSockets:

• Real-time chat (WhatsApp Web, Slack, Discord)
• Live sports scores and stock tickers
• Multiplayer online games
• Collaborative editing (Google Docs)
• Live notifications and alerts

Short Polling: The client repeatedly sends requests at fixed intervals (e.g., every 2 seconds). Most responses are empty. Simple but wasteful.

Long Polling: The client sends a request, and the server holds it open until there is new data. Reduces empty responses but ties up server resources.

In distributed systems, a Heartbeat is a periodic signal between components to indicate they are still operational. Like a doctor checking a pulse: a steady pulse means alive, a missing pulse means trouble.

• Push-Based: Each server sends "I am alive" signals to a monitor at regular intervals. Missing heartbeats trigger failure detection.
• Pull-Based: A monitoring service periodically pings each server. No response within timeout means the server is considered down.

When data travels across networks or is stored on disks, it can get corrupted. A Checksum is a small piece of data computed using a hash function (like MD5, SHA-256) used to verify data integrity. Compute before sending, recompute after receiving. If they match, data is intact.

A Bloom Filter is a space-efficient probabilistic data structure that tells you whether an element is "possibly in a set" or "definitely not in a set." It can have false positives but never false negatives.

How It Works: It uses a bit array and multiple hash functions. To add an element, pass it through all hash functions and set those bit positions to 1. To check membership, verify all positions are 1. If any is 0, the element is definitely not in the set. If all are 1, it is probably in the set (but might be a false positive).

• Latency: Time to complete a single operation. How long from click to response? Measured in ms. Lower is better.
• Throughput: Operations completed per unit time. How many requests per second? Higher is better.

• Availability: Percentage of time the system is operational. 99.9% = ~8.7 hours downtime/year.
• Reliability: Probability the system performs correctly over time. A system can be available but unreliable (responds but with wrong data).

Redundancy means having duplicate components. Failover is switching to a backup when the primary fails. Any component without a backup is a Single Point of Failure (SPOF).

• Active-Passive: Passive server sits idle, takes over when active fails. Simple but wastes resources.
• Active-Active: All servers handle traffic. If one fails, others absorb its load. Efficient but complex.

Strong Consistency: After a write, every read from any node returns the updated value. Strongest guarantee but higher latency. Use case: Banking.

Eventual Consistency: Replicas might temporarily differ but eventually converge (usually within seconds). Better performance and availability. Use case: Social media likes.

Causal Consistency: Causally related operations are seen in correct order. Unrelated operations may be seen in any order. Use case: Comment threads.

Idempotency means performing the same operation multiple times produces the same result as performing it once. Crucial in distributed systems where operations may be retried due to network failures.

Why It Matters: In a payment system, if a network timeout causes a retry, you do not want the user charged twice. The solution: include an Idempotency Key (a unique ID per request). The server checks if it already processed that key. If so, it returns the cached result without re-processing.

You have now covered the 30 most fundamental concepts in System Design. These are the building blocks that power every large-scale system on the internet. Every time you open Netflix, order food on Zomato, send a message on WhatsApp, or search on Google, these concepts are at work behind the scenes.

But knowing these concepts individually is just the beginning. The real skill lies in knowing when and how to combine them. A URL shortening service uses DNS, load balancing, databases, caching, and consistent hashing. A chat application uses WebSockets, message queues, data replication, and rate limiting.

Come to Class 1 ready to apply these concepts in real design problems. The in-class session will build directly on this foundation.