Redis Deep Dive: Data Structures, Persistence & 10 Real Use Cases

Redis shows up in almost every system design answer, usually as "and we'll put a cache here". That undersells it. Redis is a toolbox of data structures that solve rate limiting, leaderboards, sessions, locks, queues, deduplication and geo search, each with a handful of commands. Used well, it removes whole services from an architecture. Used carelessly, it becomes a single-threaded bottleneck holding data you assumed was durable. This deep dive covers the internals that matter, then ten real use cases with exact commands.

Why Redis Is Fast: The Execution Model

• In memory. Every read and write hits RAM. No disk seek on the request path (persistence happens in the background).
• Single-threaded command execution. One thread runs commands one after another. No locks, no context switches, and every command is atomic by construction: INCR can never lose an update.
• Event loop with I/O multiplexing (epoll/kqueue) serves thousands of connections from that one thread. Since Redis 6, optional I/O threads parallelise reading and writing sockets, while command execution stays single-threaded.
• Compact encodings. Small hashes, lists, sets and sorted sets are stored as contiguous listpacks (and small integer sets as intsets), switching to hash tables or skip lists only when they grow past configured thresholds.
• Simple protocol (RESP) and pipelining: clients can send many commands without waiting for each reply, cutting round trips.

The flip side: one slow command blocks everyone. KEYS * on 50 million keys, HGETALL on a million-field hash, or a long Lua script stalls every other client on that instance. Most Redis incidents come from this.

Redis Data Structures and When to Use Each

Sorted sets: the most useful structure

A sorted set maps unique members to floating-point scores and keeps them ordered by score. Internally (once large) it is a skip list for ordered operations plus a hash table for O(1) member-to-score lookup. That combination gives O(log n) insert/update, O(log n) rank lookup, and O(log n + m) range queries. Whenever a problem says "top K", "rank of X", "items between time A and B" or "next job due", think sorted set.

Hashes vs JSON strings for objects

Storing a user as a JSON string means every field update rewrites the whole value (and races with concurrent updates). A hash lets you HINCRBY user:42 orders 1 or HSET user:42 city Pune atomically per field, and small hashes are memory-efficient listpacks. Redis 7.4 added per-field expiry for hashes (HEXPIRE). For deeply nested documents, the JSON module (part of Redis Stack and included in Redis 8) supports path-level updates.

Persistence: RDB Snapshots vs AOF

Redis is in memory, but it can persist to disk so a restart does not lose everything. The two mechanisms have different trade-offs.

RDB (snapshotting)

Redis calls fork(). The child process writes the entire dataset to a compact binary .rdb file while the parent keeps serving requests. Copy-on-write means memory pages are only duplicated when the parent modifies them during the snapshot. Configured with rules like save 3600 1 300 100 60 10000 (snapshot if at least N changes in M seconds), or triggered with BGSAVE.

• Pros: compact files, ideal for backups and fast restarts; minimal overhead between snapshots.
• Cons: you lose writes since the last snapshot; fork() on a large instance can pause the parent briefly, and a write-heavy workload during the snapshot can nearly double memory usage through copy-on-write.

AOF (append-only file)

Every write command is appended to a log. On restart, Redis replays it. The appendfsync setting controls durability:

The AOF grows forever, so Redis periodically rewrites it in the background into the minimal set of commands that recreate the current state. With aof-use-rdb-preamble yes (default), the rewritten file starts with an RDB snapshot followed by incremental commands, combining fast loading with fine-grained durability. Redis 7 split the AOF into a base file plus incremental files tracked by a manifest.

What to choose

• Pure cache: persistence off, or RDB only for warm restarts.
• Sessions, rate limits, queues where losing a second is acceptable: AOF everysec + RDB for backups.
• Data you cannot lose: do not rely on Redis alone. Replication is asynchronous, so even with AOF always, a failover can promote a replica that missed acknowledged writes. Keep the source of truth in a durable database.

Replication, Sentinel and Redis Cluster

Replication

Replicas connect to a primary, receive a full snapshot, then a continuous stream of write commands. Replication is asynchronous: the primary acknowledges writes before replicas have them. The WAIT numreplicas timeout command blocks until N replicas acknowledge, reducing (not eliminating) the loss window. Replicas can serve reads that tolerate slight staleness.

Sentinel (high availability without sharding)

Redis Sentinel processes monitor a primary and its replicas, agree (by quorum) that the primary is down, promote a replica, and tell clients the new address. Run at least three Sentinels on separate machines.

Redis Cluster (sharding + HA)

• The keyspace is divided into 16,384 hash slots; each primary owns a subset. Adding a node means migrating some slots to it, online.
• Multi-key commands (MGET, SINTER, MULTI transactions, Lua scripts touching several keys) work only if all keys are in the same slot. Use hash tags: only the part inside {} is hashed, so {user:42}:profile and {user:42}:cart land together.
• Each primary has replicas; if a primary fails, the cluster promotes a replica automatically.
• Hash tags concentrate data; overusing one tag recreates a hot shard. The general theory is in database sharding strategies.

A note on the ecosystem: Redis changed its licence in 2024, which led the Linux Foundation to launch Valkey, an open-source fork that is protocol-compatible and offered by several cloud providers; Redis 8 later added an AGPLv3 licensing option. Commands in this guide work on both.

Atomicity: MULTI/EXEC, WATCH and Lua Scripts

Single commands are atomic. For multi-step logic you have three tools.

• MULTI / EXEC: queues commands and executes them together without interleaving. There is no rollback: if one command fails at runtime, the others still apply. You also cannot use a read's result to decide a later write inside the transaction.
• WATCH: optimistic locking. WATCH key, read, then MULTI ... EXEC; EXEC aborts if the watched key changed. Retry on abort.
• Lua scripts (EVAL) and functions (Redis 7+ FUNCTION LOAD / FCALL): run server-side, atomically, with full logic. The standard tool for read-modify-write operations. Keep scripts short: they block the server while running.

10 Real Redis Use Cases with Commands

1. Cache-aside for database reads

import json, random
import redis

r = redis.Redis(decode_responses=True)

def get_restaurant(rid: int) -> dict:
    key = f"restaurant:{rid}"
    cached = r.get(key)
    if cached is not None:
        return json.loads(cached)
    row = db_fetch_restaurant(rid)                    # source of truth
    ttl = 600 + random.randint(0, 120)                # jitter avoids mass expiry
    r.set(key, json.dumps(row), ex=ttl)
    return row

def update_restaurant(rid: int, fields: dict):
    db_update_restaurant(rid, fields)
    r.delete(f"restaurant:{rid}")                     # invalidate after DB write

Delete (do not overwrite) after the database write, add TTL jitter, and for very hot keys protect against cache stampede with a short lock so only one request rebuilds the value. Full treatment in distributed cache design.

2. Session store

HSET session:9f2c user_id 42 role customer city Bengaluru
EXPIRE session:9f2c 1800          # 30-minute idle timeout
HGET session:9f2c user_id         # on each request
EXPIRE session:9f2c 1800          # sliding expiry: refresh on activity
DEL session:9f2c                  # logout

Any app server can read any session, so servers stay stateless.

3. Rate limiting (fixed window and sliding window)

Fixed window, 100 requests per minute per user:

INCR rl:42:202610101532           # key includes the current minute
EXPIRE rl:42:202610101532 60 NX   # set TTL only on first increment (Redis 7+)
# reject if INCR returned > 100

Sliding window log with a sorted set, made atomic in Lua:

-- KEYS[1] = rl:{user}, ARGV: now_ms, window_ms, limit, unique_request_id
local now, window, limit = tonumber(ARGV[1]), tonumber(ARGV[2]), tonumber(ARGV[3])
redis.call('ZREMRANGEBYSCORE', KEYS[1], 0, now - window)
if redis.call('ZCARD', KEYS[1]) >= limit then
  return 0
end
redis.call('ZADD', KEYS[1], now, ARGV[4])
redis.call('PEXPIRE', KEYS[1], window)
return 1

import redis.clients.jedis.JedisPooled;
import java.util.List;
import java.util.UUID;

public class SlidingWindowLimiter {
    private static final String SCRIPT = String.join("\n",
        "local now, window, limit = tonumber(ARGV[1]), tonumber(ARGV[2]), tonumber(ARGV[3])",
        "redis.call('ZREMRANGEBYSCORE', KEYS[1], 0, now - window)",
        "if redis.call('ZCARD', KEYS[1]) >= limit then return 0 end",
        "redis.call('ZADD', KEYS[1], now, ARGV[4])",
        "redis.call('PEXPIRE', KEYS[1], window)",
        "return 1");

    private final JedisPooled jedis = new JedisPooled("localhost", 6379);

    public boolean allow(String userId, int limit, long windowMs) {
        Object res = jedis.eval(SCRIPT,
            List.of("rl:{" + userId + "}"),
            List.of(String.valueOf(System.currentTimeMillis()),
                    String.valueOf(windowMs),
                    String.valueOf(limit),
                    UUID.randomUUID().toString()));
        return Long.valueOf(1L).equals(res);
    }
}

The sliding log is exact but stores one entry per request; at high limits use a sliding window counter or token bucket instead. Algorithms compared in rate limiter system design.

4. Real-time leaderboard

ZINCRBY lb:contest:812 30 "user:42"           # user scored 30 points
ZREVRANGE lb:contest:812 0 9 WITHSCORES       # top 10
ZREVRANK lb:contest:812 "user:42"             # 0-based rank
ZSCORE lb:contest:812 "user:42"
ZCOUNT lb:contest:812 (1500 +inf              # how many above 1500

Ties: encode a tiebreaker into the score (for example score × 10¹⁰ + (MAX_TS − finish_time)) so earlier finishers rank higher. See real-time leaderboard design for sharding very large boards.

5. Distributed lock

SET lock:payout:merchant-77 "a1b2-token" NX PX 30000   # acquire for 30 s
# ... do the work ...
# release ONLY if we still own it (Lua, atomic):
EVAL "if redis.call('GET', KEYS[1]) == ARGV[1] then return redis.call('DEL', KEYS[1]) else return 0 end" 1 lock:payout:merchant-77 a1b2-token

Why the token check: if your process paused (GC, network) past 30 seconds, the lock expired and someone else acquired it; a plain DEL would release their lock. Even with this, a paused holder can still act after expiry, so for correctness-critical work use a fencing token (an increasing number issued with the lock and checked by the database on write). The multi-node Redlock algorithm exists, but its safety under clock and pause assumptions has been publicly debated (notably by Martin Kleppmann); prefer locks for efficiency (avoiding duplicate work) and rely on database constraints for correctness.

6. Job queue with lists

LPUSH jobs:thumbnails '{"video":"v-901"}'           # producer
LMOVE jobs:thumbnails jobs:processing RIGHT LEFT     # worker takes a job atomically
# BLMOVE for blocking wait; after success:
LREM jobs:processing 1 '{"video":"v-901"}'

Moving the job into a processing list (instead of a plain BRPOP) means a crashed worker's job is not lost; a reaper re-queues stale entries. For anything more involved, Streams are better.

7. Reliable event processing with Streams

XADD orders * order_id o-91 amount 450               # returns ID like 1760090000000-0
XGROUP CREATE orders billing $ MKSTREAM
XREADGROUP GROUP billing worker-1 COUNT 10 BLOCK 5000 STREAMS orders >
XACK orders billing 1760090000000-0                  # after successful processing
XAUTOCLAIM orders billing worker-2 60000 0-0 COUNT 10  # take over jobs idle > 60 s
XTRIM orders MAXLEN ~ 1000000                        # cap memory

Streams give consumer groups, per-message acks, a pending entries list and replay, like a small Kafka inside Redis. They are bounded by memory, so trim them. When to graduate to Kafka is covered in Kafka vs RabbitMQ vs SQS.

8. Pub/Sub for live fan-out

SUBSCRIBE match:ind-aus                 # each WebSocket gateway subscribes
PUBLISH match:ind-aus '{"over":"32.4","runs":187}'

Fire-and-forget: subscribers that are disconnected miss messages. Ideal for pushing live updates to WebSocket gateways when the client can re-sync on reconnect (see WebSockets vs SSE). Redis 7 added sharded Pub/Sub (SPUBLISH/SSUBSCRIBE) so cluster-wide broadcast does not hit every node.

9. Counting uniques: HyperLogLog and bitmaps

PFADD uv:product:555:2026-10-10 "user:42" "user:77"
PFCOUNT uv:product:555:2026-10-10                     # approx distinct viewers
PFMERGE uv:product:555:week uv:product:555:2026-10-04 uv:product:555:2026-10-10

SETBIT dau:2026-10-10 42 1                            # user id 42 was active
BITCOUNT dau:2026-10-10                               # exact DAU
BITOP AND active_both dau:2026-10-09 dau:2026-10-10   # active both days

A HyperLogLog uses at most ~12 KB regardless of cardinality. A bitmap for 100 million numeric user IDs is 100M bits ≈ 12.5 MB per day and gives exact counts and set operations. For membership tests with no counts, see Bloom filters (available in Redis via the Bloom module).

10. Geospatial: nearby restaurants or drivers

GEOADD drivers:blr 77.6412 12.9719 "driver:301"      # longitude, latitude
GEOSEARCH drivers:blr FROMLONLAT 77.6309 12.9352 BYRADIUS 3 km ASC COUNT 10 WITHDIST

Geo sets are sorted sets whose scores are 52-bit geohashes, so updates are O(log n). For a moving fleet, update driver positions every few seconds and expire stale drivers via a separate sorted set of last-seen timestamps. More in geohash and quadtrees.

Bonus patterns

• OTP with expiry: SET otp:+9198xxxx 482913 EX 300, plus an attempts counter to block brute force.
• Idempotency keys: SET idem:{key} processing NX EX 86400 before handling a payment request; store the response when done.
• Delayed jobs: sorted set with score = run-at timestamp; workers poll ZRANGEBYSCORE jobs 0 now LIMIT 0 10 and claim with ZREM.

Memory Management and Eviction Policies

Set maxmemory explicitly and leave headroom for fork copy-on-write, replication buffers and fragmentation. When the limit is reached, maxmemory-policy decides what happens:

Redis's LRU is approximated by sampling a few keys (maxmemory-samples, default 5) rather than maintaining an exact list; the exact data structure is what you build in LeetCode 146 LRU cache. Expired keys are removed lazily on access plus by a periodic active expiry cycle.

Production Pitfalls and How to Avoid Them

• KEYS * in production. O(n), blocks the server. Use SCAN with a cursor.
• Big keys. A million-member set or a 50 MB string blocks on read, delete and cluster migration. Split into buckets (followers:42:0..63), delete with UNLINK (asynchronous), find them with redis-cli --bigkeys or --memkeys.
• Hot keys. One key receiving a huge share of traffic saturates one node in a cluster. Add a local in-process cache in front, replicate the key under several names, or use client-side caching (Redis 6 tracking).
• Cache stampede. A popular key expires and thousands of requests rebuild it at once. Use TTL jitter, a rebuild lock, or early probabilistic refresh.
• No TTLs on cache keys. Memory fills with garbage; with noeviction, writes start failing.
• Treating Redis as durable. Async replication can lose acknowledged writes on failover.
• Too many round trips. 50 sequential GETs over a 1 ms network take 50 ms; use MGET or pipelining.
• Long Lua scripts or SORT/ZUNIONSTORE on huge sets blocking the event loop. Watch SLOWLOG GET and latency monitoring.
• Exposed instances. Bind to private networks, enable ACLs and TLS, rename or disable dangerous commands.

How Redis Comes Up in Interviews

Redis appears in system design rounds (caching, rate limiting, leaderboards, sessions, presence, queues) and backend rounds. Interviewers check whether you know which structure to use and what Redis does not guarantee.

Questions to prepare

1. Why is Redis fast if it's single-threaded? In-memory, no lock contention, event loop, efficient encodings; I/O threads for network since 6.0.
2. How would you build a leaderboard for 10 million players? Sorted set; ZINCRBY, ZREVRANK; tiebreak encoded in score; shard by score ranges or by contest if too big for one node.
3. RDB vs AOF? Snapshots vs command log; durability vs restart speed; use both.
4. What happens to data if the primary crashes? Replica promoted by Sentinel or Cluster; writes not yet replicated are lost.
5. How does Redis Cluster shard? How do you do a multi-key operation? 16,384 slots via CRC16; hash tags.
6. Implement a rate limiter. INCR + EXPIRE fixed window, or a Lua sliding window / token bucket.
7. Is a Redis lock safe? SET NX PX + token-checked release; expiry and pauses make it unsafe for correctness without fencing tokens.
8. Cache consistency? Cache-aside with delete-after-write, TTLs as a safety net, and accept brief staleness; or CDC-driven invalidation.

Frequently Asked Questions

Pranjal Jain

Ex-Microsoft SDE · IIT Kanpur · Founder of Prepflix. Helps engineers crack startup and product company interviews across India.